Enterprise knowledge graph reasoning and complementing method and reasoning and complementing system

Abstract

The invention discloses an enterprise knowledge graph inference completion method and inference completion system, and the method comprises the steps: obtaining a to-be-completed knowledge graph of a target enterprise, inputting the to-be-completed knowledge graph into a trained knowledge graph embedding model, and predicting an incomplete triple to achieve the inference completion of the knowledge graph; wherein the knowledge graph embedding model comprises an entity node coding module, a multilayer graph convolutional neural network module, a Softmax pooling layer and a gating neural network based on a self-attention mechanism. According to the method, the knowledge graph embedding model can provide more accurate semantic information for feature interaction, multi-level feature interaction and aggregation of the triple on sub-tensor kernel granularity and global granularity can be realized, and hidden semantics behind entities and relationships of the knowledge graph can be more efficiently mined. Therefore, the knowledge graph completion accuracy, the semantic modeling capability, the interpretability and the like are obviously improved.

Description

Technical Field

[0001] The present invention relates to an enterprise knowledge graph reasoning and completion method and a reasoning and completion system, and belongs to the technical field of knowledge graph reasoning and completion. Background Art

[0002] The knowledge graph is a necessary knowledge base for building an enterprise brain system. By integrating and analyzing the correlation of structured, semi-structured and unstructured data inside and outside the enterprise, a comprehensive, accurate and real-time enterprise knowledge base is formed, providing strong knowledge support for the intelligent operation and decision-making of the enterprise. However, in the process of building the knowledge graph of the enterprise brain, problems such as incomplete knowledge acquisition and insufficient knowledge association are often faced. The initial knowledge graph obtained usually contains a large number of missing or incomplete knowledge triples. Therefore, it is necessary to use the existing knowledge graph structure and semantic information to infer and fill in the missing knowledge triples.

[0003] Existing knowledge graph completion methods mainly use graph-based representation learning methods to capture semantic features and mutual relationships by learning low-dimensional embedding representations of entities and relationships, and then predict missing triples. However, existing methods usually learn a fixed embedding vector for each entity and relationship, ignoring that entities may present different semantic characteristics under different local structures. This static embedding representation is difficult to characterize the diversity and contextual relevance of entities. In addition, existing methods only consider the feature interactions of entities and relationships on a global scale, lacking modeling of local structures and fine-grained semantics, which makes it difficult for the model to capture complex and diverse semantic relationships, especially for long-tail entities and sparse relationships. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide an enterprise knowledge graph reasoning completion method and reasoning completion system. First, the multi-layer graph convolutional neural network module and gated neural network based on the knowledge graph embedding model can provide more accurate semantic information for feature interaction, and realize the multi-level feature interaction and aggregation of triples at the sub-tensor core granularity and global granularity. Secondly, the triple binary classification loss function and the adversarial regularization term loss function are introduced in the training process of the model, which improves the knowledge completion performance and generalization ability of the model, and more efficiently mines the implicit semantics behind the entities and relationships of the knowledge graph. Therefore, there are obvious improvements in the accuracy of knowledge graph completion, semantic modeling capabilities, and interpretability.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] On the one hand, the present invention discloses a method for completing enterprise knowledge graph reasoning, comprising:

[0007] Obtaining a knowledge graph to be completed for a target enterprise; the knowledge graph to be completed includes a plurality of known triples and incomplete triples;

[0008] Inputting the knowledge graph to be completed into a trained knowledge graph embedding model, and predicting the incomplete triples to achieve reasoning completion of the knowledge graph;

[0009] The knowledge graph embedding model includes an entity node encoding module, a multi-layer graph convolutional neural network module, a Softmax pooling layer, and a gated neural network based on a self-attention mechanism; the training steps of the knowledge graph embedding model include:

[0010] Get the training set;

[0011] According to the training set, with the goal of minimizing the total loss function, the pre-constructed knowledge graph embedding model is trained using a gradient descent algorithm to obtain a trained knowledge graph embedding model; the total loss function includes a triple binary classification loss function and an adversarial regularization term loss function.

[0012] Furthermore, the step of obtaining the training set includes:

[0013] Acquire a training knowledge graph, wherein the training knowledge graph includes a plurality of original triples;

[0014] The original triplet is taken as a positive triplet; for each positive triplet, a corresponding negative triplet is generated by a negative sampling strategy;

[0015] Randomly initialize the embedded representation of the head entity, the relation, and the tail entity in the positive triplet or the negative triplet to obtain a positive sample or a negative sample;

[0016] A training set is obtained according to the training knowledge graph, all positive samples and corresponding negative samples.

[0017] Furthermore, the knowledge graph embedding model includes:

[0018] An entity node module, used to obtain an entity node and an adjacency matrix of the entity node according to an input knowledge graph;

[0019] A multi-layer graph convolutional neural network, used for iteratively updating the features of the entity node based on the multi-layer graph convolutional neural network according to the adjacency matrix of the entity node to obtain a final entity embedding representation matrix of the entity node;

[0020] A Softmax pooling layer is used to perform subset partitioning based on a Softmax pooling operation according to the final entity embedding representation matrix of the entity node to obtain a soft partitioning matrix; and perform weighted summation on the final entity embedding representation matrix of the entity node according to the soft partitioning matrix to obtain an aggregated embedding matrix of multiple subsets;

[0021] A gated neural network based on a self-attention mechanism is used to obtain a final feature representation matrix after semantic interaction according to the aggregated embedding matrix of the multiple subsets.

[0022] Furthermore, in the multi-layer graph convolutional neural network, the expression for iteratively updating the features of the entity node by any layer of the graph convolutional neural network is as follows:

[0023] ;

[0024] ;

[0025] ;

[0026] In the formula, Indicates The entity embedding representation matrix obtained by updating the layer graph convolutional neural network; ReLU activation function. Indicates The entity embedding representation matrix obtained by updating the layer graph convolutional neural network; Indicates The weight matrix of the layer graph convolutional neural network;

[0027] Indicates the adjacency matrix added with self-connection; Represents the adjacency matrix of entity nodes; represents the n-order identity matrix;

[0028] Representation Matrix The degree matrix of ; Representation Matrix The element in the i-th row and i-th column of ; Representation Matrix The element at row i and column j in .

[0029] Furthermore, the expression of the soft partitioning matrix is ​​as follows:

[0030] ;

[0031] In the formula, represents the element in the i-th row and j-th column of the soft partition matrix, i.e. The entity embedding belongs to The probability of a subset; Represents the final entity embedding representation matrix Middle entity embedding vectors; represents a learnable partitioning matrix; Representation Matrix The element, i.e. The transformation vector corresponding to the subset; Representation Matrix The element, i.e. The transformation vector corresponding to the subset; Indicates the number of subsets.

[0032] Furthermore, the gated neural network based on the self-attention mechanism includes:

[0033] Input layer, used to input the aggregated embedding matrix of multiple subsets;

[0034] A gating layer, used for performing a gating operation according to the multi-subset aggregate embedding matrix to obtain a multi-subset aggregate embedding matrix after the gating operation;

[0035] An attention layer, configured to calculate an attention weight matrix and a value matrix according to the multi-subset aggregate embedding matrix after the gating operation and perform weighted summation to obtain an attention-enhanced embedding representation matrix;

[0036] A concatenation mapping layer, used for performing feature concatenation and nonlinear mapping according to the multi-subset aggregate embedding matrix after the gating operation and the attention-enhanced embedding representation matrix, to obtain a final feature representation matrix after semantic interaction;

[0037] The output layer is used to output the final feature representation matrix after semantic interaction.

[0038] Furthermore, the expression of the final feature representation matrix after the semantic interaction is as follows:

[0039] ;

[0040] ;

[0041] ;

[0042] In the formula, The final feature representation matrix after semantic interaction representing the head entity; The final feature representation matrix after semantic interaction representing the relationship; The final feature representation matrix after semantic interaction representing the tail entity; ReLU activation function.

[0043] The attention-enhanced embedding representation matrix representing the head entity; An attention-enhanced embedding representation matrix representing the relationship; The attention-augmented embedding representation matrix representing the tail entity.

[0044] The multi-subset aggregate embedding matrix after the gating operation representing the head entity; The multi-subset aggregate embedding matrix after the gating operation representing the relationship; The multi-subset aggregate embedding matrix after the gating operation representing the tail entity;

[0045] The mapping matrix representing the head entity; A mapping matrix representing a relationship; The mapping matrix representing the tail entity;

[0046] The bias term representing the head entity; The bias term that represents the relationship; Represents the bias term for the tail entity.

[0047] Furthermore, the expression of the triple binary classification loss function is as follows:

[0048] ;

[0049] In the formula, Indicates the number of training samples; Indicates The labels of samples, positive samples The value is 1, negative sample The value is 0;

[0050] Indicates The probability score of the samples;

[0051] The expression of the adversarial regularization loss function is as follows:

[0052] ;

[0053] In the formula, represents the adversarial regularization loss function; represents KL divergence; represents the embedding distribution of positive samples; represents the embedding distribution of negative samples;

[0054] Represents the KL divergence between the positive sample distribution and the average distribution;

[0055] Represents the KL divergence between the negative sample distribution and the mean distribution.

[0056] Furthermore, the step of inputting the knowledge graph to be completed into the trained knowledge graph embedding model to realize the reasoning completion of the knowledge graph includes:

[0057] According to the knowledge graph to be completed, a plurality of unknown element candidates corresponding to the incomplete triples are obtained;

[0058] For each unknown element candidate, complete the incomplete triple, and calculate the final feature representation matrix of the completed incomplete triple after semantic interaction based on the trained knowledge graph embedding model, and then calculate the corresponding probability score;

[0059] Unknown element candidates with probability scores higher than a preset threshold are taken as prediction results and added to the incomplete triples to achieve reasoning completion of the knowledge graph.

[0060] On the other hand, the present invention discloses an enterprise knowledge graph reasoning and completion system, which is applicable to the above-mentioned enterprise knowledge graph reasoning and completion method, including:

[0061] A data acquisition module, used to acquire a knowledge graph to be completed for a target enterprise; the knowledge graph to be completed includes a plurality of known triples and incomplete triples;

[0062] An inference completion module is used to input the knowledge graph to be completed into a trained knowledge graph embedding model, and predict the incomplete triples to achieve inference completion of the knowledge graph;

[0063] The knowledge graph embedding model includes an entity node encoding module, a multi-layer graph convolutional neural network module, a Softmax pooling layer, and a gated neural network based on a self-attention mechanism; the training steps of the knowledge graph embedding model include:

[0064] Get the training set;

[0065] According to the training set, with the goal of minimizing the total loss function, the pre-constructed knowledge graph embedding model is trained using a gradient descent algorithm to obtain a trained knowledge graph embedding model; the total loss function includes a triple binary classification loss function and an adversarial regularization term loss function.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] The enterprise knowledge graph reasoning completion method and reasoning completion system of the present invention, first, the pre-constructed knowledge graph embedding model, through the graph convolutional neural network, can adaptively learn the entity embedding representation and dynamically group according to the local neighborhood structure of the entity in the knowledge graph, form a sub-tensor core with higher granularity and rich semantics, and provide more accurate semantic information for feature interaction; through the gated neural network, the multi-level feature interaction and aggregation of the head entity, relationship and tail entity of the triplet can be realized at the sub-tensor core granularity and global granularity. Secondly, in the training process of the knowledge graph embedding model, not only the triplet binary classification loss function is introduced, but also the adversarial regularization term loss function is introduced, which can constrain the distribution characteristics of entity and relationship embedding, improve the quality and robustness of the knowledge graph embedding model for the embedding representation, enhance the knowledge completion performance and generalization ability of the model, and more efficiently mine the implicit semantics behind the entities and relationships of the knowledge graph, and discover long-tail and complex knowledge associations. Compared with the existing methods, the present invention has obvious improvements in the accuracy of knowledge graph completion, semantic modeling ability and interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 This is a flowchart of the enterprise knowledge graph reasoning and completion method provided in Example 1. DETAILED DESCRIPTION

[0069] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0070] Example 1

[0071] This embodiment 1 provides a method for completing enterprise knowledge graph reasoning, such as Figure 1 As shown, including:

[0072] Obtain the target enterprise's knowledge graph to be completed; the knowledge graph to be completed includes multiple known triples and incomplete triples;

[0073] The knowledge graph to be completed is input into the trained knowledge graph embedding model, and the incomplete triples are predicted to achieve reasoning completion of the knowledge graph;

[0074] The knowledge graph embedding model includes an entity node encoding module, a multi-layer graph convolutional neural network module, a Softmax pooling layer, and a gated neural network based on a self-attention mechanism; the training steps of the knowledge graph embedding model include:

[0075] Get the training set;

[0076] According to the training set, with the goal of minimizing the total loss function, the pre-built knowledge graph embedding model is trained using the gradient descent algorithm to obtain the trained knowledge graph embedding model; the total loss function includes the triple binary classification loss function and the adversarial regularization term loss function.

[0077] The technical concept of the present invention is as follows: first, based on the trained knowledge graph embedding model, through a multi-layer graph convolutional neural network module and a Softmax pooling layer, according to the local neighborhood structure of the entity in the knowledge graph, the entity embedding representation is adaptively learned and dynamically grouped to form a sub-tensor core with higher granularity and rich semantics, providing more accurate semantic information for feature interaction. Secondly, through a gated neural network with a self-attention mechanism, the multi-level feature interaction and aggregation of the head entity, relationship and tail entity of the triplet at the sub-tensor core granularity and global granularity are realized. In the training process of the knowledge graph embedding model, an adversarial regularization term loss function is introduced to constrain the distribution characteristics of entity and relationship embedding, improve the quality and robustness of the knowledge graph embedding model for embedding representation, enhance the knowledge completion performance and generalization ability of the model, and more efficiently mine the implicit semantics behind the entities and relationships of the knowledge graph, and discover long-tail and complex knowledge associations. Compared with the existing methods, the present invention has obvious improvements in the accuracy of knowledge graph completion, semantic modeling ability and interpretability.

[0078] The specific steps are as follows:

[0079] Step 1: Construction of knowledge graph embedding model.

[0080] The knowledge graph embedding model includes the following structure:

[0081] 1.1. The entity node module is used to obtain the entity nodes and the adjacency matrix of the entity nodes based on the input knowledge graph.

[0082] For the knowledge graph data of the enterprise brain, it is usually stored in the form of triples, each of which includes a head entity ,relation and tail entity , which can be expressed as For a knowledge graph with multiple entities and relations, the set of all triples can be expressed as ,in, is a collection of entities, A set of relations.

[0083] In the entity node module, the head entity and the tail entity in the input knowledge graph are regarded as two entity nodes respectively, and the relationship is the edge between the two entity nodes. For all entity nodes in the knowledge graph, the adjacency matrix of the entity nodes is constructed. , n represents the number of entity nodes in the knowledge graph. Specifically, the elements in the adjacency matrix of the entity node are Represents an entity node and When there is a relationship edge between two entities, , otherwise, its value is 0.

[0084] 1.2. A multi-layer graph convolutional neural network is used to iteratively update the features of the entity nodes based on the adjacency matrix of the entity nodes and obtain the final entity embedding representation matrix of the entity nodes.

[0085] Specifically, The expression for iteratively updating the features of entity nodes by the layer graph convolutional neural network is as follows:

[0086] ;

[0087] ;

[0088] ;

[0089] In the formula, Indicates The entity embedding representation matrix obtained by updating the layer graph convolutional neural network; ReLU activation function. Indicates The entity embedding representation matrix obtained by updating the layer graph convolutional neural network; Indicates The weight matrix of the layer graph convolutional neural network;

[0090] Indicates the adjacency matrix added with self-connection; Represents the adjacency matrix of entity nodes; represents the n-order identity matrix;

[0091] Representation Matrix The degree matrix of ; Representation Matrix The element in the i-th row and i-th column of ; Representation Matrix The element at row i and column j in .

[0092] Through the iterative update of the multi-layer graph convolutional neural network, the final entity embedding representation matrix of the entity node is obtained ,in, is the embedding dimension.

[0093] 1.3. Softmax pooling layer, which is used to perform subset partitioning based on the Softmax pooling operation according to the final entity embedding representation matrix of the entity node to obtain a soft partitioning matrix; based on the soft partitioning matrix, the final entity embedding representation matrix of the entity node is weighted summed to obtain the aggregated embedding matrix of multiple subsets.

[0094] Specifically, the final entity embedding representation matrix is ​​divided into k subsets based on the Softmax pooling operation. The expression of the soft partition matrix is ​​as follows:

[0095] ;

[0096] In the formula, represents the element in the i-th row and j-th column of the soft partition matrix, i.e. The entity embedding belongs to The probability of a subset; Represents the final entity embedding representation matrix Middle entity embedding vectors; represents a learnable partitioning matrix; Representation Matrix The element, i.e. The transformation vector corresponding to the subset; Representation Matrix The element, i.e. The transformation vector corresponding to the subset; Indicates the number of subsets.

[0097] Then, according to the soft partitioning matrix, the final entity embedding representation matrix of the entity node is weighted summed to obtain the aggregate embedding matrix of multiple subsets , represents the number of subsets, represents the embedding dimension, and each aggregated embedding vector in the aggregated embedding matrix corresponds to a dense sub-tensor kernel.

[0098] 1.4. A gated neural network based on the self-attention mechanism is used to obtain the final feature representation matrix after semantic interaction based on the aggregated embedding matrix of multiple subsets.

[0099] The gated neural network based on the self-attention mechanism can model the correlation between different embedding subsets, including the following structure:

[0100] 1.4.1. Input layer, used to input the aggregated embedding matrix of multiple subsets , the aggregate embedding matrix of multiple subsets The dimension is ; represents the number of subsets, Represents the embedding dimension.

[0101] 1.4.2. Gating layer for embedding matrix based on multiple subset aggregation , perform gating operations to control information flow and feature selection, and obtain the multi-subset aggregate embedding matrix after gating operations.

[0102] The dimension of the multi-subset aggregate embedding matrix after the gating operation is , represents the feature dimension after the gating operation, .

[0103] 1.4.3. Attention layer, which is used to calculate the attention weight matrix and value matrix according to the multi-subset aggregation embedding matrix and perform weighted summation to obtain the attention-enhanced embedding representation matrix.

[0104] First, the multi-subset aggregate embedding matrices of the head entity, relationship, and tail entity are linearly transformed to obtain the dimensions of the head entity, relationship, and tail entity: The query matrix, key matrix and value matrix are as follows:

[0105]

[0106] In the formula, The query matrix representing the head entity; A query matrix representing the relationship; The query matrix representing the tail entity; The key matrix representing the head entity; A bond matrix representing a relationship; The bond matrix representing the tail entity; A matrix of values ​​representing the head entity; A matrix of values ​​representing the relationship; A matrix of values ​​representing the tail entity; Represents the multi-subset aggregate embedding matrix of the head entity; Representing the multi-subset aggregate embedding matrix of the relationship; Represents the multi-subset aggregate embedding matrix of the tail entity; represents the first learnable linear transformation matrix; represents the second learnable linear transformation matrix; represents the third learnable linear transformation matrix, , is the dimension of the attention layer.

[0107] Secondly, calculate the attention weight matrix of the head entity, relation, and tail entity. The details are as follows:

[0108] ;

[0109] ;

[0110] ;

[0111] In the formula, Represents the attention weight matrix of the head entity; The attention weight matrix representing the relationship; Represents the attention weight matrix of the tail entity; represents the softmax activation function; ⊤ represents the transposition operation of the matrix.

[0112] Finally, the dimension is The attention weight matrix of the head entity, relationship and tail entity is weighted summed with the value matrix to obtain a dimension of Attention-enhanced embedding representation matrices of head entity, relation, and tail entity.

[0113] Specifically, the expressions of the attention-enhanced embedding representation matrices of the head entity, relation, and tail entity are as follows:

[0114] ;

[0115] ;

[0116] ;

[0117] In the formula, The attention-enhanced embedding representation matrix representing the head entity; An attention-enhanced embedding representation matrix representing the relationship; The attention-augmented embedding representation matrix representing the tail entity.

[0118] 1.4.4, splicing mapping layer, used to The multi-subset aggregate embedding matrix and dimension after the gating operation are The attention-enhanced embedding representation matrix is ​​concatenated and nonlinearly mapped to obtain a dimension of The final feature representation matrix after semantic interaction, Represents the final feature dimension after nonlinear mapping.

[0119] Specifically, the attention-enhanced embedding representation matrix of the head entity, relationship, and tail entity is concatenated with the output of the gating layer, and mapped by the activation function to obtain the final feature representation matrix after semantic interaction. The details are as follows:

[0120] ;

[0121] ;

[0122] ;

[0123] In the formula, The final feature representation matrix after semantic interaction representing the head entity; The final feature representation matrix after semantic interaction representing the relationship; The final feature representation matrix after semantic interaction representing the tail entity; ReLU activation function.

[0124] The attention-enhanced embedding representation matrix representing the head entity; An attention-enhanced embedding representation matrix representing the relationship; The attention-augmented embedding representation matrix representing the tail entity.

[0125] The multi-subset aggregate embedding matrix after the gating operation representing the head entity; The multi-subset aggregate embedding matrix after the gating operation representing the relationship; The multi-subset aggregate embedding matrix after the gating operation representing the tail entity;

[0126] The mapping matrix representing the head entity; A mapping matrix representing a relationship; The mapping matrix representing the tail entity;

[0127] The bias term representing the head entity; The bias term that represents the relationship; Represents the bias term for the tail entity.

[0128] Through attention-driven feature interaction, the correlation between head entities, relations, and tail entities in different subsets is adaptively captured, highlighting the semantic information of key embedding subsets.

[0129] 1.4.5. Output layer, used to output the final feature representation matrix after semantic interaction, including the final feature representation matrix after semantic interaction of head entity, relation and tail entity.

[0130] The structural information of the knowledge graph, that is, the connection relationship and relationship type between entities, is processed by the knowledge graph embedding model, and the final feature representation of the head entity, relationship, and tail entity is output. The final feature representation encodes the semantic information of the entities and relationships in the knowledge graph and the interaction model between them. The essence of this model is the process of mapping the entities and relationships in the knowledge graph to a low-dimensional continuous vector space. Through this mapping, the semantic relationship and structural characteristics between entities and relationships can be captured in the embedding space.

[0131] Step 2: Training of knowledge graph embedding model.

[0132] The specific steps are as follows:

[0133] 2.1. Obtain the training set.

[0134] Obtain a training knowledge graph, where the training knowledge graph includes multiple original triples;

[0135] The original triplet is taken as a positive triplet; for each positive triplet, the head entity or the tail entity in the positive triplet is replaced by a negative sampling strategy to generate a corresponding negative triplet;

[0136] Randomly initialize the embedded representations of the head entity, relation, and tail entity in the positive triplet or negative triplet to obtain positive samples or negative samples;

[0137] According to the training knowledge graph, all positive samples and corresponding negative samples, the training set is obtained.

[0138] 2.2. According to the training set, with the goal of minimizing the total loss function, the pre-built knowledge graph embedding model is trained using the gradient descent algorithm to obtain the trained knowledge graph embedding model; the total loss function includes the triple binary classification loss function and the adversarial regularization term loss function.

[0139] The principle of training is: adopt triple binary classification loss, introduce the strategy of regularization term based on generative adversarial network, and train the knowledge graph embedding model. The knowledge graph embedding space is used as the generator, and the negative sample embedding obtained by negative sampling is used as the discriminator. By minimizing the JS divergence of the positive and negative sample embedding distribution, the distribution characteristics of the embedding space are optimized, thereby improving the quality of the embedding representation and the generalization performance of the model.

[0140] 2.2.1. The expression of the triple binary classification loss function is as follows:

[0141] ;

[0142] In the formula, Indicates the number of training samples; Indicates The labels of samples, positive samples The value is 1, negative sample The value is 0; Indicates The probability score of a sample.

[0143] The calculation steps of probability score are as follows:

[0144] For any sample, it is output to the knowledge graph embedding model to obtain the final feature representation matrix after the semantic interaction of the head entity, relationship and tail entity. The final feature representation matrix after the semantic interaction of the head entity, relationship and tail entity is multiplied element by element and then the vector elements are summed. Finally, it is processed by the Sigmoid activation function to obtain the probability score of the sample.

[0145] 2.2.2. The expression of the loss function of the adversarial regularization term is as follows:

[0146] ;

[0147] In the formula, represents the adversarial regularization loss function; represents KL divergence; Represents the positive sample embedding distribution; represents the embedding distribution of negative samples;

[0148] Represents the KL divergence between the positive sample distribution and the average distribution;

[0149] Represents the KL divergence between the negative sample distribution and the mean distribution.

[0150] Step 3: Application of knowledge graph embedding model.

[0151] 3.1. Obtain the knowledge graph to be completed for the target enterprise.

[0152] An initial enterprise knowledge graph is constructed by integrating and analyzing the association of structured, semi-structured and unstructured data inside and outside the target enterprise. Due to problems such as incomplete knowledge acquisition and insufficient knowledge association, the initial enterprise knowledge graph usually contains a large number of missing or incomplete knowledge triples. It is necessary to use the existing knowledge graph structure and semantic information to infer and fill in the missing knowledge triples.

[0153] Therefore, the initial enterprise knowledge graph is used as the knowledge graph to be completed, which includes multiple known triples and incomplete triples. Among them, the incomplete triples include the following three cases: case 1, the head entity and relationship are known, and the tail entity is unknown; case 2, the head entity and tail entity are known, and the relationship is unknown; case 3, the relationship and tail entity are known, and the head entity is unknown.

[0154] 3.2. According to the knowledge graph to be completed, multiple unknown element candidates corresponding to the incomplete triples are obtained. For each unknown element candidate, the incomplete triples are completed, and the final feature representation matrix after the semantic interaction of the completed incomplete triples is calculated based on the trained knowledge graph embedding model, and then the corresponding probability score is calculated.

[0155] The calculation steps of the probability score include: element-by-element multiplication of the final feature representation matrix after the semantic interaction of the head entity, relationship and tail entity, and then summing the vector elements, and finally processing it with the Sigmoid activation function to obtain the corresponding probability score.

[0156] Unknown element candidates with probability scores higher than the preset threshold are taken as prediction results and added to the incomplete triples to achieve reasoning completion of the knowledge graph.

[0157] Specifically, assuming that the head entity and relation of the incomplete triple are known, its potential tail entity is predicted through the knowledge graph embedding model, thereby inferring and completing the tail entity of the incomplete triple.

[0158] First, all possible tail entity candidates in the knowledge graph are traversed, and the incomplete triples are completed respectively to obtain multiple completed incomplete triples.

[0159] Secondly, for any completed incomplete triple, the final feature representation matrix after the semantic interaction of the completed incomplete triple is calculated through the trained knowledge graph embedding model, and then the corresponding probability score is calculated.

[0160] Afterwards, all tail entity candidates are sorted according to the probability scores to obtain an ordered candidate list. The higher ranked tail entity candidates are considered to be the most likely tail entities given the head entity and relationship. According to a pre-set threshold, the candidate entities with triplet scores higher than the threshold are taken as the predicted tail entities.

[0161] Finally, the triple consisting of the predicted tail entity and the known head entity and relation is added to the knowledge graph to complete the missing knowledge.

[0162] Example 2

[0163] This embodiment 2 provides an enterprise knowledge graph reasoning and completion system, which is applicable to the enterprise knowledge graph reasoning and completion method of embodiment 1, including:

[0164] A data acquisition module is used to acquire the knowledge graph to be completed of the target enterprise; the knowledge graph to be completed includes multiple known triples and incomplete triples;

[0165] The reasoning completion module is used to input the knowledge graph to be completed into the trained knowledge graph embedding model, and realize the reasoning completion of the knowledge graph by predicting the incomplete triples;

[0166] The knowledge graph embedding model includes an entity node encoding module, a multi-layer graph convolutional neural network module, a Softmax pooling layer, and a gated neural network based on a self-attention mechanism; the training steps of the knowledge graph embedding model include:

[0167] Get the training set;

[0168] According to the training set, with the goal of minimizing the total loss function, the pre-built knowledge graph embedding model is trained using the gradient descent algorithm to obtain the trained knowledge graph embedding model; the total loss function includes the triple binary classification loss function and the adversarial regularization term loss function.

[0169] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0170] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0173] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for completing enterprise knowledge graph reasoning, characterized in that: include: Obtain the knowledge graph to be completed for the target enterprise; The knowledge graph to be completed includes multiple known triples and incomplete triples; Inputting the knowledge graph to be completed into a trained knowledge graph embedding model, and predicting the incomplete triples to achieve reasoning completion of the knowledge graph; The knowledge graph embedding model includes an entity node encoding module, a multi-layer graph convolutional neural network module, a Softmax pooling layer, and a gated neural network based on a self-attention mechanism; the training steps of the knowledge graph embedding model include: Get the training set; According to the training set, with the goal of minimizing the total loss function, the pre-constructed knowledge graph embedding model is trained using a gradient descent algorithm to obtain a trained knowledge graph embedding model; the total loss function includes a triple binary classification loss function and an adversarial regularization term loss function.

2. The enterprise knowledge graph reasoning and completion method according to claim 1 is characterized in that: The step of obtaining the training set comprises: Acquire a training knowledge graph, wherein the training knowledge graph includes a plurality of original triples; The original triplet is taken as a positive triplet; for each positive triplet, a corresponding negative triplet is generated by a negative sampling strategy; Randomly initialize the embedded representation of the head entity, the relation, and the tail entity in the positive triplet or the negative triplet to obtain a positive sample or a negative sample; A training set is obtained according to the training knowledge graph, all positive samples and corresponding negative samples.

3. The enterprise knowledge graph reasoning and completion method according to claim 1 is characterized in that: The knowledge graph embedding model includes: An entity node module, used to obtain an entity node and an adjacency matrix of the entity node according to an input knowledge graph; A multi-layer graph convolutional neural network, used for iteratively updating the features of the entity node based on the multi-layer graph convolutional neural network according to the adjacency matrix of the entity node to obtain a final entity embedding representation matrix of the entity node; A Softmax pooling layer is used to perform subset partitioning based on a Softmax pooling operation according to the final entity embedding representation matrix of the entity node to obtain a soft partitioning matrix; and perform weighted summation on the final entity embedding representation matrix of the entity node according to the soft partitioning matrix to obtain an aggregated embedding matrix of multiple subsets; A gated neural network based on a self-attention mechanism is used to obtain a final feature representation matrix after semantic interaction according to the aggregated embedding matrix of the multiple subsets.

4. The enterprise knowledge graph reasoning and completion method according to claim 3 is characterized in that: In the multi-layer graph convolutional neural network, the expression for iteratively updating the features of the entity node by any layer of the graph convolutional neural network is as follows: ； ； ； In the formula, Indicates The entity embedding representation matrix obtained by updating the layer graph convolutional neural network; ReLU activation function. Indicates The entity embedding representation matrix obtained by updating the layer graph convolutional neural network; Indicates The weight matrix of the layer graph convolutional neural network; Indicates the adjacency matrix added with self-connection; Represents the adjacency matrix of entity nodes; represents the n-order identity matrix; Representation Matrix The degree matrix of ; Representation Matrix The element in the i-th row and i-th column of ; Representation Matrix The element at row i and column j in .

5. The enterprise knowledge graph reasoning and completion method according to claim 3 is characterized in that: The expression of the soft partitioning matrix is ​​as follows: ； In the formula, represents the element in the i-th row and j-th column of the soft partition matrix, i.e. The entity embedding belongs to The probability of a subset; Represents the final entity embedding representation matrix Middle entity embedding vectors; represents a learnable partitioning matrix; Representation Matrix The element, namely The transformation vector corresponding to the subset; Representation Matrix The element, namely The transformation vector corresponding to the subset; Indicates the number of subsets.

6. The enterprise knowledge graph reasoning and completion method according to claim 3 is characterized in that: The gated neural network based on the self-attention mechanism includes: Input layer, used to input the aggregated embedding matrix of multiple subsets; A gating layer, used for performing a gating operation according to the multi-subset aggregate embedding matrix to obtain a multi-subset aggregate embedding matrix after the gating operation; An attention layer, configured to calculate an attention weight matrix and a value matrix according to the multi-subset aggregate embedding matrix after the gating operation and perform weighted summation to obtain an attention-enhanced embedding representation matrix; A concatenation mapping layer, used for performing feature concatenation and nonlinear mapping according to the multi-subset aggregate embedding matrix after the gating operation and the attention-enhanced embedding representation matrix, to obtain a final feature representation matrix after semantic interaction; The output layer is used to output the final feature representation matrix after semantic interaction.

7. The enterprise knowledge graph reasoning and completion method according to claim 6 is characterized in that: The expression of the final feature representation matrix after the semantic interaction is as follows: ； ； ； In the formula, The final feature representation matrix after semantic interaction representing the head entity; The final feature representation matrix after semantic interaction representing the relationship; The final feature representation matrix after semantic interaction representing the tail entity; ReLU activation function. The attention-enhanced embedding representation matrix representing the head entity; An attention-enhanced embedding representation matrix representing the relationship; The attention-enhanced embedding representation matrix representing the tail entity; The multi-subset aggregate embedding matrix after the gating operation representing the head entity; The multi-subset aggregate embedding matrix after the gating operation representing the relationship; The multi-subset aggregate embedding matrix after the gating operation representing the tail entity; The mapping matrix representing the head entity; A mapping matrix representing a relationship; The mapping matrix representing the tail entity; The bias term representing the head entity; The bias term that represents the relationship; Represents the bias term for the tail entity.

8. The enterprise knowledge graph reasoning and completion method according to claim 1 is characterized in that: The expression of the triplet binary classification loss function is as follows: ； In the formula, Indicates the number of training samples; Indicates The labels of samples, positive samples The value is 1, negative sample The value is 0; Indicates The probability score of the samples; The expression of the adversarial regularization loss function is as follows: ； In the formula, represents the adversarial regularization loss function; represents KL divergence; Represents the positive sample embedding distribution; represents the embedding distribution of negative samples; Represents the KL divergence between the positive sample distribution and the average distribution; Represents the KL divergence between the negative sample distribution and the mean distribution.

9. The enterprise knowledge graph reasoning and completion method according to claim 1 is characterized in that: The steps of inputting the knowledge graph to be completed into the trained knowledge graph embedding model to realize the reasoning completion of the knowledge graph include: According to the knowledge graph to be completed, a plurality of unknown element candidates corresponding to the incomplete triples are obtained; For each unknown element candidate, complete the incomplete triple, and calculate the final feature representation matrix of the completed incomplete triple after semantic interaction based on the trained knowledge graph embedding model, and then calculate the corresponding probability score; Unknown element candidates with probability scores higher than a preset threshold are taken as prediction results and added to the incomplete triples to achieve reasoning completion of the knowledge graph.

10. An enterprise knowledge graph reasoning and completion system, applicable to the enterprise knowledge graph reasoning and completion method according to any one of claims 1 to 9, characterized in that: include: The data acquisition module is used to obtain the knowledge graph to be completed by the target enterprise; The knowledge graph to be completed includes multiple known triples and incomplete triples; An inference completion module is used to input the knowledge graph to be completed into a trained knowledge graph embedding model, and predict the incomplete triples to achieve inference completion of the knowledge graph; The knowledge graph embedding model includes an entity node encoding module, a multi-layer graph convolutional neural network module, a Softmax pooling layer, and a gated neural network based on a self-attention mechanism; the training steps of the knowledge graph embedding model include: Get the training set; According to the training set, with the goal of minimizing the total loss function, the pre-constructed knowledge graph embedding model is trained using a gradient descent algorithm to obtain a trained knowledge graph embedding model; the total loss function includes a triple binary classification loss function and an adversarial regularization term loss function.

Images