Complementation method for internal link relationship of equipment knowledge graph

Abstract

The invention provides an equipment knowledge graph internal link relation completion method. The method comprises the steps of obtaining an instance embedding initialization matrix, a relation embedding initialization matrix and an attribute embedding initialization matrix; obtaining a first-layer instance embedding optimization matrix; obtaining a second-layer instance embedding optimization matrix; fusing the second-layer instance embedding optimization matrix with the instance embedding initialization matrix to obtain an instance embedding fusion matrix; constructing an encoder to train the instance embedding fusion matrix and the relation embedding matrix to obtain a final instance embedding matrix and a final relation embedding matrix; and constructing a multi-dimensional adaptive convolutional decoder, inputting the instance attribute embedded feature matrixes of the relationship to be complemented into the decoder, calculating scores of each pair of feature matrixes under different relationships through the decoder, and obtaining a relationship corresponding to the highest score as a final complementation relationship of the input instance pair.

Description

Technical Field

[0001] The present disclosure relates to a method for completing internal link relationships in an equipment knowledge graph, and in particular to a method for completing internal link relationships in an equipment knowledge graph based on an attention mechanism. Background Art

[0002] The knowledge triples in the knowledge graph can be divided into type triples, attribute triples and relationship triples. Among them, the relationship in the relationship triple is the key element that connects different instance nodes in the knowledge graph, and is also the main feature that distinguishes the knowledge graph from other data representation structures.

[0003] The knowledge graph is a representative of the network data structure, in which relations play the role of logically connecting different instance nodes. From a semantic point of view, relations can transform one type of node into another type of node; from a logical point of view, relations reflect people's cognition of the connection between different things in the real world. Therefore, the richness of relational information in the knowledge graph directly affects the effect of knowledge graph search, question-answering and other applications. It can be said that the number and types of relational triples in the knowledge graph restrict the ultimate application value of the knowledge graph.

[0004] In the equipment knowledge graph built with open source data as the core, the types of relationship information can be well guaranteed. However, due to the lack of attention to basic knowledge and scarcity of rare knowledge in open source data, the number of relationship triples of some types is difficult to be guaranteed from open source data. Therefore, it is necessary to use the method of link relationship completion to improve the content of the equipment knowledge graph with existing internal knowledge to enhance the application value of the open source equipment knowledge graph.

[0005] Traditional methods for completing instance link relationships within knowledge graphs mainly utilize the relationship triple knowledge in the knowledge graph and process each set of relationship triple information as an independent triple. This type of processing is simple and efficient, but it ignores the rich semantic information around the instance nodes to a certain extent, resulting in poor results in completing the relationship. Summary of the invention

[0006] In order to solve one of the above technical problems, the present disclosure provides a method for completing the internal link relationship of an equipment knowledge graph.

[0007] According to one aspect of the present disclosure, a method for completing internal link relationships in an equipment knowledge graph is provided, comprising:

[0008] Use the translation model to process the instance relationship triple information and instance attribute triple information of the equipment knowledge graph to obtain the instance embedding initialization matrix, the relationship embedding initialization matrix and the attribute embedding initialization matrix;

[0009] Use the improved first-layer graph attention network model to process the instance embedding initialization matrix and the relationship embedding initialization matrix to obtain the first-layer instance embedding optimization matrix;

[0010] The first-layer instance embedding optimization matrix and the relationship embedding matrix are processed by the improved second-layer graph attention network model to obtain the second-layer instance embedding optimization matrix;

[0011] The second-layer instance embedding optimization matrix is ​​fused with the instance embedding initialization matrix to obtain the instance embedding fusion matrix;

[0012] Constructing an encoder to train the instance embedding fusion matrix and the relation embedding matrix to obtain the final instance embedding matrix and the relation embedding matrix; and

[0013] A multi-dimensional adaptive convolutional decoder is constructed, and the instance attribute embedding feature matrix of the to-be-completed relation is input into the decoder. The decoder calculates the scores of each pair of feature matrices under different relations, and obtains the relation corresponding to the highest score as the final completion relation of the input instance pair.

[0014] According to the method for completing the internal link relationship of the equipment knowledge graph of at least one embodiment of the present disclosure, the improved first-layer graph attention network model and the improved second-layer graph attention network model are relationship-based graph attention network models.

[0015] According to the equipment knowledge graph internal link relationship completion method of at least one embodiment of the present disclosure, the relationship-based graph attention network model inputs the instance embedding initialization matrix in the first layer of the graph attention network model And the relation embedding initialization matrix

[0016] For instance relation triples (e i ,r k ,e j )'s initial embedding information The concatenation is performed, and the concatenation result is processed by the linear transformation matrix M1 to obtain a new embedding vector. The formula is shown in formula (3):

[0017]

[0018] in, Represents a triple (e i ,r k ,e j ) is the embedded vector after being processed by M1, || represents the concatenation operation, Represents instance e i The initial embedding vector of Represents instance e j The initial embedding vector of Represents the relationship k The initial embedding vector of

[0019] Then use another linear transformation matrix M2 to transform the new embedding vector Convert and use the LeakyReLU function to calculate the current triplet (e i ,r k ,e j ) is calculated as shown in formula (4):

[0020]

[0021] Finally, the calculation results are normalized using the softmax function, as shown in formula (5):

[0022]

[0023] Among them, N i is an instance i The set of all adjacent instance nodes, R in Indicates link instance e i and instance n The set of all relations of ;

[0024] After obtaining the instance node e i The attention information of each relation triplet is applied to the instance node e i The embedding representation of is updated, and the formula for updating the embedding representation is shown in formula (6):

[0025]

[0026] Where σ1 represents an arbitrary nonlinear function.

[0027] The multi-head attention mechanism is used to average the learned domain information; for the first-layer graph attention network model, the calculated multi-head attention results are integrated by splicing to obtain the first-layer instance embedding optimization matrix Among them, H′ represents the matrix composed of all instance embedding information after optimization, N e is the number of instances, T′ represents the dimension after instance embedding optimization, and the calculation formula of each instance embedding information is shown in formula (7):

[0028]

[0029] Among them, Z represents the set of attention heads, and |Z| represents the number of attention heads.

[0030] According to the method for completing the internal link relationship of the equipment knowledge graph of at least one embodiment of the present disclosure, the fully connected layer is used to transform the relationship embedding matrix. The transformation formula is shown in formula (8):

[0031] G′=GM R (8)

[0032] Among them, G represents the relationship embedding matrix, represents the transformation matrix corresponding to the fully connected layer, and P′ represents the dimension of the transformed relation embedding.

[0033] According to the method for completing the internal link relationship of the equipment knowledge graph according to at least one embodiment of the present disclosure, the first layer instance output by the first layer graph attention network model is embedded in the optimization matrix The converted relation embedding matrix G′ is input into the second-layer graph attention network model to obtain the second-layer instance embedding optimization matrix.

[0034] According to the method for completing the internal link relationship of the equipment knowledge graph in at least one embodiment of the present disclosure, the fusion matrix H″=H i +H f =M E H t +H f (10)

[0035] Among them, H t represents the initial instance embedding matrix, H i represents the transformed instance embedding matrix, represents the transformation matrix, T i represents the dimension of the initial instance embedding vector, T f represents the dimension of the final instance embedding vector, Represents the instance embedding matrix output by the second-layer graph attention network model, N e Indicates the number of instances. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0037] Figure 1 It is a flowchart of a method for completing internal link relationships in an equipment knowledge graph according to an embodiment of the present disclosure.

[0038] Figure 2 It is an architectural diagram of a model of a method for completing internal connection relationships in an assembly knowledge graph according to an embodiment of the present disclosure.

[0039] Figure 3 This is an architecture diagram of a single-head attention mechanism of a relation-based graph attention network model according to an embodiment of the present disclosure.

[0040] Figure 4 It is an architecture diagram of a multi-dimensional adaptive convolutional network model according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0042] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0043] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0044] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0045] When a component is referred to as being "on" or "over," "connected to," or "coupled to" another component, the component may be directly on, directly connected to, or directly coupled to the other component, or intervening components may be present. However, when a component is referred to as being "directly on," "directly connected to," or "directly coupled to" another component, there are no intervening components. For this purpose, the term "connected" may refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0046] For descriptive purposes, the present disclosure may use spatially relative terms such as "under," "beneath," "under," "down," "over," "upper," "above," "higher," and "side (e.g., as in "sidewall")," to describe the relationship of one component to another (other) component as shown in the accompanying drawings. The spatially relative terms are intended to encompass different orientations of the device in use, operation, and / or manufacture in addition to the orientation depicted in the accompanying drawings. For example, if the device in the accompanying drawings is turned over, components described as "under" or "beneath" other components or features would subsequently be positioned "over" the other components or features. Thus, the exemplary term "under" can encompass both the "above" and "below" orientations. Furthermore, the device may be otherwise oriented (e.g., rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein should be interpreted accordingly.

[0047] The terms used here are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used here, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used here, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0048] Figure 1 It is a structural diagram of a method for completing internal link relationships in an equipment knowledge graph according to an embodiment of the present disclosure.

[0049] The present disclosure provides a method for completing the internal link relationship of an equipment knowledge graph, which includes: using a translation model to process instance relationship triple information and instance attribute triple information of an equipment knowledge graph to obtain an instance embedding initialization matrix, a relationship embedding initialization matrix and an attribute embedding initialization matrix; using an improved first-layer graph attention network model to process the instance embedding initialization matrix and the relationship embedding initialization matrix to obtain a first-layer instance embedding optimization matrix; processing the first-layer instance embedding optimization matrix and the relationship embedding matrix through an improved second-layer graph attention network model to obtain a second-layer instance embedding optimization matrix; fusing the second-layer instance embedding optimization matrix with the instance embedding initialization matrix to obtain an instance embedding fusion matrix; constructing an encoder to train the instance embedding fusion matrix and the relationship embedding matrix to obtain a final instance embedding matrix and a relationship embedding matrix; and constructing a multi-dimensional adaptive convolution decoder, and inputting the instance attribute embedding feature matrix of the relationship to be completed into the decoder, calculating the score of each pair of feature matrices under different relationships through the decoder, and obtaining the relationship corresponding to the highest score as the final completion relationship of the input instance pair.

[0050] Figure 2 It is an architectural diagram of a model of a method for completing internal connection relationships in an assembly knowledge graph according to an embodiment of the present disclosure.

[0051] In this disclosure, a translation model (TransE model) is first used to initialize instance embedding, relationship embedding, and attribute embedding in the equipment knowledge graph, providing an initial embedding vector for subsequent acquisition of a deeper embedding representation.

[0052] Specifically, the translation model is used to train the instance relationship triple information. The basic idea of ​​the translation model is based on the translation invariance of independent knowledge triples. The score function of the translation model is shown in formula (1):

[0053]

[0054] Where e represents the embedding of the head instance, r represents the embedding of the relation, Represents the embedding of the tail instance.

[0055] Thus, the instance embedding initialization matrix can be obtained And the relation embedding initialization matrix Among them, H represents the matrix composed of all instance embedding information, N e is the number of instances, T represents the dimension of instance embedding, G represents the matrix composed of all relationship embedding information, N r is the number of relations, and P represents the dimension of relation embedding. The instance embedding initialization matrix and the relation embedding initialization matrix can be used as initialization input for subsequent acquisition of semantic information around the target instance node.

[0056] Similarly, the instance-based and attribute-based translation model (EPRTP model) is used to learn attribute embedding representation. This model introduces attribute information based on the TransE model. The score function of the EPRTP model is shown in formula (2):

[0057]

[0058] Where e represents the embedding of head instance e, r represents the embedding of relation r, Represents the tail instance Embedded, represents the embedding of the i-th attribute name related to instance e in the instance space, W e ∈W represents the set of all attribute names related to instance e, |W e | represents the number of attribute names in the attribute name set related to instance e, Indicates the instance in the instance space The embedding of the jth attribute name, Representation instance The collection of all related attribute names, Representation instance The number of attribute names in the related attribute name set. W represents the set of all attribute names in the instance space.

[0059] Thus, the attribute embedding initialization matrix can be obtained Among them, K represents the matrix composed of all attribute embedding information, N P is the number of attributes, and Q represents the dimension of attribute embedding. The attribute embedding matrix can be used to subsequently construct the attribute filter in the multi-dimensional adaptive convolution decoder.

[0060] In the present disclosure, the improved first-layer attention model and the improved second-layer attention model are the same model, for example, both models can be relation-based graph attention network models (RGAT models).

[0061] Figure 3 This is an architecture diagram of a single-head attention mechanism of a relation-based graph attention network model according to an embodiment of the present disclosure.

[0062] Relation-based graph attention network model Input instance embedding initialization matrix in the first layer of graph attention network model And the relation embedding initialization matrix

[0063] In order to obtain new instance embedding information, the instance embedding information and relation embedding information are processed through the attention mechanism.

[0064] First, for the instance relation triple (ei ,r k ,e j )'s initial embedding information The concatenation is performed, and the concatenation result is processed by the linear transformation matrix M1 to obtain a new embedding vector. The formula is shown in formula (3):

[0065]

[0066] in, Represents a triple (e i ,r k ,e j ) is the embedded vector after being processed by M1, || represents the concatenation operation, Represents instance e i The initial embedding vector of Represents instance e j The initial embedding vector of Represents the relationship k The initial embedding vector of .

[0067] Then another linear transformation matrix M2 is used to transform the new embedding vector Convert and use the LeakyReLU function to calculate the current triplet (e i ,r k ,e j ) is calculated as shown in formula (4):

[0068]

[0069] Finally, the calculation results are normalized using the softmax function, as shown in formula (5):

[0070]

[0071] Among them, N i is an instance i The set of all adjacent instance nodes, R in Indicates link instance e i and instance n The set of all relations of .

[0072] So, after getting the instance node e i The attention information of each relation triple can be used to identify the instance node e i The embedding representation of is updated, and the formula for updating the embedding representation is shown in formula (6):

[0073]

[0074] Where σ1 represents an arbitrary nonlinear function.

[0075] In order to obtain a more balanced instance embedding representation and encapsulate more adjacency information, a multi-head attention mechanism is used to average the learned domain information to obtain stable learning results.

[0076] After the optimization of the multi-head attention mechanism, the instance node e i A more accurate embedding representation can be obtained. For the first-layer graph attention network model, the calculated multi-head attention results are integrated by splicing to obtain the first-layer instance embedding optimization matrix Among them, H′ represents the matrix composed of all instance embedding information after optimization, N e is the number of instances, T′ represents the dimension after instance embedding optimization, and the calculation formula of each instance embedding information is shown in formula (7):

[0077]

[0078] Among them, Z represents the set of attention heads, and |Z| represents the number of attention heads.

[0079] Since the dimension of the instance embedding vector changes after concatenation, in order to train the second-layer graph attention network model, it is necessary to use the fully connected layer to transform the relationship embedding matrix. The transformation formula is shown in formula (8):

[0080] G′=GM R (8)

[0081] Among them, G represents the relationship embedding matrix, represents the transformation matrix corresponding to the fully connected layer, and P′ represents the dimension of the transformed relation embedding.

[0082] Embed the first layer instance output by the first layer graph attention network model into the optimization matrix The transformed relation embedding matrix G′ is input into the second-layer graph attention network model. The single-head graph attention mechanism of the second-layer graph attention network model is consistent with that of the first-layer graph attention network model, but in order to reduce the dimension of the final embedding vector, the instance embedding information is optimized by averaging the multi-head attention results in the second-layer graph attention network model. The optimization method is shown in formula (9):

[0083]

[0084] After the instance embedding information is learned by the multi-head attention model, a large amount of semantic information around the instance node will be retained, thereby obtaining the second-layer instance embedding optimization matrix H f .

[0085] Considering that after the first-layer graph attention network model and the second-layer graph attention model are processed, the final embedding representation result will lose the initial embedding information of the instance node itself. Therefore, it is necessary to optimize the second-layer instance embedding matrix H f and the instance embedding matrix.

[0086] That is, in the present disclosure, the initial embedding of the model is merged into the final attention learning result, and the formula for merging the embedding result (i.e., the fusion matrix) is shown in formula (10):

[0087] H″=H i +H f =M E H t +H f (10)

[0088] Among them, H t represents the initial instance embedding matrix, H i represents the transformed instance embedding matrix, represents the transformation matrix, T i represents the dimension of the initial instance embedding vector, T f represents the dimension of the final instance embedding vector, Represents the instance embedding matrix output by the second-layer graph attention network model, N e Indicates the number of instances.

[0089] After the multi-layer graph attention network model, the final optimized instance embedding matrix H″ (i.e., fusion matrix) and relationship embedding matrix G′ will be obtained, and the final optimized instance embedding matrix H″ (i.e., fusion matrix) and relationship embedding matrix G′ will be used as input for decoder training.

[0090] In the present disclosure, the encoder is trained by a fusion matrix to obtain the final instance embedding vector and the relationship embedding vector.

[0091] The encoder is trained by negative sampling to obtain the final instance embedding vector and relation embedding vector. The score function is shown in formula (11), and the loss function is shown in formulas (11) and (12):

[0092] S gat (h i ,g k ,h j )=||h i +g k -h j || l2 (11)

[0093]

[0094] Among them, γ3 represents the margin hyperparameter, Y represents the set of instance-relation triples in the training set, and Y′ is the set of negative samples of instance-relation triples. The set Y′ contains the content shown in formula (13):

[0095] Y′={(e i ′,r k ,e j )|(e i ,r k ,e j )∈D,e i ′∈ε,e i ′≠e i}

[0096] ∪{(e i ,r k ,e j ′)|(e i ,r k ,e j )∈D,e j ′∈ε,e j ′≠e j} (13)

[0097] Through encoding training, we can obtain instance embedding information, attribute embedding information and relationship embedding information in the equipment knowledge graph. In the decoding part, a multi-dimensional adaptive convolutional network model is used to fuse the features in the embedded information learned above.

[0098] Since the model constructs the corresponding convolution kernel through attribute embedding and relation embedding, it is referred to as ConvPR model. The architecture of the model is as follows Figure 4 shown.

[0099] Set the instance embedding vector of the instance of the relation to be completed where d e is the dimension of instance embedding vector, relation embedding vector where d r is the dimension of the relation embedding vector, and the attribute embedding vector is calculated as shown in formula (14):

[0100]

[0101] Where P is the set of attribute embedding vectors related to instance e, |P| is the number of vectors in the attribute embedding vector set, And d p is the dimension of the attribute embedding vector.

[0102] The multi-dimensional adaptive convolutional network model first embeds the vector according to the attribute Constructing attribute convolution kernel Where x and y represent the length and width of each attribute convolution kernel, and the number of attribute convolution kernels b = d p / (xy), while embedding the instance into a vector Reorganize into instance embedding matrix in and Represents the length and width of the instance embedding matrix. Through the attribute convolution kernel Get the instance attribute embedding matrix The calculation method of each matrix unit in is shown in formula (15):

[0103]

[0104] in, Represents the instance attribute embedding matrix B (l) The value at (m,n) in the figure, f(·) represents the ReLU function, e m+i-1,n+j-1 Represents the instance embedding matrix The value at (m+i-1,n+j-1) in Represents attribute convolution kernel The value at (i,j) in .

[0105] Then embed the vector according to the relationship Constructing the relational convolution kernel Where h and w represent the length and width of each convolution kernel, and the number of convolution kernels c = d r / (hw). Through the relational convolution kernel Obtained instance attribute relationship embedding matrix The calculation method of each matrix unit is shown in formula (16):

[0106]

[0107] in, Represents the instance attribute relationship embedding matrix C (l) The value at (m,n) in the figure, f(·) represents the ReLU function, b m+i-1,n+j-1 Represents the instance attribute embedding matrix B (l) The value at (m+i-1,n+j-1) in Represents the relational convolution kernel The value at (i,j) in .

[0108] For the obtained head instance attribute relation tuple (e s ,P es ,r k )’s final feature matrix Convert it into a feature vector by expanding and concatenating rows Finally, embedding vector with another instance Matrix multiplication is performed to obtain the scores of all predicted tail instances. The score function is shown in formula (17):

[0109]

[0110] in, Represents the header instance e s With the tail instance e o In the attribute set P es And the relationship is r k The score in this case, f(·) represents the ReLU function, and Represents the parameters corresponding to the fully connected layer.

[0111] The relationship corresponding to the highest score is obtained as the final completion relationship of the input instance pair.

[0112] In order to perform decoding training, the sigmoid function σ(x) = 1 / (1+e -x ), the cross entropy loss function is used for training. The loss function formula is shown in formula (18):

[0113]

[0114] Where N represents the set of all instances, |N| represents the number of instances in the instance set, If (s,P,r,o) is a correct instance attribute relation tuple, then otherwise,

[0115] The following is a specific example to illustrate the method for completing the internal link relationship of the equipment knowledge graph disclosed in the present invention.

[0116] The equipment knowledge graph data mainly comes from encyclopedia knowledge and media information under the scope of Internet open source data. The collection scale of each original data is shown in Table 1.

[0117] Table 1. Scale of raw data collection

[0118]

[0119] Among them, encyclopedia web pages with high equipment knowledge content account for 34.1% of the original data web pages, mainly providing common basic equipment knowledge triple information; while media web pages, which account for 65.9% of the original data web pages, mainly provide the latest and rare equipment knowledge triple information.

[0120] According to the nature and positioning of the equipment knowledge graph, a total of 117,185 equipment-related triplet data were extracted from the collected 67,747 web pages. The statistical results of each dimension of the data set are shown in Table 2.

[0121] Table 2 Equipment knowledge graph data statistics

[0122]

[0123] The equipment knowledge graph data is split into training set, validation set and test set. The statistical results of each data set are shown in Table 3.

[0124] Table 3 Data statistics of the data set

[0125]

[0126] The equipment knowledge graph data used this time and the data in the training set, validation set, and test set that were subsequently divided do not contain bidirectional relationships, but the above data sets all contain "one-to-many" and "many-to-one" relationships. The statistics of the number of "one-to-many" and "many-to-one" relationships in each data set are shown in Table 4.

[0127] Table 4 Statistics of the number of complex relationships in the dataset

[0128]

[0129] This paper uses indicators commonly used in the field of knowledge graph completion to evaluate model performance, namely MRR and HITS@n.

[0130] (1) MRR

[0131] The full name of MRR is Mean Reciprocal Ranking, which means the average of the reciprocal rankings. This evaluation indicator uses the reciprocal ranking for evaluation. Its calculation formula is shown in formula (19):

[0132]

[0133] Among them, S represents the set of triples, |S| represents the number of triples in the set of triples, and rank i represents the completion prediction ranking of the i-th triple. From this formula, we can see that the larger the MRR index, the higher the rank i The completion effect of the representative model is better.

[0134] (2)HITS@n

[0135] HITS@n refers to the proportion of correct answers ranked in the top n triples in the completion prediction, and its calculation formula is shown in formula (20):

[0136]

[0137] Among them, S, |S| and their meanings are consistent with those in the MRR indicator. Ⅱ represents the indicator function, which is calculated as follows: i If ≤n, the result is 1, otherwise it is 0. From the calculation method of this formula, we can see that the larger the HITS@n index is, the better the completion effect of the model is. For this evaluation index, this study uses HITS@1, HITS@3 and HITS@10 for experimental evaluation.

[0138] The hyperparameter settings during the experiment of the equipment knowledge graph internal link relationship completion model are shown in Table 5.

[0139] Table 5 Description of hyperparameter values ​​for the equipment knowledge graph internal link relationship completion model

[0140]

[0141] In order to verify the effectiveness and advancement of the equipment knowledge graph internal link relationship completion method based on the attention mechanism proposed in this disclosure, two groups of implementations are set up for comparison and explanation from different angles. The first group of experiments compares the completion effect of the internal link relationship completion model proposed in this disclosure (RGAT-ConvPR model) with the mainstream link relationship completion model in recent years on the equipment knowledge graph, and will illustrate the superiority of this method in the internal link relationship completion of the equipment knowledge graph by analyzing the experimental results; the second group of experiments, based on the experiments of the KBGAT model, respectively conduct experiments on the model combined with the adaptive convolutional network (RGAT-ConvR model), the model incorporating attribute embedding information (RGAT-ConvKBP model), and the comprehensive model including the above two components (RGAT-ConvPR model).

[0142] (1) Comparison of the effects of the RGAT-ConvPR model and mainstream link relationship completion models

[0143] Based on the equipment knowledge graph data, experiments were conducted on the mainstream knowledge graph internal link relationship completion models in recent years and the RGAT-ConvPR model proposed in this paper. The ConvE model, ConvKB model, R-GCN model, ConvR model, KG-BERT model, GenKGC model and SimKGC model were mainly compared. The characteristics of each model are shown in Table 6. By comparing the experimental results and combining the processing characteristics of the model, the advantages of the RGAT-ConvPR model in the task of completing the internal link relationship of the equipment knowledge graph are explained. The experimental results are shown in Table 7.

[0144] Table 6 Characteristics of mainstream link relationship completion models

[0145]

[0146] Table 7 Performance comparison between mainstream models and RGAT-ConvPR model

[0147]

[0148] From the experimental comparison results in Table 7, it can be seen that in the field of internal link relationship completion of equipment knowledge graph, the RGAT-ConvPR model proposed in this paper is superior to the mainstream link relationship completion model compared in this paper in terms of MRR, HITS@1, HITS@3 and HITS@10 indicators, which are improved by about 1.13%, 2.33%, 2.93% and 3.04% respectively compared with the best indicators.

[0149] Among them, by analyzing the experimental results of the ConvE model, ConvKB model, R-GCN model and ConvR model in Table 7, it can be seen that in the field of equipment knowledge graph completion, the traditional convolutional neural network-based model can capture the embedded semantic information of equipment instance triples to a certain extent. At the same time, in the above models, as the global interaction between instance embedding and relationship embedding gradually deepens, the final completion effect of the model is gradually improved. However, due to the number and sparsity of relationship triplets in the equipment knowledge graph, this type of model mainly based on convolutional neural networks is difficult to achieve a good completion effect.

[0150] According to the experimental results of the KG-BERT model in Table 7, it can be seen that using the pre-trained language model to encode the instance relationship triples in the equipment knowledge graph and make predictions can obtain better completion effects. Since the pre-trained language model encoding can obtain semantic information beyond the traditional relationship structure, and the instance relationship triple information in the equipment knowledge graph is closer to the real semantic scene, the semantic embedding information obtained by the pre-trained language model can better reflect the characteristics of the corresponding knowledge. Therefore, the final completion effect is higher than that of the ConvE, ConvKB, R-GCN and ConvR models based only on convolutional neural networks.

[0151] According to the experimental results of the GenKGC model in Table 7, it can be seen that the method of converting the completion task into a generation task to reduce the triple scoring cost can greatly reduce the problem of reduced completion effect caused by training instability due to negative samples. At the same time, the introduced relationship-guided examples and instance-aware hierarchical decoding strategies can not only help obtain better knowledge embedding representations but also speed up the reasoning completion speed. Therefore, the completion effect of this model is higher than the more basic KG-BERT pre-trained language model.

[0152] According to the experimental results of the SimKGC model in Table 7, it can be seen that effective contrastive learning can further help improve the completion effect. On the basis of the pre-trained model, the model introduces three negative sampling strategies: intra-batch negative sampling, pre-batch negative sampling, and self-negative sampling, and uses a dual encoder model to learn the instance relationship embedding representation in the equipment knowledge graph. It improves learning efficiency while obtaining more semantic information, so the completion effect is also higher than the more basic KG-BERT pre-trained language model.

[0153] The RGAT-ConvPR model proposed in this paper incorporates the instance relationship triples and instance attribute triples in the equipment knowledge graph. Compared with the above model that only uses the instance relationship triple information, it can obtain richer embedding representations, so the model effect is better than the model that only encodes the instance relationship triple information. At the same time, the RGAT-ConvPR model uses a multi-dimensional adaptive convolutional network in the decoding layer to optimize the combination of instance embedding, relationship embedding, and attribute embedding, and integrates relationship embedding and attribute embedding into the instance embedding representation through global interaction, further improving the comprehensive representation ability of the embedded information. Therefore, the completion effect of the model is further improved, and ultimately it is better than all the models compared above.

[0154] (2) The impact of multi-dimensional adaptive convolutional network model and attribute embedding information on link relationship completion

[0155] Based on the equipment knowledge graph data and taking the KBGAT model as the benchmark, experiments were conducted using a model that incorporates an adaptive convolutional network (RGAT-ConvR model), a model that incorporates attribute embedding information (RGAT-ConvKBP model), and a model that incorporates attribute embedding and a multi-dimensional adaptive convolutional network (RGAT-ConvPR model). By comparing the results of the MRR, HITS@1, HITS@3, and HITS@10 indicators, the positive effect of each component on completing the internal link relationship of the equipment knowledge graph is demonstrated. The experimental results are shown in Table 8.

[0156] Table 8 The impact of multi-dimensional adaptive convolutional network model and attribute embedding information on link relationship completion

[0157]

[0158] From Table 8, we can see that incorporating attribute embedding information or multi-dimensional adaptive convolutional network models has a positive effect on the completion of internal link relationships, and the completion effect of the RGAT-ConvPR model that combines the two components exceeds all the above models. Compared with the KBGAT baseline model, the RGAT-ConvPR model improves the MRR, HITS@1, HITS@3, and HITS@10 indicators in the link relationship completion task by approximately 3.77%, 4.5%, 5.05%, and 6.62%, respectively.

[0159] In the encoding part, the KBGAT model uses the attention mechanism in the RGAT model to supplement the embedding information of its neighborhood relationship for instance embedding, and in the decoding part, multiple one-dimensional convolution kernels in the ConvKB model are used to decode the embedding information obtained in the encoding part. The KBGAT model improves the processing method of relational information through the attention mechanism and successfully obtains a more accurate triple embedding representation result, so the equipment link relationship completion performance of the model can basically meet the application requirements.

[0160] By comparing the experiments of the KBGAT model and the RGAT-ConvR model, it can be proved that the integration of the adaptive convolutional network model improves the ability of the link relationship completion model to obtain deep features, which can enhance the features of instance embedding information. At the same time, the adaptive convolutional network model realizes the global interaction of instance embedding and relationship embedding, which reduces the problem of unbalanced feature combination caused by local interaction to a certain extent. Therefore, the final experimental results are better than the KBGAT model which does not realize the global interaction of instance embedding and relationship embedding in the decoding part.

[0161] By comparing the experiments of the KBGAT model and the RGAT-ConvKBP model, it can be proved that the incorporation of attribute embedding information can supplement the instance-relationship tuple feature embedding representation obtained by the decoding layer, so that it is no longer limited to the embedding representation of a single node itself, but obtains a more comprehensive instance embedding representation. Therefore, the final experimental results are better than the KBGAT model that only uses instance embedding information and relationship embedding information in the decoding part.

[0162] By comparing the KBGAT model, RGAT-ConvR model, RGAT-ConvKBP model and RGAT-ConvPR model, it can be proved that incorporating attribute embedding information can enhance the feature embedding representation ability of the original instance-relationship tuple. At the same time, the use of a multi-dimensional adaptive convolutional network model to realize the global interaction of instance embedding, attribute embedding and relationship embedding further improves the balance of the final embedding results. Therefore, the final experimental results are better than the KBGAT model, RGAT-ConvR model and RGAT-ConvKBP model.

[0163] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / method or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments / methods or examples described in this specification and the features of the different embodiments / methods or examples, unless they are contradictory.

[0164] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0165] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A method for completing internal link relationships in an equipment knowledge graph, characterized in that: include: Use the translation model to process the instance relationship triple information and instance attribute triple information of the equipment knowledge graph to obtain the instance embedding initialization matrix, the relationship embedding initialization matrix and the attribute embedding initialization matrix; Use the improved first-layer graph attention network model to process the instance embedding initialization matrix and the relationship embedding initialization matrix to obtain the first-layer instance embedding optimization matrix; The first-layer instance embedding optimization matrix and the relationship embedding matrix are processed by the improved second-layer graph attention network model to obtain the second-layer instance embedding optimization matrix; The second-layer instance embedding optimization matrix is ​​fused with the instance embedding initialization matrix to obtain the instance embedding fusion matrix; Construct an encoder to train the instance embedding fusion matrix and the relation embedding matrix to obtain the final instance embedding matrix and the relation embedding matrix; as well as A multi-dimensional adaptive convolutional decoder is constructed, and the instance attribute embedding feature matrix of the relation to be completed is input into the decoder. The score of each pair of feature matrices under different relations is calculated through the decoder, and the relation corresponding to the highest score is obtained as the final completion relation of the input instance pair.

2. The method for completing the internal link relationship of the equipment knowledge graph according to claim 1 is characterized in that: The improved first-layer graph attention network model and the improved second-layer graph attention network model are relationship-based graph attention network models.

3. The method for completing the internal link relationship of the equipment knowledge graph according to claim 2 is characterized in that: Relation-based graph attention network model Input instance embedding initialization matrix in the first layer of graph attention network model And the relation embedding initialization matrix For instance relation triples (e i ,r k ,e j )'s initial embedding information The concatenation is performed, and the concatenation result is processed by the linear transformation matrix M1 to obtain a new embedding vector. The formula is shown in formula (3): in, Represents a triple (e i ,r k ,e j ) is the embedded vector after being processed by M1, || represents the concatenation operation, Represents instance e i The initial embedding vector of represents the initial embedding vector of instance ej, Represents the initial embedding vector of the relation rk; Then use another linear transformation matrix M2 to transform the new embedding vector Convert and use the LeakyReLU function to calculate the current triplet (e i ,r k ,e j ) is calculated as shown in formula (4): Finally, the calculation results are normalized using the softmax function, as shown in formula (5): Among them, N i is an instance i The set of all adjacent instance nodes, R in Indicates link instance e i and instance n The set of all relations of ; After obtaining the instance node e i The attention information of each relation triplet is applied to the instance node e i The embedding representation of is updated, and the formula for updating the embedding representation is shown in formula (6): Where σ1 represents an arbitrary nonlinear function. The multi-head attention mechanism is used to average the learned domain information; for the first-layer graph attention network model, the calculated multi-head attention results are integrated by splicing to obtain the first-layer instance embedding optimization matrix Among them, H′ represents the matrix composed of all instance embedding information after optimization, N e is the number of instances, T′ represents the dimension after instance embedding optimization, and the calculation formula of each instance embedding information is shown in formula (7): Among them, Z represents the set of attention heads, and |Z| represents the number of attention heads.

4. The method for completing the internal link relationship of the equipment knowledge graph according to claim 3 is characterized in that: The fully connected layer is used to transform the relation embedding matrix. The transformation formula is shown in formula (8): G′=GM R (8) Among them, G represents the relationship embedding matrix, represents the transformation matrix corresponding to the fully connected layer, and P′ represents the dimension of the transformed relation embedding.

5. The method for completing the internal link relationship of the equipment knowledge graph according to claim 4 is characterized in that: Embed the first layer instance output by the first layer graph attention network model into the optimization matrix The converted relation embedding matrix G′ is input into the second-layer graph attention network model to obtain the second-layer instance embedding optimization matrix.

6. The method for completing the internal link relationship of the equipment knowledge graph according to claim 1 is characterized in that: The fusion matrix H″=H i +H f =M E H t +H f (10) Among them, H t represents the initial instance embedding matrix, H i represents the transformed instance embedding matrix, represents the transformation matrix, T i represents the dimension of the initial instance embedding vector, T f represents the dimension of the final instance embedding vector, Represents the instance embedding matrix output by the second-layer graph attention network model, N e Indicates the number of instances.

Images