A text-image enhanced multimodal knowledge graph embedding method
Through the text-image enhancement multimodal knowledge graph embedding method, text and image information are integrated, and the limitations of the existing technology in complex relationship processing and information fusion are solved, and more accurate knowledge representation is achieved.
Patent Information
- Application Number
- CN202210708381.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-22
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2042-06-22
AI Technical Summary
Existing knowledge graph representation learning methods have limitations in dealing with complex relationships and fail to effectively integrate rich additional information outside structured information, such as text description and image information.
Using text-image enhancement multimodal knowledge graph embedding method, the convolutional neural network extracts solid image features through Text-CNN encoding entity text description, and combines the classic translation relationship representation model to integrate text and image information for knowledge graph embedding. .
More accurate embedding of entity triplets is achieved, and the accuracy of knowledge representation is improved by using rich text and image information, overcoming the limitations of traditional methods focusing only on structured information.
Smart Images
Figure CN115099409B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of knowledge graph embedding, and specifically relates to a text-image enhanced multimodal knowledge graph embedding method. Background Art
[0002] Translation-based methods have attracted widespread attention in recent years due to their effectiveness and high efficiency in knowledge representation learning (KRL). Mikolov et al. found that word vector space is translation-invariant through the word2vec word representation learning model. Inspired by this translation-invariant phenomenon, Border et al. proposed the TransE model, which encodes the entities and relations in each fact triple (h, r, t) into the same low-dimensional continuous vector space, and regards the relation r as a translation operation from h to t in the low-dimensional space, i.e., h+r=t, where h and t are entity embeddings and r is relation embedding. TransE has good performance and scalability, but it cannot model complex relations.
[0003] In order to solve the limitations of the TransE model in dealing with one-to-many, many-to-one, and many-to-many complex relationships, the TransH model proposes to model the relationship as a hyperplane and project the head and tail entities onto the relationship-specific hyperplane, so that entities have different representations under different relationships. TransR uses a relationship-specific projection matrix to project the entity into the corresponding relationship space, and then establishes a translation relationship from the head entity to the tail entity. The TransM model allows entities to have different embeddings when involving different relationships, and assigns specific relationship weights to each fact triple.
[0004] However, the above methods only focus on the structural information in triples, but fail to consider the rich additional information that can help the representation learning of KGs. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a text-image enhanced multimodal knowledge graph embedding method based on TransE, which takes into account both entity text description and entity image, and can realize the fusion of structural information, text information and image information.
[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:
[0007] A text-image enhanced multimodal knowledge graph embedding method, comprising:
[0008] Step 1. The text encoder uses Text-CNN to encode the descriptive text of the entity to obtain the text representation of the entity;
[0009] Step 2. The image encoder uses a convolutional neural network to extract the features of the entity image, and projects the image features from the image space to the entity space through a projection matrix to obtain an image representation of the entity;
[0010] Step 3. Entity-relationship representation model uses the classic RL method based on the translation model to model the knowledge graph fact triples, regards the relationship as a translation operation from the head entity to the tail entity, and fuses the text representation and image representation of the entity to obtain the structural representation of the entity and realize the relationship space embedding;
[0011] Step 4. Construct a loss function and jointly train the entity-relationship representation model to optimize the representation of entities, relations, texts, and images and the relational space embedding.
[0012] To optimize the above technical solutions, the specific measures taken also include:
[0013] The text encoder described in step 1 above takes randomly initialized word vectors or word embeddings obtained by word2vec pre-training as input, and after convolution and maximum pooling, finally outputs the text representation vector of the entity.
[0014] The text encoder described in step 1 above includes the following structure:
[0015] 1) Input layer: Extract the text description of the entity from the annotation file of the WordNet corpus, then remove all stop words in the entity text description, and finally use the pre-trained word2vec to get all word vectors {x1, x2, ..., x n};
[0016] 2) Convolutional layer: The word vector {x1, x2, ..., x n As the input of the convolutional layer, the h word vectors are first connected through a sliding window of h words. The sliding process is as follows:
[0017]
[0018] in, is the concatenation operator, is the k-dimensional word vector of the i-th word in the text;
[0019] Apply a convolution operation to each sliding window in the sentence to generate new features:
[0020] c i =f(wx i：i+h-1 +b)
[0021] where c i Indicates that from window x i：i+h-1The features generated in represents the convolution kernel, represents the bias term, and f is a nonlinear function;
[0022] Finally, all sliding windows {x 1：h , x 2：h+1 , ..., x n-h+1：n The generated features form a feature map:
[0023] c=[c1,c2,...,c n-h+1 ]
[0024] in, These features in c have the same weight;
[0025] The above describes the process of extracting a feature vector using a convolution kernel, and using multiple convolution filters to obtain multiple feature vectors (c1, c2, ..., c k );
[0026] 3) Pooling layer: Use the maximum pooling operation on the feature map:
[0027] m i =max{c i}
[0028] Among them, m i is the feature map c i The most significant eigenvalue in , that is, the feature corresponding to the i-th convolution kernel;
[0029] Finally, we get the feature vector m=[m1, m2, ..., m k ] as the textual representation of the entity.
[0030] The image encoder described in step 2 above first uses the classic convolutional neural network AlexNet as a feature extractor for the entity image, constructs image features for the entity image, then obtains the image features of the entity space through the projection matrix, and finally calculates the average value of all image features and uses it as the image representation of the entity.
[0031] The above step 2 is specifically as follows:
[0032] 1) The image encoder inputs multiple images of each entity {img1, img2, ..., img n};
[0033] 2) Use the deep learning framework PyTorch and pre-trained AlexNet to extract features of entity images;
[0034] 3) Use a trainable shared projection matrix to project the image features from the image space to the physical space to obtain the image features of the physical space;
[0035] The image features of the i-th image entity space are:
[0036] p i =M·f(img i )
[0037] Among them, f represents AlexNet, is the trainable projection matrix, d i Represents the dimension of image features in image space, d s Represents the dimension of the entity;
[0038] 4) Perform arithmetic averaging on the feature representations of the entity’s corresponding image in the entity space to obtain the entity’s image representation.
[0039] In step 3 above, based on the learned text representation and image representation, the entity relationship representation model integrates text information and image visual information into the learning of the knowledge graph;
[0040] The entity relationship representation model is implemented based on the traditional translation-based method TransE, and its scoring function is defined as follows:
[0041] f(h, r, t) = E s +μE d +ηE i
[0042] Among them, μ and η are hyperparameters, E s The scoring function represents the structural representation, which is the same as the scoring function of TransE ||h+rt||. The head and tail entities are both structural representations, that is,
[0043] E s =||h s +rt s ||
[0044] E d Represents the scoring function of the text representation, so that E d The learning process and E s Compatible, E d The definition is as follows:
[0045] E d =d(h s , r, t d )+d(h d , r, t s )+d(h d , r, t d )
[0046] d(h s , r, t d )=||h s +rt d ||
[0047] d(h d , r, t s )=||h d +rt s ||
[0048] d(h d , r, t d )=||h d +rt d ||
[0049] Among them, d(h s , r, t d ) and d(h d , r, t s ) indicates that one of the head entity and the tail entity uses a text representation and the other uses a structural representation, such as, d(h d , r, t d ) Both h and t are in text representation.
[0050] Similarly, the scoring function E of the image representation i Defined as:
[0051] E i =d(h s , r, t i )+d(h i , r, t s )+d(h i , r, t i )
[0052] E s 、E d 、E i Share relation embeddings and project entity representations based on structure, text, and image representations into the same continuous low-dimensional vector space, allowing the structure, text, and image representations to influence each other through joint learning.
[0053] Step 3 above imposes constraints on the L2 norm of the embeddings h, r, and t to normalize the vector, i.e., ||h s ||2≤1,||t s |||2≤1,||h d |||2≤1,||t d |||2≤1,||h i ||2≤1,||t i||2≤1.
[0054] Step 4 above is based on the scoring function in step 3, and defines the training objective as minimizing the following interval-based loss function:
[0055]
[0056] where γ>0 is the margin hyperparameter, [x] + Indicates keeping the positive part of x, S is the set of correct triplets, and S′ is the negative sampling set of S, that is, the set of wrong triplets:
[0057] S′={(h′, r, t)|h′∈E}∪{(h, r, t′)|t′∈E}
[0058] S′ is constructed by destroying the correct triple (h, r, t) through probability-based entity replacement to construct an incorrect triple (h′, r, t′);
[0059] The optimization process of the loss function is implemented by stochastic gradient descent.
[0060] The present invention has the following beneficial effects:
[0061] The present invention realizes a new knowledge graph embedding method that integrates fact triples, text descriptions, and entity images, and can simultaneously utilize the rich external information in text descriptions and entity images to learn more accurate knowledge graph embedding, i.e., knowledge representation.
[0062] Different from traditional research methods such as translation-based methods that use facts observed in KG to perform embedding tasks, the method of fusing additional information in the present invention can improve knowledge representation tasks by incorporating additional information, such as entity types, association paths, entity text descriptions, entity images, and temporal information, etc.
[0063] The entity type refers to the semantic category to which the entity belongs. The semantic smooth embedding (SSE) model requires that entities of the same type are adjacent to each other in the embedding space. The association path refers to the multi-hop relationship between entities. PTransE extends TransE to model the relationship path. In most KGs, entities have concise descriptions. DKRL extends the translation-based embedding method to a text enhancement model to better integrate entity descriptions. Jiang et al. observed that KGs facts are usually sensitive to time. Based on this, they proposed a time-aware embedding model to add time order constraints to time-sensitive relationship pairs. However, traditional methods only integrate one type of additional information, while the present invention MKGE makes full use of the complementary knowledge of text descriptions and entity images to improve the accuracy of knowledge representation. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] Figure 1Embedding structure graph for multimodal knowledge graph;
[0065] Figure 2 This is the structure diagram of the text encoder;
[0066] Figure 3 This is the structure diagram of the image encoder. DETAILED DESCRIPTION
[0067] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings.
[0068] like Figure 1-3 As shown, a text-image enhanced multimodal knowledge graph embedding method includes:
[0069] Step 1. The text encoder uses Text-CNN to encode the descriptive text of the entity to obtain the text representation of the entity;
[0070] The text encoder models the entity description and generates a text representation of the entity;
[0071] The text encoder, which is implemented by a convolutional neural network, is used to model the entity text description to obtain the semantic features of the description and use it as the text-based representation of the entity;
[0072] Drawing on the CNN proposed by Kim for text analysis, namely Text-CNN, the descriptive text of the entity is encoded to obtain the text representation of the entity.
[0073] The text encoder takes randomly initialized word vectors or word embeddings obtained by word2vec pre-training as input, and finally outputs the text representation vector of the entity after convolution and maximum pooling. The overall structure of the text encoder is as follows Figure 2 shown.
[0074] 1) Input layer
[0075] First, we extract the text description of the entity from the annotation file of the WordNet corpus, then remove all stop words in the entity text description, and finally use the pre-trained word2vec to get all word vectors {x1, x2, ..., x n};
[0076] 2) Convolutional Layer
[0077] The word vector {x1, x2, ..., x n As the input of the convolutional layer, the h word vectors are first connected through a sliding window of h words. The sliding process is as follows:
[0078]
[0079] in, is the concatenation operator, is the k-dimensional word vector of the i-th word in the text;
[0080] Apply a convolution operation to each sliding window in the sentence to generate new features:
[0081] c i =f(wx i：i+h-1 +b)
[0082] where c i Indicates that from window x i：i+h-1 The features generated in represents the convolution kernel, represents the bias term, f is a nonlinear function such as the tanh function;
[0083] Finally, all sliding windows {x 1：h , x 2：h+1 , ..., x n-h+1：n The generated features form a feature map:
[0084] c=[c1,c2,...,c n-h+1 ]
[0085] in, These features in c have the same weight, which greatly reduces the number of parameters. The above describes the process of extracting one feature vector using one convolution kernel, and using multiple convolution filters to obtain multiple feature vectors (c1, c2, ..., c k );
[0086] 3) Pooling layer
[0087] In CNN, the pooling operation after the convolution layer can effectively reduce the number of parameters, speed up the calculation and prevent overfitting. In order to capture the most important features in each feature vector, the maximum pooling operation is used on the feature map:
[0088] m i =max{c i}
[0089] Among them, m i is the feature map c i The most significant eigenvalue in , that is, the feature corresponding to the i-th convolution kernel;
[0090] Finally, we get the feature vector m=[m1, m2, ..., m k ] as a textual representation of an entity whose dimensionality is independent of the sentence length.
[0091] Step 2. The image encoder uses a convolutional neural network to extract the features of the entity image, and projects the image features from the image space to the entity space through a projection matrix to obtain an image representation of the entity;
[0092] The image encoder models the entity image to generate an image representation of the entity; the image encoder is used to model the entity image. Specifically, it uses AlexNet as a feature extractor and projects the image features from the image space to the entity space through the projection matrix to obtain the image representation of the entity;
[0093] Figure 3 The structural framework of the image encoder is shown. The image encoder first uses the classic convolutional neural network AlexNet as the feature extractor of the entity image to construct image features for the entity image, then obtains the image features of the entity space through the projection matrix, and finally calculates the average value of all image features and uses it as the image representation of the entity, which is described in detail as follows.
[0094] The input to the image encoder is multiple images {img1, img2, ..., img n}.
[0095] First, the deep learning framework PyTorch and the pre-trained AlexNet are used to extract the features of entity images.
[0096] AlexNet is a deep CNN consisting of five convolutional layers, two fully connected layers, and a softmax layer.
[0097] Drawing on Shutova’s approach, the 4096-dimensional embedding before softmax, i.e. the output of the second fully connected layer (fc7), is used as the image feature representation.
[0098] After obtaining the feature representation of each image through the feature extractor, the image feature representation is projected from the image space to the entity space using a trainable shared projection matrix to obtain the image features of the entity space. The image features of the entity space of the i-th image are:
[0099] p i =M·f(img i )
[0100] Among them, f represents the feature extraction function, that is, the convolutional network mentioned above, is the trainable projection matrix, d i Represents the dimension of image features in image space, d s Represents the dimension of the entity. Note that the feature extraction function is not learned during training, only the parameters M of the encoder are learned.
[0101] Finally, in order to construct the image representation of the entity, a maximum of 10 images are selected for each entity. Since the images extracted from ImageNet are of very high quality and have less noise, the image representation of the entity is obtained by taking the arithmetic average of the feature representations of the corresponding images in the entity space.
[0102] Step 3. Entity-relationship representation model uses a classic RL method based on a translation model (such as TransE) to model the knowledge graph fact triples, treating the relationship as a translation operation from the head entity to the tail entity, fusing the text representation and image representation of the entity to obtain the structural representation of the entity and implement relational space embedding;
[0103] Based on the learned text representation and image representation, the entity relationship representation model incorporates text information and image visual information into the learning of the knowledge graph.
[0104] The entity relationship representation model is based on the traditional translation-based method. Taking TransE as an example, the scoring function is defined as follows:
[0105] f(h, r, t) = E s +μE d +ηE i
[0106] Among them, μ and η are hyperparameters, E s The scoring function represents the structural representation, which is the same as the scoring function of TransE ||h+rt||. The head and tail entities are both structural representations, that is,
[0107] E s =||h s +rt s ||
[0108] E d Represents the scoring function of the text representation, so that E d The learning process and E s Compatible, E d The definition is as follows:
[0109] E d =d(h s , r, t d )+d(h d , r, t s )+d(h d , r, t d )
[0110] d(h s , r, t d )=||h s +rt d ||
[0111] d(h d , r, t s )=||h d +rt s ||
[0112] d(h d , r, t d )=||h d +rt d ||
[0113] Among them, d(h s , r, t d ) and d(h d , r, t s ) indicates that one of the head entity and the tail entity uses a text representation and the other uses a structural representation, such as, d(h d , r, t d ) Both h and t are in text representation.
[0114] Similarly, the scoring function E of the image representation i Defined as:
[0115] E i =d(h s , r, t i )+d(h i , r, t s )+d(h i , r, t i )
[0116] E s 、E d 、E i Share relation embeddings and project entity representations based on structure, text, and image representations into the same continuous low-dimensional vector space, allowing the structure, text, and image representations to influence each other through joint learning.
[0117] In the experiment, in order to facilitate training and avoid overfitting, a constraint is imposed on the L2 norm of the embedding h, r, t to normalize the vector, that is, ||h s ||2≤1,||t s ||2≤1,||h d ||2≤1,||t d ||2≤1,||h i ||2≤1,||t i ||2≤1.
[0118] In addition, the entity relationship representation model can be easily extended to other translation-based methods, such as TransH and TransR. TransH projects the entity embeddings h and t to w r The hyperplane of the model score function is s Transformed into t s Transformed into Similarly, TransR passes through the projection matrix M r Project the entity embeddings h and t into the relational space and replace h in the scoring function with s and t s becomes h s M r and t s M r That's it.
[0119] 4. Joint Training of Representations
[0120] The model in this paper can be expressed as a parameter set θ = (E, R, X, W (1) , W (2) , M), where X, E, R represent the embedding of words, entities, and relations respectively, and W (1) , W (2) Represents the convolution kernels of different layers in the text CNN, M represents the projection matrix in the image encoder. X is randomly initialized or pre-trained by word2vec, E and R are randomly initialized or pre-trained by existing RL models, and W (1) , W (2) and M are initialized randomly.
[0121] In order to optimize the parameters, the representations of entities, relations, texts, and images are learned jointly. Based on the above scoring function, the training objective is defined as minimizing the following margin-based loss function:
[0122]
[0123] where γ>0 is the margin hyperparameter, [x] + Indicates keeping the positive part of x, S is the set of correct triplets, and S′ is the negative sampling set of S, that is, the set of wrong triplets:
[0124] S′={(h′, r, t)|h′∈E}∪{(h, r, t′)|t′∈E}
[0125] The construction method of S′ is to construct an incorrect triple (h′, r, t′) by destroying the correct triple (h, r, t) through probability-based entity replacement. Simply put, the head entity or the tail entity in the correct triple is randomly replaced with another entity according to the "unif" or "bern" strategy. In particular, if the replaced triple belongs to S, it cannot be added to S′. Since h and t have three types of representations, the entities in the loss function also have three representations, namely structural, textual and image representations. The optimization process of the loss function adopts stochastic gradient descent (SGD).
[0126] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.
Claims
1. A text-image enhanced multimodal knowledge graph embedding method, characterized in that: include: Step 1. The text encoder uses Text-CNN to encode the descriptive text of the entity to obtain the text representation of the entity; Step 2. The image encoder uses a convolutional neural network to extract the features of the entity image, and projects the image features from the image space to the entity space through a projection matrix to obtain an image representation of the entity; Step 3. Entity-relationship representation model uses the classic RL method based on the translation model to model the knowledge graph fact triples, regards the relationship as a translation operation from the head entity to the tail entity, and fuses the text representation and image representation of the entity to obtain the structural representation of the entity and realize the relationship space embedding; In step 3, based on the learned text representation and image representation, the entity relationship representation model integrates text information and image visual information into the learning of the knowledge graph; The entity relationship representation model is implemented based on the traditional translation-based method TransE, and its scoring function is defined as follows: f(h,r,t)=E s +μE d +ηE i Among them, μ and η are hyperparameters, E s The scoring function represents the structural representation, which is the same as the scoring function of TransE ||h+rt||. The head and tail entities are both structural representations, that is, E s =||h s +r-t s || E d Represents the scoring function of the text representation, so that E d The learning process and E s Compatible, E d The definition is as follows: E d =d(h s ,r,t d )+d(h d ,r,t s )+d(h d ,r,t d ) d(h s ,r,t d )=||h s +r-t d || d(h d ,r,t s )=||h d +r-t s || d(h d ,r,t d )=||h d +r-t d || Among them, d(h s , r, t d ) and d(h d , r, t s ) indicates that one of the head entity and the tail entity uses a text representation and the other uses a structural representation. d , r, t d ) In this example, both h and t are represented in text form; Similarly, the scoring function E of the image representation i Defined as: E i =d(h s ,r,t i )+d(h i ,r,t s )+d(h i ,r,t i ) E s 、E d 、E i Share relational embeddings and project entity representations based on structure, text, and image representations into the same continuous low-dimensional vector space, allowing the structure, text, and image representations to influence each other through joint learning; Step 4. Construct a loss function, jointly train the entity-relationship representation model, and optimize the representation of entities, relations, texts, and images and the relational space embedding; In step 4, the convolution kernels and image projection matrices of different layers in the word, entity, relationship, and text CNN are represented by parameter sets, and then trained by minimizing the loss function based on the interval, and the loss functions of the three representation forms of structure, text and image representation are included. Specifically: Based on the scoring function in step 3, the training objective is defined as minimizing the following margin-based loss function: where γ>0 is the margin hyperparameter, [x] + Indicates keeping the positive part of x, s is the set of correct triplets, and s′ is the negative sampling set of s, that is, the set of wrong triplets: s′={(h′, r, t)|h′∈E}U{(h, r, t′)|t′∈E} s′ is constructed by destroying the correct triple (h, r, t) through probability-based entity replacement to construct an incorrect triple (h′, r, t′); The optimization process of the loss function is implemented by stochastic gradient descent.
2. According to claim 1, a text-image enhanced multimodal knowledge graph embedding method is characterized in that: The text encoder described in step 1 takes randomly initialized word vectors or word embeddings obtained by word2vec pre-training as input, and after convolution and maximum pooling, finally outputs the text representation vector of the entity.
3. The text-image enhanced multimodal knowledge graph embedding method according to claim 1, characterized in that: The text encoder in step 1 includes the following structure: 1) Input layer: Extract the text description of the entity from the annotation file of the WordNet corpus, then remove all stop words in the entity text description, and finally use the pre-trained word2vec to get all word vectors {x1, x2, ..., x n }; 2) Convolutional layer: The word vector {x1, x2, ..., x n As the input of the convolutional layer, the h word vectors are first connected through a sliding window of h words. The sliding process is as follows: in, is the concatenation operator, is the k-dimensional word vector of the i-th word in the text; Apply a convolution operation to each sliding window in the sentence to generate new features: c i =f(w.x i：i+h-1 +b) where c i Indicates that from window x i：i+h-1 The features generated in represents the convolution kernel, represents the bias term, and f is a nonlinear function; Finally, all sliding windows {x 1：h , x 2：h+1 , ..., x n-h+1：n ]The generated features form the feature map: c=[c1,c2,...,c n-h+1 ] in, These features in c have the same weight; The above describes the process of extracting a feature vector using a convolution kernel, and using multiple convolution filters to obtain multiple feature vectors (c1, c2, ..., c k ); 3) Pooling layer: Use the maximum pooling operation on the feature map: m i =max{c i } Among them, m i is the feature map c i The most significant eigenvalue in , that is, the feature corresponding to the i-th convolution kernel; Finally, we get the feature vector m=[m1, m2,…, m k ] as the textual representation of the entity.
4. The text-image enhanced multimodal knowledge graph embedding method according to claim 1, characterized in that: The image encoder in step 2 first uses the classic convolutional neural network AlexNet as a feature extractor for the entity image, constructs image features for the entity image, then obtains the image features of the entity space through the projection matrix, and finally calculates the average value of all image features and uses it as the image representation of the entity.
5. The text-image enhanced multimodal knowledge graph embedding method according to claim 1, characterized in that: The step 2 is specifically as follows: 1) The image encoder inputs multiple images of each entity {img1, img2, ..., img n }; 2) Use the deep learning framework PyTorch and pre-trained AlexNet to extract features of entity images; 3) Use a trainable shared projection matrix to project the image features from the image space to the physical space to obtain the image features of the physical space; The image features of the i-th image entity space are: p i =M·f(img i ) Among them, f represents AlexNet, is the trainable projection matrix, d i Represents the dimension of image features in image space, d s Represents the dimension of the entity; 4) Perform arithmetic averaging on the feature representations of the entity’s corresponding image in the entity space to obtain the entity’s image representation.
6. The text-image enhanced multimodal knowledge graph embedding method according to claim 1, characterized in that: The step 3 imposes constraints on the L2 norm of the embedding h, r, t to normalize the vector, i.e., ||h s ||2≤1,||t s ||2≤1,||h d ||2≤1,||t d ||2≤1,||h i ||2≤1,||t i ||2≤1.