A method for updating out-of-vocabulary words in a knowledge graph considering precise semantic relationships

Abstract

The present invention discloses a method for updating out-of-vocabulary words in a knowledge graph considering precise semantic relationships, including: extracting entity multi-modal features such as names, name characters, synonymous features, etc. from the target graph and the graph to be matched containing out-of-vocabulary words; performing representation learning on the multi-modal features through a feature encoding method to generate embedding vectors and normalizing them; using an attention mechanism to dynamically assign weights according to the feature importance, and fusing the weighted multi-modal feature vectors to obtain the joint feature vector of the entity; evaluating the similarity by calculating the Manhattan distance between the entity joint feature vectors, completing entity alignment matching, and realizing the association of entity hierarchical relationships. The method of the present invention not only improves the ability of the knowledge graph in dealing with out-of-vocabulary words, but also enhances the semantic richness and hierarchical structure of the knowledge graph, and has important practical significance for the construction and application of the knowledge graph.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge graph construction and entity alignment, and specifically relates to a method for updating out-of-vocabulary words in a knowledge graph considering precise semantic relationships. Background Art

[0002] With the development of big data and artificial intelligence technologies, the knowledge graph, as a technology for organizing and representing structured and unstructured data through semantic associations, has been widely applied in fields such as semantic search, intelligent recommendation, and question answering systems. However, existing knowledge graph construction methods face challenges in dealing with out-of-vocabulary words. Out-of-vocabulary words are usually conceptual entities that do not appear in existing dictionaries or knowledge bases, such as newly emerging geographical names, scientific terms, etc.

[0003] In the prior art, the processing of out-of-vocabulary words mostly focuses on using natural language processing technologies for text analysis, but often ignores the importance of multi-modal feature fusion. Especially when performing entity alignment between multi-source heterogeneous knowledge graphs, relying solely on structural or text features may lead to inaccurate matching and it is difficult to reflect the hierarchical relationships and hyponymy associations between entities.

[0004] In addition, during the processing of out-of-vocabulary words, there is often a lack of an accurate measurement mechanism for semantic similarity between entities, resulting in a lack of consistency and accuracy in the update and expansion of the knowledge graph. Therefore, how to effectively process out-of-vocabulary words and achieve accurate association with existing entities has become an urgent technical problem to be solved. Summary of the Invention

[0005] The present invention aims to solve the above problems existing in the processing of out-of-vocabulary words and the entity alignment of the knowledge graph in the prior art, and provides a method for updating out-of-vocabulary words in a knowledge graph considering precise semantic relationships.

[0006] This method extracts multi-modal feature information from the target knowledge graph and the knowledge graph containing out-of-vocabulary words to be matched, performs feature encoding and feature fusion, and realizes precise entity alignment through similarity calculation.

[0007] Through the method of the present invention, it is possible to effectively process out-of-vocabulary words, and through the fusion of multi-modal features and the precise calculation of semantic similarity, ensure the precise alignment and hierarchical association of knowledge graph entities, and improve the integrity and semantic expression ability of the knowledge graph.

[0008] To achieve the above invention purpose, the technical solution adopted by the present invention is as follows:

[0009] The present application provides a method for updating out-of-vocabulary words in a knowledge graph considering precise semantic relationships. The specific steps of this method include:

[0010] S1. Extract from the target knowledge graph KGt and the knowledge graph KG to be matched containing unregistered words m Extract the multimodal feature information F of the entity multi ;

[0011] S2. Parse to obtain the target knowledge graph KG t and the knowledge graph KG to be matched m Multimodal feature information F of entities in multi , using feature encoding method to generate multimodal features F multi The embedding representation vector E mutil , and normalize it;

[0012] S3. Use attention mechanism to dynamically assign feature weights W f , perform multimodal feature fusion to generate the joint feature vector E of each entity joint ;

[0013] S4. Use Manhattan distance to measure the joint feature vector E joint The similarity between the two knowledge graphs is used to match the entities of the two knowledge graphs, and the semantic relationship between the entities is associated based on the matching results.

[0014] Wherein, the step S1 specifically includes the following steps:

[0015] S1-1. From the target knowledge graph KG t Extracting the structural information of entities Relationship Information Attribute description information Name information Name character information Synonymous information Get the target knowledge graph KG t Multimodal feature information of entities in

[0016] S1-2, using entity recognition methods, automatically extract entities that do not belong to the target knowledge graph KG from the field scientific literature t Unregistered word entities, build the knowledge graph KG to be matched m , extracting the structural information of unregistered word entities Relationship Information Attribute description information Name information Name character information Synonymous information And additionally contains the contextual text information of the text where the unregistered word is located and the text title information of the document Obtain the knowledge graph KG to be matched m entities

[0017] in the multimodal feature information

[0018] Among them, the step S2 specifically includes the following steps:

[0019] S2-1. Parse the target knowledge graph KG t and the knowledge graph KG to be matched m the structural feature F of each entity in struct , use the graph traversal algorithm to identify the hierarchical relationship of each entity, and adopt the multi-head graph attention network to learn the structural feature vector E of the entity struct ;

[0020] S2-2. First, parse the target knowledge graph KG t and the knowledge graph KG to be matched m the relationship feature F of each entity in rel and the attribute description feature F att , and obtain the set of all relationship triples of each entity:

[0021]

[0022] and the set of attribute triples:

[0023]

[0024] Among them, entity head and entity tail represent two entities participating in the relationship, Relation is the relationship type between entities, entity represents the entity with this attribute, Attribute is the attribute type, value att is the specific value of the attribute, n rel is the total number of relationship triples, n att is the total number of attribute triples;

[0025] Then, construct an N×N all-zero matrix as the relationship matrix M t for KG m and KG rel , the rows and columns of which correspond to the entities in the knowledge graph, the matrix value represents the relationship type between entities, and an N×A all-zero matrix as the attribute matrix M att , the rows correspond to entities, the columns correspond to attribute types, and the values represent the attribute values corresponding to the entity attribute types, where N is the total number of knowledge graph entities and A is the number of knowledge graph attribute types;

[0026] For each relational triple (entity head , Relation, entity tail ), construct a relation dictionary {key rel : Relation} that maps the relation type Relation to a unique index value key rel . Determine the row and column positions in the relation matrix based on the entities entity head and entity tail , and set the value at this position to the index value of the relation type, indicating that there is a certain relation between the entities entity head and entity tail ;

[0027] For each attribute triple (entity, Attribute, value att ), construct an attribute dictionary {key att : value att}, and update the attribute information of this entity to the corresponding position in the attribute matrix based on the attribute dictionary;

[0028] Finally, traverse all relational triples and attribute triples, and update the relation matrix M rel and the attribute matrix M att according to the following rules:

[0029]

[0030] The relation feature vector E rel and the attribute description feature vector E att can be obtained respectively by specific transformations of the relation matrix M rel and the attribute matrix M att ;

[0031] S2-3. First, traverse each entity name in the target knowledge graph KG t and the knowledge graph KG m to be matched, split it into combinations of adjacent two words (word k , word k+1 ) and characters . At the same time, use the synonym dictionary WordNet to find the synonym set Syn i = {syn1,..., syn s}, where n name represents the number of words in the entity name, n char represents the number of characters in the entity name, l represents the window size when extracting grams, and s represents the number of synonyms in the entity name;

[0032] Then, use the pre-trained language model to encode bigrams and character grams respectively, and average and aggregate the encoding results of all bigrams to obtain the name feature vector Average and aggregate the encoding results of all character grams to finally obtain the entity name character feature vector

[0033] Subsequently, encode the synonym set Syn i = {Syn1,..., syn s} using the pre-trained language model and convert it into an embedding representation Finally, by weighted summing the feature vectors of the entity name and its synonyms, aggregate all the synonym embedding representation vectors of the entity name. Let the name feature vector of the i-th entity be The embedding representation vector of the j-th synonym of the name feature of the i-th entity be Then the synonym feature vector E syn The calculation process is as follows:

[0034]

[0035] where |Syn i | represents the size of the synonym set;

[0036] S2-4. For the knowledge graph KG m to be matched, additionally obtain the context text feature vector and text title feature vector of the out-of-vocabulary entity:

[0037] First, split the context text included in the context text feature F ctx of the out-of-vocabulary entity into individual sentences S m . Let the total number of sentences be m. For each sentence S m , use the pre-trained language model to calculate its feature vector Obtain the global context text feature vector E ctx through average pooling aggregation operation. The calculation process is as follows:

[0038] [[ID=5Y]]

[0039] where represents the embedding of the i-th sentence;

[0040] Then, input the title information included in the text title feature F title of the out-of-vocabulary entity into the pre-trained language model, and through a specific mapping rule fmodel (F title )→E title , get the text title feature vector E title , where f model (·) is the internal mapping mechanism using a pre-trained language model;

[0041] S2-5, the obtained entity multimodal feature vector E mutil , that is, the structural characteristic vector E struct , relational eigenvector E rel , attribute description feature vector E att , name feature vector E name , name character feature vector E char , name synonym feature vector E syn , context text feature vector E ctx and text title feature vector E title Normalize them separately. For a given eigenvector E mutil , the normalization formula is:

[0042]

[0043] Where M represents the total number of eigenvectors, represents the i-th eigenvector.

[0044] Wherein, the step S3 specifically includes the following steps:

[0045] S3-1. Given a target knowledge graph KG t and the knowledge graph KG to be matched m Normalized entity multimodal feature vector Use independent linear transformations to map the features of each modality to the same feature space, and perform padding operations on each modal feature to make its dimension consistent. This process can be expressed as:

[0046]

[0047] Among them, W linear represents the linear transformation matrix, represents the zero filling process, d max_size Indicates the maximum dimension of the first dimension of all feature vectors, d i represents the first dimension of the i-th eigenvector;

[0048] S3-2. After completing the linear transformation and filling of the features, the BERT self-attention mechanism and the cross-modal attention module are used to dynamically allocate the weights of multimodal features and perform deep fusion. Let the feature vector after linear transformation and filling be Its dimensions are d, W Q, W K and W V are the weight matrices corresponding to the query vector, key vector, and value vector after the feature vector is mapped, and W S is the weight matrix of the modality-specific weight adjustment factor, d k is the dimension of the key vector, d v is the dimension of the value vector. sigmoid(·) is used to limit the output of the weight adjustment factor between 0 and 1. M is the total number of feature vectors, and ε is a constant used to prevent division by zero. The specific operations are as follows:

[0049] For the i-th feature vector, first calculate its attention score with other feature vectors, that is, the cross-modal attention score between the i-th feature vector and the j-th feature vector:

[0050]

[0051] Normalize the attention scores to obtain the attention probability distribution:

[0052]

[0053] Weighted sum the value vectors according to the attention probabilities to obtain the weighted feature vector:

[0054]

[0055] Furthermore, by the self-attention mechanism, focus on the feature relationships at different positions. Let the self-attention matrix be M Attn Perform a non-linear transformation on the weighted feature vector to finally obtain the deep fusion feature:

[0056] E fused = f nonlinear (M Attn ·E weighted )

[0057] = σ(W nonlinear ·(M Attn ·E weighted ) + b nonlinear )

[0058] where f nonlinear (·) is the non-linear transformation function σ, W nonlinear and b nonlinear are the weight matrix and bias vector of the non-linear transformation layer;

[0059] S3-3. After obtaining the high-level fusion feature, let this feature be E fused , whose dimension is d fused , first, through the linear layer (the weight matrix is set as W gate , and the bias vector is set as bgate ) and the Sigmoid activation function are used to transform the mean of the modal embeddings to obtain the gating weights for each modality:

[0060]

[0061] Next, multiply the embedding of each modality by the corresponding gating weight to obtain the modality embedding after gating processing:

[0062] E gated = G weights ⊙ E fused

[0063] where ⊙ represents element-wise multiplication;

[0064] Finally, calculate the weight normalization factor Multiply the modality embedding after gating processing by the weight normalization factor and perform a normalization operation to obtain the final joint feature representation:

[0065] F joint = α ⊙ E gated .

[0066] Among them, the step S4 specifically includes the following steps:

[0067] S4-1. For the target knowledge graph KG t in the entity entity t and the entity entity m to be matched in the knowledge graph KG m , their joint feature vectors are respectively represented as and Let d be the dimension of the joint feature vectors and , and are the values of the vectors in the i-th dimension, then the Manhattan distance between and is calculated as follows:

[0068]

[0069] S4-2. Evaluate the similarity between the entity entity t in the target knowledge graph KG t and the entity entity m to be matched in the knowledge graph KG m through the calculated Manhattan distance D. For the entity pair (entity t , entity m ), if D is less than or equal to the preset threshold θ, it is considered that these two entities are similar in the feature space and can be regarded as a match, that is:​

[0070]

[0071] S4-3. Through the matching results, the target knowledge graph KG can be further analyzed t and the knowledge graph KG to be matched m for the hyponymy and hypernymy relationships of entities. Let the matching result be

[0072]

[0073] To determine the hyponymy and hypernymy relationships, sort the entity pairs in R according to the Manhattan distance D. For the target entity entity ti , select the top three entities to be matched with the smallest Manhattan distance in all matching pairs {entity mi,1 , entity mi,2 , entity mi,3}. Based on their proximity in the feature space, speculate that they are potential hyponym entities of entity ti , and may be specific types, subclasses or specific instances of the target entity semantically. Further, a hypernym relationship credibility index can be defined to measure the possibility of the hypernym relationship of the entity pair (entity ti , entity mi,j ) (j = 1, 2, 3). The calculation formula is:

[0074]

[0075] The larger the index value, the higher the possibility that entity mi,j is a hyponym entity of entity ti , thus more precisely depicting the semantic hierarchical relationship between entities.

[0076] The beneficial effects of the present invention are as follows:

[0077] 1. By extracting and analyzing features such as the context information and text titles of out-of-vocabulary entities, the method of the present invention can effectively cope with the diversity and complexity of out-of-vocabulary words, achieve precise alignment and association of new concept entities, significantly improve the expansion ability of the knowledge graph, and greatly enhance the dynamic expansion ability of the knowledge graph.

[0078] 2. Through the extraction and embedded representation learning of multi-modal features such as structural features, relationship features, attribute descriptions, names and character features, synonym features, context texts and titles, and using the attention mechanism to dynamically allocate weights to different features, the multi-modal features can be more flexibly fused and expressed in entity alignment, improving the accuracy of entity alignment.

[0079] 3. Through semantic - related reasoning, the present invention can better reveal the semantic relationships in the knowledge graph, enhance the semantic expression ability of the knowledge graph, and contribute to providing more in - depth semantic reasoning and knowledge services in the application scenarios of the knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 It is a schematic diagram of the overall process of this method;

[0081] Figure 2 It is an example of the content of the knowledge graph implemented by this method. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0082] The following describes the specific embodiments of the present invention to facilitate those skilled in the art of this technology to understand the present invention, but it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art of this technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions created using the concept of the present invention are within the scope of protection.

[0083] As Figure 1 shown in the technical route, in an embodiment of the present invention, taking the Figure 2 knowledge graph as an implementation example, it generally includes four steps: extracting multi - modal feature information of entities from the knowledge graph, converting the multi - modal feature information of entities into an embedded representation vector, fusing multi - modal feature vectors using dynamic attention, and performing entity matching using the Manhattan distance. Specifically, it includes:

[0084] S11. From the target knowledge graph KG tExtract the multimodal feature information of the entity "Glacial Erosion Landform Type", specifically including: extract the structural information from the hierarchical relationship triples (such as <Moraine, subClassOf, Glacial Erosion Landform Type>), extract the relationship information from the semantic relationship triples (such as <Glacial Erosion, hasPossibleCause, Glacial Erosion Landform Type>), extract the attribute description information from the attribute triples (such as <Glacial Erosion Landform Type, definition, "Glacier erosion landform refers to the surface morphology formed by glacier erosion.">), and extract the entity name information and entity name character information from the entity name "Glacial Erosion Landform Type".

[0085] S12. Through the entity recognition method, extract from the domain literature those that do not belong to the target knowledge graph KG tUnregistered entity terms, such as "Moraine Chain", are identified, and their multimodal feature information is extracted, specifically including: structural information extracted from hierarchical relationship triples (such as <Moraine Chain, subClassOf, Moraine>), relationship information extracted from semantic relationship triples (such as <Moraine Chain, hasPart, Moraine Mound>), attribute description information extracted from attribute triples (such as <MoraineChain, definition, "Moraine Chain is a geological term referring to a series of glacial deposits formed by glacial transport and accumulation.">), entity name information and entity name character information extracted from the entity name "Moraine Chain", context text information (such as "Here we present a comprehensive dataset of glacial geomorphological features covering an area of 65 000km(2) in central west Poland, located along the southern sector of the last Scandinavian Ice Sheet, within the limits of the Baltic Ice Stream Complex…") extracted from the context text where the unregistered entity "Moraine Chain" is located, and text title information (such as "GIS dataset: geomorphological record of terrestrial terminating ice streams, southern sector of the Baltic Ice Stream Complex, last Scandinavian Ice Sheet, Poland") of the literature where the unregistered entity "Moraine Chain" is located.

[0086] S21. Identify the hierarchical relationships of entities in the target knowledge graph and the knowledge graph to be matched using the depth - first search algorithm (such as <Moraine, subClassOf, Glacial Erosion Landform Type>, <Moraine Chain, subClassOf, Moraine>, etc.), and use the multi - head graph attention network to learn the structural feature vector E struct .

[0087] S22. Construct a relationship matrix through the relationship information represented by triples (such as <Glacial Erosion, hasPossibleCause, Glacial Erosion Landform Type>). Each row in the matrix represents a triple relationship, and convert the relationship information into a relationship feature vector E rel =[1., 0., 0.,..., 0., 0., 0.] 1000 . Construct an attribute matrix through the attribute information represented by attribute triples (such as <Glacial Erosion Landform Type, definition, "Glacier erosion landform refers to the surface morphology formed by glacier erosion.">). Similarly, convert the attribute information into an attribute description feature vector E att =[1., 1., 1.,..., 0., 0., 0.] 1000 .

[0088] S23. Combine the entity names (such as "Glacial Erosion Landform Type" and "Moraine Chain") in the target knowledge graph and the knowledge graph to be matched into bigrams and extract character n - grams features. Obtain the synonym set of the entity name through WordNet (such as the synonyms of "Moraine Chain" include "Glacial Deposits", "Glacial Sediment", etc.). Use the DeBERTa model to encode the bigrams, n - grams, and synonym set to obtain the name feature vector E name =[0.0118, - 0.0009, 0.0126,…, - 0.0060, 0.0151, - 0.0017] 768 , the name character feature vector E char = 479 and the synonym feature vector E syn= [0.0155, 0.0004, 0.0110,..., -0.0052, 0.0151, -0.0066] 768 。

[0089] S24. For the out-of-vocabulary entity "Moraine Chain" in the knowledge graph to be matched, split its context text (such as "Here we present a comprehensive dataset of glacial geomorphological features…") into several clauses S m , and use the DeBERTa model to extract the feature vector E of each sentence S m . ctxi , and aggregate the feature vectors of all sentences into a global context text feature vector E through average pooling ctx . Then, use the DeBERTa model to encode the literature title ("GIS dataset: geomorphological record of terrestrial terminating ice streams, southern sector of the Baltic Ice Stream Complex, last Scandinavian Ice Sheet, Poland") to obtain the text title feature vector E title .

[0090] S25. Normalize all the extracted feature vectors to eliminate the influence of the dimension between different features. Obtain the normalized structural feature vector:

[0091] E struct=[[-7.1052e-03,4.7153e-03,6.2186e-03,...,-5.2834e-03,-4.5784e-03,-8.4631e-03],[-7.6812e-05,-3.0986e-03,-4.7079e-03,...,1.8698e-03,-2.1913e-03,5.3486e-03],[-1.6952e-04,-4.7226e-03,3.7893e-03,...,1.7191e-03,2.8094e-03,5.8236e-03],...,[1.2027e-03,1.7389e-03,1.4869e-03,...,-2.6818e-03,-3.6241e-04,4.4808e-03],[7.8364e-03,2.6386e-02,-1.2385e-02,...,8.1529e-03,-1.4877e-02,1.2660e-02],[-1.0174e-02,-1.8471e-03,-2.1599e-03,...,-1.2836e-03,-6.1202e-03,1.7901e-02]] 256 , the relationship feature vector E rel =[[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0174,-0.0863,-0.0230,...,0.0308,0.0444,-0.0292],...,[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0059,-0.0262,-0.0179,...,0.0008,0.0220,-0.0302],[-0.0059,-0.0262,-0.0179,...,0.0008,0.0220,-0.0302]] 256 , the attribute description feature vector E att= [[-0.0399, 0.0259, -0.0305,..., 0.0444, 0.0139, -0.0509], [-0.0150, 0.0094, -0.0165,...,-0.0613, -0.0146, -0.0008], [-0.0031, 0.0294, 0.0169,...,-0.0749, -0.0113, -0.0506],..., [0.0619, -0.0036, -0.0078,...,-0.0461, -0.0258, -0.0513], [0.0153, 0.0165, 0.0004,...,-0.0184, -0.0291, -0.0183], [0.0153, 0.0165, 0.0004,...,-0.0184, -0.0291, -0.0183]] 256 ,

[0092] Name feature vector E name = [[-0.0070, -0.0267, -0.0204,...,-0.0191, -0.0026, 0.0345], [-0.0044, -0.0307, -0.0354,...,-0.0058, -0.0021, 0.0408], [-0.0096, -0.0451, -0.0181,...,-0.0095, 0.0224, 0.0466],..., [-0.0105, -0.0389, -0.0350,...,-0.0139, 0.0109, 0.0525], [-0.0297, -0.0294, -0.0133,...,-0.0064, -0.0074, 0.0453], [-0.0146, -0.0332, -0.0271,...,-0.0200, -0.0009, 0.0214]] 256 ,

[0093] The name character feature vector and the synonymous feature vector are represented as follows:

[0094] Name character feature vector E char=[[-0.0460, 0.0307, 0.0028, ..., -0.0439, -0.0166, -0.0401], [-0.0186, 0.0238, 0.0331, ..., -0.0553, 0.0370, -0.0587], [-0.0121, 0.0634, 0.0044, ..., -0.0174, 0.0552, -0.0664], ..., [-0.0279, -0.0103, -0.0310, ..., -0.0024, 0.0451, -0.0446], [-0.0320, 0.0284, -0.0041, ..., -0.0327, 0.0136, -0.0137], [-0.0320, 0.0284, -0.0041, ..., -0.0327, 0.0136, -0.0137]] 256 , the synonymous feature vector E syn =[[-0.0079, -0.0272, 0.0331, ..., 0.0363, 0.0302, 0.0144], [-0.0084, -0.0292, 0.0384, ..., 0.0403, 0.0338, 0.0240], [-0.0086, -0.0264, 0.0383, ..., 0.0363, 0.0347, 0.0248], ..., [-0.0122, -0.0290, 0.0406, ..., 0.0423, 0.0390, 0.0237], [-0.0115, -0.0257, 0.0424, ..., 0.0422, 0.0411, 0.0230], [-0.0083, -0.0282, 0.0388, ..., 0.0394, 0.0344, 0.0232]] 256 , the context text feature vector E ctx=[[-0.0040, 0.0243, 0.0292,...,-0.0121, -0.0312, 0.0220], [-0.0040, 0.0243, 0.0292,...,-0.0121, -0.0312, 0.0220], [-0.0040, 0.0243, 0.0292,...,-0.0121, -0.0312, 0.0220],..., [-0.0040, 0.0243, 0.0292,...,-0.0121, -0.0312, 0.0220], [-0.0040, 0.0243, 0.0292,...,-0.0121, -0.0312, 0.0220], [-0.0040, 0.0243, 0.0292,...,-0.0121, -0.0312, 0.0220]] 256 and the text title feature vector E title =[[0.0205, 0.0108, -0.0317,..., 0.0236, -0.0292, -0.0326], [0.0205, 0.0108, -0.0317,..., 0.0236, -0.0292, -0.0326], [0.0205, 0.0108, -0.0317,..., 0.0236, -0.0292, -0.0326],..., [0.0205, 0.0108, -0.0317,..., 0.0236, -0.0292, -0.0326], [0.0205, 0.0108, -0.0317,..., 0.0236, -0.0292, -0.0326], [0.0205, 0.0108, -0.0317,..., 0.0236, -0.0292, -0.0326]] 256 .

[0095] S31. Use linear transformation to map the features of different modalities to the same feature space, and perform zero-padding operation on the features to make the dimensions of the features of each modality consistent, obtaining the zero-padded structural feature vector:

[0096] E struct=[[-7.1052e-03,4.7153e-03,6.2186e-03,...,-5.2834e-03,-4.5784e-03,-8.4631e-03],[-7.6812e-05,-3.0986e-03,-4.7079e-03,...,1.8698e-03,-2.1913e-03,5.3486e-03],[-1.6952e-04,-4.7226e-03,3.7893e-03,...,1.7191e-03,2.8094e-03,5.8236e-03],...,[1.2027e-03,1.7389e-03,1.4869e-03,...,-2.6818e-03,-3.6241e-04,4.4808e-03],[7.8364e-03,2.6386e-02,-1.2385e-02,...,8.1529e-03,-1.4877e-02,1.2660e-02],[-1.0174e-02,-1.8471e-03,-2.1599e-03,...,-1.2836e-03,-6.1202e-03,1.7901e-02]] 256 ,

[0097] Relationship feature vector E rel =[[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0174,-0.0863,-0.0230,...,0.0308,0.0444,-0.0292],...,[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0059,-0.0262,-0.0179,...,0.0008,0.0220,-0.0302],[-0.0059,-0.0262,-0.0179,...,0.0008,0.0220,-0.0302]] 256 , Attribute description feature vector E att=[[-0.0399, 0.0259, -0.0305,..., 0.0444, 0.0139, -0.0509], [-0.0150, 0.0094, -0.0165,...,-0.0613, -0.0146, -0.0008], [-0.0031, 0.0294, 0.0169,...,-0.0749, -0.0113, -0.0506],..., [0.0619, -0.0036, -0.0078,...,-0.0461, -0.0258, -0.0513], [0.0153, 0.0165, 0.0004,...,-0.0184, -0.0291, -0.0183], [0.0153, 0.0165, 0.0004,...,-0.0184, -0.0291, -0.0183]] 256 , named feature vector E name =[[-0.0070, -0.0267, -0.0204,...,-0.0191, -0.0026, 0.0345], [-0.0044, -0.0307, -0.0354,...,-0.0058, -0.0021, 0.0408], [-0.0096, -0.0451, -0.0!81,...,-0.0095, 0.0224, 0.0466],..., [-0.0146, -0.0332, -0.0271,...,-0.0200, -0.0009, 0.0214], [0.0000, 0.0000, 0.0000,..., 0.0000, 0.0000, 0.0000], [0.0000, 0.0000, 0.0000,..., 0.0000, 0.0000, 0.0000]] 256 , named character feature vector E char= [[-0.0460, 0.0307, 0.0028,..., -0.0439, -0.0166, -0.0401], [-0.0186, 0.0238, 0.0331,..., -0.0553, 0.0370, -0.0587], [-0.0121, 0.0634, 0.0044,..., -0.0174, 0.0552, -0.0664],..., [-0.0279, -0.0103, -0.0310,..., -0.0024, 0.0451, -0.0446], [-0.0320, 0.0284, -0.0041,..., -0.0327, 0.0136, -0.0137], [-0.0320, 0.0284, -0.0041,..., -0.0327, 0.0136, -0.0137]] 256 ,

[0098] The synonymous feature vector E syn = [[-0.0079, -0.0272, 0.0331,..., 0.0363, 0.0302, 0.0144], [-0.0084, -0.0292, 0.0384,..., 0.0403, 0.0338, 0.0240], [-0.0086, -0.0264, 0.0383,..., 0.0363, 0.0347, 0.0248],..., [-0.0083, -0.0282, 0.0388,..., 0.0394, 0.0344, 0.0232], [0.0000, 0.0000, 0.0000,..., 0.0000, 0.0000, 0.0000], [0.0000, 0.0000, 0.0000,..., 0.0000, 0.0000, 0.0000]] 256 ,

[0099] The context text feature vector and the text title feature vector are as follows:

[0100] The context text feature vector E ctx=[[-0.0040, 0.0243, 0.0292,...,-0.0121, -0.0312, 0.0220],[-0.0040, 0.0243, 0.0292,...,-0.0121, -0.0312, 0.0220],[-0.0040, 0.0243, 0.0292,...,-0.0121, -0.0312, 0.0220],...,[0.0000, 0.0000, 0.0000,...,0.0000, 0.0000, 0.0000],[0.0000, 0.0000, 0.0000,...,0.0000, 0.0000, 0.0000],[0.0000, 0.0000, 0.0000,...,0.0000, 0.0000, 0.0000]] 256 and the text title feature vector E title =[[0.0205, 0.0108, -0.0317,...,0.0236, -0.0292, -0.0326],[0.0205, 0.0108, -0.0317,...,0.0236, -0.0292, -0.0326],[0.0205, 0.0108, -0.0317,...,0.0236, -0.0292, -0.0326],...,[0.0000, 0.0000, 0.0000,...,0.0000, 0.0000, 0.0000],[0.0000, 0.0000, 0.0000,...,0.0000, 0.0000, 0.0000],[0.0000, 0.0000, 0.0000,...,0.0000, 0.0000, 0.0000]] 256 .

[0101] S32. After completing the linear transformation and padding of the features, use the BERT self-attention mechanism and the cross-modal attention module to perform dynamic allocation of multi-modal feature weights to obtain the deep fusion feature E fused .

[0102] S33. Multiply the gated modal embedding by the weight normalization factor and perform a normalization operation to obtain the final joint feature representation:

[0103] E joint=[[-0.0062, 0.0040, -0.0047,..., 0.0097, 0.0080, 0.0038], [-0.0026, 0.0016, -0.0028,..., 0.0109, 0.0092, 0.0065], [-0.0005, 0.0050, 0.0029,..., 0.0097, 0.0093, 0.0067],..., [0.0134, -0.0008, -0.0017,..., 0.0109, 0.0095, 0.0064], [0.0065, 0.0070, 0.0002,..., 0.0000, 0.0000, 0.0000], [0.0065, 0.0070, 0.0002,..., 0.0000, 0.0000, 0.0000]] 256 。

[0104] S41. Calculate the Manhattan distance between the target knowledge graph entity and the knowledge graph entity to be matched according to the joint feature vector Measure the similarity between entities.

[0105] S42. If the Manhattan distance is less than or equal to the preset threshold θ = 0.5, the two entities are considered to be matched.

[0106] S43. According to the matching result, analyze the hyponym and hypernym relationships of the entities. For example, for the entity to be matched "MoraineChain", the top three prediction results with the smallest Manhattan distance among all its matching pairs are "Moraine Mound", "GlacialErosion", and "Glacial Erosion Landform Type". Through the calculation of the hypernym relationship credibility index, it can be obtained that "Moraine Chain" has a subordinate relationship with "Glacial Erosion Landform Type". Therefore, it can be inferred that "Moraine Chain" is a hyponym of "Glacial Erosion Landform Type".

[0107] In summary, the present invention provides an efficient solution for the alignment of knowledge graph entities of out-of-vocabulary words by performing multi-modal feature extraction, feature encoding, feature fusion on the entities in the target knowledge graph and the knowledge graph to be matched, and performing entity matching and hyponym and hypernym relationship reasoning through the Manhattan distance. Through this method, efficient entity alignment and association can be achieved between knowledge graphs in different fields, providing support for the construction and optimization of knowledge graphs.

Claims

1. A method for updating out-of-vocabulary words in a knowledge graph considering precise semantic relationships, characterized in that Including the following steps: S1. Extract the multi-modal feature information F of entities from the target knowledge graph KG t and the knowledge graph KG to be matched containing out-of-vocabulary words m ; multi ​ Including: S1-1. Extract the structural information, t relationship information, attribute description information, name information, name character information, and name synonymy information from the target knowledge graph KG Obtain the multi-modal feature information of the entities in the target knowledge graph KG t S1-2. Using entity recognition methods, automatically extract out-of-vocabulary entity that does not belong to the target knowledge graph KG t from the domain scientific and technological literature, and construct a knowledge graph KG to be matched m , and extract the structural information relationship information attribute description information name information name character information and name synonymy information from the out-of-vocabulary entities, and additionally include the context text information of the text where the out-of-vocabulary words are located as well as the text title information of the literature Obtain the multi-modal feature information of the entities in the knowledge graph KG to be matched m ​ S2. Parse to obtain the target knowledge graph KG t and the multi-modal feature information F of the entities in the knowledge graph KG to be matched m Use the feature encoding method to generate the embedded representation vector F of the multi-modal feature F multi and perform normalization processing on it; multi mutil ​​ S3. Dynamically allocate feature weights W using the attention mechanism f for multimodal feature fusion to generate the joint feature vector E of each entity joint ; S4. Measure the similarity between the combined feature vectors E using the Manhattan distance, perform entity matching on the entities of the two knowledge graphs, and realize the association of semantic relationships between entities based on the matching results. joint Between them, perform entity matching on the entities of the two knowledge graphs, and realize the association of semantic relationships between entities based on the matching results.

2. The method for updating out-of-vocabulary words in a knowledge graph considering precise semantic relationships according to claim 1, wherein, The specific steps of step S2 include the following steps: S2-1. Analyze the target knowledge graph KG t with the knowledge graph KG to be matched m for the structural feature F of each entity struct . Use the graph traversal algorithm to identify the hierarchical relationship of each entity, and adopt the multi-head graph attention network to learn the structural feature vector F of the entity struct ; S2-2. First, parse the target knowledge graph KG t and the relationship features F m of each entity in the knowledge graph KG to be matched rel and the attribute description features F att , and obtain all relationship triple sets of each entity: And the set of attribute triples: Among them, entity head and entity tail represent two entities participating in the relationship, Relation is the relationship type between entities, entity represents the entity with this attribute, Attribute is the attribute type, and value att is the specific value of the attribute, n rel is the total number of relationship triples, and n att is the total number of attribute triples; Then, for KG t and KG m Construct an N×N all-zero matrix as the relationship matrix M rel , where the rows and columns correspond to the entities in the knowledge graph, the matrix values represent the relationship types between entities, and an N×A all-zero matrix as the attribute matrix M att , where the rows correspond to the entities, the columns correspond to the attribute types, and the values represent the attribute values corresponding to the entity attribute types. Here, N is the total number of entities in the knowledge graph, and A is the number of attribute types in the knowledge graph; For each relational triple (entity head , Relation, entity tail ), construct a relation dictionary {key rel : Relation} that maps the relation type Relation to a unique index value key rel . Determine the row and column positions in the relation matrix based on the entities entity head and entity tail , and set the value at that position to the index value of the relation type, indicating that there is a certain relationship between the entities entity head and entity tail ; For each property triple (entity, Attribute, value att ), construct a property dictionary {key att : value att}, and update the property information of the entity to the corresponding position in the property matrix based on the property dictionary; Finally, traverse all relational triples and attribute triples, and update the relational matrix M according to the following rules rel and the attribute matrix M att : Relationship feature vector E rel and attribute description feature vector E att are respectively obtained by converting rel the relationship matrix M att and the attribute matrix M S2-3. First, traverse the target knowledge graph KG t with the knowledge graph KG to be matched m for each entity name of each entity in it, split it into combinations of adjacent two words (word k , word k+1 ). and characters At the same time, use the synonym dictionary WordNet to find the synonym set Syn of the entity name i ={syn1,..., syn s}, where n name represents the number of words contained in the entity name, n char represents the number of characters contained in the entity name, l represents the window size when extracting grams, and s represents the number of synonyms contained in the entity name; Then, use the pre-trained language model to encode bigrams and character grams respectively, and average and aggregate the encoding results of all bigrams to obtain the name feature vector Average and aggregate the encoding results of all character grams to finally obtain the entity name character feature vector Subsequently, for the synonym set Syn i ={syn1,..., syn s}, it is encoded using a pre-trained language model and converted into an embedding representation Finally, by weighted summation of the feature vectors of the entity name and its synonyms, the embedding representation vectors of all synonyms of the entity name are aggregated. Let the name feature vector representing the i-th entity be The embedding representation vector of the j-th synonym of the name feature of the i-th entity be Then the synonym feature vector E syn The calculation process is as follows: Among them, |Syn i | represents the size of the synonym set; S2-4. For the knowledge graph KG to be matched m , additionally obtain the context text feature vector and text title feature vector of the out-of-vocabulary entity: First, the context text feature F of the unregistered entity ctx The included context text is split into individual sentences S by periods m , assuming the total number of sentences is m. For each sentence S m , use a pre-trained language model to calculate its feature vector Obtain the global context text feature vector E through average pooling aggregation operation ctx , and the calculation process is as follows: Among them, represents the embedding of the i-th sentence; Then, input the text title feature F of the unregistered entity title The included title information into the pre-trained language model, and through the mapping rule f model (F title ) → E title , obtain the text title feature vector E title , where f model (·) is the internal mapping mechanism using the pre-trained language model; S2-5. Normalize the obtained entity multi-modal feature vectors E mutil , namely the structural feature vector E struct , the relational feature vector E rel , the attribute description feature vector E att , the name feature vector E name , the name character feature vector E char , the name synonymy feature vector E syn , the context text feature vector E ctx and the text title feature vector E title respectively. For a given feature vector E mutil , the normalization formula is: Among them, M represents the total number of feature vectors, represents the i-th feature vector.

3. The method for updating out-of-vocabulary words in a knowledge graph considering precise semantic relationships according to claim 1, wherein The specific steps of step S3 include the following steps: S3-1. Given the target knowledge graph KG t and the knowledge graph KG to be matched m Normalized entity multi-modal feature vectors Use independent linear transformations to map the features of each modality to the same feature space, and perform padding operations on the features of each modality to make their dimensions consistent. This process is expressed as: Among them, W linear represents a linear transformation matrix, represents a zero-padding process, d max_size represents the maximum dimension of the first dimension among all eigenvectors, d i represents the first dimension of the i-th eigenvector; S3-2. After completing the linear transformation and padding of the features, use the BERT self-attention mechanism and cross-modal attention module for dynamic allocation of multi-modal feature weights and deep fusion. Let the feature vector after linear transformation and padding be with dimension d, W Q , W K and W V be the weight matrices corresponding to the query vector, key vector, and value vector after mapping the feature vector. W S is the weight matrix of the weight adjustment factor. d k is the dimension of the key vector. d v is the dimension of the value vector. sigmoid(·) is used to limit the output of the weight adjustment factor between 0 and 1. M is the total number of feature vectors. ε is a constant used to prevent division by zero. The specific operations are as follows: For the i-th feature vector, first calculate its attention score with other feature vectors, that is, the cross-modal attention score between the i-th feature vector and the j-th feature vector: Normalize the attention scores to obtain the attention probability distribution: Perform weighted summation on the value vectors according to the attention probabilities to obtain the weighted feature vectors: Pay attention to the feature relationships at different positions through the self-attention mechanism. Let the self-attention matrix be M Attn , perform a non-linear transformation on the weighted feature vectors, and finally obtain the deep fusion features: E fused = f nonlinear (M Attn ·E weighted ) = σ(W nonlinear ·(M Attn ·E weighted ) + b nonlinear ) Among them, f nonlinear (·) is the non-linear transformation function σ, W nonlinear and b nonlinear are the weight matrix and bias vector of the non-linear transformation layer; S3-3. After obtaining the high-level fusion feature, let this feature be E fused , and its dimension is d fused . First, the mean of the modal embeddings is transformed through a linear layer and a Sigmoid activation function to obtain the gating weights for each modality. The weight matrix of the linear layer is set as W gate , and the bias vector is set as b gate : Next, multiply the embedding of each modality by the corresponding gating weight to obtain the modality embedding after gating processing: E gated = G weights ☉E fused Where, ⊙ represents element-wise multiplication; Finally, calculate the weight normalization factor Multiply the gated modal embeddings by the weight normalization factor and perform a normalization operation to obtain the final joint feature representation: E joint = α☉E gated .

4. The method for updating out-of-vocabulary words in a knowledge graph considering precise semantic relationships according to claim 1, wherein The specific steps of step S4 include the following steps: S4-1. For the target knowledge graph KG t the entity t and the entity m in the knowledge graph KG to be matched m , their joint feature vectors are respectively denoted as and Let d be the dimension of the joint feature vectors and , and are the values of the vectors in the i-th dimension, then the Manhattan distance between and is calculated as follows: S4-2. Evaluate the target knowledge graph KG by the calculated Manhattan distance D t entity in t and the knowledge graph KG to be matched m entity in m The similarity between them. For the entity pair (entity t , entity m ), if D is less than or equal to the preset threshold θ, it is considered that these two entities are similar in the feature space and can be regarded as a match, that is: S4-3. Analyze the target knowledge graph KG based on the matching results t and the knowledge graph KG to be matched m for the hypernym and hyponym relationships of entities, and let the matching result be To determine the hyponymy relationship, sort the entity pairs in R according to the Manhattan distance D. For the target entity entity ti , select the top three candidate entities with the smallest Manhattan distance from all the matching pairs {entity mi，1 , entity mi，2 , entity mi，3}. Based on their proximity in the feature space, infer that they are potential hyponym entities of entity ti , which are specific types, subclasses, or concrete instances of the target entity semantically. Define a hypernym relationship credibility index to measure the possibility of the hypernym relationship of the entity pair (entity ti , entity mi，j ). The calculation formula is as follows: The larger the value of this indicator, the more it indicates entity mi，j As entity ti The higher the possibility of a subordinate entity.

Images