Knowledge graph unlisted word updating method considering precise semantic relationship

Abstract

The invention discloses a knowledge graph unregistered word updating method considering a precise semantic relationship, which comprises the following steps of: extracting entity multi-modal features, such as names, name characters, synonymous features and the like, from a target graph and a to-be-matched graph containing unregistered words; performing representation learning on the multi-modal features through a feature coding method, and generating and normalizing embedded vectors; dynamic empowerment is carried out according to feature importance by utilizing an attention mechanism, and weighted multi-modal feature vectors are fused to obtain a joint feature vector of the entity; the similarity is evaluated by calculating the Manhattan distance between entity joint feature vectors, entity alignment matching is completed, and entity hierarchical relation association is achieved. According to the method disclosed by the invention, the capability of the knowledge graph in processing the unregistered words is improved, the semantic richness and hierarchical structure of the knowledge graph are enhanced, and the method has important practical significance on construction and application of the knowledge graph.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge graph construction and entity alignment, and specifically to a method for updating unregistered words in a knowledge graph while taking into account precise semantic relationships. Background Art

[0002] With the development of big data and artificial intelligence technologies, knowledge graphs, a technology that organizes and represents structured and unstructured data through semantic associations, have been widely used in fields such as semantic search, intelligent recommendation, and question-answering systems. However, existing knowledge graph construction methods face challenges when dealing with unregistered terms. Unregistered terms are typically conceptual entities that do not appear in existing dictionaries or knowledge bases, such as newly emerged geographical terms and scientific terms.

[0003] Existing technologies for handling unregistered words often focus on text analysis using natural language processing techniques, but often overlook the importance of multimodal feature fusion. In particular, when aligning entities across heterogeneous, multi-source knowledge graphs, relying solely on structural or textual features can lead to inaccurate matches and make it difficult to reflect the hierarchical relationships and hyponymy between entities.

[0004] Furthermore, when processing unregistered words, there's often a lack of precise mechanisms for measuring semantic similarity between entities, leading to a lack of consistency and accuracy when updating and expanding knowledge graphs. Therefore, effectively processing unregistered words and accurately associating them with existing entities has become a pressing technical challenge. Summary of the Invention

[0005] The present invention aims to solve the above-mentioned problems existing in the prior art in the processing of unregistered words and the alignment of knowledge graph entities, and to provide a method for updating unregistered words in the knowledge graph that takes into account precise semantic relationships.

[0006] This method extracts multimodal feature information from the target knowledge graph and the knowledge graph to be matched containing unregistered words, performs feature encoding and feature fusion, and achieves accurate entity alignment through similarity calculation.

[0007] Through the method of the present invention, it is possible to effectively process unregistered words, and through the fusion of multimodal features and the precise calculation of semantic similarity, the precise alignment and hierarchical association of knowledge graph entities can be ensured, thereby improving the integrity and semantic expression ability of the knowledge graph.

[0008] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0009] This application provides a method for updating unregistered words in a knowledge graph that takes into account precise semantic relationships. The specific steps of the method include:

[0010] S1. From the target knowledge graph KGt and the knowledge graph KG to be matched containing unregistered words m Extract the multimodal feature information F of the entity multi ;

[0011] S2. Parse to obtain the target knowledge graph KG t and the knowledge graph KG to be matched m Multimodal feature information F of entities in multi , using feature encoding method to generate multimodal features F multi The embedding representation vector E mutil , and normalize it;

[0012] S3. Use attention mechanism to dynamically assign feature weights W f , perform multimodal feature fusion to generate the joint feature vector E of each entity joint ;

[0013] S4. Use Manhattan distance to measure the joint feature vector E joint The similarity between the two knowledge graphs is used to match the entities of the two knowledge graphs, and the semantic relationship between the entities is associated based on the matching results.

[0014] Wherein, the step S1 specifically includes the following steps:

[0015] S1-1. From the target knowledge graph KG t Extracting the structural information of entities Relationship Information Attribute description information Name information Name character information Synonymous information Get the target knowledge graph KG t Multimodal feature information of entities in

[0016] S1-2, using entity recognition methods, automatically extract entities that do not belong to the target knowledge graph KG from the field scientific literature t Unregistered word entities, build the knowledge graph KG to be matched m , extracting the structural information of unregistered word entities Relationship Information Attribute description information Name information Name character information Synonymous information And additionally contains the contextual text information of the text where the unregistered word is located and the text title information of the document Get the knowledge graph KG to be matched m Medium Entity

[0017] Multimodal feature information

[0018] Wherein, the step S2 specifically includes the following steps:

[0019] S2-1. Parsing the target knowledge graph KG t and the knowledge graph KG to be matched m The structural features F of each entity in struct , using graph traversal algorithm to identify the hierarchical relationship of each entity, and using multi-head graph attention network to learn the structural feature vector E of the entity struct ;

[0020] S2-2. First, parse the target knowledge graph KG t and the knowledge graph KG to be matched m The relation feature F of each entity in rel and attribute description feature F att , get the set of all relationship triples for each entity:

[0021]

[0022] And the set of attribute triples:

[0023]

[0024] Among them, entity head and entity tail Represents two entities involved in a relationship. Relation is the relationship type between entities. Entity represents the entity with the attribute. Attribute is the attribute type. Value att is the specific value of the attribute, n rel is the total number of relation triples, n att is the total number of attribute triples;

[0025] Then, KG t and KG m Construct an N×N all-zero matrix as the relationship matrix M rel , whose rows and columns correspond to entities in the knowledge graph, matrix values ​​represent the relationship types between entities, and an N×A all-zero matrix is ​​used as the attribute matrix M att , where rows correspond to entities, columns correspond to attribute types, and values ​​represent attribute values ​​corresponding to entity attribute types, where N is the total number of knowledge graph entities and A is the number of knowledge graph attribute types;

[0026] For each relation triple (entity head ,Relation,entity tail ), build the relationship dictionary {key rel :Relation} maps the relation type Relation to a unique index value key rel , according to entity head and entity tail Determine the row and column position in the relationship matrix and set the value of the position to the index value of the relationship type, representing the entity entity head and entity tail There is a relationship between them;

[0027] For each attribute triple (entity,Attribute,value att ), build the attribute dictionary {key att :value att}, update the attribute information of the entity to the corresponding position of the attribute matrix based on the attribute dictionary;

[0028] Finally, traverse all relationship triples and attribute triples and update the relationship matrix M according to the following rules rel and the attribute matrix M att :

[0029]

[0030] Relation eigenvector E rel and attribute description feature vector E att The relationship matrix M rel and the attribute matrix M att Obtained through specific transformation;

[0031] S2-3. First, traverse the target knowledge graph KG t and the knowledge graph KG to be matched m Each entity name of each entity in is split into two adjacent words (word k ,word k+1 ) and characters At the same time, the synonym dictionary WordNet is used to find the synonym set Syn i ={syn1,…,syn s}, where n name Indicates the number of words contained in the entity name, n char Indicates the number of characters contained in the entity name, l indicates the window size when extracting gram, and s indicates the number of synonyms contained in the entity name;

[0032] Then, use the pre-trained language model to encode bigram and character gram respectively, and encode the encoding results of all bigrams Perform average aggregation to obtain the name feature vector Encoding results for all character gram Perform average aggregation to finally obtain the entity name character feature vector

[0033] Then, the synonym set Syn i ={Syn1,…,syn s}Use the pre-trained language model to encode and convert it into an embedded representation Finally, by weighted summing the feature vectors of the entity name and its synonyms, all synonym embedding representation vectors of the entity name are aggregated. Let the name feature vector representing the i-th entity be The embedding representation vector of the jth synonym of the name feature of the i-th entity is Then the synonymous eigenvector E syn The calculation process is as follows:

[0034]

[0035] Among them, |Syn i | indicates the size of the synonym set;

[0036] S2-4. For the knowledge graph KG to be matched m , additionally obtain the context text feature vector and text title feature vector of the unregistered entity:

[0037] First, the contextual text feature F of the unregistered entity ctx The context text is divided into separate sentences S by periods. m , let the total number of sentences be m, for each sentence S m , use the pre-trained language model to calculate its feature vector The global context text feature vector E is obtained through average pooling aggregation operation ctx , the calculation process is as follows:

[0038]

[0039] in, represents the embedding of the i-th sentence;

[0040] Then, the text title feature F of the unregistered entity title The title information contained is input into the pre-trained language model, and the specific mapping rule fmodel (F title )→E title , get the text title feature vector E title , where f model (·) is the internal mapping mechanism using a pre-trained language model;

[0041] S2-5, the obtained entity multimodal feature vector E mutil , that is, the structural characteristic vector E struct , relational eigenvector E rel , attribute description feature vector E att , name feature vector E name , name character feature vector E char , name synonym feature vector E syn , context text feature vector E ctx and text title feature vector E title Normalize them separately. For a given eigenvector E mutil , the normalization formula is:

[0042]

[0043] Where M represents the total number of eigenvectors, represents the i-th eigenvector.

[0044] Wherein, the step S3 specifically includes the following steps:

[0045] S3-1. Given a target knowledge graph KG t and the knowledge graph KG to be matched m Normalized entity multimodal feature vector Use independent linear transformations to map the features of each modality to the same feature space, and perform padding operations on each modal feature to make its dimension consistent. This process can be expressed as:

[0046]

[0047] Among them, W linear represents the linear transformation matrix, represents the zero filling process, d max_size Indicates the maximum dimension of the first dimension of all feature vectors, d i represents the first dimension of the i-th eigenvector;

[0048] S3-2. After completing the linear transformation and filling of the features, the BERT self-attention mechanism and the cross-modal attention module are used to dynamically allocate the weights of multimodal features and perform deep fusion. Let the feature vector after linear transformation and filling be Its dimensions are d, W Q、W K and W V It is the weight matrix corresponding to the feature vector after mapping to query vector, key vector and value vector, W S is the weight matrix of the modality-specific weight adjustment factors, d k is the dimension of the key vector, d v is the dimension of the value vector, sigmoid(·) is used to limit the output of the weight adjustment factor between 0 and 1, M is the total number of feature vectors, and ε is a constant used to prevent division by zero. The specific operation is as follows:

[0049] For the i-th feature vector, first calculate its attention score with other feature vectors, that is, the cross-modal attention score of the i-th feature vector and the j-th feature vector:

[0050]

[0051] Normalize the attention scores to obtain the attention probability distribution:

[0052]

[0053] The value vector is weighted and summed according to the attention probability to obtain the weighted feature vector:

[0054]

[0055] Furthermore, we use the self-attention mechanism to focus on the relationship between features at different positions, and set the self-attention matrix as M Attn Perform nonlinear transformation on the weighted feature vector to finally obtain deep fusion features:

[0056] E fused =f nonlinear (M Attn ·E weighted )

[0057] =σ(W nonlinear ·(M Attn ·E weighted )+b nonlinear )

[0058] Among them, f nonlinear (·) is the nonlinear transformation function σ, W nonlinear and b nonlinear is the weight matrix and bias vector of the nonlinear transformation layer;

[0059] S3-3, after obtaining the high-level fusion feature, let the feature be E fused , whose dimension is d fused , first, through the linear layer (weight matrix is ​​set to W gate , the bias vector is set to bgate ) and the Sigmoid activation function transform the mean of the modal embedding to obtain the gating weight of each modality:

[0060]

[0061] Next, the embedding of each modality is multiplied by the corresponding gating weight to obtain the gated modality embedding:

[0062] E gated =G weights ⊙E fused

[0063] Among them, ⊙ represents the multiplication of corresponding elements;

[0064] Finally, calculate the weight normalization factor The gated modal embedding is multiplied by the weight normalization factor and normalized to obtain the final joint feature representation:

[0065] F joint =α⊙E gated .

[0066] Wherein, the step S4 specifically includes the following steps:

[0067] S4-1. For the target knowledge graph KG t entity t and the knowledge graph KG to be matched m entity m , and their joint eigenvectors are expressed as and Let d be the joint eigenvector and Dimensions, and is the value of the vector in the i-th dimension, then and The Manhattan distance between them is calculated as follows:

[0068]

[0069] S4-2. Evaluate the target knowledge graph KG by calculating the Manhattan distance D t entity t and the knowledge graph KG to be matched m entity m Similarity between entities t ,entity m ) If D is less than or equal to the preset threshold θ, the two entities are considered similar in the feature space and can be considered as matching, that is:

[0070]

[0071] S4-3. Through the matching results, the target knowledge graph KG can be further analyzed t and the knowledge graph KG to be matched m The upper and lower relationship of the entity in , let the matching result be

[0072]

[0073] In order to determine the hierarchical relationship, the entity pairs in R are sorted according to the Manhattan distance D. For the target entity ti , select the first three matching entities with the smallest Manhattan distance among all matching pairs {entity mi,1 ,entity mi,2 ,entity mi,3}, based on their proximity in feature space, it is inferred that they are entities ti The potential subordinate entity of the target entity may be a specific type, subclass or specific instance of the target entity in semantics. Furthermore, a hyperlink credibility index can be defined to measure the entity pair. ti ,entity mi,j The possibility of the hyperstasis relationship of )(j=1,2,3) is calculated as follows:

[0074]

[0075] The larger the value of this indicator, the mi,j As an entity ti The higher the probability of subordinate entities, the more accurately the semantic hierarchical relationship between entities can be portrayed.

[0076] The beneficial effects of the present invention are:

[0077] 1. By extracting and analyzing the contextual information, text titles and other features of unregistered word entities, the method of the present invention can effectively cope with the diversity and complexity of unregistered words, achieve precise alignment and association of new concept entities, significantly improve the expansion capability of the knowledge graph, and greatly enhance the dynamic expansion capability of the knowledge graph.

[0078] 2. The present invention extracts and embeds multimodal features such as structural features, relationship features, attribute descriptions, name and character features, synonymous features, contextual text and titles for representation learning, and uses the attention mechanism to dynamically assign weights of different features, so that multimodal features can be more flexibly integrated and expressed in entity alignment, thereby improving the accuracy of entity alignment.

[0079] 3. Through reasoning about semantic associations, the present invention can better reveal the semantic relationships in the knowledge graph, enhance the semantic expression ability of the knowledge graph, and help the knowledge graph provide more in-depth semantic reasoning and knowledge services in application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 Schematic diagram of the overall process of this method;

[0081] Figure 2 Example of knowledge graph content implemented for this method. DETAILED DESCRIPTION

[0082] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0083] like Figure 1 As shown in the technical route, in one embodiment of the present invention, Figure 2 The knowledge graph of is used as an implementation example. The overall process includes four steps: extracting multimodal feature information of entities from the knowledge graph, converting the multimodal feature information of entities into embedded representation vectors, using dynamic attention to fuse multimodal feature vectors, and using Manhattan distance to match entities. Specifically, it includes:

[0084] S11. From the target knowledge graph KG tExtract the multimodal feature information of the entity "Glacial Erosion Landform Type" from the hierarchical relationship triples (such as<Moraine,subClassOf,Glacial ErosionLandform Type> ) extracts structural information from semantic relationship triples (such as<Glacial Erosion,hasPossibleCause,Glacial Erosion Landform Type> ) extracts relation information from attribute triples (such as <Glacial Erosion Landform Type,definition,"Glacier erosion landform refersto the surface morphology formed by glacier erosion."> ) extracts attribute description information, and extracts entity name information and entity name character information from the entity name "Glacial Erosion Landform Type".

[0085] S12, through the entity recognition method, extract the entities that do not belong to the target knowledge graph KG from the domain literature tUnregistered word entities, such as "Moraine Chain", and extract their multimodal feature information, specifically including: from hierarchical relationship triples (such as<Moraine Chain,subClassOf,Moraine> ) extracts structural information from semantic relationship triples (such as<Moraine Chain,hasPart,Moraine Mound> ) extracts relation information from attribute triples (such as <MoraineChain,definition,"Moraine Chain is a geological term referring to a series ofglacial deposits formed by glacial transport and accumulation."> ) extracts attribute description information, entity name information and entity name character information from the entity name "Moraine Chain", context text information (such as "Here we present a comprehensive dataset of glacial geomorphological features covering an area of ​​65,000 km(2)in central west Poland,located along the southern sector of the last Scandinavian Ice Sheet,within the limits of the Baltic Ice Stream Complex…") from the context text where the unregistered entity "Moraine Chain" is located, and text title information (such as "GIS dataset:geomorphological record of terrestrial terminating ice streams,southern sector of the Baltic Ice Stream Complex,last Scandinavian Ice Sheet,Poland") from the document where the unregistered entity "Moraine Chain" is located.

[0086] S21. Use the depth-first search algorithm to identify the hierarchical relationship between the target knowledge graph and the entities in the knowledge graph to be matched (such as<Moraine,subClassOf,Glacial Erosion Landform Type><Moraine Chain,subClassOf,Moraine> etc.), using the multi-head graph attention network to learn the structural feature vector E struct .

[0087] S22, relationship information represented by triples (such as<Glacial Erosion,hasPossibleCause,Glacial Erosion Landform Type> ) Construct a relationship matrix, each row in the matrix represents a triple relationship, and convert the relationship information into a relationship feature vector E rel =[1.,0.,0.,...,0.,0.,0.] 1000 Attribute information represented by attribute triples (such as <Glacial Erosion Landform Type,definition,"Glacier erosionlandform refers to the surface morphology formed by glacier erosion."> ) constructs the attribute matrix, and similarly, transforms the attribute information into the attribute description feature vector E att =[1.,1.,1.,...,0.,0.,0.] 1000 .

[0088] S23. Perform bigram combination and character n-gram feature extraction on the entity names in the target knowledge graph and the knowledge graph to be matched (such as "Glacial ErosionLandform Type" and "Moraine Chain"). Obtain the synonym set of the entity name through WordNet (such as the synonyms of "Moraine Chain" include "GlacialDeposits" and "GlacialSediment", etc.), use the DeBERTa model to encode the bigram, n-gram and synonym set, and obtain the name feature vector E name =[0.0118,-0.0009,0.0126,…,-0.0060,0.0151,-0.0017] 768 , name character feature vector E char =479 and synonymous eigenvector E syn=[0.0155,0.0004,0.0110,...,-0.0052,0.0151,-0.0066] 768 .

[0089] S24. For the unregistered entity “Moraine Chain” in the knowledge graph to be matched, its context text (such as “Here we present a comprehensive dataset of glacial geomorphological features…”) is divided into several clauses according to the period S m , use the DeBERTa model to extract each sentence S m The eigenvector E ctxi , aggregate the feature vectors of all sentences into a global contextual text feature vector E by average pooling ctx Then, the DeBERTa model is used to encode the document title ("GIS dataset: geomorphological record of terrestrialterminating ice streams, southern sector of the Baltic Ice Stream Complex, last Scandinavian Ice Sheet, Poland") to obtain the text title feature vector E title .

[0090] S25. Normalize all extracted feature vectors to eliminate the dimensionality effects between different features. The normalized structural feature vector is obtained:

[0091] E struct=[[-7.1052e-03,4.7153e-03,6.2186e-03,...,-5.2834e-03,-4.5784e-03,-8.4631e-03],[-7.6812e-05,-3.0986e-03,-4.7079e-03 ,...,1.8698e-03,-2.1913e-03,5.3486e-03],[-1.6952e-04,-4.7226e-03,3.7893e-03,...,1.7191e-03,2.8094e-03,5.8236e-03], ...,[1.2027e-03,1.7389e-03,1.4869e-03,...,-2.6818e-03,-3.6241e-04,4.4808e-03],[7.8364e-03,2.6386e-02,-1.2385e-02,. ..,8.1529e-03,-1.4877e-02,1.2660e-02],[-1.0174e-02,-1.8471e-03,-2.1599e-03,...,-1.2836e-03,-6.1202e-03,1.7901e-02]] 256 , relational eigenvector E rel =[[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0174,-0.0863,-0.0230,...,0.0308,0.0444,-0.0292],. ..,[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0059,-0.0262,-0.0179,...,0.0008,0.0220,-0.0302],[-0.0059,-0.0262,-0.0179,...,0.0008,0.0220,-0.0302]] 256 , attribute description feature vector E att=[[-0.0399,0.0259,-0.0305,...,0.0444,0.0139,-0.0509],[-0.0150,0.0094,-0.0165,...,-0.0613,-0.0146,-0.0008],[-0.0031,0.0294,0.0169,...,-0.0749,-0.0113,-0.0506], ...,[0.0619,-0.0036,-0.0078,...,-0.0461,-0.0258,-0.0513],[0.0153,0.0165,0.0004,...,-0.0184,-0.0291,-0.0183],[0.0153,0.0165,0.0004,...,-0.0184,-0.0291,-0.0183]] 256 ,

[0092] Name feature vector E name =[[-0.0070,-0.0267,-0.0204,...,-0.0191,-0.0026,0.0345],[-0.0044,-0.0307,-0.0354,...,-0.0058,-0.0021,0.0408],[-0.0096,-0.0451,-0.0181,...,-0.0095,0.0224,0.0466],. ..,[-0.0105,-0.0389,-0.0350,...,-0.0139,0.0109,0.0525],[-0.0297,-0.0294,-0.0133,...,-0.0064,-0.0074,0.0453],[-0.0146,-0.0332,-0.0271,...,-0.0200,-0.0009,0.0214]] 256 ,

[0093] Name character feature vectors and synonym feature vectors are expressed as follows:

[0094] Name character feature vector E char=[[-0.0460,0.0307,0.0028,...,-0.0439,-0.0166,-0.0401],[-0.0186,0.0238,0.0331,...,-0.0553,0.0370,-0.0587],[-0.0121,0.0634,0.0044,...,-0.0174,0.0552,-0.0664],.. .,[-0.0279,-0.0103,-0.0310,...,-0.0024,0.0451,-0.0446],[-0.0320,0.0284,-0.0041,...,-0.0327,0.0136,-0.0137],[-0.0320,0.0284,-0.0041,...,-0.0327,0.0136,-0.0137]] 256 , synonymous eigenvector E syn =[[-0.0079,-0.0272,0.0331,...,0.0363,0.0302,0.0144],[-0.0084,-0.0292,0.0384,...,0.0403,0.0338,0.0240],[-0.0086,-0.0264,0.0383,...,0.0363,0.0347,0.0248],. ..,[-0.0122,-0.0290,0.0406,...,0.0423,0.0390,0.0237],[-0.0115,-0.0257,0.0424,...,0.0422,0.0411,0.0230],[-0.0083,-0.0282,0.0388,...,0.0394,0.0344,0.0232]] 256 , context text feature vector E ctx=[[-0.0040,0.0243,0.0292,...,-0.0121,-0.0312,0.0220],[-0.0040,0.0243,0.0292,...,-0.0121,-0.0312,0.0220],[-0.0040,0.0243,0.0292,...,-0.0121,-0.0312,0.0220],. ..,[-0.0040,0.0243,0.0292,...,-0.0121,-0.0312,0.0220],[-0.0040,0.0243,0.0292,...,-0.0121,-0.0312,0.0220],[-0.0040,0.0243,0.0292,...,-0.0121,-0.0312,0.0220]] 256 and text title feature vector E title =[[0.0205,0.0108,-0.0317,...,0.0236,-0.0292,-0.0326],[0.0205,0.0108,-0.0317,...,0.0236,-0.0292,-0.0326],[0.0205,0.0108,-0.0317,...,0.0236,-0.0292,-0.0326],. ..,[0.0205,0.0108,-0.0317,...,0.0236,-0.0292,-0.0326],[0.0205,0.0108,-0.0317,...,0.0236,-0.0292,-0.0326],[0.0205,0.0108,-0.0317,...,0.0236,-0.0292,-0.0326]] 256 .

[0095] S31. Use linear transformation to map the features of different modes to the same feature space, and perform zero-filling operation on the features to make the dimensions of each modal feature consistent, and obtain the structural feature vector after zero filling:

[0096] E struct=[[-7.1052e-03,4.7153e-03,6.2186e-03,...,-5.2834e-03,-4.5784e-03,-8.4631e-03],[-7.6812e-05,-3.0986e-03,-4.7079e-03 ,...,1.8698e-03,-2.1913e-03,5.3486e-03],[-1.6952e-04,-4.7226e-03,3.7893e-03,...,1.7191e-03,2.8094e-03,5.8236e-03], ...,[1.2027e-03,1.7389e-03,1.4869e-03,...,-2.6818e-03,-3.6241e-04,4.4808e-03],[7.8364e-03,2.6386e-02,-1.2385e-02,. ..,8.1529e-03,-1.4877e-02,1.2660e-02],[-1.0174e-02,-1.8471e-03,-2.1599e-03,...,-1.2836e-03,-6.1202e-03,1.7901e-02]] 256 ,

[0097] Relation eigenvector E rel =[[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0174,-0.0863,-0.0230,...,0.0308,0.0444,-0.0292],. ..,[-0.0116,-0.0563,-0.0204,...,0.0158,0.0332,-0.0297],[-0.0059,-0.0262,-0.0179,...,0.0008,0.0220,-0.0302],[-0.0059,-0.0262,-0.0179,...,0.0008,0.0220,-0.0302]] 256 , attribute description feature vector E att=[[-0.0399,0.0259,-0.0305,...,0.0444,0.0139,-0.0509],[-0.0150,0.0094,-0.0165,...,-0.0613,-0.0146,-0.0008],[-0.0031,0.0294,0.0169,...,-0.0749,-0.0113,-0.0506], ...,[0.0619,-0.0036,-0.0078,...,-0.0461,-0.0258,-0.0513],[0.0153,0.0165,0.0004,...,-0.0184,-0.0291,-0.0183],[0.0153,0.0165,0.0004,...,-0.0184,-0.0291,-0.0183]] 256 , name feature vector E name =[[-0.0070,-0.0267,-0.0204,...,-0.0191,-0.0026,0.0345],[-0.0044,-0.0307,-0.0354,...,-0.0058,-0.0021,0.0408],[-0.0096,-0.0451,-0.0181,...,-0.0095,0.0224,0.04 66],...,[-0.0146,-0.0332,-0.0271,...,-0.0200,-0.0009,0.0214],[0.0000,0.0000,0.0000,...,0.0000,0.0000,0.0000],[0.0000,0.0000,0.0000,...,0.0000,0.0000,0.0000]] 256 , name character feature vector E char=[[-0.0460,0.0307,0.0028,...,-0.0439,-0.0166,-0.0401],[-0.0186,0.0238,0.0331,...,-0.0553,0.0370,-0.0587],[-0.0121,0.0634,0.0044,...,-0.0174,0.0552,-0.0664],.. .,[-0.0279,-0.0103,-0.0310,...,-0.0024,0.0451,-0.0446],[-0.0320,0.0284,-0.0041,...,-0.0327,0.0136,-0.0137],[-0.0320,0.0284,-0.0041,...,-0.0327,0.0136,-0.0137]] 256 ,

[0098] Synonymous eigenvector E syn =[[-0.0079,-0.0272,0.0331,...,0.0363,0.0302,0.0144],[-0.0084,-0.0292,0.0384,...,0.0403,0.0338,0.0240],[-0.0086,-0.0264,0.0383,...,0.0363,0.0347,0.0248] ,...,[-0.0083,-0.0282,0.0388,...,0.0394,0.0344,0.0232],[0.0000,0.0000,0.0000,...,0.0000,0.0000,0.0000],[0.0000,0.0000,0.0000,...,0.0000,0.0000,0.0000]] 256 ,

[0099] The context text feature vector and text title feature vector are as follows:

[0100] Context text feature vector E ctx=[[-0.0040,0.0243,0.0292,...,-0.0121,-0.0312,0.0220],[-0.0040,0.0243,0.0292,...,-0.0121,-0.0312,0.0220],[-0.0040,0.0243,0.0292,...,-0.0121,-0.0312,0.0220 20],...,[0.0000,0.0000,0.0000,...,0.0000,0.0000,0.0000],[0.0000,0.0000,0.0000,...,0.0000,0.0000,0.0000],[0.0000,0.0000,0.0000,...,0.0000,0.0000,0.0000]] 256 and text title feature vector E title =[[0.0205,0.0108,-0.0317,...,0.0236,-0.0292,-0.0326],[0.0205,0.0108,-0.0317,...,0.0236,-0.0292,-0.0326],[0.0205,0.0108,-0.0317,...,0.0236,-0.0292,-0.03 26],...,[0.0000,0.0000,0.0000,...,0.0000,0.0000,0.0000],[0.0000,0.0000,0.0000,...,0.0000,0.0000,0.0000],[0.0000,0.0000,0.0000,...,0.0000,0.0000,0.0000]] 256 .

[0101] S32. After completing the linear transformation and filling of the features, the BERT self-attention mechanism and the cross-modal attention module are used to dynamically allocate the multimodal feature weights to obtain the deep fusion feature E fused .

[0102] S33. Multiply the gated modal embedding by the weight normalization factor and perform normalization to obtain the final joint feature representation:

[0103] E joint=[[-0.0062,0.0040,-0.0047,...,0.0097,0.0080,0.0038],[-0.0026,0.0016,-0.0028,...,0.0109,0.0092,0.0065],[-0.0005,0.0050,0.0029,...,0.0097,0.0093,0.0067] ,...,[0.0134,-0.0008,-0.0017,...,0.0109,0.0095,0.0064],[0.0065,0.0070,0.0002,...,0.0000,0.0000,0.0000],[0.0065,0.0070,0.0002,...,0.0000,0.0000,0.0000]] 256 .

[0104] S41. Calculate the Manhattan distance between the target knowledge graph entity and the knowledge graph entity to be matched based on the joint feature vector Measure the similarity between entities.

[0105] S42. If the Manhattan distance is less than or equal to a preset threshold θ=0.5, the two entities are considered to be matched.

[0106] S43. Analyze the entity's hyponym relationship based on the matching results. For example, for the entity to be matched, "MoraineChain," the top three predictions with the smallest Manhattan distance among all matching pairs are "Moraine Mound," "GlacialErosion," and "GlacialErosionLandformType." Calculating the hypernym credibility index reveals that "Moraine Chain" and "GlacialErosionLandformType" exhibit a subordinate relationship. Therefore, it can be inferred that "Moraine Chain" is a hyponym of "GlacialErosionLandformType."

[0107] In summary, this invention provides an efficient solution for knowledge graph entity alignment of unregistered words by performing multimodal feature extraction, feature encoding, and feature fusion on entities in the target knowledge graph and the knowledge graph to be matched, and performing entity matching and hyponym / hypernym reasoning using Manhattan distance. This method can achieve efficient entity alignment and association between knowledge graphs in different fields, providing support for the construction and optimization of knowledge graphs.

Claims

1. A method for updating unregistered words in a knowledge graph taking into account precise semantic relationships, characterized in that: The following steps are involved: S1. From the target knowledge graph KG t and the knowledge graph KG to be matched containing unregistered words m Extract the multimodal feature information of the entity F multi ; S2. Parse to obtain the target knowledge graph KG t And the knowledge graph KG to be matched m The multimodal feature information F of the entity in multi , using feature encoding method to generate multimodal features F multi The embedding representation vector E mutil , and normalize it; S3. Use the attention mechanism to dynamically assign feature weights W f , perform multimodal feature fusion to generate the joint feature vector E of each entity joint ; S4. Use Manhattan distance to measure the joint feature vector E joint The similarity between the two knowledge graphs is used to match the entities of the two knowledge graphs, and the semantic relationship between the entities is associated based on the matching results.

2. According to claim 1, the method for updating unregistered words in the knowledge graph taking into account precise semantic relationships is characterized in that: The step S1 specifically includes the following steps: S1-1. From the target knowledge graph KG t Extracting structural information of entities Relationship Information Attribute description information Name information Name character information Synonymous information with name Get the target knowledge graph KG t Multimodal feature information of entities in S1-2. Use entity recognition methods to automatically extract entities that do not belong to the target knowledge graph KG from the scientific literature in the field t Unregistered word entities to build the knowledge graph KG to be matched m , extracting the structural information of unregistered word entities Relationship Information Attribute description information Name information Name character information Synonymous information with name And additionally contains the contextual text information of the text where the unregistered word is located and the text title information of the document Get the knowledge graph KG to be matched m Multimodal feature information of entities in 3. The method for updating unregistered words in a knowledge graph taking into account precise semantic relationships according to claim 1, characterized in that: The step S2 specifically includes the following steps: S2-1. Parsing the target knowledge graph KG t And the knowledge graph KG to be matched m The structural features F of each entity in struct , using the graph traversal algorithm to identify the hierarchical relationship of each entity, and using the multi-head graph attention network to learn the structural feature vector E of the entity struct ; S2-2. First, parse the target knowledge graph KG t And the knowledge graph KG to be matched m The relation feature F of each entity in rel and attribute description feature F att , get the set of all relationship triples for each entity: And a set of attribute triples: Among them, entity head and entity tail Represents two entities involved in a relationship. Relation is the type of relationship between entities. Entity indicates an entity with the attribute. Attribute is the attribute type. Value att is the specific value of the attribute, n rel is the total number of relation triples, n att is the total number of attribute triples; Then, KG t and KG m Construct an N×N all-zero matrix as the relationship matrix M rel , whose rows and columns correspond to entities in the knowledge graph, matrix values ​​represent the relationship types between entities, and a ×A all-zero matrix as the attribute matrix M att , where rows correspond to entities, columns correspond to attribute types, and values ​​represent attribute values ​​corresponding to entity attribute types, where N is the total number of knowledge graph entities and A is the number of knowledge graph attribute types; For each relation triple (entity head ,Relation,entity tail ), build a relation dictionary {key rel :Relation} maps the relation type Relation to a unique index value key rel , according to entity head and entity tail Determine the row and column position in the relationship matrix and set the value of the position to the index value of the relationship type, representing the entity entity head and entity tail There is a relationship between them; For each attribute triple (entity,Attribute,value att ), construct the attribute dictionary {key att :value att }, based on the attribute dictionary, update the attribute information of the entity to the corresponding position of the attribute matrix; Finally, traverse all relationship triplets and attribute triplets and update the relationship matrix M according to the following rules rel and the attribute matrix M att : Relation feature vector E rel and attribute description feature vector E att The relationship matrix M rel and the attribute matrix M att Obtained by specific transformation; S2-3. First, traverse the target knowledge graph KG t And the knowledge graph KG to be matched m Each entity name of each entity in is split into two adjacent words (word k ,word k+1 ) and characters At the same time, the synonym dictionary WordNet is used to find the synonym set Syn i ={syn1,…,syn s }, where n name Indicates the number of words contained in the entity name, n char Indicates the number of characters contained in the entity name, l indicates the window size when extracting gram, and s indicates the number of synonyms contained in the entity name; Then, the pre-trained language model is used to encode bigrams and character gram respectively, and the encoding results of all bigrams are Perform average aggregation to obtain the name feature vector Encoding results for all character gram Perform average aggregation to finally obtain the entity name character feature vector Then, for the synonym set Syn i ={syn1,…,syn s } Encode using a pre-trained language model and convert it into an embedded representation Finally, by weighted summing the feature vectors of the entity name and its synonyms, all synonym embedding representation vectors of the entity name are aggregated. Let the name feature vector representing the i-th entity be The embedding representation vector of the jth synonym of the name feature of the i-th entity is Then the synonymous eigenvector E syn The calculation process is as follows: Among them, |Syn i | indicates the size of the synonym set; S2-4. For the knowledge graph KG to be matched m , additionally obtain the context text feature vector and text title feature vector of the unregistered entity: First, the contextual text features F of the unregistered entity ctx The context text contained is split into separate sentences S by periods. m , let the total number of sentences be m, for each sentence S m , using the pre-trained language model to calculate its feature vector The global context text feature vector E is obtained through average pooling aggregation operation ctx , the calculation process is as follows: in, represents the embedding of the i-th sentence; Then, the text title feature F of the unregistered entity title The title information contained is input into the pre-trained language model, and the specific mapping rule f model (F title ) → E title , get the text title feature vector E title , where f model (·) is the internal mapping mechanism using a pre-trained language model; S2-5. Obtain the entity multimodal feature vector E mutil , that is, the structural feature vector E struct , relation feature vector E rel , attribute description feature vector E att , name feature vector E name , name character feature vector E char , name synonym feature vector E syn , context text feature vector E ct x and the text title feature vector E mutil Normalize them separately. For a given eigenvector E mutil , the normalization formula is: Where M represents the total number of eigenvectors, represents the i-th eigenvector.

4. The method for updating unregistered words in a knowledge graph taking into account precise semantic relationships according to claim 1, characterized in that: The step S3 specifically includes the following steps: S3-1. Given the target knowledge graph KG t and the knowledge graph KG to be matched m Normalized entity multimodal feature vector Use independent linear transformations to map the features of each modality to the same feature space, and fill each modal feature to make its dimension consistent. This process can be expressed as: Among them, W linear represents the linear transformation matrix, represents the zero-filling process, d max_size represents the maximum dimension of the first dimension of all feature vectors, d i represents the first dimension of the i-th eigenvector; S3-2. After completing the linear transformation and filling of the features, the BERT self-attention mechanism and the cross-modal attention module are used to dynamically allocate and deeply integrate the multimodal feature weights. Suppose the feature vector after linear transformation and filling is Its dimensions are d, W Q , W K and W V is the weight matrix corresponding to the feature vector after being mapped into query vector, key vector and value vector, W S is the weight matrix of the modality-specific weight adjustment factors, d k is the dimension of the key vector, d v is the dimension of the value vector, sigmoid(·) is used to limit the output of the weight adjustment factor between 0 and 1, M is the total number of feature vectors, and ε is a constant used to prevent division by zero. The specific operation is as follows: For the i-th feature vector, first calculate its attention score with other feature vectors, that is, the cross-modal attention score of the i-th feature vector and the j-th feature vector: Normalize the attention scores to get the attention probability distribution: The value vector is weighted and summed according to the attention probability to obtain the weighted feature vector: Furthermore, the self-attention mechanism is used to focus on the relationship between features at different positions. Let the self-attention matrix be M Attn The weighted feature vector is transformed nonlinearly to obtain deep fusion features: HAVE BEEN fused =f nonlinear (M Attn ·HAVE BEEN weighted ) =σ(W nonlinear ·(M Attn ·E weighted )+b nonlinear ) Among them, f nonlinear (·) is the nonlinear transformation function σ, W nonlinear and b nonlinear is the weight matrix and bias vector of the nonlinear transformation layer; S3-3. After obtaining the high-level fusion feature, let this feature be E fused , whose dimension is d fused , first, through the linear layer (weight matrix is ​​set to W gate , the bias vector is set to b gate ) and the Sigmoid activation function transform the mean of the modal embedding to obtain the gating weight of each modality: Next, the embedding of each modality is multiplied by the corresponding gating weight to obtain the gated modality embedding: E gated =G weights ⊙E fused Among them, ⊙ represents the multiplication of corresponding elements; Finally, the weight normalization factor is calculated The gated modal embedding is multiplied by the weight normalization factor and normalized to obtain the final joint feature representation: AND joint =α⊙E gated 。 5. The method for updating unregistered words in a knowledge graph taking into account precise semantic relationships according to claim 1, characterized in that: The step S4 specifically includes the following steps: S4-1. For the target knowledge graph KG t entity t and the knowledge graph KG to be matched m entity m , and their joint eigenvectors are expressed as and Let d be the joint eigenvector and The dimension of and is the value of the vector in the i-th dimension, then and The Manhattan distance between is calculated as follows: S4-2. Evaluate the target knowledge graph KG by calculating the Manhattan distance D t entity t and the knowledge graph KG to be matched m entity m The similarity between entities t ,entity m ) If D is less than or equal to the preset threshold θ, the two entities are considered similar in the feature space and can be considered as matching, that is: S4-3. Through the matching results, the target knowledge graph KG can be further analyzed t and the knowledge graph KG to be matched m The upper and lower relations of the entities in , let the matching result be In order to determine the hierarchical relationship, the entity pairs in R are sorted according to the Manhattan distance D. For the target entity ti , select the first three matching entities with the smallest Manhattan distance among all matching pairs {entity mi,1 ,entity mi,2 ,entity mi,3 }, based on their proximity in feature space, it is inferred that they are entities ti The potential subordinate entity of a target entity may be a specific type, subclass or specific instance of the target entity in terms of semantics. Furthermore, a hyperlink credibility index can be defined to measure the entity pair. ti ,entitym i,j The possibility of the hypernymy of )(j=1,2,3) is calculated as follows: The larger the value of this indicator, the mi,j As entity ti The higher the probability of subordinate entities, the more accurately the semantic hierarchical relationship between entities can be portrayed.

Images