Entity alignment method and device suitable for multi-modal knowledge graph
What is Al technical title?
Al technical title is built by PatSnap Al team. It summarizes the technical point description of the patent document.
A knowledge graph, multi-modal technology, applied in the field of knowledge graph in natural language processing, can solve the problems of insufficient extraction of VGG model and limited alignment effectiveness.
Active Publication Date: 2021-01-29
NAT UNIV OF DEFENSE TECH
View PDF7 Cites 15 Cited by
Summary
Abstract
Description
Claims
Application Information
AI Technical Summary
This helps you quickly interpret patents by identifying the three key elements:
Problems solved by technology
Method used
Benefits of technology
Problems solved by technology
However, the VGG model cannot sufficiently extract useful features from images, thus limiting the effectiveness of the alignment
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more
Image
Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
Click on the blue label to locate the original text in one second.
Reading with bidirectional positioning of images and text.
Smart Image
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0044] Such as figure 1 As shown, an entity alignment method suitable for multi-modal knowledge graphs includes the following steps:
[0045] Step 1, get two multimodal knowledge graphs and The data;
[0046] Step 2, project the data of each modality to the hyperbolic space;
[0047] Step 3, use the hyperbolic graph convolutional neural network to learn the structural features and visual features of the entity;
[0048] Step 4, fusing multimodal features;
[0049] Step 5, express entity similarity with distance in hyperbolic space;
[0050] Step 6, perform entity recognition alignment according to the similarity.
[0051] The entity alignment method of this embodiment, that is, the method operating in a hyperbolic space, will be described in detail below.
[0052] First, a hyperbolic graph convolutional neural network is adopted to learn the structural information of entities.
[0053] Then, the densenet model is used to convert the images associated with the entitie...
Embodiment 2
[0086] An entity alignment device suitable for multimodal knowledge graphs, including:
[0087] processor;
[0088] And, a memory for storing executable instructions of the processor;
[0089] Wherein, the processor is configured to execute the above entity alignment method by executing the executable instruction in the first embodiment.
[0090] Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
PUM
Login to view more
Abstract
The invention discloses an entity alignment method and device suitable for a multi-modal knowledge graph. The method comprises the following steps of: acquiring data of two multi-modal knowledge graphs; projecting the data of each mode to a hyperbolic space; learning structural features and visual features of entities by using a hyperbolic graph convolutional neural network; fusing the multi-modalfeatures; representing the entity similarity by using the distance in the hyperbolic space; and performing entity identification alignment according to the similarity. According to the method, Euclidean representation is expanded to a hyperboloid manifold, and structural representation of the entities is learned by adopting the hyperbolic graph convolutional network; for visual information, imageinserts are generated by using a densenet model, and the image inserts are inserted into the hyperbolic curve space by using the hyperbolic graph convolutional network; finally, structure inserts andthe image inserts are combined in the hyperbolic space to predict a potential alignment mode. and therefore, the method is particularly suitable for entity alignment and fusion of the multi-modal knowledge graph.
Description
technical field [0001] The present invention relates to the technical field of knowledge graphs in natural language processing, in particular to an entity alignment method and device suitable for multimodal knowledge graphs. Background technique [0002] In recent years, Knowledge Graph (KG) has become a popular data structure for representing factual knowledge in the form of RDF (Resource Description Framework, Resource Description Framework) triples, which can facilitate a series of downstream practical applications, such as question answering, information extraction Wait. Currently, there are a large number of common KGs (eg, DBpedia, YAGO, Google's Knowledge Vault) and domain-specific KGs (eg, pharmaceutical and molecular KGs). Meanwhile, the trend of incorporating multimedia information into KG is growing to support cross-modal tasks involving data interaction in multiple modalities, such as image and video retrieval, video summarization, visual entity disambiguation, ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
Application Information
Patent Timeline
Application Date:The date an application was filed.
Publication Date:The date a patent or application was officially published.
First Publication Date:The earliest publication date of a patent with the same application number.
Issue Date:Publication date of the patent grant document.
PCT Entry Date:The Entry date of PCT National Phase.
Estimated Expiry Date:The statutory expiry date of a patent right according to the Patent Law, and it is the longest term of protection that the patent right can achieve without the termination of the patent right due to other reasons(Term extension factor has been taken into account ).
Invalid Date:Actual expiry date is based on effective date or publication date of legal transaction data of invalid patent.
Login to view more 
Login to view more