Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Entity alignment method and device suitable for multi-modal knowledge graph

A knowledge graph, multi-modal technology, applied in the field of knowledge graph in natural language processing, can solve the problems of insufficient extraction of VGG model and limited alignment effectiveness.

Active Publication Date: 2021-01-29
NAT UNIV OF DEFENSE TECH
View PDF7 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the VGG model cannot sufficiently extract useful features from images, thus limiting the effectiveness of the alignment

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Entity alignment method and device suitable for multi-modal knowledge graph
  • Entity alignment method and device suitable for multi-modal knowledge graph
  • Entity alignment method and device suitable for multi-modal knowledge graph

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0044] Such as figure 1 As shown, an entity alignment method suitable for multi-modal knowledge graphs includes the following steps:

[0045] Step 1, get two multimodal knowledge graphs and The data;

[0046] Step 2, project the data of each modality to the hyperbolic space;

[0047] Step 3, use the hyperbolic graph convolutional neural network to learn the structural features and visual features of the entity;

[0048] Step 4, fusing multimodal features;

[0049] Step 5, express entity similarity with distance in hyperbolic space;

[0050] Step 6, perform entity recognition alignment according to the similarity.

[0051] The entity alignment method of this embodiment, that is, the method operating in a hyperbolic space, will be described in detail below.

[0052] First, a hyperbolic graph convolutional neural network is adopted to learn the structural information of entities.

[0053] Then, the densenet model is used to convert the images associated with the entitie...

Embodiment 2

[0086] An entity alignment device suitable for multimodal knowledge graphs, including:

[0087] processor;

[0088] And, a memory for storing executable instructions of the processor;

[0089] Wherein, the processor is configured to execute the above entity alignment method by executing the executable instruction in the first embodiment.

[0090] Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an entity alignment method and device suitable for a multi-modal knowledge graph. The method comprises the following steps of: acquiring data of two multi-modal knowledge graphs; projecting the data of each mode to a hyperbolic space; learning structural features and visual features of entities by using a hyperbolic graph convolutional neural network; fusing the multi-modalfeatures; representing the entity similarity by using the distance in the hyperbolic space; and performing entity identification alignment according to the similarity. According to the method, Euclidean representation is expanded to a hyperboloid manifold, and structural representation of the entities is learned by adopting the hyperbolic graph convolutional network; for visual information, imageinserts are generated by using a densenet model, and the image inserts are inserted into the hyperbolic curve space by using the hyperbolic graph convolutional network; finally, structure inserts andthe image inserts are combined in the hyperbolic space to predict a potential alignment mode. and therefore, the method is particularly suitable for entity alignment and fusion of the multi-modal knowledge graph.

Description

technical field [0001] The present invention relates to the technical field of knowledge graphs in natural language processing, in particular to an entity alignment method and device suitable for multimodal knowledge graphs. Background technique [0002] In recent years, Knowledge Graph (KG) has become a popular data structure for representing factual knowledge in the form of RDF (Resource Description Framework, Resource Description Framework) triples, which can facilitate a series of downstream practical applications, such as question answering, information extraction Wait. Currently, there are a large number of common KGs (eg, DBpedia, YAGO, Google's Knowledge Vault) and domain-specific KGs (eg, pharmaceutical and molecular KGs). Meanwhile, the trend of incorporating multimedia information into KG is growing to support cross-modal tasks involving data interaction in multiple modalities, such as image and video retrieval, video summarization, visual entity disambiguation, ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/36G06F16/28G06K9/62G06N3/04G06N3/08
CPCG06F16/367G06F16/288G06N3/08G06N3/045G06F18/22G06F18/25
Inventor 赵翔唐九阳郭浩曾维新谭真徐浩张鑫
Owner NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com