Knowledge base completion method based on multi-modal representation learning

Abstract

The invention relates to a knowledge base completion method based on multi-modal representation learning, and the method comprises the steps of giving a knowledge base KB which comprises two parts: aknown knowledge set and an unknown knowledge set; performing data preprocessing on the data in the knowledge base; proposing a knowledge base completion model ConvAt, and firstly generating multi-modal representation of a head entity and a tail entity for the acquired data; splicing the multi-modal representation of the head entity, the structural feature vector of the relationship and the multi-modal representation of the tail entity according to columns, respectively processing through a convolutional neural network module, a channel attention module and a spatial attention module, and finally multiplying by a weight matrix to obtain a score of a triple (h, r, t); and training the completion model in the step S2 by using a loss function, and performing knowledge base completion by usingthe trained model. According to the algorithm provided by the invention, external information can be fused, and richer semantic information can be utilized.

Description

technical field [0001] The invention relates to the field of knowledge base completion, in particular to a knowledge base completion method based on multimodal representation learning. Background technique [0002] In recent years, a variety of knowledge base completion methods have emerged, among which the method based on knowledge representation learning is currently an active research field for knowledge base completion. A key problem in representation learning is learning low-dimensional distributed embeddings of entities and relations. [0003] At present, there are mainly two types of information that can be used for knowledge representation learning. The first is triples that already exist in knowledge graphs. It mainly includes: knowledge graph representation learning methods based on translation / translation, such as TransE; methods based on tensor / matrix decomposition, such as the RESCAL model; representation learning models based on neural networks, such as ConvE....

AI Technical Summary

Problems solved by technology

Most of the existing methods for integrating external information only consider a single modal information, most of which are text modals, and fail to use the complementary characteristics between multiple modalities to learn more comprehensive features.

[0004] At present, most knowledge graph representation learning only considers the structural knowledge between entities and relationships. The performance of this type of model is limited by the display and storage of knowledge, resulting in unstable knowledge base completion. Possess knowledge of multiple modes, such as text, picture, audio and video, etc.

These external knowledge of different modalities can enrich and expand the existing knowledge base to a certain extent, and then provide richer semantic information for downstream tasks such as question answering and link prediction; existing representation learning methods that incorporate external information , most of them only consider a single modality information, and fail to use the complementary characteristics between multimodalities to learn more comprehensive features

Images