Multimodal knowledge representation method fusing entity image information and entity category information

Abstract

The invention discloses a multi-modal knowledge representation method fusing entity image information and entity category information. A model carries out united joint training on structure-based knowledge representation, image-based knowledge representation and category-based knowledge representation of entities to complete multi-modal knowledge graph representation. The method includes an entity image information embedding method which is responsible for extraction of entity image feature information and conversion from an image space to a knowledge space; according to the entity category information embedding method, modeling is carried out on a semantic relation between an entity category and a corresponding triple relation, and representation of an entity based on the category is constructed; and the multi-modal knowledge representation model fusing the entity image information and the entity category information is responsible for multi-modal knowledge representation learning fusing the entity image information and the entity category information. By constructing the multi-modal knowledge representation method, a new thought is provided for solving the problem of data sparseness in a knowledge graph reasoning technology and efficiently calculating semantic relation, fusion and reasoning performance of entities and relations.

Description

technical field [0001] The invention belongs to knowledge map reasoning technology, and in particular relates to artificial intelligence related fields such as information retrieval, question answering system and intelligent dialogue. Background technique [0002] With the advent of the Fifth Generation (5G) era, Internet technology has developed rapidly, and the volume and dimension of data have grown explosively. In the face of massive data, users are increasingly demanding more precise and intelligent searches. In order to provide users with intelligent services that can understand user needs, this problem needs to be solved urgently. Knowledge graph emerged as an intuitive knowledge expression method for discovering, managing and utilizing knowledge. To understand the knowledge map, we must first explain the knowledge base. The knowledge base (knowledge base, KB) is a knowledge system formed by structuring human knowledge, which contains basic facts, general rules and ...

AI Technical Summary

Problems solved by technology

Network-based knowledge representation has problems such as low computational efficiency and data sparseness under large-scale knowledge graphs.

At present, most of the open knowledge graphs are mainly constructed manually or semi-automatically, which leads to the serious lack of knowledge of these graphs. At the same time, due to the storage method of the graph structure of the knowledge graph itself, the calculation efficiency is low.

Images