Knowledge reasoning method based on multi-modal knowledge graph

Abstract

The invention discloses a knowledge reasoning method based on a multi-modal knowledge graph, and aims to enable knowledge reasoning reliability and accuracy to be higher and enable the knowledge reasoning method to have stronger modeling and reasoning capabilities. The method is realized through the following technical scheme: different information is fused based on multi-hop reasoning of a large-scale knowledge base; attribute completion is performed on the attribute missing graph through attribute graph embedding, structured information is extracted from unstructured and semi-structured documents or sentences, and a dynamic heterogeneous graph embedding model is constructed for multi-type characteristics of the multi-modal knowledge graph through heterogeneous graph embedding; feature learning of semi-structured knowledge, structured knowledge and different types of non-structured knowledge is achieved, and multi-modal knowledge graph features are obtained and serve as input for knowledge reasoning based on a graph neural network GNN; an inference path is generated, and a plurality of types of inference paths are constructed; and classification, edge prediction and frequent subgraphs of node types are calculated on the graph, a knowledge reasoning task is generated, and multi-step complex knowledge reasoning is completed.

Description

technical field [0001] The invention relates to a knowledge reasoning method in the technical field of knowledge engineering, in particular to a knowledge reasoning method based on a multimodal knowledge map. Background technique [0002] Artificial intelligence is moving from perceptual intelligence to cognitive intelligence. At present, artificial intelligence is still in the state of weak artificial intelligence. To make it form a brain, have the ability to understand and reason, the core is to have "knowledge"; in terms of learning knowledge, machines are mainly divided into end-to-end deep learning and There are two categories of structured representation and learning. The former mainly focuses on active learning. What people learn is the underlying feature space of things, while what humans can understand is the semantic space of things. Knowledge graphs can bridge the gap between the two and transform human thinking. It provides a possible way for machine path thinki...

AI Technical Summary

Problems solved by technology

However, there are still relatively few researches on knowledge map learning, calculation and application based on graph neural network, and there is still huge room for development in the future.

[0004]However, the current research on knowledge is more focused on the construction of knowledge graphs, and the research on related reasoning and mining prediction based on knowledge graphs has made slow progress

The main problems are as follows: there are many types of knowledge, difficult to express, and it is impossible to conduct unified cognition and reasoning analysis from a multi-dimensional perspective; Knowledge Reasoning is to further mine hidden knowledge on the basis of the existing knowledge base, so as to enrich and expand knowledge base

Due to the complexity of the graph data structure (such as power law degree distribution, etc.), in the process of migrating the convolutional neural network in the image domain to the graph data, it is impossible to directly define the convolution operator in the node domain

Images