Multi-modal entity alignment method based on triple screening fusion

Abstract

The invention discloses a multi-modal entity alignment method based on triple screening fusion. The method comprises the following steps of: acquiring data of two multi-modal knowledge maps; quantifying the importance of triples by using an unsupervised triple screening module, and filtering part of invalid triples based on importance scores; respectively learning structure vectors of entities of the two multi-modal knowledge maps by using a graph convolutional neural network, and generating a structure feature representation of each entity; respectively generating visual feature representations of respective entities; and performing entity alignment by combining entity structure features and entity visual features of the two multi-modal knowledge maps. According to the multi-modal entity alignment method, for the problem of poor visual information utilization, entity-picture similarity scores are calculated, and more accurate entity visual feature representations are obtained based on the similarity; and triple scores are generated based on a relationship PageRank score and entity degrees, the triples are filtered, and the structural difference of different knowledge maps are alleviated, so that the alignment effect is better.

Description

technical field [0001] The invention relates to the technical field of knowledge graphs in natural language processing, in particular to a multimodal entity alignment method based on triple group screening and fusion. Background technique [0002] In recent years, knowledge graphs have become a widely used representation of structured data. It represents real-world knowledge or events in the form of triples, and is widely used in downstream tasks of various artificial intelligence. At present, multimodal knowledge graphs are often constructed from limited data sources, and there are problems of missing information and low coverage, which makes the knowledge utilization rate low. Considering the high cost and low efficiency of manual completion of knowledge graphs, in order to improve the coverage of knowledge graphs, a feasible method is to automatically integrate useful knowledge from other knowledge graphs. As a hub linking different knowledge graphs, entities are crucia...

AI Technical Summary

Problems solved by technology

However, in the real world, due to different construction methods, different knowledge graphs may have large structural differences.

For such problems, triplets can be generated based on link prediction to enrich structural information. Although the problem of structural diversity is alleviated to a certain extent, the reliability of the generated triplets needs to be considered, and the number of triplets varies greatly. Double the situation is very difficult to complete

Second, poor use of visual information

The current method cannot distinguish the noise pictures in the entity-related pictures, so that the visual information of the entity is mixed with some noise, which in turn reduces the accuracy of visual information for entity alignment

Images