Knowledge Graph Vector Representation via Multi-Modal Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional algorithms for knowledge graphs generate vector representations of entities and relations solely based on topological structure, resulting in inaccurate and incomplete representations.
Innovation Solution
A method and device that acquire text and image information related to entities from a database, generate structural, text, and image information vectors, and optimize a joint loss function to produce more accurate and comprehensive vector representations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If vector representations are generated solely from topological structure, then the processing method is simple, but the representation accuracy and comprehensiveness are insufficient
Solution Approach 1:
The patent merges multiple information sources (topological structure, text information, image information) into a unified vector representation framework. The loss function combines structural loss, text loss, and image loss components to jointly optimize the embedding vectors, thereby improving representation accuracy while integrating diverse data types.
Solution Approach 2:
The patent creates composite information representations by combining structural vectors, text vectors, and image vectors into a comprehensive entity representation. This composite approach integrates heterogeneous information sources to produce more accurate and comprehensive vector representations that capture both structural and semantic characteristics.
2Loss of information
If only topological structure is used for vector generation, then the processing complexity is low, but the information completeness is insufficient
Solution Approach 1:
The patent extends the traditional topological structure dimension by adding text and image dimensions. This multi-dimensional approach allows the system to capture entity information from multiple perspectives (structural relationships, semantic content, visual characteristics), thereby reducing information loss while managing complexity through structured processing.
Solution Approach 2:
The patent creates a universal processing framework that handles multiple information types (structure, text, images) through a unified loss function and optimization process. This multi-functional system can process diverse data types using the same architectural components, improving information completeness without proportionally increasing system complexity.
Data Source
AI summary
Knowledge graph processing method and device are disclosed. The method includes steps of obtaining an entity set containing a first entity, a second entity, and relation information; acquiring text information and image information related to the first entity and the second entity; generating a first structural information vector of the first entity and a second structural information vector of the second entity, and creating a first text information vector of the first entity, a first image information vector of the first entity, a second text information vector of the second entity, and a second image information vector of the second entity; and building a joint loss function so as to attain a first target vector of the first entity, a second target vector of the second entity, and a target relation vector of the relation information between the first entity and the second entity.