Knowledge Base Population via Multi-Granularity Embedding Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models for populating knowledge bases lose information when determining facts based on character string embeddings, as they rely on word division methods that do not account for finer granularities.
Innovation Solution
The method involves determining two sets of embeddings for a character string using different vocabularies, with meta-embeddings created by combining corresponding embeddings from these sets, allowing for a more detailed representation by accounting for embeddings without direct matches through independent assignment or averaging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If word division methods are used to determine parts of character strings for embedding, then the processing is simplified and computationally efficient, but information is lost due to coarse granularity
Solution Approach 1:
The patent applies segmentation by dividing character strings into multiple parts using different word division methods (e.g., character-level, subword-level, word-level segmentation). This creates multiple sets of embeddings from the same input string, each capturing different granularities of information. The multiple segmentations allow the system to preserve fine-grained details while maintaining processing efficiency through systematic division strategies.
Solution Approach 2:
The patent combines multiple embedding sets into a composite embedding structure. By integrating embeddings from different segmentation levels (character, subword, word), the system creates a composite representation that preserves information from all granularities. This composite approach allows the model to utilize both coarse and fine-grained features simultaneously, preventing information loss while maintaining computational feasibility.
2Loss of information
If multiple embeddings are created per part of character string, then information loss is reduced through finer granularity, but device complexity increases
Solution Approach 1:
The patent implements multi-functionality by designing embedding layers that serve multiple purposes simultaneously. The same embedding infrastructure processes different segmentation levels and vocabularies, with each embedding set serving both as standalone representation and as part of the integrated composite structure. This universal approach reduces redundancy and manages complexity by making the embedding system perform multiple functions through a unified architecture.
Solution Approach 2:
The patent applies nesting by organizing embeddings at different granularities in a hierarchical structure. Finer-grained embeddings (character-level) are nested within coarser-grained embeddings (word-level), with each level containing and building upon the previous level. This nested organization allows the model to access information at multiple granularities without duplicating the entire embedding process, thereby managing complexity while preserving information.
3Measurement precision
If embeddings from different vocabularies are integrated, then representation accuracy improves, but computational requirements increase
Solution Approach 1:
The patent applies preliminary action by pre-processing and organizing embeddings from different vocabularies before integration. Embeddings are generated and structured in advance using multiple segmentation strategies and vocabularies, then systematically combined. This preliminary organization reduces the computational burden during the actual integration phase, as the data is already prepared and aligned for efficient processing.
Solution Approach 2:
The patent combines multiple embedding sets from different vocabularies and segmentation levels into a unified composite embedding. By merging these representations through systematic integration (e.g., concatenation, averaging, or attention-based fusion), the model achieves enhanced representation accuracy. The merging process is optimized to reduce computational overhead by leveraging the structured nature of the multiple embedding sets.
Data Source
AI summary
A device and a computer-implemented method for machine learning a fact in particular for populating a knowledge base. A character string is provided. A first set of embeddings of parts of the character string is determined. A second set of embeddings of parts of the character string is determined. For mutually corresponding embeddings from the sets, one of the variables for predicting the fact is determined in each case. The fact is determined, in particular in the knowledge base, depending on the variables.