Knowledge Graph Generation Using Inverted Index Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing knowledge graph generation techniques are slow and inefficient due to the need for full natural language processing over every passage in a corpus, making them resource-intensive and time-consuming.
Innovation Solution
A computer-implemented method that receives a search query, identifies a subset of documents containing the query using an inverted index, processes these documents to identify entities and relations, and generates a knowledge graph based on the entities and relations, thereby reducing the scope of natural language processing and enabling real-time generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full natural language processing is performed over every passage in a corpus to extract knowledge, then comprehensive knowledge extraction is achieved, but processing time and resource consumption increase significantly
Solution Approach 1:
The patent segments the corpus into relevant and irrelevant portions based on the search query. An inverted index is used to identify documents containing the query terms, creating a segmented subset for full NLP processing while excluding other documents from intensive processing.
Solution Approach 2:
The patent extracts only the necessary documents from the full corpus using the inverted index, rather than processing the entire corpus. This extraction of relevant documents eliminates unnecessary processing of irrelevant passages while maintaining knowledge extraction completeness.
2Measurement precision
If full natural language processing is performed over every passage in a corpus to extract knowledge, then comprehensive knowledge extraction is achieved, but resource consumption increases significantly
Solution Approach 1:
The patent segments the corpus into relevant and irrelevant portions based on the search query. An inverted index is used to identify documents containing the query terms, creating a segmented subset for full NLP processing while excluding other documents from intensive processing.
Solution Approach 2:
The patent extracts only the necessary documents from the full corpus using the inverted index, rather than processing the entire corpus. This extraction of relevant documents eliminates unnecessary processing of irrelevant passages while maintaining knowledge extraction completeness.
3Measurement precision
If the entire corpus is processed to generate a knowledge graph, then comprehensive knowledge coverage is achieved, but real-time generation is not possible
Solution Approach 1:
The patent segments the corpus into relevant and irrelevant portions based on the search query. An inverted index is used to identify documents containing the query terms, creating a segmented subset for full NLP processing while excluding other documents from intensive processing.
Solution Approach 2:
The patent applies partial action by processing only the subset of documents relevant to the specific query rather than the entire corpus. This partial processing achieves sufficient knowledge graph coverage for the query context while enabling real-time generation performance.
Data Source
AI summary
A computer-implemented method that includes receiving a search query and, after receiving the search query, identifying a first subset of documents that include the search query by searching, based on the search query, an inverted index of a corpus of documents that includes the first subset of documents. The computer-implemented method additionally includes processing the first subset of documents to identify entities related to the search query and relations between the entities and the search query. The computer-implemented method additionally includes generating a knowledge graph based on the entities and the relations.