Text Concept Linking via Differential Language Model Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional information retrieval technologies based on keyword indexing are inefficient, especially with rich ontologies, and semantic analysis techniques like LSA do not effectively utilize large volumes of crowd-sourced data.
Innovation Solution
A method for automatically linking text to concepts in a knowledge base by selecting relevant data sources, calculating probabilities, and creating links based on differential analysis of language models, enabling efficient conceptual searching and indexing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If query expansion techniques are applied to improve search quality, then the potential quality of returned results increases, but the computational cost and time required for searching increases significantly
Solution Approach 1:
The patent pre-computes and stores language models for multiple data sources before search queries are executed. These pre-built models are saved for rapid retrieval during actual searches, eliminating the need to compute them in real-time. This preliminary action allows the system to perform expensive probabilistic calculations ahead of time, making the actual search process much faster while maintaining high search quality through comprehensive query expansion.
2Adaptability or versatility
If rich ontologies are used to improve conceptual searching, then the ability to perform semantic analysis improves, but the number of required searches increases from one to potentially tens of thousands
Solution Approach 1:
The patent introduces language models as intermediary representations between the ontology and search queries. Instead of directly navigating the complex ontology structure during searches, the system uses pre-computed language models that capture semantic relationships. These models act as mediators, translating ontology knowledge into probabilistic frameworks that can be efficiently queried without traversing the entire ontology, thus reducing search complexity while maintaining semantic analysis capabilities.
3Measurement precision
If multiple data sources are analyzed to improve concept linking accuracy, then the precision of text-to-concept linking improves, but the computational resources required increase
Solution Approach 1:
The system pre-computes language models for multiple data sources and stores them for later use. During actual concept linking tasks, it retrieves these pre-built models and performs differential analysis on the stored probabilities, rather than computing from scratch. This preliminary computation of language models allows the system to maintain high linking accuracy by analyzing multiple data sources while significantly reducing the computational resources needed during actual operations.
Data Source
AI summary
According to an aspect, automatically linking text to concepts in a knowledge base using differential analysis includes receiving a text string and selecting, based on contents of the text string, a plurality of data sources that correspond to concepts in the knowledge base. In a further aspect, automatically linking the text to the concepts includes calculating, for each of the selected data sources, a probability that the text string is output by a language model built using the selected data source, calculating a probability that the text string is output by a generic language model, calculating link confidence scores for each concept based on a differential analysis of the probabilities, and creating a link from the text string to one of the concepts in the knowledge base. The creating is based on a link confidence score of the concept being more than a threshold value away from a prescribed threshold.