Domain-Specific Thesaurus Generation via Ensemble ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The generation of high-quality domain-specific thesauri is laborious and expensive, requiring human expertise and not scalable, as existing open domain thesauri like WordNet lack domain-specific terms and synonyms.
Innovation Solution
A computer-implemented method generates a domain-specific thesaurus by receiving data from a domain-specific corpus, splitting it into sentences, retrieving candidate sentences with synonyms, forming a frequency map, and sorting them based on occurrence frequency, using ensemble machine learning techniques and distributional semantic systems like Word2Vec, DiSSect, and GLoVE, and dependency parsing to automatically create a high-quality thesaurus without human involvement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If human expert annotators are used to create domain-specific thesauri, then the quality and accuracy of the thesaurus is improved, but the cost and time required increase significantly
Solution Approach 1:
The system creates synthetic training data by copying and adapting existing thesaurus structures and linguistic patterns, then uses machine learning models to generate domain-specific thesauri without requiring manual annotation of every term. The models learn from existing linguistic resources and apply that knowledge automatically.
Solution Approach 2:
The patent replaces the mechanical process of manual human annotation with automated machine learning systems. Ensemble models including Word2Vec, DiSSect, GLoVE, and Glimpse work together to automatically identify synonyms and relationships, substituting human cognitive labor with computational processes.
2Measurement precision
If manual annotation methods are used to generate thesauri, then the quality of the thesaurus is maintained, but the scalability to different domains is severely limited
Solution Approach 1:
The system develops a universal framework that can generate thesauri across multiple domains using the same ensemble of machine learning models. The approach is domain-agnostic, allowing the same technical infrastructure to produce high-quality thesauri for medicine, law, technology, and other fields by simply changing the input corpus.
Solution Approach 2:
The system enables automatic self-service thesaurus generation where the machine learning models independently process domain corpora and produce thesauri without human intervention. The automated pipeline handles everything from text processing to synonym identification and thesaurus construction.
3Adaptability or versatility
If existing open domain thesauri like WordNet are used, then the coverage of general terms is good, but domain-specific terms and synonyms are missing
Solution Approach 1:
The system applies local quality by tailoring the thesaurus to specific domain requirements. Instead of using a one-size-fits-all approach, the ensemble models analyze domain-specific corpora to identify and prioritize synonyms and relationships that are relevant to particular fields, ensuring the thesaurus reflects local domain characteristics.
Solution Approach 2:
The patent changes the parameters of thesaurus generation by adjusting the training data, model weights, and processing parameters based on the target domain. The system modifies its behavior to capture domain-specific linguistic patterns, frequency distributions, and semantic relationships that differ from general language.
Data Source
AI summary
Embodiments provide a computer implemented method for generating a domain-specific thesaurus on a cognitive system, comprising: receiving data of the domain-specific corpus and a plurality of terms of interest from a user; splitting the data of the domain-specific corpus into a plurality of sentences using natural language processing techniques; for each term in the plurality of terms of interest, retrieving a plurality of candidate sentences containing a corresponding term, from the plurality of sentences; for each candidate sentence, providing a list of synonyms of the corresponding term, wherein the synonyms are contextual alternatives in the corresponding candidate sentence; for each term in the plurality of terms of interest, tracking a frequency of each synonym, and forming a frequency map including all the synonyms of a corresponding term and the frequency of each synonym; and generating a domain-specific thesaurus based on a combination of all the synonyms in the frequency map.