Ontology Learning via Predictive Language Model Priming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional automatic extraction and formulation of ontologies from natural language text are error-prone and require hand-engineered methods, limiting their scope and expressivity.
Innovation Solution
The method involves using a pretrained predictive language model primed with ontological rules to generate natural language text, which is then processed to extract ontology rules, compared to a database, and evaluated for accuracy, enabling automated ontology learning without labeled training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If conventional automatic extraction methods are used to formulate ontologies from natural language text, then the process can be automated, but the extraction is error-prone and requires hand-engineered methods with limited scope and expressivity
Solution Approach 1:
The patent introduces an intermediary component - a language model - that mediates between natural language text and ontology extraction. The language model generates intermediate representations and facilitates the transformation of unstructured text into structured ontology rules, thereby improving both automation and reliability without requiring hand-engineered methods
Solution Approach 2:
The patent changes the parameters of the extraction process by using a language model with adjustable temperature, top-k, and top-p parameters. These parameter changes enable flexible control over the generation process, allowing the system to produce reliable ontology extractions while maintaining full automation
2Reliability
If hand-engineered extraction methods are used, then some level of accuracy can be achieved, but the scope and expressivity are significantly limited
Solution Approach 1:
The language model serves multiple functions: it performs text understanding, generates intermediate representations, extracts ontology rules, and validates the extracted content. This multi-functionality allows the system to achieve high accuracy while simultaneously expanding the scope and expressivity of the extracted ontologies beyond what hand-engineered methods can provide
3Adaptability or versatility
If predictive language models are used to generate natural language text from ontology rules, then expressive ontology generation is enabled, but the process requires guidance components to ensure appropriateness
Solution Approach 1:
The patent implements feedback mechanisms where the guidance component monitors the language model's generation process and provides real-time corrections and steering. The system compares generated text against the original ontology rules and adjusts the generation parameters accordingly, enabling expressive ontology generation while managing complexity through iterative refinement
Data Source
AI summary
An ontology topic is selected and a pretrained predictive language model is primed to create a predictive primed model based on one or more ontological rules corresponding to the selected ontology topic. Using the predictive primed model, natural language text is generated based on the ontology topic and guidance of a prediction steering component. The predictive primed model is guided in selecting text that is predicted to be appropriate for the ontology topic and the generated natural language text. The generated natural language text is processed to generate extracted ontology rules and the extracted ontology rules are compared to one or more rules of an ontology rule database that correspond to the ontology topic. A check is performed to determine if a performance of the ontology extractor is acceptable.