Task-Specific Text Segment Selection for Multi-Modal Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models face challenges in effectively processing multi-modal inputs that combine natural language and structured data due to the complexity of interpreting natural language, leading to increased resource utilization and minimal predictive performance improvements.
Innovation Solution
A domain-specific language model is trained using a balanced training dataset to encode semantic information from text sequences, which are then combined with structured data to enhance predictive tasks by generating semantically dense embeddings and identifying task-specific text segments for improved model training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multi-modal inputs combining natural language and structured data are used, then predictive performance may improve, but processing complexity and resource utilization increase
Solution Approach 1:
The patent extracts and selects only the most relevant text segments from large text corpora using a domain-specific language model, rather than processing all text data. This extraction process filters out redundant information and identifies only the text segments that complement structured data, reducing processing complexity while maintaining predictive performance.
Solution Approach 2:
The patent applies local quality by creating semantically dense embeddings for specific text segments rather than processing entire documents uniformly. The domain-specific language model generates embeddings tailored to the specific predictive task, allowing the system to handle different text segments with appropriate semantic density based on their relevance to the prediction target.
2Loss of information
If natural language extraction is performed from large text corpora, then more information may be available, but redundant information increases and processing resources are consumed
Solution Approach 1:
The system extracts only the most informative text segments from large corpora by comparing embeddings against prompts designed to extract complementary information. This selective extraction avoids processing redundant text while maintaining access to valuable predictive information.
Solution Approach 2:
The patent changes the parameter of text representation by transforming raw text into semantically dense embeddings. This transformation consolidates information density, allowing the system to represent large amounts of text data in a more compact and efficient format that reduces processing resource requirements.
3Measurement precision
If enterprise-level language models are used, then language understanding improves, but model training complexity and data requirements increase
Solution Approach 1:
The patent trains a domain-specific language model tailored to the particular predictive task and domain, rather than using a generic enterprise-level model. This domain-specific approach concentrates computational resources on learning the specific language patterns and relationships relevant to the target prediction, reducing overall training complexity while achieving high language understanding for the specific domain.
Solution Approach 2:
The system segments the language modeling process into two stages: pre-training a domain-specific language model on balanced training data, and then fine-tuning it on task-specific prompts. This segmentation allows the model to learn general domain patterns efficiently before being adapted to specific predictive tasks, reducing the computational burden of training from scratch for each task.
Data Source
AI summary
Various embodiments of the present disclosure provide a text interpretation technique. The text interpretation technique includes generating a plurality of text segment embeddings from a plurality of input text documents and identifying prompt embeddings associated with a predictive task. The technique includes generating task-specific similarity scores for the plurality of text segments based on a comparison between the plurality of text segment embeddings and the prompt embeddings, The technique includes identifying a set of task-specific text segments from the plurality of text segments based on the plurality of task-specific similarity scores and training a target machine learning model based on the set of task-specific text segments.