Summary Model for Text Embedding Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text embedding models face limitations in processing long text strings and are prone to misclassifications due to noise, as they have input length limits and are influenced by irrelevant information.
Innovation Solution
A summary model is used to generate concise summaries of text, which are then processed by embedding models, allowing for the determination of embeddings that exceed input length limits and improving precision by denoising the text.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If text strings with considerable noise are processed directly by embedding models, then the embedding models can process the text, but the embeddings become imprecise leading to misclassifications
Solution Approach 1:
The patent applies preliminary action by introducing a summarization step before the embedding model processes the text. The summarization model pre-processes the noisy text to extract relevant information and remove noise, creating a cleaned summary that is then fed to the embedding model. This preliminary cleaning action prevents noise from degrading embedding precision.
Solution Approach 2:
The patent uses an intermediary approach by inserting a summarization model between the noisy text input and the embedding model. This intermediary component acts as a mediator that transforms the noisy text into a refined summary, which then serves as input to the embedding model, protecting it from noise while preserving essential information.
2Length of stationary object
If long text strings exceeding input length limits are fed directly to embedding models, then the complete text is processed, but the embedding models cannot handle inputs beyond their token limits
Solution Approach 1:
The patent applies preliminary action by using a summarization model to condense long text strings into shorter summaries before they are fed to the embedding model. This pre-compression action reduces the token count to within acceptable limits while preserving the essential meaning and information needed for accurate embedding generation.
Solution Approach 2:
The patent changes the parameter of text length by transforming the original long text into a compressed summary form. The summarization model alters the length parameter while maintaining the semantic content, enabling the embedding model to process texts of varying original lengths within its fixed token capacity.
3Measurement precision
If summarization is applied to reduce text length and remove noise, then embedding precision improves, but additional processing steps are required
Solution Approach 1:
The patent applies universality by designing a summarization model that performs multiple functions simultaneously: it reduces text length to fit token limits and removes noise to improve precision. This multi-functional approach consolidates what could be separate operations into a single processing step, minimizing the increase in overall system complexity.
Data Source
AI summary
Aspects of the present disclosure provide techniques for improved text classification. Embodiments include providing, based on a text string, one or more first inputs to a summary model. Embodiments include determining, based on one or more first outputs from the summary model in response to the one or more first inputs, a summarized version of the text string. In some embodiments the summarized version of the text string comprises a number of tokens that is less than or equal to a maximum number of input tokens for a machine learning model. Embodiments include providing, based on the summarized version of the text string, one or more second inputs to the machine learning model. Embodiments include determining one or more attributes of the text string based on one or more second outputs received from the machine learning model in response to the one or more second inputs.