Text-Based Image Classification Using Relative Vector Distances
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional object and action recognition methods face low accuracy due to the inherent disconnect between textual and visual representations, and the high costs and time required for data gathering, annotation, and training of large language models (LLMs).
Innovation Solution
A relative vector distance process is used to classify images and text by comparing multiple classes of text to an image, selecting the text with the smallest distance relative to other classes, eliminating the need for new dataset collection and LLM fine-tuning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If cosine similarity is used to compare textual descriptions and visual data, then object and action recognition can be performed, but accuracy is low due to inherent differences between text and visual representations
Solution Approach 1:
The patent transforms the classification approach by changing the parameter being measured: instead of using absolute cosine similarity thresholds, it computes relative vector distances by comparing the image vector's distance to multiple text vectors and selecting the minimum. This parameter transformation resolves the contradiction by making the classification more robust to the inherent differences between text and visual representations.
2Measurement precision
If a dataset is acquired and annotated to fine tune a large language model, then classification accuracy can be improved, but costs and time for data gathering, annotation, and training become prohibitively high
Solution Approach 1:
The patent uses existing pre-trained text vectors from a large language model without creating new training data. Instead of copying and fine-tuning the entire model with new datasets, it copies the vector representation capability and applies it directly to image classification through relative distance computation, eliminating the time-consuming data gathering and annotation process while maintaining accuracy.
Solution Approach 2:
The patent leverages the preliminary work already done by pre-trained large language models that have learned robust text representations. By using these pre-computed text vectors and applying relative distance comparison, the system avoids the need for preliminary data gathering, annotation, and model training steps, directly achieving accurate classification.
3Measurement precision
If a dataset is acquired and annotated to fine tune a large language model, then classification accuracy can be improved, but financial costs become prohibitively high
Solution Approach 1:
The patent copies only the necessary vector representations from pre-trained models rather than investing resources in creating and training new models. This copying approach achieves high classification accuracy without the prohibitive financial costs associated with dataset acquisition, annotation, and extensive model training.
Solution Approach 2:
The patent uses inexpensive text vector representations from existing models as disposable intermediaries for classification. Instead of investing in expensive, long-term model training infrastructure, it leverages cheap, readily-available pre-computed vectors that can be used directly for classification tasks without further training investment.
Data Source
AI summary
Example implementations include a method, apparatus and computer-readable medium of object/action recognition using a text-based classification model, comprising generating an image vector configured to represent one or more features of the first image. Additionally, the implementations further include computing a vector distance between the image vector and each of a first text vector and a second text vector, wherein the first text vector is configured to represent a first text, and wherein the second text vector is configured to represent a second text. Additionally, the implementations further include classifying the first image according to the first text or the second text based on which computed vector distance indicates a highest similarity between the image vector and either the first text vector or the second text vector relative to the other of the first text vector or the second text vector.