Zero-Shot Classification Using Dynamic Text Prompt Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing zero-shot classifiers simply average encoded text prompts, which may not effectively capture the similarity between data and class queries, leading to suboptimal classification performance.
Innovation Solution
The method employs a weighted average of encoded text prompts, where weights are automatically determined based on the similarity between the data embedding and the embeddings of the text prompts, enhancing the classification process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a simple average of encoded text prompts is used, then the computation is fast and simple, but the classification performance is suboptimal because it does not capture similarity between data and class queries
Solution Approach 1:
The patent transforms the uniform weighting parameter into dynamic similarity-based weights. Instead of using equal weights for all text prompts, the system computes cosine similarity between the encoded sensor data and each text prompt embedding, then uses these similarity scores as weights in the averaging process. This parameter change allows the system to adaptively emphasize more relevant prompts while maintaining computational efficiency through the use of standard embedding operations.
2Measurement precision
If weighted averaging with similarity-based weights is used, then the classification accuracy improves, but the inference time increases
Solution Approach 1:
The patent implements a balanced approach by computing similarity weights for all text prompts but using them in a computationally efficient manner. Rather than performing complex iterative optimization or multiple passes, the system computes a single round of cosine similarities and applies them directly to the weighted averaging. This partial action approach achieves most of the performance benefit without the full computational cost of more elaborate weighting schemes.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method of classifying a sensor signal (30) by a sensor signal encoder and a text encoder, wherein said encoders are configured to encoder their inputs into a latent representation, comprising the steps of: Encoding the sensor signal (30) to a first latent representation (e(x)) by the sensor signal encoder. Generating a plurality of text prompts (tij), wherein for each class (cj) several text prompts characterizing the corresponding class are instantiated (ti ← cj). Encoding the generated text prompts (tij) into second latent representations ( eijt) by the text encoder. Determining a class query (qj) for each class by weighted averaging over the second representations ( eijt) corresponding to the same class (ci). Computing a similarity between the first latent representation (e(x)) and each of the class queries (qj). Assigning the sensor signal (30) to the class with the highest similarity.