Multimodal Embedding Modifier for Search Intent Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search systems limit users' ability to receive search results closely tailored to their intent, as they rely on predefined filters or new search queries, leading to inefficient processing and potential mischaracterization of items, and do not effectively utilize multimodal data like images and text for accurate item identification.
Innovation Solution
A multimodal machine learning model that modifies seed search embeddings using positive and negative modifiers, allowing users to pivot searches based on various attributes, such as image and text features, to provide more relevant search results without the need for new queries or filters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If users rely on predefined filters or new search queries to modify search results, then search systems can provide modified results, but the user's ability to locate search results closely tailored to user intent is constrained and processing efficiency decreases
Solution Approach 1:
The system pre-generates embeddings for multiple seed search selections from the initial search results. These embeddings are prepared in advance and stored, so when a user applies a modifier, the system can quickly compute modified embeddings by combining pre-computed seed embeddings with modifier embeddings, avoiding the need to re-process entire search results or generate new queries.
Solution Approach 2:
The patent introduces embeddings as an intermediary representation between search queries/results and user intent modifiers. Instead of directly filtering or re-querying, the system uses vector space embeddings to represent search selections and modifiers, enabling smooth combination and transformation that preserves user intent while improving processing efficiency.
2Measurement precision
If search systems use predefined filters to modify results, then results can be narrowed down, but the system may mischaracterize items and fail to accurately reflect user intent
Solution Approach 1:
The system changes the parameter representation from categorical filter tags to continuous vector embeddings. By representing search selections and modifiers as embeddings in a vector space, the system can perform nuanced combinations that preserve item characteristics while accurately reflecting user intent, avoiding the loss of information inherent in rigid predefined filters.
3Measurement precision
If users provide detailed modifiers to refine search results, then search accuracy improves, but the complexity of the search process increases
Solution Approach 1:
The system automatically computes modified embeddings by combining seed search embeddings with modifier embeddings using vector operations. This self-service approach handles the complexity of processing detailed modifiers in the background, presenting users with a simple interface where they can apply modifiers without needing to understand the underlying computational complexity.
Data Source
AI summary
A multimodal embedding modifier generates a modified seed search selection embedding for providing a set of search results. The multimodal embedding modifier enhances the ability and accuracy of identifying a user's true intent when searching the online marketplace. For example, embodiments disclosed herein can allow a user to navigate multiple modalities for an item. In some embodiments, a user may select a search result corresponding to an initial search query, and further modify the selected search result by inputting a modifier (e.g., a textual modifier). The multimodal embedding modifier can be trained using a training dataset including a text embedding, an image embedding, another type of embedding, or a combination thereof.