Multi-Modal Search Using Virtual Vector Space Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search engines struggle to accurately provide search results that match user intent, especially when dealing with multi-modal inputs like images and text, as they often fail to linearly express and combine attributes effectively, leading to suboptimal results.
Innovation Solution
A multi-modal search method that transforms multi-modal embeddings into a vector space allowing linear expression, enabling vector operations such as addition and subtraction of attributes using a virtual space, with a correction function to address non-linear errors, thereby providing search results that better align with user queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-modal embedding is used to represent product attributes, then search accuracy for multi-modal inputs is improved, but the ability to perform linear vector operations is lost due to non-linear errors
Solution Approach 1:
The patent introduces a virtual attribute vector space as an intermediary representation layer between the multi-modal embedding space and the query processing system. This virtual space allows linear vector operations to be performed on transformed representations, while a correction function maps the results back to the original multi-modal embedding space, thus resolving the contradiction between maintaining non-linear embedding accuracy and enabling linear operations.
Solution Approach 2:
The patent transforms the multi-modal embedding space into a virtual attribute vector space with different mathematical properties. In this transformed space, attributes can be represented and operated on linearly, while a correction function accounts for the non-linear relationship between the two spaces. This parameter transformation enables both accurate multi-modal representation and linear operability.
2Productivity
If vector operations are performed directly in multi-modal embedding space, then computation is simplified, but non-linear errors cause inaccurate search results
Solution Approach 1:
The virtual attribute vector space serves as an intermediary that enables efficient linear vector operations while a correction function ensures accuracy by mapping back to the original embedding space. This resolves the contradiction by separating the computation efficiency function from the accuracy function.
Solution Approach 2:
The correction function acts as a feedback mechanism that accounts for non-linear errors by using the relationship between the multi-modal embedding space and the virtual attribute vector space. This feedback ensures that the final search results remain accurate despite the simplified linear operations performed in the virtual space.
3Measurement precision
If multi-modal inputs (image and text) are combined for search, then user intent matching is improved, but system complexity increases
Solution Approach 1:
The patent merges image and text attribute representations into a unified virtual attribute vector space where both modalities can be processed using the same linear vector operations. This unification simplifies the system architecture compared to handling modalities separately, while still enabling accurate multi-modal user intent matching through the correction function.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of providing search results based on multi-modal features includes performing a vector operation between attributes according to a user query on a multi-modal embedding space; and providing search results corresponding to the user query based on an embedding vector acquired through the vector operation.