Cross-Modal Object Similarity Screening
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently screening associated objects and recommending same style products, as they often rely solely on image or text similarities without considering the combined impact of both modalities.
Innovation Solution
A method that utilizes feature vectors for both images and text descriptions, ensuring they correspond to the same feature space, allowing for the calculation of vector similarities between image and text features. This method determines object similarity by combining image similarity, text similarity, and cross-modal similarity, leading to more accurate recommendations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If only image similarity or text similarity is used for object screening, then the screening process is simple, but the accuracy of determining object similarity is insufficient
Solution Approach 1:
The patent merges image feature vectors and text feature vectors into a unified feature space, enabling cross-modal similarity calculation. By combining multiple similarity dimensions (image-image, text-text, and image-text similarities), the system achieves more accurate object similarity determination while managing complexity through structured integration of multiple data modalities.
2Measurement precision
If cross-modal similarity calculation is implemented, then the accuracy of object similarity determination is improved, but the computational complexity increases
Solution Approach 1:
The patent segments the similarity calculation process into distinct components: image feature extraction, text feature extraction, image-image similarity calculation, text-text similarity calculation, and image-text cross-modal similarity calculation. This segmentation allows for optimized computation of each component independently and facilitates selective application based on specific needs.
Solution Approach 2:
The system can selectively apply cross-modal similarity calculation depending on the specific application requirements. For certain use cases, only partial similarity calculations (e.g., image-image or text-text) may be sufficient, while full cross-modal calculation is applied only when higher precision is needed, thus balancing computational resources with accuracy requirements.
Data Source
AI summary
The application provides methods for screening associated objects and recommending same style products. The method for screening associated objects includes: obtaining a first feature vector set corresponding to a first object and at least one second feature vector set corresponding to at least one second object; determining an object similarity between the first object and the second object based on multiple vector similarities between the feature vectors in the first feature vector set and the feature vectors in the second feature vector set, wherein the multiple vector similarities include vector similarity between the image feature vectors and the text feature vectors; and screening the at least one second object based on the object similarity to obtain an associated object of the first object. The solution proposed by this application improves the accuracy of determining object similarity, thereby allowing for more precise screening of the associated objects for the first object.