Product Search Embedding Visual Text Relevance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
E-commerce platforms face inaccuracies and biased search results due to reliance on product titles and descriptions alone, leading to suboptimal customer experiences and noisy behavioral feedback.
Innovation Solution
Incorporating product images into the search process by extracting image features and using an end-to-end relevance model that combines text and image embeddings, allowing for the determination of relevance between search queries and product information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If product search relies only on product titles and descriptions, then the search system is simple to implement, but the search accuracy deteriorates
Solution Approach 1:
The patent combines multiple information sources (product titles, descriptions, and images) into a unified search system. The image embedding module converts product images into vector representations that are merged with text embeddings, allowing the system to leverage both visual and textual information for improved search accuracy while maintaining a integrated architecture.
Solution Approach 2:
The patent introduces image embeddings as an intermediary representation that bridges visual information and text-based search queries. The embedding modules transform images into vector spaces that can be compared with query embeddings, enabling accurate visual-based search without requiring complex image processing in the main search pipeline.
2Measurement precision
If product search uses only text information, then the processing speed is fast, but the search result accuracy deteriorates
Solution Approach 1:
The patent pre-computes and stores image embeddings for products in advance. When a search query is received, the system retrieves pre-computed embeddings rather than processing images in real-time, significantly reducing search processing time while maintaining high accuracy through the use of pre-extracted visual features.
3Measurement precision
If product search uses comprehensive product information including images, then the search accuracy improves, but the computational resources required increase
Solution Approach 1:
The patent extracts only the essential visual features from product images through embedding modules, rather than processing entire high-resolution images during search operations. This extraction approach captures the most relevant visual information (colors, patterns, shapes) while discarding redundant data, reducing computational energy consumption while maintaining search accuracy.
4Ease of operation
If product search relies on traditional text-based methods, then the system complexity is low, but the customer experience deteriorates
Solution Approach 1:
The patent creates a universal search system that handles both traditional text queries and visual-based queries through a unified embedding comparison mechanism. The same infrastructure supports multiple search modalities (text-to-text, image-to-text, text-to-image), providing enhanced customer experience without requiring separate specialized systems for each search type.
Data Source
AI summary
A computer-implemented method for searching a product corresponding to a query from a customer. The method includes: embedding the query to obtain a query embedding; retrieving product information having a product text and a product image; embedding the product text to obtain a product text embedding, embedding the product image to obtain a product image embedding, and combining the product text embedding and the product image embedding to obtain a product embedding, where the product image embedding has a same format as the product text embedding; subjecting the query embedding and the product embedding to a transformer to determine whether the query and the product are relevant; and providing the product as a search result of the query when the query and the product are relevant.