Multi-Task Visual Search for Accurate Similar Fashion Products
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing similar product search methods struggle to accurately identify fashion products based on image queries due to limitations in reflecting various attributes, particularly in text-based searches, leading to inclusion of non-similar products.
Innovation Solution
A multi-task learning model utilizing a unified backbone network is employed to convert images into single vectors incorporating multiple fashion attributes, enabling detailed feature comparison and similarity searches across different categories.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If text-based search methods are used to search for similar products, then the search process is simple and fast, but the search accuracy deteriorates because detailed fashion attributes such as style, length, and pattern cannot be properly specified
Solution Approach 1:
The patent replaces text-based search mechanisms with image-based visual search. Instead of using keyword input and text matching, the system captures product images and uses deep learning models (CNNs) to extract visual features and perform similarity comparison. This substitution enables the system to automatically recognize detailed fashion attributes like style, length, and pattern that are difficult to specify through text, thereby improving search accuracy while maintaining operational simplicity.
Solution Approach 2:
The patent introduces an intermediate feature extraction layer that bridges the gap between image input and search results. The system uses pre-trained CNN models (such as VGG16, ResNet, or Inception) to extract high-dimensional feature vectors from product images. These feature vectors serve as intermediaries that capture detailed visual attributes and enable accurate similarity comparison, resolving the contradiction between simple search operation and precise attribute matching.
2Measurement precision
If multiple separate models are used to detect different fashion attributes, then each attribute can be detected with high precision, but the system complexity increases
Solution Approach 1:
The patent employs a universal deep learning architecture that can detect multiple fashion attributes simultaneously. Instead of using separate specialized models for each attribute (color, pattern, style, length), the system uses a single multi-functional CNN model that extracts comprehensive visual features from the image. This unified approach maintains high detection precision for all attributes while significantly reducing system complexity compared to multiple separate models.
Solution Approach 2:
The patent merges multiple attribute detection tasks into a single integrated feature extraction process. The system combines color detection, pattern recognition, style classification, and length estimation into one unified deep learning model that processes the image once and generates all attribute information simultaneously. This merging eliminates the need for multiple separate processing pipelines, reducing computational overhead and system complexity while maintaining precision.
3Device complexity
If a unified backbone network is used in the multi-task learning model, then the system complexity is reduced and training efficiency is improved, but the ability to capture task-specific features may be limited
Solution Approach 1:
The patent implements a segmented multi-task learning architecture where a shared backbone network extracts common visual features, and separate task-specific head networks process different fashion attributes. The backbone network (e.g., VGG16, ResNet) provides a unified foundation that reduces model complexity, while the modular task-specific heads (for color, pattern, style, length) capture nuanced attribute-specific features. This segmentation resolves the contradiction by maintaining both simplicity and precision.
Solution Approach 2:
The patent applies local quality by making different parts of the network specialized for different functions. The shared backbone network uses generic feature extraction suitable for all tasks, while the task-specific head networks are customized to capture local nuances of each fashion attribute. For example, the color detection head focuses on chromatic features, while the pattern detection head emphasizes texture and repetition patterns. This local specialization maintains high feature extraction precision while keeping the overall model manageable in complexity.
Data Source
AI summary
A method of searching for similar products using a multi-task learning (MTL) model is provided. The method includes converting, by using a multi-task learning model utilizing a unified backbone network, each of a plurality of original images including a fashion item into a single vector including at least two multi-task attributes, and generating a visual search database by storing the original image and the single vector of the original image together.