Relation Network for Order-Invariant Outfit Compatibility Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing fashion recommendation systems struggle to determine compatibility between multiple items in images, as they often rely on pair-wise matching and are not order invariant, requiring category labels, and fail to achieve sufficient accuracy in predicting compatibility for entire outfits with an arbitrary number of items.
Innovation Solution
A computer-implemented method using a relation network that processes image data from multiple images of items through convolutional neural networks to generate feature vectors, which are then input into a multi-layer perceptron to produce compatibility scores, allowing for order-invariant processing and integration of textual descriptions to improve compatibility prediction without requiring category labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pair-wise matching methods are used to determine item compatibility, then the system can process simple item pairs, but it fails to achieve sufficient accuracy for entire outfits with arbitrary numbers of items and is not order invariant
Solution Approach 1:
The patent segments the compatibility determination process into two distinct stages: (1) pair-wise compatibility scoring between individual items using CNNs, and (2) aggregate outfit compatibility scoring using an LSTM network that processes the sequence of item pairs. This segmentation allows the system to handle variable numbers of items while maintaining accuracy through hierarchical processing.
Solution Approach 2:
The patent transitions from two-dimensional pair-wise comparisons to three-dimensional outfit-level compatibility assessment by introducing a temporal dimension through LSTM processing. The LSTM network processes the sequence of pair-wise compatibility scores over time/steps, enabling the system to evaluate overall outfit compatibility while considering the arbitrary number of items in the outfit.
2Adaptability or versatility
If traditional recommendation systems use category labels for item matching, then classification is simplified, but the system loses flexibility in determining semantic compatibility and visual harmony
Solution Approach 1:
The patent replaces the mechanical classification system based on category labels with an intelligent visual processing system using convolutional neural networks. The CNNs automatically extract visual features and determine compatibility based on visual harmony and semantic relationships, eliminating the need for predefined category structures while improving adaptability.
Solution Approach 2:
The system enables items to self-describe their visual characteristics through learned feature representations from CNNs. Instead of requiring external category labels, the network automatically learns to represent items in a feature space where compatibility can be determined through visual and semantic analysis, allowing the system to serve itself without predefined taxonomies.
3Measurement precision
If the system processes each item individually through CNNs, then visual feature extraction is accurate, but the computational complexity increases with the number of items
Solution Approach 1:
The patent divides the computational workload into efficient segments: CNNs process individual item images in parallel to extract visual features, then the LSTM network processes the resulting compatibility scores in a sequential manner. This segmentation allows computationally intensive visual feature extraction to be performed once per item, rather than repeatedly for each pair or outfit configuration.
Solution Approach 2:
The system performs preliminary visual feature extraction through CNNs for each item before the compatibility assessment stage. By pre-processing and storing visual features of individual items, the system avoids redundant computations when evaluating multiple outfit combinations, significantly reducing the computational power required for subsequent compatibility analyses.
Data Source
AI summary
Systems, methods, and computer program products for determining compatibility between items in images obtain image data associated with a plurality of images, each image being associated with a single item of a plurality of items, each image including a plurality of elements of a matrix, and the plurality of elements of the matrix of each image including the image data associated with that image; for each image, input the image data associated with that image as a single object in a set of objects to a relation network; and generate a compatibility score for the plurality of items from the relation network based on the set of objects, the compatibility score including a prediction of whether the plurality of items is compatible.