Conditional Generative Adversarial Network for Complementary Image Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing recommendation engines struggle to accurately suggest complementary digital items, such as fashion apparel, due to noisy click count data and the complexity of fashion collocation, which requires nuanced understanding and professional expertise.
Innovation Solution
A computer-implemented technique using a machine-trained generator component based on a conditional generative adversarial network (cGAN) to transform first-part image information into complementary second-part generated image information, allowing for the retrieval and presentation of recommended output images that complement the initial selection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If recommendation engines use click count information to determine relatedness between digital items, then the system can provide recommendations based on user interaction data, but the recommendations become noisy and insufficiently nuanced to accurately identify complementary items
Solution Approach 1:
The patent transforms the recommendation approach by changing from using click count statistics to using image feature representations. The system extracts visual features from images and uses deep learning models to compute similarity based on visual characteristics rather than interaction metrics, thereby improving the precision of relatedness measurement while maintaining recommendation productivity
Solution Approach 2:
The patent replaces the mechanical counting of user interactions with a computational model based on image feature analysis. Instead of relying on discrete click events, the system uses continuous visual feature vectors and neural network-based similarity computation to determine item relatedness, achieving more nuanced and accurate recommendations
2Manufacturing precision
If the system narrows down to fashion images from fashion websites for training, then the model can learn discriminative features for fashion collocation, but the system requires professional expertise and has large uncertainty in fashion collocation
Solution Approach 1:
The patent implements self-service through automated training data generation. The system automatically crawls fashion websites, downloads images, and uses segmentation models to extract relevant fashion items without requiring manual curation by experts. This automation reduces both the complexity of data preparation and the dependency on professional fashion expertise while maintaining high discrimination accuracy
Solution Approach 2:
The patent performs preliminary actions by pre-processing and segmenting fashion images during the training phase. The system uses segmentation models to isolate fashion items from complex fashion show images before training the recommendation model, thereby preparing high-quality training data in advance and reducing the need for expert intervention during deployment
3Manufacturing precision
If the system uses deep neural networks with hard-aware online exemplar mining, then the network can learn discriminative features at early and later training stages, but the training process becomes complex and computationally intensive
Solution Approach 1:
The patent segments the training process into distinct phases: initial training with raw fashion images to learn basic features, and subsequent training with segmented and curated images to refine discriminative capabilities. This segmentation of the training pipeline allows the system to achieve high feature learning accuracy while managing computational complexity through staged processing
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented technique is described herein for retrieving at least one recommended output image. In one implementation, the technique uses a generator component to transform first-part image information, associated with a first-part image selected by a user, into one or more instances of second-part generated image information. Each instance of the second-part generated image information complements the first-part image information. The generator component is trained by a computer-implemented training system using a conditional generative adversarial network (cGAN). The technique further includes: retrieving one or more second-part output images from a data store based on the instance(s) of second-part generated image information; generating a user interface presentation that presents the first-part image and the second-part output image(s); and displaying the user interface presentation on a display device. In one example, the first-part image and the second-part output images show complementary apparel items.