Image-Based Complementary Item Recommendations From Scene Photos
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Online catalog shopping lacks the immersive experience of discovering complementary products that are visually or stylistically similar to items in a physical store setting, as customers cannot easily find related products like in physical retail environments.
Innovation Solution
A two-stage machine learning framework using a visual similarity learning module and a complementary set learning module with a flexible bidirectional transformer to analyze images and recommend complementary products based on visual similarity and compatibility, leveraging self-supervised training with datasets of real-world scenes and product images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If online catalog shopping is used, then shopping speed and convenience are improved, but the ability to discover complementary products is worsened
Solution Approach 1:
The system uses image copying technology to capture visual information from physical store environments and recreate it in the online catalog interface. When a customer views a product online, the system generates visual recommendations by copying and displaying images of complementary products that would typically be displayed together in physical stores, thereby transferring the discovery capability from physical to digital environment
Solution Approach 2:
The patent introduces an image-based recommendation system as an intermediary between the customer and the catalog. This intermediary analyzes product images and automatically generates visual recommendations for complementary items, bridging the gap between fast online shopping and the traditional advantage of physical store product discovery
2Ease of operation
If traditional online catalog interfaces are used, then ease of operation is improved, but the immersive experience of related products is worsened
Solution Approach 1:
The system adds a visual dimension to the traditional text-based online catalog interface. By incorporating image-based recommendations that display complementary products visually alongside the main product, the system creates a multi-dimensional shopping experience that combines the ease of online browsing with the visual context of physical store arrangements
Solution Approach 2:
The patent merges the advantages of online shopping (convenience and speed) with the advantages of physical stores (visual product discovery and context). The interface combines traditional catalog browsing with overlaid visual recommendations, creating a hybrid shopping experience that maintains ease of operation while restoring visual context
Data Source
AI summary
A two-stage learning framework provides image-based complementary item recommendations. After receiving an image of a real-world scene depicting at least one object, the first stage generates feature embeddings for the real-world scene and the at least one object. The second stage generates a predicted feature embedding for at least one predicted object from the feature embeddings obtained from the first stage, generates a category embedding for the at least one predicted object from the predicted feature embedding, generates a recommended feature embedding for at least one recommended object from the predicted feature embedding and the category embedding, and outputs an identifier of the at least one recommended object, wherein the at least one recommended object is complementary to the at least one object.