Neural Tensor Network for Scalable Multi-Item Visual Compatibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current recommendation engines struggle with scalability and difficulty in determining multi-item compatibility and exploiting visual signals from items, particularly in fashion, where existing methods are not efficient in time and memory usage.
Innovation Solution
A scalable neural tensor network (TensorNet) is introduced to capture visual compatibility among item types by employing a cross-attention message passing segment and wide and deep tensor interaction segment, which characterizes local region-based patterns and global compatibility of outfits, using unsupervised region-wise feature extraction and cross-attention mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current recommendation methods are used to determine multi-item compatibility and exploit visual signals, then recommendation quality can be achieved, but time and memory scalability deteriorates
Solution Approach 1:
The patent segments images into multiple regions and processes each region independently to generate region-wise feature maps. This segmentation allows the system to handle visual signals at a granular level while maintaining scalability, as each region can be processed in parallel rather than requiring full-image processing for all item combinations.
Solution Approach 2:
The patent transitions from traditional matrix-based recommendation models to tensor-based models that incorporate an additional dimension for visual features. This dimensional expansion enables the system to simultaneously process user preferences, item attributes, and visual compatibility signals without proportionally increasing computational complexity.
2Measurement precision
If region-wise feature extraction and cross-attention mechanisms are employed to capture visual compatibility, then compatibility prediction accuracy is improved, but computational complexity increases
Solution Approach 1:
By dividing images into regions and processing them separately, the system captures local visual features that are computationally less intensive than processing entire images. This segmentation strategy maintains prediction accuracy while reducing the overall computational burden.
Solution Approach 2:
The patent introduces region-wise feature maps as intermediary representations that bridge raw images and final compatibility predictions. These feature maps serve as compressed intermediaries that retain essential visual information while reducing data volume and computational requirements for subsequent processing stages.
3Adaptability or versatility
If tensor network models are used to model multi-item interactions, then multi-aspect feature interactions are captured, but memory requirements increase
Solution Approach 1:
The patent uses tensor factorization to decompose high-dimensional interaction tensors into lower-dimensional factor matrices. This dimensional transformation enables the model to capture complex multi-item interactions while storing parameters in a more memory-efficient format, avoiding the exponential memory growth that would result from explicitly representing all possible item combinations.
Data Source
AI summary
A method includes determining a set of regions for each of a first plurality of images of a first item type, a second plurality of images of a second item type, and a third plurality of images of a third item type. The method also includes for each region in each set of regions of the images, generating, by the processing computer, a vector representing the region, and then generating a plurality of aggregated messages using the vectors corresponding to combinations of images of different types of items, the images being from the first, second, and third plurality of images. Then, unified embeddings are generated for the images in the first, second, and third plurality of images, respectively, using aggregated messages in the plurality of aggregated messages. Matching scores associated with combinations of the images are created using the unified embeddings and a machine learning model.