SKU Embedding Compression for Fast Catalog Search Across Customer Groups
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
E-commerce platforms face challenges with managing large numbers of stock keeping units (SKUs) due to increased memory storage and computational demands, leading to inefficient search and retrieval times, especially when considering multiple customer groups and varying prices.
Innovation Solution
A compression scheme encodes catalog data into fingerprints, clusters similar items in an embedding space, and uses an autoencoder to create a thin matrix, allowing for efficient search by computing distances to cluster centroids.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If catalog data is stored with full detail for each SKU including customer groups and pricing variants, then search accuracy is improved, but storage requirements and computational resources increase significantly
Solution Approach 1:
The patent segments catalog data into multiple hierarchical levels: product categories, customer groups, and pricing variants. This segmentation allows the system to store data in a structured format where common attributes are shared across segments, reducing redundant storage while maintaining search accuracy through multi-level indexing.
Solution Approach 2:
The patent introduces a new dimensional organization by creating a multi-dimensional data structure that indexes SKUs across different customer groups and pricing variants simultaneously. This dimensional approach transforms the storage problem from linear redundancy to hierarchical sharing, reducing overall storage requirements while enabling precise multi-criteria searches.
2Measurement precision
If all SKU variants across all customer groups are tracked individually, then pricing accuracy for each customer segment is improved, but the number of effective SKUs explodes to billions
Solution Approach 1:
The patent merges individual SKU tracking with customer group and pricing variant information into a unified data model. Instead of creating separate SKU entries for each customer-group-SKU combination, the system combines these dimensions into a single indexed structure, reducing the effective SKU count from billions to manageable levels while preserving pricing accuracy through the unified index.
Solution Approach 2:
The patent creates a universal data structure that serves multiple functions simultaneously: it tracks individual SKUs, manages customer group specificities, handles pricing variants, and enables searches across all these dimensions. This multi-functional design eliminates the need for separate tracking systems for each dimension, reducing overall system complexity.
3Loss of information
If the data management system stores complete catalog information for billions of SKUs, then product availability information is complete, but search and retrieval time increases
Solution Approach 1:
The patent implements preliminary indexing and pre-computation of product availability information across all customer groups and pricing variants. By pre-organizing data into the multi-dimensional structure and pre-computing availability status, the system eliminates the need for real-time computation during searches, significantly reducing retrieval time while maintaining complete information.
4Loss of information
If traditional data structures are used to manage multi-customer group catalogs, then data completeness is maintained, but memory storage and throughput requirements increase
Solution Approach 1:
The patent changes the fundamental parameters of data organization from traditional flat structures to hierarchical multi-dimensional structures. This parameter change transforms how data is stored and accessed, enabling the same complete information to be represented more compactly in memory, thereby reducing storage requirements and throughput needs while maintaining data completeness.
Data Source
AI summary
An effective stock keeping unit (SKU) management system encodes catalog data into an embedding per catalog item. An embedding space is created by encoding catalog item data into an embedding per catalog item. The embedding is created by generating an index, where a number of rows represents a number of catalog items and a number of columns represents a number of fields associated with each catalog item. The index is then denormalized using customer groups and transformed by compressing the number of columns, to create the embedding space. In some configuration, a machine learning model is trained using catalog data. In the embedding space, item similarity is encoded by clustering catalog SKUs into groups in the embedding space, by placing similarly related items close to each other in the embedding space. Catalog items are then searched for in the embedding, with the closest clusters searched for a particular catalog item.