SKU Embedding Compression for Customer-Specific Catalog Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
E-commerce data management systems face challenges in efficiently tracking and managing large numbers of stock keeping units (SKUs) due to exploding index sizes, leading to increased memory storage needs and prolonged search times, especially when dealing with multiple customer groups and varying product features.
Innovation Solution
The system encodes catalog data into fingerprints, compresses them into a thin matrix, and uses an autoencoder to derive an embedding space for clustering, allowing for efficient search and retrieval by computing distances to cluster centroids, thereby reducing the search space and improving computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the data management system stores and tracks all effective SKUs for multiple customer groups and product variants, then the system can provide comprehensive product information and pricing, but the index size explodes leading to increased memory storage needs and prolonged search times
Solution Approach 1:
The patent segments the massive SKU index into hierarchical groups and clusters. Instead of storing all SKUs in a single flat index, the system organizes them into parent-child relationships with cluster assignments. This segmentation allows the search algorithm to navigate through hierarchical levels and search only relevant clusters rather than the entire index, dramatically reducing search time while preserving comprehensive product information.
Solution Approach 2:
The patent introduces a new dimensional organization by assigning cluster identifiers and hierarchical levels to SKUs. This adds spatial dimensions (hierarchy level, cluster group) to the traditional flat SKU structure, enabling multi-dimensional search optimization. The system can now search along hierarchical paths and cluster boundaries rather than linearly through all SKUs, reducing the effective search space.
2Reliability
If the data management system tracks billions of SKUs across multiple customer groups, then complete inventory tracking is achieved, but computing resources such as memory storage and throughput are excessively consumed
Solution Approach 1:
The patent merges multiple related SKUs into cluster groups with shared characteristics. By grouping SKUs that share common attributes, pricing rules, or inventory patterns into the same cluster, the system can store and manage them more efficiently. The cluster structure allows shared metadata and indexing information to serve multiple SKUs simultaneously, reducing redundant storage while maintaining complete inventory tracking capability.
3Measurement precision
If the system maintains detailed catalog data for every product variant and customer group, then accurate pricing and inventory information is provided, but the index complexity and search computational load increase
Solution Approach 1:
The patent applies local quality by assigning different levels of detail and indexing granularity to different parts of the SKU hierarchy. Not all SKUs require the same level of indexing depth or cluster assignment complexity. The system can optimize cluster assignments and hierarchical levels based on local characteristics such as product category, demand patterns, and pricing complexity, reducing overall index complexity while maintaining precise pricing information where needed.
Data Source
AI summary
An effective stock keeping unit (SKU) management system encodes catalog data into an embedding per catalog item. An embedding space is created by encoding catalog item data into an embedding per catalog item. The embedding is created by generating an index, where a number of rows represents a number of catalog items and a number of columns represents a number of fields associated with each catalog item. The index is then denormalized using customer groups and transformed by compressing the number of columns, to create the embedding space. In some configuration, a machine learning model is trained using catalog data. In the embedding space, item similarity is encoded by clustering catalog SKUs into groups in the embedding space, by placing similarly related items close to each other in the embedding space. Catalog items are then searched for in the embedding, with the closest clusters searched for a particular catalog item.