Duplicate Product Detection Using Brand and Visual Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing duplicate product detection methods for e-commerce platforms are inadequate, as they fail to account for brand-specific and category-specific features and do not effectively identify duplicate products using traditional string similarity algorithms.
Innovation Solution
A system and method utilizing e-commerce specific text and visual similarity metrics, combined with binary classification models, to detect duplicate products by training on raw data, incorporating brand-specific and image features, and reducing pairwise comparisons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional string similarity algorithms are used to detect duplicate products, then the detection process is simple, but the detection accuracy is insufficient and cannot identify duplicate products effectively
Solution Approach 1:
The patent transforms the duplicate detection problem from simple string matching to a multi-parameter classification task. It extracts multiple features including text similarity, image similarity, price ratio, and brand information, then uses these parameters as inputs to a binary classification model (Random Forest) to determine duplicate status, thereby significantly improving detection accuracy
Solution Approach 2:
The patent creates a composite detection system that combines multiple types of data (text, images, pricing, brand information) and multiple analysis methods (string similarity, image recognition, statistical analysis) into a unified detection framework. This composite approach leverages the strengths of each component to achieve superior detection performance
2Reliability
If all possible product pairs are compared to detect duplicates, then no duplicate is missed, but the computational time and resources increase significantly
Solution Approach 1:
The patent segments the detection process into multiple stages: first filtering products by brand and category, then by basic text similarity thresholds, and finally applying the complex binary classification model only to candidate pairs. This multi-level segmentation dramatically reduces the number of pairs requiring full analysis while maintaining detection completeness
Solution Approach 2:
The patent applies a tiered analysis approach where not all product pairs receive the same level of analysis. Simple filtering rules are applied first to eliminate obvious non-duplicates, and the more computationally intensive classification model is applied only to borderline cases, achieving efficient use of resources while maintaining reliability
3Measurement precision
If traditional detection methods are used without brand-specific features, then the detection method is general and simple, but it cannot distinguish between same-name products from different brands
Solution Approach 1:
The patent applies different detection strategies to different local characteristics of products. It specifically extracts and analyzes brand information, category information, and product attributes separately, allowing the system to adapt its detection logic to the specific features of each product pair rather than applying a uniform approach
Data Source
AI summary
Disclosed is a duplicate product entry detection system and method for detecting duplicate products in product catalogs of e-commerce platforms running on mobile devices or IT devices. The duplicate product entries can be searched by a duplicate product detection engine running on a deduplication engine server, the similarity calculations can be carried out, and the duplicate product detection can be performed for the finally detected product pairs.