Deep Learning Multi-Modal Product Matching System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing product matching algorithms fail to effectively match commercial products across different platforms due to missing or non-standard descriptive data attributes, relying heavily on numeric codes like UPC and SKU, which are not always available, and struggle with multimodal data types such as text, images, and audio.
Innovation Solution
A deep learning-based multi-modal product matching system that processes and matches products using multimodal data including text, categorical, audio, and image data, selecting appropriate models based on product category and user preferences to generate match scores and probabilities, incorporating both structured and unstructured data for accurate matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional matching algorithms rely on numeric product codes (UPC, SKU), then matching can be performed when codes are available, but matching fails or provides poor results when codes are missing or non-standard
Solution Approach 1:
The patent segments the matching process into multiple independent modalities (text analysis, image analysis, audio analysis, categorical analysis) that can process different types of data separately. Each modality handles specific data types independently, allowing the system to adapt to missing or non-standard data in any single modality while relying on other modalities to provide matching capability.
Solution Approach 2:
The patent creates a universal matching system that handles multiple data types and formats through a single multi-modal framework. The system can process text, images, audio, and categorical data through unified deep learning models, making it adaptable to various data formats and platforms while maintaining consistent matching reliability across different product representation styles.
2Loss of information
If product data is represented by descriptive attributes (text, images, audio), then more information is available for matching, but existing algorithms fail to effectively process and match this multi-modal data
Solution Approach 1:
The patent introduces deep learning models as intermediary components between raw multi-modal data and the matching decision. These intermediary models (text encoders, image encoders, audio encoders, categorical encoders) transform diverse data types into unified feature representations that can be effectively compared, bridging the gap between complex multi-modal inputs and simple matching outcomes.
Solution Approach 2:
The patent combines multiple types of data (text, images, audio, categorical attributes) into a composite multi-modal representation for each product. By fusing information from different data types through deep learning models, the system creates a richer, more comprehensive product representation that leverages the strengths of each data type while compensating for weaknesses in individual modalities.
3Device complexity
If a single matching model is used for all product categories, then the system is simpler to implement, but matching precision decreases for category-specific products
Solution Approach 1:
The patent implements a dynamic model selection mechanism that adapts the matching approach based on the specific product category and data availability. The system dynamically selects or weights different deep learning models depending on the product type (e.g., emphasizing image analysis for visual products, text analysis for service products), allowing optimal matching precision for each category while maintaining a unified system architecture.
Data Source
AI summary
Methods and systems for generating a plurality of matching items that match a reference item are disclosed. The method includes first determining reference attribute data for the reference item, where the reference attribute data is multimodal. Next, selecting a deep learning multimodal matching model from a plurality of candidate multimodal matching models. The selected deep learning multimodal matching model has a first deep learning neural network (DLNN) for processing data having a first data mode and a second DLNN analyzer for processing data having a second data mode. Then, matching a potential matching item to the reference item using the selected deep learning multimodal matching model to generate a match score, where the match score is computed based on the reference attribute data for the reference item and attribute data for the potential matching item. Finally, adding the potential matching item to the plurality of matching items based on the match score.