Product Matching Model Using Bi-Encoder and Cross-Encoder Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing product matching techniques in e-commerce platforms, particularly those using deep learning models, face challenges in efficiency due to resource-intensive cross-encoder models when dealing with large product catalogs, making it difficult to accurately match products in a timely and cost-effective manner.
Innovation Solution
A method involving a bi-encoder model to generate initial vector representations of products, followed by a cross-encoder to refine similarity assessments, and a third model for learning using these refined data sets, reducing computational load and improving accuracy by leveraging both models' strengths.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a cross-encoder model is used for product matching, then matching accuracy is improved, but computational resources and computing time increase significantly
Solution Approach 1:
The patent segments the product matching process into two distinct stages: a pre-filtering stage using a bi-encoder model to quickly identify candidate product pairs, and a refinement stage using a cross-encoder model only on these candidates. This segmentation allows the system to leverage the speed of bi-encoders for broad coverage while applying the accuracy of cross-encoders only where needed, thus resolving the contradiction between matching accuracy and computational efficiency.
2Measurement precision
If a cross-encoder model is used for product matching, then matching accuracy is improved, but the complexity of the system increases
Solution Approach 1:
The patent divides the matching system into two specialized components with distinct functions: bi-encoders for initial candidate generation and cross-encoders for final accuracy refinement. This segmentation allows each component to be optimized independently and simplifies the overall system architecture compared to using a single complex cross-encoder system for all matching tasks.
Solution Approach 2:
The patent applies cross-encoder processing only partially - specifically, only to candidate product pairs identified by the bi-encoder, rather than to all possible product pairs. This partial application of the more complex cross-encoder model reduces system complexity while maintaining high accuracy for the most relevant matches.
3Productivity
If traditional product matching methods are used, then computational resources are saved, but matching accuracy decreases
Solution Approach 1:
The patent performs preliminary filtering using bi-encoder models before applying the more computationally intensive cross-encoder models. This preliminary action identifies and eliminates non-matching product pairs early in the process, ensuring that computational resources are focused only on promising candidates, thereby achieving both efficiency and accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method according to this disclosure is a method of learning a model used upon searching for a product corresponding to a predetermined search condition from among multiple products, including: a step of generating first vector representations based on product information of the multiple products and generating first data including one or more combinations of products highly similar to each other from among the multiple products based on similarity between the first vector representations, with using a first model 16; a step of generating second vector representations from product information of two products included in each of the combinations of products included in the first data and generating second data including combinations of products highly similar to each other based on the second vector representations with using a second model 18; and a step of executing learning of a third model 19 using the second data as training data.