Image and Text Embedding Training from Search Click Signals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for training image and text embedding models are limited by the quality and scalability of training data, often relying on manual labeling or social media captions, which can be time-consuming and produce less accurate results, and fail to effectively characterize highly specific concepts or handle multiple languages.
Innovation Solution
A training system that processes historical query logs from search systems to generate query-image and image-image training examples, using selection data from user interactions to jointly train image and text embedding models, leveraging co-click rates and similar-image click rates to adjust model parameters and improve embedding accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If manual labeling or social media captions are used for training data, then the training process is simpler to implement, but the quality and accuracy of training data deteriorates
Solution Approach 1:
The patent introduces an automatic training data generation system that acts as an intermediary between raw search query logs and the training data needed for embedding models. This system automatically processes search queries, image search results, and user interaction data to generate structured training examples, eliminating the need for manual labeling while maintaining high data quality through algorithmic processing of real user behavior patterns.
2Device complexity
If conventional training methods are used, then the implementation is straightforward, but the ability to characterize highly specific concepts deteriorates
Solution Approach 1:
The patent segments the training data generation process into distinct components: search query processing, image search result analysis, user interaction data extraction (clicks, co-clicks, similar-image clicks), and training example construction. This segmentation allows each component to be optimized independently, enabling the system to capture highly specific concepts through focused processing of different data dimensions while maintaining manageable system complexity.
3Measurement precision
If large-scale user interaction data is processed, then the embedding accuracy improves, but the data processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary processing of search query logs and user interaction data to pre-structure training examples before actual model training. By pre-computing image pairs, co-click rates, and similar-image click rates from historical data, the system prepares high-quality training data in advance, reducing the computational burden and processing time during the actual model training phase while maintaining high embedding accuracy.
4Ease of manufacture
If conventional training data sources are used, then data collection is easier, but the scalability of training data deteriorates
Solution Approach 1:
The patent implements a self-service training data generation system that automatically collects, processes, and structures training data from search query logs and user interaction records without requiring manual intervention. The system continuously ingests new search data and user interactions, automatically generating scaled training examples that grow with the search system's data accumulation, enabling scalable training data production that adapts to increasing data volumes.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image embedding model. In one aspect, a method comprises: obtaining training data comprising a plurality of training examples, wherein each training example comprises: an image pair comprising a first image and a second image; and selection data indicating one or more of: (i) a co-click rate of the image pair, and (ii) a similar-image click rate of the image pair; and using the training data to train an image embedding model having a plurality of image embedding model parameters.