Multimodal Image Retrieval With Embedding Filters for Campaign Relevance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manually curating images for marketing campaigns is labor-intensive and highly subjective, making it difficult to align images with campaign objectives and resonate with the target audience effectively.
Innovation Solution
A multimodal image retrieval system utilizing circular, text-heavy, and deduplication filters, along with advanced multimodal embedding models, automates the image selection process by filtering and retrieving relevant, visually appealing images based on text and image inputs, and is fine-tuned through incremental task scaling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual curation of images is performed, then image selection can be customized according to personal judgment, but the process becomes labor-intensive and time-consuming
Solution Approach 1:
The patent replaces the manual mechanical process of image curation with an automated computer-based system that uses machine learning models and embedding comparisons to retrieve relevant images, eliminating the need for manual selection while maintaining or improving selection quality
Solution Approach 2:
The system enables self-service image retrieval by automatically understanding user requirements through text or image inputs and autonomously selecting appropriate images from the database without requiring manual intervention or expertise in image selection
2Reliability
If manual curation is used, then subjective judgment can guide image selection, but consistency and alignment with campaign objectives become difficult to achieve
Solution Approach 1:
The system transforms the subjective parameter of image selection into objective measurable parameters by comparing embedding vectors and calculating similarity scores, allowing consistent and reliable image selection based on quantifiable metrics rather than subjective judgment
Solution Approach 2:
The system incorporates feedback mechanisms where user selections and campaign performance data are used to refine and improve the image retrieval algorithm, continuously enhancing alignment with campaign objectives through iterative learning
3Adaptability or versatility
If a comprehensive image database is maintained for various campaigns, then image variety and relevance can be improved, but computational complexity and processing time increase
Solution Approach 1:
The system performs preliminary action by pre-computing and storing embedding vectors for all images in the database during an offline process, so that during online query processing, only simple embedding comparisons are needed, significantly reducing real-time computational complexity
Solution Approach 2:
The system segments the image database into different categories or collections based on embedding similarities or metadata, allowing efficient retrieval by querying only relevant segments rather than processing the entire database, thus reducing computational complexity while maintaining versatility
Data Source
AI summary
Systems and methods for retrieving and providing images are disclosed. An example system receives, from a user device, a request for an image. The system determines, using a machine-learning model, search embeddings based on the request; filters image data based on the request to identify a filtered set of the image data; and obtains a subset of the image embeddings corresponding to the filtered set of the image data. The system further determines based on a comparison of the search embeddings and the subset of the image embeddings, recommended image data, and causes presentation of the recommended image data at the user device. In response to selection of the recommended image data, the system provides the recommended image data to the user device.