Arbitrary Image Feature Search via Region Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image services fail to allow users to specify the aspects of similarity when searching for similar images, relying on general visual features and assuming a single notion of similarity, which limits the user's ability to find images based on specific characteristics.
Innovation Solution
A cloud-based image search system that allows users to upload images and define search criteria for arbitrary image features, using a trained model to generate similarity criterion masks and determine Euclidean distances between images to identify similar images based on user-defined criteria, enabling detailed and context-specific similarity searches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image services use general visual features for similarity determination, then the system is simple and easy to operate, but the search precision and relevance to user needs deteriorates
Solution Approach 1:
The patent segments the image into multiple regions (e.g., foreground, background, sky, ground) and allows users to select specific regions for similarity search. This segmentation enables precise control over which parts of the image contribute to similarity determination, resolving the contradiction by allowing high precision through region-specific matching while keeping the overall system manageable through modular region-based processing
Solution Approach 2:
The patent introduces an additional dimension of control by allowing users to specify not just the uploaded image but also weightings for different image regions. This transforms the search from a single holistic similarity metric to a multi-dimensional search space where users can adjust the importance of different regions, thereby achieving higher search precision without overwhelming system complexity
2Adaptability or versatility
If conventional image services assume a single notion of similarity, then the system is simple to implement, but the adaptability to different user needs deteriorates
Solution Approach 1:
The patent makes the similarity determination dynamic by allowing users to adjust weightings for different image regions based on their specific needs. The system adapts to different user requirements by dynamically changing which regions are emphasized in the similarity calculation, enabling the same system to serve multiple search purposes without requiring multiple separate systems
Solution Approach 2:
The patent creates a universal image search system that can handle multiple types of similarity searches through a single interface. By providing region selection and weighting capabilities, the system becomes multi-functional, capable of performing various search tasks (e.g., finding images with similar skies, similar backgrounds, or similar overall compositions) without requiring separate specialized systems for each search type
3Measurement precision
If conventional image services provide only general filtering options, then the ease of operation is maintained, but the search precision for specific features deteriorates
Solution Approach 1:
The patent performs preliminary segmentation of the uploaded image into distinct regions before the user initiates the search. This preliminary action prepares the image data in advance, allowing users to simply select from pre-defined regions without having to manually define complex search parameters, thereby maintaining ease of operation while achieving precise feature matching
Solution Approach 2:
The patent enables the system to automatically analyze the uploaded image and generate suggested region weightings or selections based on the image content. This self-service capability reduces the burden on users by providing intelligent defaults, allowing them to achieve precise feature matching with minimal manual input, thus balancing search precision with ease of operation
Data Source
AI summary
In implementations of digital image search based on arbitrary image features, a server computing device maintains an images database of digital images, and includes an image search system that receives a search input as a digital image depicting image features, and receives search criteria of depicted image features in the digital image. The image search system can then determine similar images to the received digital image based on similarity criterion corresponding to the search criteria. A trained image model of the image search system is applied to determine an image feature representation of the received digital image. A feature mask model of the image search system is applied to the image feature representation to determine a masked feature representation of the received digital image. The masked feature representation of the received digital image is compared to a masked feature representation of each respective database image to identify the similar images.