Per-Object Semantic Feature Extraction for Digital Image Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional digital image search techniques are inadequate in locating specific objects within images due to their reliance on entire image representations, leading to inefficiencies and inaccuracies, especially when expressing complex features like feelings or themes, and fail to support searches for objects not at the image's center.
Innovation Solution
The technique extracts semantic features on a per-object basis using global and local segmentation masks, allowing for location-aware object searches, size, and rotation-based queries, enabling the composition and arrangement of objects in search queries to find matching images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image similarity techniques use a single representation of the entire digital image, then the system can perform basic image searches, but it cannot accurately locate objects within particular subsections of the image
Solution Approach 1:
The patent divides the digital image into multiple segments by generating segmentation masks that identify different objects and regions within the image. Each segmentation mask isolates specific objects, allowing the system to extract features from individual objects rather than treating the entire image as a single unit. This segmentation enables precise object localization while maintaining manageable computational complexity through targeted processing of divided regions.
2Adaptability or versatility
If conventional techniques rely on text tags to describe digital images, then the system can perform text-based searches, but it fails when it is difficult to express what is desired in the image using text
Solution Approach 1:
The patent replaces the text-based mechanical search system with an image-based visual search system. Instead of requiring users to input text descriptions and match them against text tags, the system allows users to upload query images and performs similarity comparisons based on visual features extracted from segmentation masks. This substitution enables users to search for visual characteristics, colors, compositions, and object arrangements that are difficult or impossible to express through text alone.
3Reliability
If conventional image similarity techniques perform repeated searches to locate digital images of interest, then the system can attempt to improve accuracy, but it results in inefficient use of computational resources
Solution Approach 1:
The patent performs preliminary processing by generating segmentation masks and extracting visual features from all images in the database before search queries are executed. These pre-extracted features are stored and organized for efficient retrieval. When a search query is received, the system compares query features against the pre-processed database features directly, avoiding repeated full-image analysis and significantly reducing computational resources required during actual search operations while maintaining high accuracy.
Data Source
AI summary
Object search techniques for digital images are described. In the techniques described herein, semantic features are extracted on a per-object basis form a digital image. This supports location of objects within digital images and is not limited to semantic features of an entirety of the digital image as involved in conventional image similarity search techniques. This may be combined with indications a location of the object globally with respect to the digital image through use of a global segmentation mask, use of a local segmentation mask to capture post and characteristics of the object itself, and so on.