Visual Object Graphs for Cross-Scene Object Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems struggle to efficiently link objects across visually different scenes based on the presence of specific objects, limiting users' ability to find relevant content in vast digital collections.
Innovation Solution
A method involving image processing and deep neural networks to segment objects, generate embedding vectors, cluster similar objects, and link scenes containing these objects, allowing users to select query objects and receive matching scenes that may be visually distinct but include the same or similar objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional image search methods are used, then visual similarity can be maintained, but the ability to find objects across visually different scenes is limited
Solution Approach 1:
The patent segments images into individual objects using object detection algorithms. Each detected object is then processed separately to extract features and generate embeddings, allowing the system to focus on specific objects rather than treating the entire image as a single unit. This segmentation enables accurate object matching across visually different scenes.
Solution Approach 2:
The patent introduces an intermediary representation layer using embedding vectors that capture object characteristics independently of scene context. These embeddings serve as a mediator between object detection and scene matching, enabling the system to link objects across visually different scenes while maintaining matching accuracy through the intermediary embedding space.
2Productivity
If visual content is prioritized for discovery, then relevant content can be found, but the complexity of processing and indexing increases
Solution Approach 1:
The patent performs preliminary processing by pre-training deep neural networks to generate object embeddings and pre-indexing objects with their embeddings and scene associations. This preliminary action enables fast query processing during actual search operations, as the system can directly query the pre-built index rather than processing images in real-time.
Solution Approach 2:
The patent replaces traditional mechanical image processing approaches with deep learning-based neural networks that automatically extract object features and generate embeddings. This substitution reduces manual processing complexity while improving the speed and accuracy of content discovery through automated intelligent processing.
3Adaptability or versatility
If objects are linked based on visual similarity, then matching scenes can be found, but visually distinct scenes with same objects are missed
Solution Approach 1:
The patent applies local quality analysis by extracting and processing only the relevant local regions corresponding to detected objects, rather than analyzing the entire image globally. Each object region is processed independently to extract its specific visual characteristics and generate its embedding, enabling precise identification of objects even when they appear in visually distinct scenes with different overall compositions.
Data Source
AI summary
Disclosed are implementations that enable the linking or connection of objects and different scenes in which those objects are represented. For example, a corpus of scenes (e.g., digital images) that include a representation of one or more objects may be processed using the disclosed implementations to segment from those scenes the individual objects represented in those scenes. The disclosed implementations may further determine clusters of visually similar object segments and form object clusters for those object segments. The scenes that include those object segments are also linked to the object cluster. With scenes linked to different object clusters, a user may select one or more query objects or a query scene and be presented with other scenes that include visually similar objects, even though the overall scenes may be visually different.