Visual Search Queries Using Hashed RGB-D Local Descriptors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mobile visual search technologies face challenges in efficiently processing and matching RGB-D images due to the complexity of encoding both color and depth information, leading to high bandwidth requirements and limited flexibility in handling large databases.
Innovation Solution
A method and apparatus that extract local descriptors from digital image and depth information, generate global representations, and hash these representations to create visual search queries that fuse appearance and shape description information, allowing for selective variation of information amounts based on response signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If both color and depth information are encoded in visual search queries, then recognition accuracy is improved, but bandwidth requirements increase
Solution Approach 1:
The visual search system segments the encoding process into separate handling of color information and depth information. Local descriptors are extracted independently from color channels and depth channels, then processed through separate global encoding pipelines before being combined in the query. This segmentation allows optimized compression of each modality independently, reducing total bandwidth while preserving recognition accuracy.
Solution Approach 2:
The patent extracts only the essential features needed for recognition rather than transmitting complete color and depth images. Local descriptors capture salient visual characteristics, and global encoding extracts dominant patterns from these descriptors. This extraction process removes redundant information, significantly reducing bandwidth requirements while maintaining recognition performance.
2Measurement precision
If complex encoding is used to process RGB-D images, then recognition rates improve, but device complexity increases
Solution Approach 1:
The complex encoding process is divided into modular stages: local descriptor extraction, global encoding, and query formation. Each stage processes specific aspects of the RGB-D data independently, making the overall complex task manageable through structured decomposition. This modular approach allows mobile devices to handle processing in discrete, optimized steps rather than overwhelming monolithic computation.
Solution Approach 2:
The system performs preliminary processing of color and depth information separately before combining them. Local descriptors are pre-computed from raw sensor data, and global encoding is performed on these descriptors before query formation. This preliminary action prepares data in advance, reducing the computational burden during actual search operations and making the system feasible for mobile deployment.
3Productivity
If appearance and shape information are fused early in processing, then search efficiency improves, but flexibility in updating databases decreases
Solution Approach 1:
The system maintains dynamic flexibility by allowing appearance and shape information to be processed and combined at different stages depending on application needs. The architecture supports both early fusion (improving search efficiency) and late fusion (maintaining update flexibility). This dynamic adaptability allows the same system to optimize for different operational requirements without sacrificing either search performance or database maintenance capability.
Data Source
AI summary
Local descriptors are extracted from digital image information and digital depth information related to digital images. The local descriptors convey appearance description information and shape description information related to the digital images. Global representations of the one or more digital images are generated based on the extracted local descriptors, and are hashed. Visual search queries are generated based on the hashed global representations. The visual search queries include fused appearance description information and shape description information conveyed in the local descriptors. The fusing may occur before the global representations are generated, before the hashing or after the hashing.