Tap-less Visual Search Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current visual search apps require users to manually tap on a screen to capture images, which can be cumbersome and interrupt the search process, especially in crowded scenes, necessitating a more intuitive and efficient method for product matching.
Innovation Solution
A 'tap-less' visual search system that uses on-device object detection and tracking to automatically identify and focus on objects of interest, providing real-time visual cues and triggering product matching in the cloud without further user interaction, allowing continuous data recording for improved matching accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual tap-to-search approach is used, then user can initiate product search, but user experience is degraded due to cumbersome interaction and process interruption
Solution Approach 1:
The system performs preliminary object detection and tracking continuously in the camera viewfinder before a search is initiated. This allows the system to pre-identify objects of interest and their locations, so when the user points the camera at an object, the search can be immediately triggered without requiring manual tapping or additional user actions to capture the image.
Solution Approach 2:
The system automatically detects objects, tracks them, determines when an object is the intended search target, and triggers the product matching process without requiring user intervention. The mobile device performs these functions autonomously by monitoring camera input and automatically initiating cloud-based visual search when confidence thresholds are met.
2Extent of automation
If automatic object detection is implemented, then user interaction is eliminated, but system complexity increases
Solution Approach 1:
The system divides the automation into distinct functional modules: object detection module that identifies objects in the camera view, object tracking module that monitors object position across frames, confidence assessment module that determines when the intended object is found, and search trigger module that initiates cloud-based product matching. This segmentation allows each component to be optimized independently and reduces overall system complexity.
Solution Approach 2:
The system uses visual cues displayed in the camera viewfinder as an intermediary between the automatic detection system and the user. These cues indicate which objects are detected, their confidence levels, and guide the user to point the camera at the correct object, thereby simplifying the automation while maintaining user control.
3Measurement precision
If visual cues are provided to guide object selection, then object identification accuracy improves, but display resources are consumed
Solution Approach 1:
The system provides visual cues selectively rather than continuously. Visual feedback is displayed only when objects are detected and tracked, and only in the specific regions of the viewfinder where objects are located. This partial action approach improves object identification accuracy while minimizing display resource consumption compared to full-screen continuous visual feedback.
Data Source
AI summary
A system and method detects that an object within an image frame being captured via use of an imaging element associated with a computing device is an object of interest, tracks the object of interest within the image frame while determining if the object within the image frame remains the object of interest within the image frame for a predetermined amount of time, and, when the object within the image frame fails to remain the object of interest within the image frame for the predetermined amount of time causes the steps to be repeated. Otherwise, the system and method will automatically provide at least of part of the image frame to a cloud-based visual search process for the purpose of locating one or more matching products from within a product database for the object of interest with the located one or more matching products being returned to a customer as a product search result.