Server-Assisted Visual Object Identification for Low-Lag AR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems struggle to efficiently identify and provide additional information about objects in real-time, especially when capturing images at varying rates and angles, leading to lag and inefficiencies in displaying relevant data.
Innovation Solution
A system and method utilizing a camera-enabled mobile device that captures images at sufficient frequency, sends them to a server for analysis, and displays additional information synchronized with image capture, using optical flow algorithms to track object locations and aggregate data across frames for optimal annotation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If images are captured at high frequency to enable real-time augmented reality display, then the responsiveness and user interaction quality improve, but the processing load and system complexity increase
Solution Approach 1:
The patent extracts the image analysis and object identification tasks from the mobile device and relocates them to a remote server. The mobile device only captures images and transmits them to the server, which performs the computationally intensive processing and returns annotated results. This extraction reduces the processing load and complexity on the mobile device while maintaining high capture rates for real-time augmented reality display.
2Loss of information
If additional information is retrieved and displayed for every detected object, then the information completeness and user utility improve, but the display complexity and processing time increase
Solution Approach 1:
The patent applies partial action by having the server retrieve and process additional information only for objects that are detected and deemed relevant in the captured images. Instead of processing all possible objects or performing exhaustive searches, the system focuses on objects of interest identified through image analysis, thereby reducing unnecessary processing time while maintaining information completeness for relevant objects.
Solution Approach 2:
The server acts as an intermediary between the mobile device and the information retrieval systems. It receives images from the mobile device, performs object detection and information aggregation, filters relevant information, and returns annotated results. This intermediary role reduces the processing burden on the mobile device and optimizes the timing of information retrieval to minimize overall processing time.
3Measurement precision
If object tracking and location aggregation are performed across multiple frames, then the identification accuracy and annotation quality improve, but the computational load and processing delay increase
Solution Approach 1:
The system performs preliminary object detection and location identification in each individual frame as it is captured, rather than waiting to aggregate data from multiple frames before identifying objects. This preliminary action in each frame enables the system to maintain identification accuracy while reducing the overall processing delay, as the server can start annotating objects as they are detected across sequential frames rather than waiting for complete frame sequences.
Data Source
AI summary
A system and method of identifying objects is provided. In one aspect, the system and method includes a hand-held device with a display, camera and processor. As the camera captures images and displays them on the display, the processor compares the information retrieved in connection with one image with information retrieved in connection with subsequent images. The processor uses the result of such comparison to determine the object that is likely to be of greatest interest to the user. The display simultaneously displays the images the images as they are captured, the location of the object in an image, and information retrieved for the object.