Offloading Image Tracking to Hardware Accelerator

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional image-based tracking systems in augmented and virtual reality require significant processing and memory resources, diverting these resources away from other essential operations like rendering environments and processing audio/video data, thus impacting the overall user experience.

Innovation Solution

Offloading image-based tracking operations from a general processing unit to a hardware accelerator unit, such as a digital signal processor (DSP) or application-specific integrated circuit (ASIC), which generates a multi-scale representation of imaging data and executes tracking operations like patch matching and corner detection, allowing the general processing unit to focus on other tasks.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If image-based tracking operations are performed on a general processing unit, then tracking accuracy is maintained, but power consumption increases and other operations (rendering, audio/video processing) are impaired

Engineering Contradiction:

Improvetracking accuracyVSAvoidpower consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system segments tracking operations into two parts: feature detection/correspondence matching (computationally intensive) handled by the hardware accelerator, and higher-level tracking logic (computationally lighter) handled by the general processing unit. This segmentation allows the hardware accelerator to handle the power-intensive tasks while maintaining tracking accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A dedicated hardware accelerator unit acts as an intermediary between the imaging device and the general processing unit. This intermediary handles the computationally intensive image processing tasks, freeing the general processing unit for other operations and reducing overall power consumption while maintaining tracking functionality.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If image-based tracking operations are performed on a general processing unit, then full processing capability is available, but other operations (rendering, audio/video processing) suffer from resource allocation

Engineering Contradiction:

Improveprocessing capabilityVSAvoidoverall system performance

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The system divides processing responsibilities between two units: the hardware accelerator handles image processing tasks (feature detection, matching), while the general processing unit handles higher-level tracking logic and other system operations. This segmentation enables parallel processing and improves overall system productivity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The hardware accelerator serves as a specialized intermediary that handles computationally intensive image processing tasks, allowing the general processing unit to focus on rendering, audio/video processing, and other operations that contribute to overall system performance.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Measurement precision

If dedicated processing resources are allocated to image-based tracking, then tracking accuracy is improved, but resources for other operations are reduced

Engineering Contradiction:

Improvetracking accuracyVSAvoidresource allocation

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

A dedicated hardware accelerator unit is introduced as an intermediary component that specializes in image processing tasks. This adds a specialized component but reduces the burden on the general processing unit, allowing it to allocate resources more efficiently across multiple functions including rendering, audio/video processing, and tracking.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10916031B2Systems and methods for offloading image-based tracking operations from a general processing unit to a hardware accelerator unit

Publication Date: 2021.02.09 META PLATFORMS TECHNOLOGIES LLC

AI summary

The disclosed computer-implemented method for offloading image-based tracking operations from a general processing unit to a hardware accelerator unit may include (1) sending imaging data from an imaging device to a hardware accelerator unit, and (2) directing the hardware accelerator unit to generate a multi-scale representation of the imaging data sent from the imaging device, (3) preparing a set of input data for a set of image-based tracking operations, and (4) directing the hardware accelerator unit to execute the set of image-based tracking operations using the generated multi-scale representation of the imaging data and the prepared set of input data. Various other methods, systems, and computer-readable media are also disclosed.