Reactive Processing System for Augmented Reality Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality systems face latency issues due to continuous processing of input video facets, which hinders the realistic integration of computer-generated sound, video, and graphics into the output video stream.
Innovation Solution
A reactive processing system utilizing a dependency graph, expression templates, and an effects engine with data bindings and programmable video effects, allowing functions to be executed only when corresponding video input signal changes, thereby reducing unnecessary processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If continuous processing of video facets is performed to maintain realistic integration, then integration quality is improved, but latency increases
Solution Approach 1:
The system transitions from continuous periodic processing to event-driven periodic processing, where video facet processing is triggered only when changes are detected in input signals or data bindings. This maintains integration quality by ensuring processing occurs when necessary while reducing unnecessary periodic operations that contribute to latency.
Solution Approach 2:
The system dynamically changes the processing state based on parameter changes in input signals. When parameters such as video region coordinates, object attributes, or data binding values change, the system activates processing for affected facets. This parameter-driven approach ensures realistic integration only when input conditions warrant it, thereby reducing overall processing latency.
2Reliability
If all video facets are processed continuously regardless of change, then integration completeness is improved, but processing efficiency deteriorates
Solution Approach 1:
The system applies different processing strategies to different video facets based on their individual change states. Only facets associated with changed data bindings or input signals undergo processing, while unchanged facets are skipped. This localized processing approach maintains integration completeness for changed regions while dramatically improving overall processing efficiency by avoiding redundant operations on static regions.
Solution Approach 2:
The video processing is segmented into independent facets, each associated with specific data bindings. The system evaluates and processes only those facets whose associated data have changed, rather than treating the entire video stream as a single processing unit. This segmentation enables selective processing that preserves completeness where needed while enhancing efficiency by excluding unchanged segments.
3Stability of the object's composition
If video effects are updated every frame, then visual continuity is improved, but system performance deteriorates
Solution Approach 1:
Video effect updates are performed periodically only when triggered by detected changes in input signals or data bindings, rather than on every frame. This change-triggered periodic update mechanism maintains visual continuity by ensuring effects are updated when their underlying data changes, while avoiding unnecessary updates that would degrade system performance during static periods.
Data Source
AI summary
In one or more embodiments, one or more systems, methods, and/or processes may utilize a reactive processing system in an augmented reality (AR) system. The AR system may execute program instructions based on a region of an input video signal and produce an output video signal that includes a first output that includes a video effect from the program instructions. The AR system may determine if the region of the input video signal changes. If so, the program instructions may be executed based on the region of the input video signal and a change of the region of the input video and produce the output video signal that includes the second output that includes the video effect. If not, the output video signal that includes the first output that includes the video effect may continue to be produced.