Head-Mounted Passthrough and HDR Recording Without Frame Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Challenges exist in recording high dynamic range content with a wide depth of field in head-mounted devices without affecting the passthrough feed.
Innovation Solution
A method for operating a head-mounted device that includes outputting passthrough content at a passthrough frame rate, activating a recording mode to capture images at a recording frame rate, warping images based on device poses, and acquiring bracketed images with adjusted settings to produce enhanced recorded content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If bracketed images are acquired with adjusted image settings to achieve high dynamic range and wide depth of field, then recording quality is improved, but the passthrough feed may be affected due to shared hardware resources
Solution Approach 1:
The patent divides the image processing pipeline into separate pathways: one for passthrough content and another for recorded content. The image sensor captures raw images that are then routed to different processing streams, allowing bracketed images to be processed exclusively for recording without interfering with the passthrough feed.
Solution Approach 2:
The patent extracts the bracketed image acquisition and processing operations from the passthrough pipeline. By capturing bracketed images at specific intervals and processing them separately through a dedicated recording pipeline, the system achieves high dynamic range recording without affecting the continuous passthrough feed.
2Measurement precision
If images are warped based on device poses to maintain temporal alignment, then visual accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent performs warping operations on captured images to align them with the device pose at the time of display. By pre-warping images based on measured or predicted poses before composition, the system maintains temporal alignment without requiring complex real-time adjustments during playback.
Solution Approach 2:
The patent introduces pose information as an intermediary element that mediates between the captured image and the display timing. The warping operation uses pose data to transform images into the correct spatial configuration, simplifying the overall processing by decoupling image capture from display timing.
3Manufacturing precision
If multiple bracketed images are captured and processed to produce enhanced recorded content, then recording capability is improved, but frame rate may be reduced
Solution Approach 1:
The patent applies bracketed image capture selectively rather than continuously. Bracketed images are captured at specific intervals or under specific conditions (such as when scene complexity warrants enhanced dynamic range), allowing the system to maintain high frame rates for standard recording while occasionally enhancing quality when needed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A head-mounted device (10) is provided that includes a variety of subsystems for acquiring content, displaying the content, and recording the content. A method of operating the head-mounted device (10) can include outputting passthrough content at a passthrough frame rate with one or more displays (14), activating a recording mode to capture recorded content at a recording frame rate while the one or more displays (14) are outputting the passthrough content at the passthrough frame rate, acquiring an image with one or more image sensors (50), and warping the image to produce a plurality of warped images, corresponding to different points in time, that are used to generate the passthrough content. The method can further include acquiring bracketed images associated with the image, where the bracketed images are not displayed. The images being displayed as part of the passthrough content can be enhanced using information in at least some of the bracketed images.