Gaze Point Detector Pixel Readout Priority for Latency Reduction

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing imaging technologies experience significant delays between image capture and display due to increased data processing requirements, leading to impaired realistic sensation and user stress in real-time applications like augmented and virtual reality.

Innovation Solution

An electronic device with a control circuit, readout processing circuit, and output circuit that determines an area of interest in an imaging plane and prioritizes the reading and processing of pixel data based on the user's gaze point, allowing for efficient data output and reduced latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the amount of data of the original captured image is increased to improve image processing accuracy and display definition, then image quality is improved, but processing delay increases significantly

Engineering Contradiction:
Improveimage processing accuracyVSAvoidprocessing delay
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The imaging plane is divided into multiple regions including a first region (area of interest) and a second region. Different readout priorities are assigned to different regions, allowing the system to process high-priority areas faster while still capturing lower-priority areas, thus reducing overall processing delay while maintaining image quality

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

Different regions of the imaging plane are assigned different qualities or priorities. The area of interest (first region) receives higher priority for readout and processing, ensuring that critical image data is processed quickly with high accuracy, while other regions are processed with lower priority, balancing image quality and processing speed

Inventive Principle:
Principle #3Local quality

2Manufacturing precision

If the amount of data of the original captured image is increased to improve display definition, then display quality is improved, but transmission and processing bandwidth requirements increase, causing delay

Engineering Contradiction:
Improvedisplay definitionVSAvoidprocessing speed
Core Design Contradiction:
Manufacturing precisionVSProductivity

Solution Approach 1:

The image data is segmented into high-priority (first region) and low-priority (second region) portions. The high-priority region is read out and transmitted first, ensuring that critical display information is delivered quickly, while the low-priority region follows, improving overall processing speed without sacrificing display definition

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The control circuit determines the area of interest and establishes readout priorities before the actual image capture and processing begins. This preliminary configuration allows the system to optimize data flow and processing sequence in advance, reducing real-time processing delays while maintaining high display definition

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS11190714B2Electronic device, head-mounted display, gaze point detector, and pixel data readout method
Publication Date: 2021.11.30 SONY INTERACTIVE ENTERTAINMENT LLC
  • US11190714B2 patent drawing
  • US11190714B2 patent drawing
  • US11190714B2 patent drawing

AI summary

In a plane of a captured image 202, a gaze-corresponding point 204 corresponding to a user's gaze point with respect to a displayed image is identified. Then, an area of interest 206 is determined based on the identified gaze-corresponding point 204. A space imaging section determines rows or pixel blocks in an imaging plane 208 that include the area of interest 206 and reads out and outputs data in pixel columns included in the rows or pixel blocks prior to other areas.