Gaze-Based Audio Switching With 3D Sight-Line Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing media delivery systems face inefficiencies and resource wastage due to duplicate content streaming, audio interference, and cumbersome device switching when multiple users share media streams, leading to bandwidth consumption and decreased resolution.
Innovation Solution
A media application that modifies audio delivery based on user gaze detection, utilizing existing camera devices to create a 3D map of the environment, allowing accurate gaze tracking without additional equipment, and adjusting audio distribution accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If duplicate copies of audio/video streams are created and transmitted to multiple devices, then multiple users can consume media simultaneously, but network bandwidth is consumed and resolution of each stream is compromised
Solution Approach 1:
The patent creates a virtual copy of the audio stream for the second user by routing it through the first user's device rather than creating a duplicate physical stream. The server sends a single audio stream to the first user's device, which then relays it to the second user's device, eliminating the need for duplicate encoding and transmission.
Solution Approach 2:
The first user's device acts as an intermediary between the server and the second user's device. Instead of the server directly transmitting duplicate streams to both devices, the first device receives the stream and forwards it to the second device, reducing network bandwidth consumption.
2Adaptability or versatility
If duplicate copies of audio/video streams are created and transmitted to multiple devices, then multiple users can consume media simultaneously, but processing power of each device is consumed
Solution Approach 1:
The patent eliminates the need for the second user's device to decode a duplicate stream by having it receive the already-decoded audio stream from the first user's device. This reduces the processing power requirements of the second device.
3Ease of operation
If users switch devices to borrow headphones or connect headphones temporarily, then audio can be shared between users, but device switching is cumbersome and playback is delayed
Solution Approach 1:
The patent replaces the mechanical process of physically switching devices or connecting headphones with a digital solution. The audio stream is electronically routed through the network from the first user's device to the second user's device, eliminating the need for physical device switching or headphone connections.
4Ease of operation
If users switch devices to borrow headphones or connect headphones temporarily, then audio can be shared between users, but synchronization errors and transmission errors increase
Solution Approach 1:
The patent replaces the error-prone mechanical process of manually switching devices and connecting headphones with a reliable digital audio routing system. The audio stream is transmitted through the network with proper synchronization, eliminating manual errors and transmission errors associated with physical device switching.
Data Source
AI summary
Systems and methods of gaze-based switching of audio components of content streams are disclosed. A first audio/video stream is provided to a first device of a first user. A second audio/video stream is provided to a second device of a second user. Based on determining that the gaze of the second user is directed to a display of the first device, the system pauses the second audio/video stream at the second device and causes an audio portion of the first audio/video stream to be played at the second device. The gaze can be determined by maintaining and modifying a 3D gaze map of an environment indicating the real-time 3D locations of the user and target object of the user's gaze. The 3D gaze map is updated based on cross-referencing video feeds of position data of the user and the target object and of gaze data of the user captured by a plurality of camera devices.