Event-Based Video Streaming With Hybrid Camera Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing video transmission techniques face challenges in efficiently transmitting high-quality video data over communication channels due to limitations in data compression, computational complexity, and latency, especially in real-time scenarios, which are exacerbated by the increasing resolution of video recordings.

Innovation Solution

Combining high-quality visual cameras with event cameras to transmit image and event data, where event cameras capture scene changes, allowing for reduced data transfer rates and adaptive frame rates, and using neural networks to generate updated images based on this data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If high-resolution video data is transmitted, then video quality is improved, but data transmission requirements increase

Engineering Contradiction:

Improvevideo qualityVSAvoiddata transmission requirements

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent segments video information into two distinct types: full image data captured at lower resolution and event data capturing only changes in the scene. This segmentation allows the system to transmit comprehensive visual information without transmitting redundant static data, thereby maintaining high video quality while reducing overall data transmission requirements.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The event camera extracts only the essential changes in the scene (events) rather than transmitting complete high-resolution frames. By taking out only the dynamic portions of the video content and combining them with periodic full frames, the system achieves high-quality video reconstruction with significantly reduced data transmission.

Inventive Principle:

Principle #2Taking out (Extraction)

2Quantity of substance

If compression algorithms are applied, then data transmission requirements are reduced, but loss of data occurs

Engineering Contradiction:

Improvedata transmission requirementsVSAvoiddata loss

Core Design Contradiction:

Quantity of substanceVSLoss of information

Solution Approach 1:

The event camera inherently captures only changing portions of the scene, providing a form of intelligent compression without traditional lossy algorithms. The event data self-selects important information (changes) while naturally discarding redundant static information, achieving compression without the need for complex compression algorithms that cause data loss.

Inventive Principle:

Principle #25Self-service

3Productivity

If look-ahead algorithms are used, then compression efficiency is improved, but latency increases

Engineering Contradiction:

Improvecompression efficiencyVSAvoidlatency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent replaces traditional mechanical look-ahead compression algorithms with an event-based sensing approach. The event camera provides real-time change detection without requiring buffering or look-ahead processing, thereby achieving efficient compression while maintaining minimal latency suitable for real-time transmissions.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Measurement precision

If high-quality video analysis is performed, then video quality is maintained, but computational cost increases

Engineering Contradiction:

Improvevideo qualityVSAvoidcomputational cost

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system extracts only the essential dynamic information through event cameras, significantly reducing the amount of data requiring computational analysis. By processing event data instead of complete high-resolution video frames, the computational cost and energy consumption are dramatically reduced while maintaining the ability to reconstruct high-quality video.

Inventive Principle:

Principle #2Taking out (Extraction)

5Quantity of substance

If event cameras are used, then data transmission requirements are reduced, but device complexity increases

Engineering Contradiction:

Improvedata transmission requirementsVSAvoiddevice complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent merges the outputs of two different camera types (traditional image camera and event camera) into a unified video stream. This combination allows the system to leverage the strengths of both technologies while managing complexity through integrated processing that reconstructs high-quality video from the complementary data sources.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS12579809B2Information processing devices, information processing methods and video streaming system

Publication Date: 2026.03.17 SONY GROUP CORP

AI summary

An information processing device, including circuitry configured to:obtain image data representing an image of a scene;obtain event image data representing a change in the scene after the image is captured; andtransmit the image data and the event image data via a communication channel to a second information processing device for generating updated image data representing an updated image of the scene.