Video Frame Segmentation for Adaptive Streaming Encoding

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing video streaming technologies face inefficiencies in resource utilization and processing load due to the need to maintain multiple encoding profiles for varying network conditions and device capabilities, leading to wastage of resources and abrupt changes in content streaming experiences.

Innovation Solution

The solution involves segmenting image and audio frames into portions, assigning tags for encoding characteristics like resolution and bitrate, and using machine learning to optimize encoding based on viewer cohorts, reducing storage, bandwidth, and processing requirements while ensuring seamless streaming.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If multiple encoding profiles are maintained for varying network conditions and device capabilities, then streaming quality adaptability is improved, but resource utilization deteriorates due to wastage of storage and processing

Engineering Contradiction:
Improvestreaming quality adaptabilityVSAvoidresource utilization
Core Design Contradiction:
Adaptability or versatilityVSLoss of energy

Solution Approach 1:

The patent segments video frames into multiple portions (e.g., foreground, background, text regions) and applies different encoding profiles to each segment. This allows the system to maintain adaptability across varying network conditions while reducing overall resource consumption by applying aggressive compression only to less important segments like backgrounds, rather than uniformly encoding entire frames at multiple quality levels.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

Different quality levels and encoding parameters are applied to different spatial regions within video frames. Critical regions (foreground objects, text) receive higher quality encoding while non-critical regions (backgrounds) receive lower quality encoding. This local differentiation maintains perceived streaming quality adaptability while significantly reducing the computational and storage resources needed compared to maintaining multiple complete encoding profiles.

Inventive Principle:
Principle #3Local quality

2Adaptability or versatility

If multiple encoding profiles are maintained for varying network conditions and device capabilities, then streaming quality adaptability is improved, but device complexity deteriorates

Engineering Contradiction:
Improvestreaming quality adaptabilityVSAvoidprocessing load
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

Video frames are divided into multiple segments with different importance levels, allowing the encoding system to focus computational resources on critical segments. This segmentation reduces the processing load on streaming devices while maintaining the ability to adapt quality based on network conditions, as the system only needs to intelligently encode important regions rather than processing entire frames at multiple quality levels.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements local quality differentiation where encoding parameters are optimized for specific regions of video frames. This approach reduces overall device complexity by eliminating the need to maintain and switch between multiple complete encoding profiles, while still providing adaptability through region-specific quality control that responds to network conditions.

Inventive Principle:
Principle #3Local quality

3Ease of operation

If uniform encoding is applied to all video frames, then processing simplicity is improved, but streaming quality deteriorates due to abrupt changes and inability to adapt to network conditions

Engineering Contradiction:
Improveprocessing simplicityVSAvoidstreaming quality
Core Design Contradiction:
Ease of operationVSReliability

Solution Approach 1:

The patent applies different encoding qualities to different spatial regions within video frames, with higher quality for important content (foreground, text) and lower quality for less important content (backgrounds). This local differentiation maintains processing simplicity by working within a single encoding pass while significantly improving streaming quality and reducing abrupt changes, as the encoder can smoothly adjust parameters for different regions rather than making abrupt switches between complete encoding profiles.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The encoding approach dynamically adjusts quality parameters for different frame segments based on content importance and network conditions. This dynamic adaptation within a unified encoding framework maintains processing simplicity while improving streaming reliability, as the system can respond to changing conditions by adjusting regional encoding parameters rather than requiring complex profile switching.

Inventive Principle:
Principle #15Dynamics

Data Source

PatentUS20230370683A1Method and system for providing encoded streaming content to content viewers
Publication Date: 2023.11.16 JIOSTAR INDIA PTE LTD
  • US20230370683A1 patent drawing
  • US20230370683A1 patent drawing
  • US20230370683A1 patent drawing

AI summary

Techniques for encoding video content are discloses. These techniques include generating an encoded video content corresponding to a first video content by encoding a plurality of image frames. This includes segmenting an image frame into image portions associated with a corresponding portion identifier, assigning a tag for each of the image portions based on: (i) information relating to a viewer for the video content or (ii) a corresponding image portion different from the respective image portion, where the tag relates to: (i) a resolution or (ii) a bitrate for encoding the respective image portion, and encoding each image portion based on the respective assigned tag. The techniques further include generating a record including the encoded video content and the plurality of portion identifiers and transmitting the encoded video content over a communication network. The record is used to reconstruct the first video content from the encoded video content.