Low Latency Streaming Media via Playlist Annotation and HTTP/2

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing streaming media content delivery systems face challenges in delivering live events with low latency, resulting in significant delays, often exceeding 15 seconds, which is disappointing for users.

Innovation Solution

The proposed solution involves updated playlist requests from client devices that include requests for future media segments, allowing servers to block responses until the segments are available, and using HTTP/2 for efficient transmission, enabling simultaneous requests for updated playlists and media segments to reduce latency. This approach maintains socket connections open for continuous data transfer and employs techniques like concurrent media segment requests and rendition switching to minimize delays.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If conventional HTTP compliant requests are used to retrieve media segments from servers, then the system can deliver streaming media content, but the latency relative to live events becomes large (15 seconds or more)

Engineering Contradiction:
ImprovelatencyVSAvoiddelivery speed
Core Design Contradiction:
Loss of timeVSProductivity

Solution Approach 1:

The client device sends a playlist request that includes a request for a future media segment before that segment is available. The server blocks the response until the future segment becomes available, then pushes it to the client. This preliminary action eliminates the need for separate requests and reduces latency.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system combines the playlist request and the future media segment request into a single HTTP request. This merging of requests eliminates round trip time between separate requests and allows the server to push both the updated playlist and the future media segment in one response.

Inventive Principle:
Principle #5Merging (Combining)

2Loss of time

If the server blocks responses until future media segments are available, then latency is reduced, but the complexity of the transfer protocol increases

Engineering Contradiction:
ImprovelatencyVSAvoidprotocol complexity
Core Design Contradiction:
Loss of timeVSDevice complexity

Solution Approach 1:

The HTTP/2 protocol is used to provide multiple functions within a single request framework. It enables both playlist retrieval and future media segment requests to be combined in one request, and supports server push functionality, reducing the need for separate protocols while maintaining low latency.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Reliability

If separate requests are made for updated playlists and media segments, then the system can handle available content, but round trip time increases latency

Engineering Contradiction:
Improvecontent delivery reliabilityVSAvoidround trip time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The client device combines the playlist request and the future media segment request into a single HTTP request. This merging eliminates the round trip time between separate requests while maintaining reliable content delivery through the server's blocking and push mechanisms.

Inventive Principle:
Principle #5Merging (Combining)

Data Source

PatentUS11757965B2Low latency streaming media
Publication Date: 2023.09.12 APPLE INC
  • US11757965B2 patent drawing
  • US11757965B2 patent drawing
  • US11757965B2 patent drawing

AI summary

Content streaming systems, such as systems that use HTTP compliant requests to obtain media segments for presentation of the content on a device. These content streaming systems can be optimized to reduce latency to a low level so that live events can be streamed to receiving devices in such a manner so that the time between an action in the live event and the presentation of the action on a receiving device that receives the streamed content is less than about 10 seconds. A client device can use rendition reports to tune-in to a new rendition (at a first bit rate) after presenting a prior rendition (of a second bit rate) when switching between the different bit rates; also, for example, a client device can use playlist annotations that indicate independent frames to avoid downloading depending frame media segments when switching between different renditions.