Fragmented MP4 Video Streaming for HTML5 Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video streaming technologies face challenges in delivering real-time video streams to client devices that support playback of video files, particularly those using HTML5, as they do not natively support real-time live video protocols like RTMP or RTSP, leading to unwanted latencies and inefficiencies.
Innovation Solution
The system processes and delivers real-time video streams by converting them into file-based formats like MP4, fragmenting them, and transmitting these fragments to client devices for playback, using a video server with a relay service, fragmented video generator, and streaming service to ensure real-time playback, even on devices that initially buffer the content before displaying it.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If real-time live video protocols (RTMP/RTSP) are used for video streaming, then real-time video deliverance is achieved, but client devices that support only video file playback (like HTML5 video element) cannot be used
Solution Approach 1:
The video stream is divided into small fragmented segments (e.g., 100ms chunks) that are delivered sequentially to the client. Each fragment is a complete MP4 file containing a moov atom followed by moof and mdat atoms, allowing the client to process and playback small portions of video at a time while maintaining real-time delivery to compatible devices
Solution Approach 2:
The system changes the delivery format parameter from continuous stream protocols (RTMP/RTSP) to file-based format (fragmented MP4). This parameter change allows HTML5 video elements to accept the content as valid video files while the fragmentation ensures real-time delivery characteristics are maintained
2Adaptability or versatility
If adaptive streaming protocols (HLS/MPEG-DASH) are used to support live streaming into HTML5 video element, then client device compatibility is achieved, but real-time video deliverance is not maintained and unwanted latencies are introduced
Solution Approach 1:
The video is segmented into small fragmented MP4 files with controlled duration (e.g., 100ms per fragment). This segmentation allows the HTML5 video element to process video in real-time manageable chunks rather than waiting for large segment files, significantly reducing the latency inherent in traditional HLS/MPEG-DASH approaches while maintaining compatibility
Solution Approach 2:
The system implements dynamic fragment delivery where fragments are pushed to clients in real-time as they are encoded, with the fragmentation structure adapting to network conditions and client playback capabilities. This dynamic approach allows real-time delivery to HTML5 elements without the fixed-latency structure of traditional adaptive protocols
3Speed
If video is buffered initially and played back at maximum speed without displaying to user, then real-time playback is achieved after buffering, but initial latency is introduced
Solution Approach 1:
The client performs preliminary buffering of a small number of video fragments (e.g., first 3-5 fragments of 100ms each) before initiating playback. This preliminary action ensures the video element has enough data to start playback immediately at maximum speed without stalling, while keeping the initial delay minimal due to the small buffer size required
Solution Approach 2:
While the initial buffer plays through at maximum speed, the system continuously receives and processes new video fragments in the background. This continuous action ensures that after the initial buffer is consumed, playback transitions seamlessly to real-time delivery without interruption or additional latency, maintaining continuous useful action throughout
Data Source
AI summary
Real-time video streaming is essential in many types of systems. A system for streaming real-time video provides a low-latency solution allows systems and humans to respond to real-time events as they occur. Client application platforms, such as HTML5, may be used for web development; however, support for real-time video is very limited. The system provides the ability to stream real-time video from an originating video source to an HTML5 web application with low latency. A real-time video server includes a relay service configured to receive a video stream from a video source, a fragmented video generator configured to convert the video stream into a file-based video compression format including a plurality of video fragments, and a streaming service configured to transmit the fragmented video file to a video client for real-time playback.