NBMP Stream Splitting and Merging for Common Header Segments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The Network Based Media Processing (NBMP) standard fails to address media streams with common headers and start codes, limiting its ability to process media streams effectively.
Innovation Solution
The proposed solution involves extending the NBMP splitter and merger functions to support common headers and sequence numbering of metadata, allowing for the replication and handling of common headers in sub-streams and ensuring proper merging of segments using start codes and sequence numbers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the NBMP splitter and merger functions are used for parallel processing of media streams, then processing efficiency is improved, but the ability to handle media streams with common headers and start codes deteriorates
Solution Approach 1:
The patent segments the media stream processing by introducing a header extraction mechanism that separates common headers from the main segment data. The splitter function extracts and buffers common headers separately, allowing parallel processing of segment bodies while headers are handled independently and reassembled during merging, thus enabling both efficient parallel processing and proper header handling.
Solution Approach 2:
The patent introduces an intermediary buffer mechanism that temporarily stores extracted common headers and metadata. This buffer acts as a mediator between the parallel processing streams and the final merger function, allowing segments with common headers and start codes to be properly reassembled after parallel processing, thereby extending adaptability without compromising processing efficiency.
2Ease of operation
If the NBMP functions process media streams with clear boundaries, then processing simplicity is improved, but the capability to process media streams with headers at their beginnings deteriorates
Solution Approach 1:
The patent applies preliminary action by extracting and buffering common headers and metadata before the main parallel processing begins. The splitter function performs this extraction upfront, storing headers in a buffer, so that the subsequent parallel processing of segment bodies remains simple and straightforward, while the header handling is prepared in advance to accommodate streams with headers at their beginnings.
3Measurement precision
If timing metadata is used for finding segment boundaries, then boundary detection is improved, but compatibility with streams that have start codes deteriorates
Solution Approach 1:
The patent enhances the universality of the splitter function by making it capable of handling multiple types of segment boundary indicators. The function can now process both timing metadata-based boundaries and start code-based boundaries, as well as streams with common headers. This multi-functionality is achieved through the header extraction and buffering mechanism that works alongside the existing timing metadata processing, allowing the same function to serve multiple purposes.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and apparatus for processing a media stream including segments with common headers via a network based media processing (NBMP) function, performed by at least one processor. The method includes obtaining an input stream including one or more segments, identifying a start code in the segments, wherein the start code is present at a beginning of a segment and the start code is a unique code that indicates a start of the segment, identifying a sequence number in the segments, wherein the sequence number indicates a location of a segment in the segments, splitting the input stream into sub-streams and signaling a location of a respective common header in each of the sub-streams, based on the start code and the sequence number, and merging the sub-streams and signaling a location of a common header, based on the start code and the sequence number, and generate an output stream.