Interleaved Video Stream Encoding for Standard Decoder Interoperability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in producing a combined video stream from separate video streams captured by multiple image sensors, particularly when using codecs like H.264 and MJPEG, as they may encounter issues with interoperability and efficient data handling.
Innovation Solution
A digital video camera system that includes multiple image sensors and encoders, where each image frame is encoded into slices, and a data combining unit interleaves these slices to form a combined image frame, which can be decoded by a standard decoder, and further optimizes by replacing subsequent skip encoded slices with a new slice to reduce metadata and file size.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If separate video streams from multiple image sensors are produced and handled separately, then each image sensor can be processed independently with standard encoders, but the combined video stream cannot be decoded by standard decoders and interoperability issues arise
Solution Approach 1:
The patent divides each encoded video stream into multiple slices, where each slice corresponds to a specific image sensor's data. This segmentation allows independent processing of each sensor while maintaining the ability to reassemble them into a format compatible with standard decoders. The slices are then combined in a specific order and format that standard decoders can interpret, resolving the interoperability issue.
Solution Approach 2:
The patent introduces a data combining unit as an intermediary component that receives slices from multiple encoders and reformats them into a combined video stream. This intermediary layer translates the separate sensor formats into a unified format that standard decoders can process, enabling interoperability without requiring changes to the encoders or decoders themselves.
2Area of stationary object
If multiple encoders process image data from different image sensors, then comprehensive scene coverage is achieved, but the data volume and processing complexity increase significantly
Solution Approach 1:
The patent segments the video stream data into slices that can be independently processed by separate encoders for different image sensors. Each encoder only processes its designated sensor's data, reducing individual processing complexity while maintaining comprehensive scene coverage through the combination of multiple segmented streams.
Solution Approach 2:
The patent merges the output of multiple encoders into a single combined video stream through a data combining unit. This merging process consolidates the data from multiple sensors while managing the complexity centrally, allowing the system to handle comprehensive scene coverage through a unified processing architecture rather than requiring complex independent processing for each sensor.
3Reliability
If skip encoded coding units are handled as individual slices, then decoding flexibility is maintained, but metadata redundancy increases and file size grows
Solution Approach 1:
The patent merges multiple consecutive skip encoded coding units into a single slice structure. By combining these redundant skip units, the system maintains decoding flexibility (as skip encoding preserves the ability to decode efficiently) while eliminating repetitive metadata headers that would otherwise be present for each individual skip unit, thereby reducing overall file size.
Solution Approach 2:
The patent discards redundant metadata information that would be repeated for each individual skip encoded unit, while recovering the essential decoding functionality through the combined slice structure. This approach removes unnecessary data (redundant metadata) while preserving the functional requirements (decoding flexibility).
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
The present invention relates to a method for forming of a combined image frame of a combined video stream. The method comprising: capturing image frames of a first video stream; capturing, image frames of a second video stream; encoding image data of the image frames of the first video stream, wherein each image frame of the first video stream is encoded into first encoded data comprising a plurality of rows wherein each row has a height of a single coding unit and a width equal to a width of the image frame and is encoded as one or more slices; encoding image data of the image frames of the second video stream, wherein each image frame of the second video stream is encoded into second encoded data comprising a plurality of rows wherein each row has a height of a single coding unit and a width equal to a width of the image frame and is encoded as one or more slices; and combining the first encoded data of an image frame of the first video stream and second encoded data of an image frame of the second video stream into combined encoded data by interleaving rows of the first encoded data of the image frame of the first video stream and rows of the second encoded data of the image frame of the second video stream. The combined encoded data representing the combined image frame of the combined video stream. Also, a digital video camera configured to form the combined image frame of the combined video stream is presented.