Chroma-Subsampled Video Processing Using Separate Luma and Chroma CNN Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI-based video processing methods are inefficient for chroma-subsampled video formats, as they require conversion to and from RGB 444, increasing computational requirements and reducing accuracy.
Innovation Solution
The use of convolutional neural networks (CNNs) for separate convolutions and downsampling of luma and chroma channels in chroma-subsampled video, eliminating the need for RGB 444 conversion and enhancing processing efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If AI-based methods convert chroma-subsampled video to RGB 444 for processing, then processing can be performed using existing AI models, but computational requirements increase and processing efficiency decreases
Solution Approach 1:
The patent segments the video processing into separate luma and chroma channel processing streams. The luma channel is processed independently through convolution and downsampling, while chroma channels are processed separately. This segmentation allows the system to maintain chroma-subsampled format throughout processing without conversion to RGB 444, thereby improving processing efficiency while still achieving effective video compression through specialized neural network operations on each channel type.
2Adaptability or versatility
If chroma-subsampled video is converted to RGB 444 for AI processing, then standard AI models can be used, but processing accuracy decreases
Solution Approach 1:
The patent changes the processing parameters by operating directly in the chroma-subsampled color space rather than converting to RGB 444. The neural network processes video frames with their original luma and chroma channel structures, applying different convolution operations tailored to each channel type. This parameter change preserves the inherent efficiency of chroma subsampling while achieving effective compression through learned transformations in the native color space.
3Ease of manufacture
If conversion to RGB 444 is performed for AI-based video compression, then existing encoding frameworks can be utilized, but computational complexity increases
Solution Approach 1:
The patent extracts the chroma channels from the standard video processing pipeline and handles them separately from the luma channel. By taking out the chroma processing path and applying specialized convolution and downsampling operations specifically designed for chroma data, the system avoids the computational overhead of converting to and from RGB 444 format, thereby reducing overall computational complexity while maintaining implementation feasibility through modular neural network architecture.
Data Source
AI summary
Efficient processing of chroma-subsampled video is performed using convolutional neural networks (CNNs) in which the luma and chroma channels are processed separately. The luma channel is independently convolved and downsampled and, in parallel, the chroma channels are convolved and then merged with the downsampled luma to generate encoded chroma-subsampled video. Further processing of the encoded video that involves deconvolution and upsampling, splitting into two sets of channels, and further deconvolutions and upsampling is used in CNNs to generate decoded chroma-subsampled video in compression-decompression applications, to remove noise from chroma-subsampled video, or to upsample chroma-subsampled video to RGB 444 video. CNNs with separate luma and chroma processing in which the further processing includes additional convolutions and downsampling may be used for object recognition and semantic search in chroma-subsampled video.