Inter-Predicted Block Processing With STRN Neighborhood Signals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in efficiently processing inter-predicted blocks due to sequential dependency on reconstructed neighboring samples, leading to high computational complexity and decreased coding efficiency, particularly when using convolutional neural networks (CNNs) for inter prediction.
Innovation Solution
The proposed spatio-temporal residual network (STRN) decouples the processing of current blocks from spatial dependencies by generating a neighborhood signal independent of spatial interdependencies, allowing parallel processing of multiple blocks and using polyphase decomposition to reduce computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If convolutional neural networks are used for inter prediction processing, then coding performance is improved, but computational complexity increases
Solution Approach 1:
The patent applies polyphase decomposition to segment the input signal into multiple polyphase components, each processed by separate convolutional neural network filters. This segmentation allows the complex prediction task to be divided into manageable parts, reducing the computational burden on any single processing unit while maintaining the overall coding performance benefits of CNN-based prediction.
2Reliability
If sequential processing is used for inter-predicted blocks, then dependency on reconstructed samples is satisfied, but processing time increases
Solution Approach 1:
The patent generates a neighborhood signal in advance that captures spatial neighborhood information without requiring sequential processing of all blocks. This preliminary preparation of neighborhood data allows the convolutional neural network to process inter-predicted blocks in parallel, satisfying dependency requirements while significantly reducing processing time through advance computation.
3Measurement precision
If neighborhood signal is generated using reconstructed samples, then spatial information is captured accurately, but processing dependencies increase
Solution Approach 1:
The patent introduces a neighborhood signal as an intermediary that mediates between the requirement for accurate spatial information and the need to reduce processing dependencies. This intermediate representation captures essential spatial neighborhood characteristics while being generated in a way that decouples the processing of current blocks from the sequential reconstruction of neighboring samples, thus reducing processing dependencies.
Data Source
AI summary
Video decoder and encoder using a neighborhood signal generated by using a contribution signal in a version not post-processed and/or substituting a contribution signal by a substitute signal generated independent from spatial signal-interdependencies. Picture-processing tool configured to polyphase-wisely split luma samples and subject a tensor of cascaded matrices of the polyphase-components to a neural network or a convolution. Video decoder and encoder applying a post-processing only to certain inter-predicted blocks.