Multi-threading Video Hardware Engine Controller
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video hardware engines face inefficiencies in multithread processing due to lack of real-time OS and limited on-chip memory, leading to lower performance in applications requiring high computation power, such as video surveillance and ultra-HD resolutions, especially in portable devices.
Innovation Solution
A video hardware engine with a controller and a video hardware accelerator unit that operates in both encode and decode modes, utilizing a multi-threading approach without an operating system, where threads communicate through shared memory and activation semaphores to process frames and attributes in parallel, optimizing video processing tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a simple sequential and non-pipelined model is used for video processing, then the device complexity is reduced and ease of operation is improved, but the productivity and performance are degraded
Solution Approach 1:
The video processing system is divided into multiple independent threads (first thread, second thread, third thread) that operate in parallel. Each thread handles specific processing tasks such as frame decoding, slice processing, and attribute processing, allowing simultaneous execution of multiple operations without increasing overall system complexity.
Solution Approach 2:
The system implements dynamic thread scheduling where threads can be activated and deactivated based on processing needs. The controller dynamically manages thread execution states, enabling the system to adapt between sequential and parallel processing modes as required by different video processing scenarios.
2Productivity
If multi-threading is implemented to improve productivity, then the processing performance is improved, but the device complexity increases due to lack of real-time OS
Solution Approach 1:
A dedicated controller acts as an intermediary between multiple processing threads and the video hardware engine. The controller manages thread activation, coordination, and synchronization without requiring a full operating system, thereby enabling multi-threading while controlling the increase in system complexity.
Solution Approach 2:
Each thread is designed to be self-contained with its own processing logic and state management. Threads independently handle their specific tasks (frame decoding, slice processing, attribute handling) and automatically synchronize through predefined protocols, reducing the burden on the controller and enabling scalable thread management.
3Productivity
If more on-chip memory is provided to support multi-threading, then the productivity is improved, but the device complexity and cost increase
Solution Approach 1:
The on-chip memory structures are designed to serve multiple threads simultaneously through shared access protocols. The same memory resources are universally utilized by different threads for storing intermediate results, attributes, and control data, eliminating the need for dedicated memory per thread and reducing overall memory complexity.
Solution Approach 2:
Multiple thread-specific data structures and buffers are merged into shared memory pools that are efficiently managed by the controller. This consolidation reduces the total memory footprint required for multi-threading while maintaining the parallel processing capabilities needed for high productivity.
Data Source
AI summary
A video hardware engine with multi-threading functionality is disclosed. The video hardware engine includes a video hardware accelerator unit and a controller. The controller is coupled to the video hardware accelerator unit. The controller operates in an encode mode and a decode mode. In the encode mode, the controller receives a plurality of frames and encode attributes associated with each frame of the plurality of frames. The encode attributes associated with a frame of the plurality of frames is processed to generate encode parameters associated with the frame. The video hardware accelerator unit is configured to process the frame based on the encode parameters to generate an output. The output of the video hardware accelerator unit is processed to generate a compressed bit-stream and an encode status. In decode mode, the controller receives a compressed bit-stream and decode attributes and generates a plurality of frames and a decode status.