Applet-Driven FPGA GPU for Flexible Multistream Video Graphics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing GPUs on FPGAs are hard-coded and inflexible, consuming FPGA resources and limiting scalability, as they can only process predefined input commands and data, making them unsuitable for scalable video processing in broadcast environments.
Innovation Solution
An FPGA with a configurable graphical processing unit (GPU) that uses applets to execute drawing commands, allowing flexible and hardware-accelerated graphics generation without occupying excessive FPGA resources, featuring a shader device with an execution unit and scratch pad memory for applet management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a hard-coded GPU is implemented on FPGA, then graphical processing capability is provided, but FPGA resources are consumed and scalability is limited
Solution Approach 1:
The GPU functionality is made dynamic through software applets that can be loaded and unloaded from external memory. The shader device can execute different applets to perform various graphical processing tasks, transforming the static hard-coded GPU into a dynamic, reconfigurable system that adapts to different processing needs without permanently occupying FPGA resources.
Solution Approach 2:
An intermediary software layer (applets) is introduced between the hardware shader device and the graphical processing tasks. These applets act as mediators that translate high-level graphical commands into hardware-executable instructions, enabling flexible graphical processing without hard-coding specific functions into the FPGA fabric.
2Productivity
If a hard-coded GPU is implemented on FPGA, then predefined graphical processing is enabled, but adaptability to different commands is reduced
Solution Approach 1:
The system dynamically loads different software applets into the shader device's working memory to handle various graphical processing commands. This dynamic approach maintains high processing speed by executing optimized applets while providing adaptability to different command types without requiring a completely different hardware design.
Solution Approach 2:
The graphical processing capability is changed from fixed hardware parameters to software-configurable parameters. By loading different applets with varying instructions and parameters, the same hardware shader device can adapt to different graphical processing requirements while maintaining optimal performance.
3Power
If FPGA resources are allocated for a fixed function GPU, then graphics processing is hardware accelerated, but resources are unavailable for video stream processing
Solution Approach 1:
The shader device provides dynamic hardware acceleration by loading applets only when graphical processing is needed. When graphics processing is not required, the same FPGA resources can be reallocated for video stream processing, enabling time-multiplexed resource utilization that provides both hardware acceleration and resource flexibility.
Solution Approach 2:
The shader device is designed as a universal processing unit that can execute different types of operations through software applets. It serves both graphical processing functions and general-purpose video stream processing tasks, eliminating the need for dedicated fixed-function hardware blocks for each task type.
Data Source
AI summary
A Field Programmable Gate Array (FPGA) for video processing including a graphical processing unit. The FPGA comprises an interface for receiving drawing commands, a VRAM for storing video data, a shader device configured to load a software applet into a working memory for executing the drawing commands. The shader device generates graphical elements that are written into the VRAM. The FPGA further comprises an output for providing a composed video output signal including at least one video stream and at least one graphical element associated with the video stream(s). The FPGA can be implemented in a video processing device, such as a multiviewer system.