FPGA Shader GPU Applets for Flexible Multiviewer Graphics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional GPUs on FPGAs are hard-coded and inflexible, consuming FPGA resources and limiting scalability, as they can only process predefined input commands and data, making them unsuitable for flexible video processing in multiviewer systems.
Innovation Solution
An FPGA with a configurable graphical processing unit (GPU) that uses applets to execute drawing commands, allowing flexible and efficient graphical processing without occupying excessive FPGA resources, capable of handling multiple video streams and graphical elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a hard-coded GPU is implemented on an FPGA, then graphical processing functionality is provided, but FPGA resources are consumed and flexibility is reduced
Solution Approach 1:
The patent implements a dynamic GPU architecture where the shader device can be reconfigured by loading different software applets into working memory. This allows the same hardware resources to perform different graphical processing functions dynamically, rather than being statically dedicated to a single function. The shader device transitions between different processing modes by loading appropriate applets, providing flexibility without requiring multiple dedicated hardware units.
Solution Approach 2:
The shader device is designed as a universal processing unit that can execute multiple types of graphical processing tasks through software applets. A single shader device can perform vertex shading, fragment shading, geometry processing, and other graphical functions by loading different applets, replacing the need for multiple specialized hard-coded GPU units and reducing overall FPGA resource consumption.
2Productivity
If multiple dedicated GPUs are implemented for different drawing commands, then specific graphical functions are optimized, but FPGA resources are excessively occupied
Solution Approach 1:
The patent merges multiple dedicated GPU functions into a single shader device that can execute different software applets. Instead of having separate hard-coded hardware units for vertex processing, fragment processing, and geometry processing, a single unified shader device performs all these functions by loading appropriate applets, thereby consolidating FPGA resource usage while maintaining processing capabilities.
Solution Approach 2:
The shader device changes its operational parameters and processing behavior by loading different software applets with varying instructions and configurations. The same physical hardware unit adapts its function by changing the software parameters (applets) it executes, allowing it to optimize for different graphical processing tasks without requiring dedicated hardware for each task type.
3Adaptability or versatility
If a software-programmable CPU is used for video processing, then flexibility is achieved, but processing performance is insufficient
Solution Approach 1:
The patent replaces the mechanical/software-based CPU processing system with a hardware-accelerated shader device that executes software applets. This substitution provides both the flexibility of software programming and the high-speed performance of hardware execution. The shader device is implemented in FPGA logic, providing hardware-level speed while maintaining software-like reconfigurability through applet loading.
4Speed
If the GPU operates at high clock rate to process multiple frames, then frame processing speed increases, but processing time per frame decreases
Solution Approach 1:
The shader device operates continuously at high clock rates, processing multiple video frames without interruption. By maintaining continuous high-speed operation and efficiently managing applet execution across frames, the system achieves high overall throughput. The device can execute applets for multiple frames back-to-back, ensuring that the useful processing action continues without idle periods, maximizing the utilization of the high clock rate capability.
Data Source
Figure 1~2
Figure 3A~3B
Figure 4~5
AI summary
An FPGA for video processing including a graphical processing unit. The FPGA comprises an interface (206) for receiving drawing commands, a VRAM (207) for storing video data, a shader device (301) configured to load a software applet into a working memory for executing the drawing commands. The shader device generates graphical elements that are written into the VRAM. The FPGA further comprises an output (303) for providing a composed video output signal including at least one video stream and at least one graphical element associated with the video stream(s). The FPGA can be implemented in a video processing device, such as a multiviewer system.