ML-Accelerator SoC Video Decoding for Format-Matched Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning-accelerators face inefficiencies due to mismatched video decoder output and input formats, leading to significant drops in inference per second efficiency, particularly in video/imaging centric applications, and require a next-generation system-on-chip solution with an on-chip video decoder to address these issues.
Innovation Solution
A machine learning-accelerator system-on-chip (SoC) architecture that includes ML-accelerator cores, a network-on-chip (NoC), an inference video post-processing module, and a video decoder, supporting specified video/image processing, format conversion, data scaling, and data packing without performance drops, utilizing a packed dense depth data dump (D4) format for efficient data handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If video decoder output formats are used directly for ML inference, then hardware acceleration is achieved, but format mismatch between video decoder and ML model inputs causes significant drop in inference efficiency
Solution Approach 1:
The patent merges the video decoder and ML accelerator into a single integrated SoC chip, with the video decoder outputting data directly in the format required by the ML accelerator. This eliminates the need for separate format conversion operations and reduces the computational burden on the ML accelerator, thereby maintaining high inference efficiency while handling format mismatches internally.
Solution Approach 2:
The patent introduces an intermediary buffer/memory structure between the video decoder and ML accelerator that handles format conversion. This buffer receives video decoder output in one format and stores it in the format expected by the ML accelerator, acting as a mediator that prevents direct format mismatch without requiring real-time conversion during inference.
2Adaptability or versatility
If format conversion, data scaling and data packing are executed on ML accelerator, then video/imaging centric applications are supported, but considerable computation resources are consumed leading to performance drop
Solution Approach 1:
The patent performs format conversion, data scaling and data packing operations before the data reaches the ML accelerator, ideally in a dedicated preprocessing stage or within the video decoder itself. This preliminary action ensures that when data enters the ML accelerator, it is already in the correct format, minimizing the computational resources required during the actual inference process.
Solution Approach 2:
The patent extracts the format conversion and data preparation functions from the ML accelerator's primary inference workload. By separating these tasks into dedicated hardware modules or preprocessing stages, the ML accelerator can focus solely on inference operations, thereby maintaining high inference per second efficiency while still supporting video/imaging centric applications.
3Power
If multi-chip architecture is used with separate video decoder and ML accelerator, then hardware acceleration is achieved, but format mismatch requires additional processing steps reducing efficiency
Solution Approach 1:
The patent combines the video decoder and ML accelerator into a single integrated SoC chip, eliminating inter-chip communication overhead and reducing the time required for data transfer and format conversion. The integrated architecture allows for tighter coupling between the video decoder output and ML accelerator input, minimizing processing delays while maintaining hardware acceleration capabilities.
Data Source
AI summary
A machine learning (ML)-accelerator system-on-chip (SoC) is described. The ML-accelerator SoC includes a set of ML-accelerator cores. The ML-accelerator SoC also includes a network-on-chip (NoC) coupled to the set of ML-accelerator cores. The ML-accelerator SoC further includes an inference video post processing (infVPP) module coupled to the NoC. The ML-accelerator SoC also includes a video decoder coupled to the NoC.