Graph-Based Heterogeneous Parallel Processing System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image and video processing systems face inefficiencies due to lack of flexibility and programmability, leading to suboptimal performance, power usage, and data loss, despite attempts to achieve high parallelism using various hardware approaches like ASICs, FPGAs, and GPUs, which often require complex design skills and result in communication bottlenecks.
Innovation Solution
A system and method that interconnects heterogeneous compute resources using a user-programmable graph-based data flow, enabling time multiplexing and context switching of compute resources, with message-based routing to facilitate seamless reuse of compute engines and DMA resources, allowing for high parallel processing with minimal programming overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If ASICs or FPGAs with hardcoded data processing structures are used to achieve high parallelism, then processing performance is improved, but flexibility and programmability are lost
Solution Approach 1:
The patent implements a universal data processing structure that can be configured to perform multiple different image and video processing functions. The homogeneous processing units can be programmed to execute various operations (convolution, pooling, activation functions, etc.) through a unified instruction set architecture, allowing the same hardware to serve multiple purposes without requiring dedicated customized circuits for each function.
Solution Approach 2:
The system employs dynamic configuration capabilities where the data processing structures can be reprogrammed and reconfigured at runtime to adapt to different processing tasks. The instruction memory and control logic allow the hardware to dynamically change its behavior based on the specific processing requirements, enabling flexibility while maintaining high-performance parallel execution.
2Adaptability or versatility
If multiple SIMD or VLIW processors are used to achieve parallelism, then programmability is improved, but communication overhead and programming complexity increase causing performance bottlenecks
Solution Approach 1:
The patent merges multiple processing units into a unified homogeneous array that shares common control logic, instruction memory, and data pathways. This consolidation reduces the number of independent communication channels needed compared to heterogeneous multi-processor systems, simplifying the inter-processor communication infrastructure while maintaining high parallelism through the unified architecture.
Solution Approach 2:
The system uses homogeneous processing units with identical instruction set architectures and uniform data pathways throughout the array. This homogeneity simplifies programming by providing consistent access patterns and uniform communication protocols, eliminating the need for complex heterogeneous programming models and reducing communication overhead through standardized interfaces.
3Productivity
If fixed-function pipeline blocks are used for specific tasks, then image/video specific parallelism is improved, but flexibility to add heterogeneous compute engines is reduced
Solution Approach 1:
The patent segments the image and video processing functions into distinct programmable stages within the homogeneous processing array. Each processing unit can be independently configured to perform specific operations (filtering, transformation, feature detection) while remaining part of the unified reconfigurable system, allowing optimization for specific tasks without sacrificing overall flexibility.
Solution Approach 2:
The system achieves task-specific optimization by changing operational parameters and configuration settings of the homogeneous processing units rather than using fixed-function hardware. Through parameter configuration and instruction programming, the same hardware structure can be adapted to perform different image/video processing functions, maintaining flexibility while achieving high parallelism for specific applications.
Data Source
AI summary
A system and method to interconnect heterogeneous compute resources using graph based execution flow for image and video processing is disclosed. The system includes a mix of compute engines (SIMDs, fixed-function pipelines and hardware kernels) which work on pipelined streams of data based on programming instructions and/or configuration data. The nodes act as proxies for compute engines in graph and message based routing system to establish graph based flow control mechanism between these nodes. A message based routing mechanism to assign tasks to heterogeneous compute engines and receive their status in time-multiplexed manner thus allowing compute engines to be reused. A configuration method that allows hardware compute engines are uniquely configured based on graph node thus allows reusability and programming or selecting heterogeneous compute system for physical implementation using simple standard definition file generation method.