Virtual-Channel Stream Switch for Scalable I/O Port Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional bus-based architectures and network-on-chip (NoC) solutions in hardware accelerators for learning/inference machines face scalability and efficiency issues due to bottlenecks, low link utilization, and increased latency, making them impractical for large-scale interconnectivity.
Innovation Solution
A stream switch employing virtual channels is introduced, utilizing a data router, configuration registers, and arbitration logic to manage bandwidth allocation and data streaming between processing elements, enabling efficient data transfer through virtual input and output channels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional bus-based architectures or network-on-chip solutions are used for interconnectivity in hardware accelerators, then basic data transfer functionality is achieved, but scalability is limited due to bottlenecks and low link utilization
Solution Approach 1:
The patent segments the communication channels into multiple virtual channels (VC0, VC1, VC2, etc.) within each physical link. This segmentation allows simultaneous data flows to use different virtual channels, eliminating bottlenecks and improving link utilization while maintaining scalability for large numbers of processing elements
Solution Approach 2:
The patent adds a virtual channel dimension to the traditional single physical link. By introducing virtual channels as an additional dimension, the system can support multiple concurrent data streams over the same physical infrastructure, thereby improving scalability and link utilization without requiring proportional increases in physical connections
2Adaptability or versatility
If conventional crossbar switches are used to improve interconnectivity, then data transfer capability is enhanced, but area and power consumption increase significantly
Solution Approach 1:
The stream switch enables a single physical link to serve multiple functions by implementing multiple virtual channels. Each virtual channel can handle different data streams independently, providing crossbar-like interconnectivity capabilities with shared physical resources, thereby reducing the area required compared to a full crossbar switch
Solution Approach 2:
The patent creates virtual copies of communication channels through virtual channels instead of physical copies. These virtual channels replicate the functionality of separate physical links without requiring duplicate physical infrastructure, reducing area and power consumption while maintaining high interconnectivity
3Area of stationary object
If network-on-chip solutions are implemented to reduce area, then device area is reduced, but latency increases due to additional routing hops
Solution Approach 1:
The stream switch ensures continuous data flow by allowing multiple virtual channels to operate simultaneously without blocking. Configuration data pre-establishes routing paths, and arbitration logic continuously allocates bandwidth to active virtual channels, eliminating idle time and reducing latency while maintaining compact area
Solution Approach 2:
The patent uses configuration data to pre-establish routing paths and mappings between virtual input channels and virtual output channels. This preliminary configuration enables direct routing decisions at the stream switch without complex runtime routing computations, reducing latency while maintaining a compact NoC structure
4Productivity
If more virtual channels are added to improve link utilization, then bandwidth efficiency increases, but device complexity increases
Solution Approach 1:
The stream switch implements self-service through automated arbitration logic that dynamically allocates bandwidth based on configuration data and active virtual channels. The system automatically manages the increased complexity of multiple virtual channels without requiring external control, maintaining high link utilization while keeping the control mechanism compact and integrated
Data Source
AI summary
A stream switch includes a data router, configuration registers, and arbitration logic. The data router has a plurality of input ports, each having a plurality of associated virtual input channels, and a plurality of output ports, each having a plurality of associated virtual output channels. The data router transmits data streams from input ports to one or more output ports of the plurality of output ports. The configuration registers store configuration data associated with the virtual output channels of the respective output ports of the plurality of output ports. The stored configuration data identifies a source input port and virtual input channel ID associated with the virtual output channel of the output port. The arbitration logic allocates bandwidth of the data router based on request signals associated with virtual input channels of the input ports and the configuration data associated with the virtual output channels.