Multicast Master Port for Neural Network Data Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Integrated circuit devices face inefficiencies in data transfer due to the lack of multicast capabilities in existing bus protocols like AXI, leading to reduced memory bandwidth utilization and computational array underutilization, especially in neural network processing where data needs to be loaded simultaneously across multiple accelerators.
Innovation Solution
Implementing a communication fabric that enables multicast transactions and staggered data distribution across memory banks, using a master port with multicast address ranges to generate multiple transactions for multiple targets, and a target port that adjusts data placement within accelerators to optimize systolic array computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a transaction initiator generates a write transaction for each location that is to receive the data, then the data can be written to multiple targets, but the memory bandwidth utilization is reduced and the loading time is increased
Solution Approach 1:
The patent segments the data transfer process by introducing a multicast transaction mechanism that divides the data distribution task into a single multicast write operation instead of multiple individual write transactions. The master port generates one multicast transaction that is then distributed to multiple target ports, each handling a portion of the data distribution to different memory locations or accelerators, thereby reducing the total number of transactions and improving bandwidth utilization.
Solution Approach 2:
The patent merges multiple individual write transactions into a single multicast transaction. Instead of generating separate write transactions for each target location, the system combines these operations into one unified multicast transaction that carries the same data payload to multiple destinations simultaneously. This merging reduces transaction overhead and improves memory bandwidth utilization.
2Productivity
If data is loaded sequentially into multiple accelerators, then the system is simpler to implement, but the computational array underutilization occurs and accelerators are not kept busy
Solution Approach 1:
The patent implements preliminary action by pre-configuring the master port with multicast address ranges that identify multiple target accelerators before the data transfer begins. The system prepares the multicast transaction parameters in advance, including the list of target addresses and the data payload, so that when the transfer is initiated, all accelerators can begin loading data simultaneously without sequential delays. This keeps accelerators busy and maximizes computational array utilization.
3Adaptability or versatility
If standardized bus protocol is used, then the implementation is simplified and interface circuits can be reused, but multicast capabilities are not supported
Solution Approach 1:
The patent applies universality by designing the master port and target port interface circuits to handle both traditional unicast transactions and multicast transactions through the same physical interface. The ports are configured to recognize multicast address ranges and automatically route transactions appropriately. This multi-functionality allows the same hardware interface to support multiple communication modes without requiring separate dedicated circuits for multicast operations.
Data Source
AI summary
Provided are integrated circuit devices and methods for operating integrated circuit devices. In various examples, an integrated circuit device can include a master port operable to send transactions to a target components of the device. The master port can have point-to-point connections with each of the targets. The master port can be configured with a first address range for a first target, a second address range for a second target, and a multicast address range for both the first and second target. When the master port receive a request with an address that is in the multicast address range, the master port can generate, for the one request, a transaction for each of the first and second transactions.