Streaming AI Chip Buffer Unit for Matrix Concatenation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Streaming-based artificial intelligence chips experience low computational efficiency due to computation units with lower calculation parallelism performing calculations before those with higher parallelism, leading to idle operators and reduced efficiency.
Innovation Solution
A streaming-based computation circuit with multiple groups of computation units, where a buffer unit buffers and concatenates matrices outputted by computation units with lower parallelism to match the calculation parallelism of units with higher parallelism, allowing for consecutive output and improved operator utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If computation units with lower calculation parallelism perform calculations before those with higher parallelism, then the calculation order follows data dependency requirements, but the amount of data fed into high-parallelism units is less than their capacity, leading to idle operators and reduced computational efficiency
Solution Approach 1:
The buffer unit pre-processes data by concatenating multiple first matrices into a second matrix that matches the calculation parallelism of high-parallelism computation units. This preliminary action ensures that when data is fed to high-parallelism units, their operators are fully utilized without idle time, resolving the contradiction between maintaining calculation order and maximizing computational efficiency
Solution Approach 2:
The buffer unit acts as an intermediary between low-parallelism and high-parallelism computation units. It receives data from low-parallelism units, performs concatenation operations to reshape data dimensions, and outputs processed data that fully utilizes the capacity of high-parallelism units, thereby eliminating operator idle time while preserving calculation dependencies
2Productivity
If matrices are repeatedly retrieved from external storage for each calculation, then data can be accessed for multiple computations, but the repeated access increases power consumption and reduces processing speed
Solution Approach 1:
The buffer unit performs preliminary concatenation of multiple first matrices into a second matrix before they are needed for subsequent calculations. This pre-processing allows the same concatenated data to be reused across multiple computation operations without returning to external storage, thereby reducing power consumption and improving processing speed through data reuse
Solution Approach 2:
The buffer unit maintains a continuous supply of pre-processed concatenated matrices to computation units, eliminating interruptions caused by repeated external storage access. This continuity ensures that computation units can operate continuously at high speed without idle waiting periods, while avoiding the energy costs of repeated memory access operations
Data Source
AI summary
The present disclosure provides a streaming-based computation circuit, method and artificial intelligence chip. The computation circuit includes: multiple groups of computation units. Multiple groups of computation units include a first group of computation units and a second group of computation units, the second group of computation units is configured to output a first matrix after each calculation; and a buffer unit, configured to perform one or more first operation. The first operations include: buffering M first matrices consecutively outputted by the second group of computation units for M times, concatenating the M first matrices into a second matrix, the number of elements in the second matrix is not greater than the calculation parallelism of the first computation unit in the first group of computation units, and consecutively outputting the second matrix to the first computation unit for N times to perform N calculation.