Vector Computing Device Scratchpad Memory Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vector computing devices face performance bottlenecks due to inter-chip communication issues, insufficient on-chip cache, and inflexible support for varying vector lengths, particularly in executing large-scale vector operations.
Innovation Solution
A vector computing device with a storage unit, register unit, and vector operations unit that utilizes a Scratchpad Memory to temporarily store vector data, allowing for flexible support of different data widths and improving performance by arranging pipeline stages for efficient execution of vector operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple general-purpose processors are used for concurrent execution, then vector operation capability is improved, but intercommunication bottleneck occurs
Solution Approach 1:
The patent extracts the vector operation functionality from general-purpose processors into a dedicated vector processing unit. This separates the vector computation function from scalar processing, allowing vector operations to be performed in parallel without requiring intercommunication between multiple general-purpose processors, thereby eliminating the intercommunication bottleneck while maintaining high vector operation capability.
Solution Approach 2:
The vector processing unit is designed with a configurable register file that can be dynamically adjusted to support different vector lengths and data widths. This multi-functional design allows a single vector processing unit to handle various vector operation requirements without requiring multiple specialized processors, thus avoiding intercommunication bottlenecks while maintaining versatility.
2Adaptability or versatility
If GPU with small on-chip cache is used, then vector operation is supported, but off-chip bandwidth bottleneck occurs
Solution Approach 1:
The patent introduces a scratchpad memory as an intermediate storage layer between the vector processing unit and the main memory system. This adds a new memory dimension that is larger than GPU's on-chip cache but faster and more directly accessible than off-chip memory, allowing large-scale vector operations to be performed with reduced off-chip memory access requirements and improved bandwidth utilization.
Solution Approach 2:
The scratchpad memory acts as an intermediary storage buffer between the vector processing unit and the main memory system. It temporarily holds vector data during computation, reducing the frequency and volume of off-chip memory transactions. This mediator structure enables efficient large-scale vector operations without being constrained by off-chip bandwidth limitations.
3Productivity
If specialized vector operation device with customized register file is used, then vector operation performance is improved, but flexibility for different vector lengths is lost
Solution Approach 1:
The vector processing unit incorporates a configurable register file whose parameters (such as number of registers, register width, and associativity) can be dynamically adjusted based on the specific vector operation requirements. This dynamic configurability allows the system to optimize performance for different vector lengths and data widths while maintaining the high performance characteristics of a specialized vector processor.
Solution Approach 2:
The patent implements a register file with configurable parameters including register count, register width, and cache line size that can be modified according to the vector operation being performed. By changing these parameters dynamically, the vector processing unit maintains high performance across different vector lengths and data formats, resolving the contradiction between specialization and flexibility.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure relates to a vector computing device comprising a storage unit, a register unit and a vector operations unit, wherein vectors are stored in the storage unit, vector storage addresses are stored in the register unit, and the vector operations unit acquires a vector address from the register unit according to a vector operation instruction, and then acquires a corresponding vector from the storage unit according to the vector address, subsequently performing a vector operation according to the acquired vector so as to obtain a vector operation result. The device temporarily stores vector data involved in computation in a Scratchpad Memory, so that different data widths can be supported more flexibly and effectively during vector operations, and execution of computing tasks comprising a large number of vectors can be improved.