Configurable Vector Operation Units for Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vector operation technologies in deep learning, such as those using GPUs or CPUs, face limitations in execution performance and flexibility due to limited on-chip cache space, frequent data movement from off-chip memory, and the inability to support complex vector operations, which are divided into multiple basic operations, resulting in low performance and inefficiency.
Innovation Solution
A method for configuring vector operations by obtaining configurable parameters, allowing vector operation units to execute composite operations defined by these parameters, which can be scalar or vector typed, and storing them in dedicated memory to improve execution efficiency and support complex operations without relying on off-chip memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If vector operations are executed using traditional GPU or CPU architectures, then basic vector operations can be performed, but execution performance is limited due to frequent data movement from off-chip memory and inability to support complex vector operations
Solution Approach 1:
The patent implements a hierarchical memory architecture where on-chip cache memory is nested within the vector processing unit, and further nested within the GPU architecture. This nested structure allows vector operation units to access frequently used data from on-chip cache without leaving the processing unit, eliminating the time loss associated with off-chip memory access while maintaining support for complex vector operations.
Solution Approach 2:
The patent introduces a new dimension of memory hierarchy by adding on-chip cache memory at the vector processing unit level, transforming the traditional two-level memory architecture (on-chip/register and off-chip/memory) into a three-level architecture. This dimensional addition provides a intermediate storage layer that resolves the contradiction between fast access and large capacity.
2Adaptability or versatility
If complex vector operations are divided into multiple basic operations for execution, then the operations can be processed by existing hardware, but the flexibility and execution efficiency are reduced
Solution Approach 1:
The patent implements a universal vector processing unit that can execute both basic vector operations and complex vector operations natively. The vector operation unit is designed with configurable parameters that allow it to adapt to different operation types without requiring decomposition into basic operations, thereby maintaining high execution efficiency while supporting diverse vector operation types including complex operations.
Solution Approach 2:
The patent introduces dynamic configurability to the vector processing unit through configurable parameters that can be adjusted based on the specific vector operation being executed. This dynamic adaptation allows the same hardware unit to efficiently handle different operation types (basic and complex) by reconfiguring its internal logic and data flow paths, eliminating the need for operation decomposition.
3Device complexity
If on-chip cache space is limited, then the device complexity is reduced, but the ability to execute complex vector operations without off-chip memory access is compromised
Solution Approach 1:
The patent applies local quality optimization by allocating on-chip cache memory specifically to the vector processing unit where it is most needed for complex vector operations. Rather than providing uniform cache across the entire GPU, the patent concentrates cache resources locally at the vector processing level, achieving high execution capability for vector operations while maintaining overall device complexity at acceptable levels.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Embodiments of the present disclosure provide a method, executed by a computing device, for configuring a vector operation, an apparatus, a device, and a storage medium. The method includes obtaining information indicating at least one configurable vector operation parameter. The information indicating the at least one configurable vector operation parameter indicates a type and a value of the configurable vector operation parameter. The method further includes: based on the type and the value of the configurable vector operation parameter, configuring multiple vector operation units to enable each of the vector operation units to execute a target vector operation including two or more basic vector operations and defined based on the type and value of the configurable vector operation parameter.