Matrix Multiplication Integrated Circuit with Data Transfer Loop
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix multiplication methods in artificial intelligence, particularly in deep learning, face inefficiencies due to high data interaction with external memory, leading to I/O bottlenecks and reduced performance caused by limited bus bandwidth and resource constraints.
Innovation Solution
A hardware architecture with an integrated circuit apparatus that divides matrices into blocks, utilizing multiple master computing units connected in a data transfer loop to perform parallel matrix multiplication tasks, minimizing external memory interactions and optimizing data transfer through a control unit.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a processor (CPU/GPU) is used for matrix multiplication, then computing tasks can be executed, but the limited internal register capacity causes frequent data interaction with external memory, leading to I/O bottlenecks and reduced efficiency
Solution Approach 1:
The patent divides large matrices into multiple matrix blocks and distributes them across multiple master computing units. Each master computing unit processes a specific matrix block independently, enabling parallel computation and reducing the frequency of data interactions with external memory, thereby resolving the I/O bottleneck problem.
Solution Approach 2:
The patent introduces a spatial dimension by arranging multiple master computing units in a two-dimensional array structure with inter-unit data transfer paths. This allows data to be transferred between computing units through adjacent connections rather than all accessing external memory sequentially, adding a new dimension to data flow and reducing I/O delays.
2Productivity
If the bandwidth of the I/O bus between processor and external memory is limited, then system performance is constrained, but increasing data interaction with external memory is necessary for large-scale matrix multiplication
Solution Approach 1:
The patent segments the matrix multiplication task into multiple independent sub-tasks, each handled by a dedicated master computing unit. This segmentation allows simultaneous processing of multiple matrix blocks, increasing overall system performance without proportionally increasing data interaction volume with external memory.
Solution Approach 2:
The patent establishes continuous data flow between adjacent master computing units through dedicated transfer paths. Once data enters the computing unit array, it circulates and is processed continuously through adjacent units without returning to external memory, maintaining continuous useful action and reducing total data interaction volume.
3Productivity
If large-scale matrix multiplication is performed, then computing capability is utilized, but the amount of data that must be transferred to and from external memory increases significantly
Solution Approach 1:
The patent divides large-scale matrix multiplication into multiple smaller matrix block operations distributed across several master computing units. Each unit processes its assigned blocks independently, enabling high computing capability utilization while limiting the data volume each unit must transfer with external memory, thereby reducing overall power consumption.
Solution Approach 2:
The patent creates additional data transfer pathways between adjacent master computing units, forming a two-dimensional data flow network. This dimensional expansion allows data to be shared and processed across the array without repeatedly accessing external memory, reducing energy-consuming I/O operations while maintaining high computing utilization.
Data Source
AI summary
An integrated circuit apparatus may be included in a computing processing apparatus of a combined processing apparatus. The computing processing apparatus includes one or a plurality of integrated circuit apparatuses. The combined processing apparatus may further include an interface apparatus and other processing apparatus. The computing processing apparatus interacts with other processing apparatus to jointly complete a computing operation specified by a user. The combined processing apparatus further includes a storage apparatus. The storage apparatus is connected to the apparatus and other processing apparatus, respectively. The storage apparatus is used to store data of the apparatus and other processing apparatus. The solution of the present disclosure may reduce the amount of data transferred between an internal device and an external storage apparatus, thus minimizing the I/O bottleneck caused by bandwidth limitations and then improving the overall performance of the integrated circuit apparatus.