Memory Processor Floating-Point Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The Von Neumann bottleneck occurs frequently in systems like neural networks and big data applications, where a large amount of data is transmitted between memory devices and processors, leading to inefficiencies in computational tasks, particularly in performing double-precision floating-point operations required for high-performance computing.
Innovation Solution
A method and apparatus that utilize a memory processor to perform floating-point operations by determining whether to use emulation and which stages of the operation to offload to the memory processor based on matrix size, sub-matrix size, and memory bandwidth, allowing for direct memory access and efficient processing of double-precision floating-point matrix multiplication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transmitted between memory device and processor for double-precision floating-point operations, then computational tasks can be performed, but bandwidth limits cause delays and reduce operational performance
Solution Approach 1:
The patent segments the floating-point operation into multiple stages (e.g., splitting stage, matrix multiplication operation stage, summation stage) that can be individually determined to use the memory processor. This allows selective offloading of computation-intensive stages to the memory processor, reducing data transmission requirements and bandwidth-dependent delays while maintaining computational accuracy.
Solution Approach 2:
The patent introduces an intermediary mechanism (the memory processor with its own processing capabilities) that can perform computational tasks directly within the memory device. This intermediary approach eliminates the need for constant data shuttling between memory and processor, thereby reducing bandwidth limits and associated delays.
2Productivity
If double-precision floating-point operations are performed using traditional processor-memory architecture, then computational accuracy is maintained, but the Von Neumann bottleneck reduces efficiency
Solution Approach 1:
The memory processor is designed with multi-functionality, capable of performing both traditional memory functions and floating-point computational tasks. This universal approach allows the same hardware to adaptively handle different operation types (emulation or direct computation) based on operational requirements, improving efficiency without requiring entirely separate specialized hardware.
Solution Approach 2:
The patent implements dynamic determination of whether to use the memory processor for each stage of the floating-point operation. The system can adaptively switch between emulation mode and direct memory processor execution based on operational characteristics, matrix size, and resource availability, thereby optimizing computational efficiency while managing system complexity.
3Device complexity
If emulation is used to perform floating-point operations, then hardware requirements are reduced, but additional overhead is introduced
Solution Approach 1:
The patent changes the operational parameters of the memory processor by allowing it to dynamically switch between emulation mode and direct execution mode. When emulation is determined to be beneficial (e.g., for compatibility or resource constraints), the memory processor emulates the required floating-point operations; otherwise, it executes directly. This parameter change approach balances hardware requirements against operational overhead based on real-time conditions.
Data Source
AI summary
A method of performing a floating-point operation using a memory processor (the floating-point operation being a multiplication of a first matrix and a second matrix that are double-precision floating-point matrices) includes: determining whether an emulation is to be used to perform the floating-point operation, based on a result of the determining whether the emulation is to be used, determining whether to use the memory processor for the emulation, the emulation comprising stages, based on a result of the determining whether to use the memory processor for the emulation, individually determining whether to use the memory processor for each stage of the emulation, and multiplying the first matrix and the second matrix based on a result of the individually determining whether to use the memory processor.