Local ANN Dataset Memory Management for Efficient Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies, including CPUs and GPUs, face inefficiencies in processing artificial neural networks (ANNs) due to high computational loads and power consumption, while Field Programmable Gate Arrays (FPGAs) are challenging to program for ANN computations and lack integration in AI software environments, failing to address the massive data movement requirements of real industrial applications.
Innovation Solution
A system and method for performing multiple locally stored ANN computations using one or more processing units and a memory, where ANN datasets are stored in memory until needed, allowing for sequential or prioritized computations based on input datasets, and utilizing integrated circuits, FPGAs, or computer systems to manage memory efficiently, reducing the number of required processing units and improving data reuse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPUs are used to process ANNs, then programming effort is minimized, but computing power is insufficient
Solution Approach 1:
The patent introduces a memory management system that acts as an intermediary between the CPU and multiple ANN datasets. The system loads ANN datasets into memory and manages their storage and retrieval, allowing the CPU to process ANNs efficiently without direct management of multiple datasets. This mediator approach enables the CPU to maintain its programming simplicity while gaining access to enhanced computing capabilities through efficient memory management of multiple ANNs.
2Productivity
If GPUs are used to process ANNs, then throughput capacity is improved, but power consumption increases excessively
Solution Approach 1:
The patent implements a memory management system that loads only the necessary ANN datasets into memory based on computation requirements. Instead of loading all possible ANN datasets simultaneously (excessive action), the system selectively loads only those ANNs that are currently needed for processing (partial action). This approach provides improved throughput capacity by having multiple ANNs readily available in memory while avoiding excessive power consumption by not maintaining all datasets in high-speed memory at once.
3Power
If FPGAs are used to compute ANNs, then parallel computing capability is enhanced, but programming complexity increases significantly
Solution Approach 1:
The patent introduces a memory management system that serves as an intermediary layer between the complex FPGA hardware and the ANN computation tasks. The system manages the loading, storage, and retrieval of ANN datasets in memory, shielding the complexity of FPGA programming from the user. This allows FPGAs to provide enhanced parallel computing capability while the memory management system handles the complexity of data organization and access, reducing the perceived programming complexity.
4Productivity
If multiple ANN datasets are stored in memory, then data reuse is improved, but memory management complexity increases
Solution Approach 1:
The patent implements a memory management system that automatically manages multiple ANN datasets through self-service mechanisms. The system autonomously loads ANN datasets into memory when needed, manages their storage, and retrieves them for processing without requiring manual intervention. This automatic self-management improves data reuse by keeping multiple ANNs readily available in memory while reducing memory management complexity by eliminating the need for manual tracking and management of multiple datasets.
Data Source
AI summary
Systems and methods for performing multiple locally stored artificial neural network (ANN) computations are provided. An example method comprises receiving, by one or more processing units, an ANN dataset associated with at least one ANN of a plurality of ANNs; storing, by processing units, the ANN dataset in a memory coupled to the processing units; associating, by the processing units, a base address with the at least one ANN, wherein the base address is to be used to locate the ANN dataset in the memory; keeping, by the processing units, the ANN dataset in the memory; receiving, by the processing units, an input dataset and the base address; determining, by the processing units and based on the base address, a location of the ANN dataset in the memory; and performing, by the processing units, ANN computation using the ANN dataset and input dataset.