Storage Compute Device Neural Network Context Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional computing setups face bottlenecks in processing large data sets due to limitations in random access memory (RAM) and slow data transfer through channels like disk drives and network interfaces, leading to underutilization of processing resources and inefficiencies in handling large matrices and neural networks.
Innovation Solution
A storage compute device with configurable neural network circuitry that can adapt its configuration to process different inputs by switching between contexts, utilizing non-volatile memory for large data sets and intermediate results, and optimizing data access patterns to maintain throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional computing setups use RAM and disk drives for data storage and transfer, then data processing can be performed, but processing speed is limited by slow data transfer through channels and bottlenecks occur
Solution Approach 1:
The patent merges storage and computation functions by implementing neural network processing directly within the storage device's non-volatile memory array, eliminating the need to transfer data between separate storage and processing units. This integration allows data to be processed in-place, dramatically reducing data transfer time and eliminating communication bottlenecks between RAM, disk drives, and processing units.
2Adaptability or versatility
If neural network size is increased to handle large data sets, then processing capability improves, but device complexity and memory requirements increase
Solution Approach 1:
The patent implements a universal processing architecture where the same non-volatile memory array can be dynamically reconfigured to implement different neural network sizes and configurations. By using context switching between different configuration sets stored in the memory, a single memory array can serve multiple neural network functions, eliminating the need for separate dedicated memory resources for each network size and achieving multi-functionality without proportionally increasing device complexity.
3Adaptability or versatility
If context switching is implemented to process different inputs, then adaptability improves, but processing time may increase due to configuration changes
Solution Approach 1:
The patent pre-loads multiple neural network configurations into the non-volatile memory array before processing begins. When context switching is required, the system can rapidly switch between pre-configured networks by simply changing which configuration set is active, rather than performing time-consuming reconfiguration operations. This preliminary preparation of multiple configurations minimizes the time penalty associated with adaptability.
4Productivity
If data is kept close to processing units, then data access efficiency improves, but device architecture complexity increases
Solution Approach 1:
The patent segments the processing function into distributed processing elements that are embedded within or directly coupled to the non-volatile memory array. Rather than having a single centralized processing unit far from storage, the computation capability is segmented and distributed across multiple processing elements located at different locations within the memory structure, enabling data access efficiency while managing architecture complexity through modular segmentation.
Data Source
AI summary
A first input is processed via a first configuration of a neural network to produce a first output. The first configuration defines attributes of the neural network, such as connections between neural elements of the neural network. If the neural network requires a context switch to process a second input, a second configuration is applied to the neural network to change the attributes, and the second input is processed via the second configuration of the neural network to produce a second output.