Neural Network Processing With Dedicated and Shared Storage Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing systems face inefficiencies in data storage and transfer between processing elements, particularly in resource-constrained devices like mobile devices, due to limited CPU resources and suboptimal use of dedicated neural network processing hardware.
Innovation Solution
A processor architecture is introduced that includes neural processing elements, dedicated storage elements, shared storage elements, and a control element to manage data storage and transfer efficiently, utilizing a directed graph of operations and selecting between neural and shared storage elements for in-process data handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If dedicated neural storage elements are used for each processing element, then data access efficiency is improved, but device complexity increases
Solution Approach 1:
The storage system is segmented into two distinct types: dedicated neural storage elements accessible only by neural processing elements, and shared storage elements accessible by multiple processing elements. This segmentation allows the system to optimize for both fast dedicated access and flexible shared access, resolving the contradiction between data access efficiency and device complexity.
Solution Approach 2:
The control element acts as an intermediary that manages the selection between dedicated and shared storage elements. It receives data access requests and determines whether to route to dedicated or shared storage based on the processing requirements, thereby simplifying the architecture while maintaining high performance.
2Device complexity
If shared storage elements are used for data transfer between processing elements, then device complexity is reduced, but data transfer speed decreases
Solution Approach 1:
By segmenting storage into dedicated and shared types, the system can maintain simple shared storage for general purposes while providing fast dedicated storage pathways for critical neural network data transfers, thus achieving both low complexity and high speed.
Solution Approach 2:
The control element dynamically selects between dedicated and shared storage elements based on the specific data transfer requirements of each operation. This dynamic routing allows the system to optimize performance for each individual data transfer while maintaining overall architectural simplicity.
3Productivity
If more processing elements are added to handle neural network tasks, then processing capability is improved, but resource consumption increases
Solution Approach 1:
The system segments processing into neural processing elements that execute neural network operations and CPU elements that handle general tasks. By adding more neural processing elements while keeping CPU resources separate and optimized for coordination tasks, the system improves neural network processing capability without proportionally increasing CPU resource consumption.
Solution Approach 2:
Neural network processing functions are extracted from the CPU and implemented in dedicated neural processing elements. This extraction allows the CPU to be freed from computational burdens and focused on resource management and coordination, thereby improving overall processing capability while reducing CPU resource consumption.
4Loss of time
If data is stored in dedicated neural storage elements, then data access time is reduced, but storage flexibility decreases
Solution Approach 1:
The dual-storage architecture segments data storage into dedicated neural storage for fast access during neural network processing and shared storage for flexible data management. This segmentation allows the system to achieve both reduced data access time for critical operations and maintained storage flexibility for diverse data requirements.
Solution Approach 2:
Shared storage elements serve multiple functions: they can store data for neural network processing, act as buffers for data transfer, and provide flexible memory management. This multi-functionality compensates for the reduced flexibility of dedicated storage elements, maintaining overall system adaptability while enabling fast dedicated access where needed.
Data Source
AI summary
A processor, method and computer program for artificial neural network processing and comprising a neural processing element operable to perform processing operations on data; a dedicated neural storage element accessible only by the neural processing element; a shared storage element accessible by the neural processing element and one or more other processing elements configured to perform separate processing operations; and a control element operable to control task processing by the processor, respective tasks to be executed as a graph of operations, wherein each operation maps to a corresponding neural processing element, and wherein each connection between operations in the graph maps to a corresponding storage element for storing in-process data; and wherein, for a given task connection, the control element is configured to select between neural storage and shared storage and to direct reading and writing of in-process data corresponding to the connection to the selected storage.