Command Processor Context Data Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processing systems face challenges in efficiently processing neural network models due to high memory load requirements for computation descriptions, particularly in systems that lack specialized data structures to minimize memory usage.
Innovation Solution
A command processor and a neural core SoC are designed to utilize a separate data structure pool, allowing for the activation of contexts using context dynamic and static objects, which reduces memory load by storing and retrieving only object IDs instead of the entire objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If computation descriptions for each layer are performed separately, then processing completeness is improved, but memory load increases
Solution Approach 1:
The patent segments computation descriptions into layer-specific components and introduces a data structure pool that separates static objects (reusable across layers) from dynamic objects (layer-specific). This segmentation allows the system to maintain complete processing capabilities while reducing redundant memory storage by sharing static computation descriptions across multiple layers.
2Quantity of substance
If specialized data structure pool is introduced, then memory efficiency is improved, but device complexity increases
Solution Approach 1:
The patent introduces a data structure pool as an intermediary component between the computation engine and memory storage. This pool acts as a mediator that manages the separation between static and dynamic objects, providing efficient memory access patterns while isolating the complexity of memory management from the core computation engine.
3Speed
If context activation using object IDs is implemented, then data access speed is improved, but processing complexity increases
Solution Approach 1:
The patent implements a copying mechanism where object IDs (lightweight references) are used to access full computation descriptions in the data structure pool. Instead of directly accessing or duplicating large computation description data, the system copies only the compact object IDs, which then serve as keys to retrieve the actual computation descriptions from the pool, significantly reducing data access overhead.
Data Source
AI summary
A command processor determines whether a command descriptor describing a current command is in a first format or in a second format, wherein the first format includes a source memory address pointing to a memory area in a shared memory having a binary code to be accessed according to direct memory access (DMA) scheme, and the second format includes one or more object indices, a respective one of the one or more object indices indicating an object in an object database. If the command descriptor describing the current command is in the second format, the command processor converts a format of the command descriptor to the first format, generates one or more task descriptors describing neural network model tasks based on the command descriptor in the first format, and distributes the one or more task descriptors to the one or more neural processors.