Neural Network Memory Management via Signature Comparison

Solution Overview

Problem

The processing of neural networks on lower-powered portable devices is hindered by high power consumption, increased processing burden, and memory bandwidth usage due to large memory transactions required for data arrays, which are often too large for local storage and necessitate frequent transfers between main and local memory.

Innovation Solution

A method involving the generation and comparison of signatures for blocks of data arrays to determine whether to write or read them to memory, allowing for the elimination of unnecessary memory transactions by identifying similar data regions already stored in memory, thereby reducing power and bandwidth consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If data arrays are stored in main memory for neural network processing, then memory capacity is sufficient, but power consumption and memory bandwidth usage increase

Engineering Contradiction:

Improvememory capacityVSAvoidpower consumption

Core Design Contradiction:

Quantity of substanceVSUse of energy by moving object

Solution Approach 1:

The data array is divided into multiple blocks, where frequently accessed blocks are kept in local memory and less frequently accessed blocks are stored in main memory. This segmentation allows the system to maintain only necessary portions in high-power local memory while storing others in lower-power main memory, reducing overall power consumption.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Data blocks are pre-loaded into local memory before they are needed for processing. By anticipating future access patterns and loading data in advance, the system avoids frequent reads from main memory during actual processing, thereby reducing memory bandwidth usage and power consumption.

Inventive Principle:

Principle #10Preliminary action

2Speed

If data arrays are kept in local memory, then processing speed increases, but memory bandwidth usage and power consumption increase

Engineering Contradiction:

Improveprocessing speedVSAvoidpower consumption

Core Design Contradiction:

SpeedVSLoss of energy

Solution Approach 1:

The data array is divided into multiple blocks, where frequently accessed blocks are kept in local memory and less frequently accessed blocks are stored in main memory. This segmentation allows the system to maintain only necessary portions in high-power local memory while storing others in lower-power main memory, reducing overall power consumption.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different portions of the data array are stored in different memory locations based on their access patterns. Hot data blocks are placed in local memory for fast access, while cold data blocks are kept in main memory, creating a hierarchical storage structure optimized for both speed and energy efficiency.

Inventive Principle:

Principle #3Local quality

3Reliability

If data arrays are frequently transferred between main and local memory, then data availability is ensured, but processing burden and memory bandwidth usage increase

Engineering Contradiction:

Improvedata availabilityVSAvoidprocessing burden

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

Data blocks are pre-loaded into local memory before they are needed for processing. By anticipating future access patterns and loading data in advance, the system avoids frequent reads from main memory during actual processing, thereby reducing memory bandwidth usage and power consumption.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system monitors access patterns of data blocks and dynamically adjusts which blocks are kept in local memory versus main memory. This feedback mechanism ensures that frequently accessed data remains in local memory for fast access, while less frequently accessed data is stored in main memory, optimizing both data availability and processing efficiency.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS11093168B2Processing of neural networks on electronic devices

Publication Date: 2021.08.17 ARM LTD

AI summary

A data processing system comprising a processor operable to execute a neural network, and associated methods for operating data processing systems. The data processing system comprises the processor and a memory for storing data relating to neural network processing being performed by the processor. The data processor is operable to write to the memory and to read from the memory blocks of data corresponding to regions of a data array generated by the neural network. The writing (or reading) is performed based on a signature comparison between the block of data that is to be written to (or read from) memory and a block of data that is already stored in memory.