Data layout conscious processing in memory architecture for executing neural network model

Abstract

The present disclosure relates to a device supporting in-memory processing (PIM) for executing a neural network model. The device supporting PIM comprises a storage block component, and the storage block component comprises a first storage block array; a second memory block array adjacent to the first memory block array; and a plurality of first data links associated with the first memory block array and the second memory block array, wherein each data link of the plurality of first data links is communicatively connected to two respective memory blocks from the first memory block array and the second memory block array, respectively; and a second data link communicatively connected to the plurality of first data links, the data from the first memory block of the first memory block array being transmittable to the second memory block of the second memory block array via the plurality of first data links and the second data link.

Description

Background technique [0001] Machine learning has been widely used in various fields such as natural language processing, speech recognition, image classification, etc. In machine learning, neural network models are constantly proliferating and becoming more complex. As neural network models, especially deep neural network (DNN) models, become large and complex, with hundreds of layers and millions of weights, executing a neural network model is not only computationally intensive, but also memory-intensive. Because a conventional von Neumann architecture has separate processing units and storage units, when a neural network model is processed on a conventional von Neumann architecture, a large amount of data occurs between the processing unit and the storage unit transmission, which becomes the bottleneck for processing neural network models. [0002] Processing In Memory (PIM) technology, which enables calculations to be performed in storage units, has emerged to address suc...

AI Technical Summary

Problems solved by technology

As neural network models, especially deep neural network (DNN) models, become large and complex, with hundreds of layers and millions of weights, executing a neural network model is not only computationally intensive, but also memory-intensive

Because a conventional von Neumann architecture has separate processing units and storage units, when a neural network model is processed on a conventional von Neumann architecture, a large amount of data occurs between the processing unit and the storage unit transmission, which becomes the bottleneck for processing neural network models

Images