Hardware neural network batch normalization system

Abstract

The invention discloses a hardware neural network batch normalization system. The hardware neural network batch normalization system comprises a cascaded C-layer neural network circuit, the output control circuit of the p layer of neural network circuit is connected with the weight area input coding circuit in the (p + 1) layer of neural network circuit; wherein p = 1, 2,..., C-1; the p layer neural network circuit comprises a weight area input coding circuit, a batch normalization area input coding circuit, a weight area synaptic unit, a batch normalization area synaptic unit, anactive layer circuit and an output control circuit. Derivation and simplification are performed by combining a batch normalization formula with neural network activation function characteristics, batch normalization parameter information of the neural network is stored by adopting a batch normalization region synapse unit, and a normalization process corresponds to a process of summing output of aweight region synapse unit and the batch normalization parameter information of the neural network according to lines. Originally complex hardware functions are enabled to adapt to a memory storage and calculation integrated framework, the circuit complexity for realizing batch normalization hardware functions is greatly simplified, and relatively high network precision can be realized with relatively low circuit area consumption.

Description

technical field [0001] The invention belongs to the technical field of artificial neural networks, and more specifically relates to a hardware neural network batch normalization system. Background technique [0002] In the era of big data, more and more artificial intelligence and deep learning are applied to daily life, but limited by the traditional von Neumann architecture with separated memory and processor, the existing neural network hardware implementation and acceleration system are facing increasingly severe problems. The growing "storage wall" problem. The neural network in-memory computing hardware system based on mature memory and new memory has the characteristics of high parallelism, low latency, low power consumption, and no clear boundary between storage and computing. It is expected to break through the von Neumann bottleneck problem of traditional computer architecture. It has great potential and significance in the context of today's era. [0003] With t...

AI Technical Summary

Problems solved by technology

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a hardware neural network batch normalization system, the purpose of which is to solve the technical problem that the prior art cannot achieve higher network accuracy with lower circuit area consumption

Images