Hardware-Implemented Lookup Tables for Neural Network Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer architectures for executing artificial neural networks face challenges in efficiently performing matrix-vector multiplications and executing additional mathematical functions like activation functions, leading to high computational demands and power consumption.
Innovation Solution
A hardware system with a neural processing apparatus, lookup table circuits, and processing units is designed to efficiently compute neuron outputs and apply mathematical functions, using hardware-implemented lookup tables to accelerate computations by accessing parameter values directly from memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional von Neumann computing architecture is used, then processing capability and data storage are separated, but this leads to congestion and high power consumption due to continuous data transfer
Solution Approach 1:
The patent merges processing capability and data storage into a unified in-memory computing architecture. The crossbar array structure integrates memory cells with compute-in-memory circuits, allowing MAC operations to be performed directly within the memory array. This eliminates the need for continuous data transfer between separate processing and storage units, thereby reducing power consumption and improving computational efficiency.
2Measurement precision
If mathematical functions like activation functions are executed using conventional methods, then computational accuracy is maintained, but computational resources and latency increase significantly
Solution Approach 1:
The patent pre-computes and stores lookup tables containing pre-calculated activation function values in the memory array before inference. During neural network execution, the system performs a simple table lookup instead of computing activation functions in real-time. This preliminary preparation eliminates complex computations during inference, significantly reducing latency while maintaining accuracy through the pre-stored precise values.
Solution Approach 2:
The patent creates a simplified copy of the activation function computation by storing pre-computed results in lookup tables. Instead of executing the full mathematical computation during inference, the system copies the necessary function values from the LUTs into the computation pipeline. This copying approach replaces complex real-time calculations with simple data retrieval operations, reducing latency while preserving computational accuracy.
3Adaptability or versatility
If activation functions are offloaded to a digital signal processor, then functional flexibility is improved, but area and energy requirements increase dramatically
Solution Approach 1:
The patent implements universal in-memory computing circuits within the crossbar array that can perform multiple functions including MAC operations and activation function evaluation. The same memory array and compute-in-memory circuits used for matrix-vector multiplication are also utilized for storing and retrieving activation function values. This multi-functionality eliminates the need for separate DSP hardware, reducing area requirements while maintaining functional flexibility through reconfigurable memory content.
Data Source
AI summary
The invention is notably directed to a hardware system (1) designed to implement an artificial neural network (ANN). The hardware system basically includes a neural processing apparatus (15), e.g., involving as crossbar array structure, one or more lookup table circuits (17), and one or more processing units (18). The neural processing apparatus is configured to implement M artificial neurons, where M≥1. The lookup table circuits are configured to implement a lookup table (LUT). The system further includes M′ processing units, where M≥M′≥1. Each processing unit is connected by at least one neuron, in order to be able to access a first value outputted by each connected neuron. In addition, each processing unit is connected to a LUT circuit, in order to efficiently access parameter values of a set of parameters from the LUT. Finally, each processing unit is configured to output a second value, corresponding to a value of a mathematical function taking said first value as argument. The mathematical function is otherwise determined by the set of parameters, the parameter values of which are accessed by each processing unit from the LUT, in operation. I.e., the mathematical function is defined (and thus determined) by a set of parameters, the values of which are efficiently retrieved from the hardware-implemented LUT. This results in a substantial acceleration of the computations of the function outputs, beyond the acceleration that may already be achieved within the neural processing apparatus and the processing units themselves. As a result, the neuron outputs can be more efficiently processed, prior to being passed to a next neuron layer. The invention is further directed to a method of operating such a hardware system.