Memory-Resident Activation Functions for Low-Latency AI Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory devices experience increased latency and power consumption when defining activation functions for artificial intelligence operations due to the need for data transfer and host involvement in selecting these functions.
Innovation Solution
The solution involves pre-defining or customizing activation functions directly on the memory device, reducing the need for data transfer and host interaction by processing and selecting activation functions locally, thereby reducing latency and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If activation functions are defined on the host and transferred to the memory device, then the memory device can perform AI operations, but latency and power consumption increase due to data transfer and host involvement
Solution Approach 1:
The patent pre-defines activation functions directly on the memory device in activation function registers, so that when AI operations are needed, the functions are already available locally without requiring transfer from the host. This preliminary positioning of the activation functions resolves the technical contradiction by eliminating the data transfer step that causes latency.
Solution Approach 2:
The memory device is designed to autonomously select and apply activation functions from its own internal registers without requiring host involvement in the selection process. The device serves itself by having the capability to define and execute activation functions locally, which reduces both latency and power consumption associated with host-device communication.
2Productivity
If activation functions are transferred from host to memory device, then AI operations can be performed, but power consumption increases due to data transfer and host involvement
Solution Approach 1:
The activation functions are pre-loaded into the memory device's internal registers during device initialization or manufacturing, eliminating the need for repeated transfers from the host during AI operations. This preliminary action reduces power consumption by avoiding unnecessary data transfer operations.
Solution Approach 2:
The memory device autonomously manages activation functions using its own internal resources, selecting and applying them without requiring host participation. This self-service approach eliminates the power consumption associated with host-device communication protocols and data transfer operations.
3Productivity
If the memory device stores activation function data, then local processing is enabled, but device complexity increases
Solution Approach 1:
The memory device is designed with multi-functionality by incorporating activation function registers that allow it to both store data and execute AI operations with local processing. This universal design enables the device to perform multiple functions (data storage, AI computation, activation function execution) without requiring separate dedicated hardware for each function, thus managing complexity while enhancing productivity.
Solution Approach 2:
The activation function data is nested within the memory device's internal register structure, integrating the processing capability directly into the storage architecture. This nesting approach allows the device to contain activation functions within its existing structural framework rather than requiring separate external processing units, thereby enabling local processing with minimal increase in overall device complexity.
Data Source
AI summary
The present disclosure includes apparatuses and methods related to defining activation functions for artificial intelligence (AI) operations. An example apparatus can include a number of memory arrays and a controller, wherein the controller includes a number of activations function registers, wherein the number of activation function registers define activation functions for artificial intelligence (AI) operations performed by the apparatus.