Compressed AI Model Storage in Electronic Apparatus
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
RNN-trained AI models require a large amount of data, leading to increased arithmetic delay and energy consumption due to data transfer from external memory sources like DRAM, which is not efficiently managed by existing systems.
Innovation Solution
An electronic apparatus with a processor and external memory is designed to store and manage AI models by compressing data, using a first memory for a full AI model and a second memory for a compressed version, allowing for efficient data transmission and processing by selecting the appropriate model based on output element sizes and performing arithmetic operations to optimize energy efficiency and speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If an RNN-trained AI model is used to achieve accurate voice recognition, then the recognition accuracy is improved, but the data transmission volume increases leading to higher energy consumption and arithmetic delay
Solution Approach 1:
The AI model data is segmented into two parts: frequently accessed data stored in internal memory and less frequently accessed data stored in external memory. This segmentation allows the system to maintain high recognition accuracy while reducing the volume of data that needs to be continuously transmitted, thereby lowering energy consumption.
Solution Approach 2:
The patent implements a nested memory structure where internal memory (second memory) is embedded within the processor and external memory (first memory) contains the complete AI model. The internal memory holds a compressed or subset version of the model, creating a nested arrangement that enables fast access to critical data while storing comprehensive data externally.
2Reliability
If an RNN-trained AI model is used to achieve accurate voice recognition, then the recognition accuracy is improved, but the data transmission volume increases leading to longer arithmetic delay
Solution Approach 1:
The system performs preliminary action by pre-loading frequently accessed AI model data into internal memory before it is needed for processing. This advance preparation ensures that when voice recognition processing occurs, the necessary data is already in fast memory, eliminating transmission delays and reducing arithmetic delay.
Solution Approach 2:
By segmenting the AI model data and storing only the most frequently accessed portions in internal memory, the system reduces the amount of data that needs to be transmitted during real-time processing, thereby reducing arithmetic delay while maintaining recognition accuracy.
3Use of energy by moving object
If a compressed AI model is stored in internal memory to reduce data transmission, then energy consumption is reduced, but the model size is reduced potentially affecting accuracy
Solution Approach 1:
The patent applies local quality by storing different versions or portions of the AI model in different memory locations with different compression levels. The internal memory contains a compressed version optimized for energy efficiency, while the external memory stores the full-precision model. The system selectively accesses appropriate data based on processing needs, maintaining accuracy when required while reducing energy consumption during normal operation.
Data Source
AI summary
An electronic apparatus is provided. The electronic apparatus includes a first memory configured to store a first artificial intelligence (AI) model including a plurality of first elements and a processor configured to include a second memory. The second memory is configured to store a second AI model including a plurality of second elements. The processor is configured to acquire output data from input data based on the second AI model. The first AI model is trained through an AI algorithm. Each of the plurality of second elements includes at least one higher bit of a plurality of bits included in a respective one of the plurality of first elements.