Memory Attention Module for Scalable Image Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models, particularly transformer models, face challenges in scaling and updating due to their large number of parameters, leading to memory and computational resource issues, making it difficult to store and fine-tune them effectively for changing contexts and new data.
Innovation Solution
The approach involves transforming knowledge into a massive-scale index/memory rather than compiling it statically into model weights, using a low-compute classifier neural network that accesses a large database for predictions, and employing a memory attention module to fuse retrieved knowledge with input queries for efficient image classification tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of parameters in foundation models is increased to achieve improved performance, then model accuracy and information storage capacity improve, but memory requirements and computational resources increase excessively
Solution Approach 1:
The patent segments the model into two parts: a small classifier neural network and a separate memory database. The memory database stores the knowledge that would otherwise require billions of parameters, while the classifier remains small and computationally efficient. This segmentation allows high accuracy through access to extensive stored information without requiring excessive memory resources for the model itself.
Solution Approach 2:
The patent introduces a memory database as an intermediary between the input data and the classifier. This memory database acts as a mediator that stores and retrieves relevant information, allowing the small classifier to achieve high accuracy without needing to internally encode all the knowledge. The memory serves as the intermediary that bridges the gap between limited model parameters and extensive knowledge requirements.
2Quantity of substance
If foundation models are scaled to tens of billions of parameters, then the models can store more information and achieve state-of-the-art results, but scaling becomes challenging due to large memory requirements
Solution Approach 1:
By segmenting the information storage function from the computational function, the patent enables independent scaling of the memory database without proportionally increasing model complexity. The memory database can be scaled to store vast amounts of information using standard database technologies, while the classifier neural network remains a simple, manageable component that doesn't require complex infrastructure to deploy and maintain.
3Adaptability or versatility
If the model parameters are increased to improve performance, then the model can handle more complex tasks, but updating and fine-tuning the model becomes difficult
Solution Approach 1:
The patent extracts the knowledge storage function from the model parameters and places it in an external memory database. This extraction allows the classifier neural network to remain small and highly adaptable. When new tasks or information are needed, the system can simply add or update entries in the memory database without requiring complex retraining or fine-tuning of large model parameters. The small classifier can quickly adapt to new patterns by leveraging updated memory contents.
4Quantity of substance
If a massive-scale memory is used to store knowledge, then the memory dataset can be scaled up to 1 billion image-text pairs, but inference-time latency and computational resource consumption increase
Solution Approach 1:
By extracting only the essential features and representations into compact memory entries, the patent enables efficient retrieval from large-scale memory. The memory stores processed embeddings and key features rather than raw data, allowing rapid comparison and retrieval during inference. This extraction approach maintains the benefits of large-scale memory while minimizing the computational overhead of accessing and processing stored information.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a classification task on a data item. In particular, a system classifying an input data item using key and value embeddings of memory data items.