Episodic Memory Training for New-Class AI Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep neural network (DNN) technology requires extensive training with hundreds to thousands of example datasets and accurate class labels, incurring high visual and economic costs, and necessitates retraining for new classes, which is inconvenient.
Innovation Solution
A method and apparatus for training artificial intelligence using episodic memory, enabling a model to perform tasks for new classes without further learning by constructing an episodic memory with feature vectors, deriving similarity, and using a combination of hard-attention and soft-attention loss functions, along with knowledge distillation from a pretrained teacher model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural network is trained with hundreds to thousands of example datasets to achieve human-level accuracy, then inference accuracy is improved, but visual and economic costs increase significantly
Solution Approach 1:
The patent pre-processes training data to extract and store feature vectors in an episodic memory structure before actual inference is needed. This preliminary action allows the model to access pre-computed features during inference, reducing the need for extensive retraining while maintaining accuracy
Solution Approach 2:
The patent creates copies of essential feature representations and stores them in episodic memory. Instead of retraining the full model on large datasets, the system copies and retrieves relevant feature vectors from memory, achieving accurate inference with minimal training data
2Reliability
If deep neural network is retrained with大量 data to perform tasks for new classes, then task performance is improved, but training time and computational resources increase
Solution Approach 1:
The system performs preliminary processing to extract feature vectors from training data and stores them in episodic memory organized by class. When new classes need to be handled, the system can quickly retrieve pre-computed features from memory rather than retraining from scratch, significantly reducing training time
Solution Approach 2:
The patent segments the training process into distinct phases: feature extraction, episodic memory construction, and inference. This segmentation allows the computationally intensive feature extraction to be performed once, with rapid retrieval for subsequent tasks on new classes
3Measurement precision
If accurate class labels are assigned to numerous example datasets to enable supervised learning, then model learning capability is improved, but annotation costs increase
Solution Approach 1:
The patent extracts essential feature vectors from training data and stores them in episodic memory, separating the essential learning information from the full annotated datasets. This extraction allows the model to learn from compressed feature representations, reducing the need for extensive annotated data while maintaining learning capability
Data Source
AI summary
The present disclosure relates to a method and apparatus for training artificial intelligence based on an episodic memory. According to an embodiment of the present disclosure, a method for training artificial intelligence based on an episodic memory may include: constructing an episodic memory by using a feature vector of a training dataset stored in a full memory; obtaining output data by inputting query data into an artificial intelligence model; deriving a similarity between the output data and a feature vector in the constructed episodic memory; and deriving an episode loss function based on the similarity.