Incremental Model Training Using Encoded Exemplars
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Classical batch machine learning approaches for device identification in network security are hindered by the need for complete data sets, which are impractical due to data privacy laws and cost constraints, leading to catastrophic forgetting and limitations in updating input features and device classes.
Innovation Solution
An incremental learning approach that stores exemplar representations of fingerprint data in an encoded form, allowing for privacy compliance and reducing storage and processing costs, while updating input features and device classes without retraining from scratch.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complete data sets are used for batch machine learning training, then model accuracy is improved, but storage costs and processing time increase significantly
Solution Approach 1:
The patent extracts only the most informative samples (exemplars) from the complete data set, storing them in an encoded form. This selective extraction maintains model training accuracy while dramatically reducing the volume of data that needs to be stored and processed in subsequent incremental training batches.
Solution Approach 2:
The patent creates encoded representations (copies) of the original fingerprint data that preserve the essential information needed for model training. These encoded exemplars serve as simplified substitutes for the complete raw data set, reducing storage requirements while maintaining training effectiveness.
2Measurement precision
If complete data sets are stored for incremental training, then model updating accuracy is improved, but data privacy compliance becomes difficult
Solution Approach 1:
The patent extracts only essential characteristic features from the fingerprint data to create exemplars, omitting personally identifiable information. This selective extraction maintains the information needed for accurate device identification while removing privacy-sensitive details, enabling compliance with data protection regulations.
Solution Approach 2:
The patent uses encoded representations instead of storing original raw data. These encoded exemplars serve as sufficient substitutes for training purposes, allowing the system to discard or anonymize the original sensitive data while retaining the essential patterns needed for model training.
3Measurement precision
If retraining from scratch is performed with new device classes, then model accuracy is improved, but training time and computational resources increase
Solution Approach 1:
The patent performs preliminary encoding and selection of exemplars during initial training, organizing the data in a way that enables efficient incremental updates. This preliminary preparation allows subsequent model updates to proceed quickly by simply incorporating new exemplars rather than reprocessing the entire data set from scratch.
Solution Approach 2:
The patent implements a dynamic training approach where the model can be incrementally updated with new device classes by adding new exemplars to the existing training batch. This dynamic update mechanism allows the model to adapt to new devices efficiently without requiring complete retraining, maintaining accuracy while reducing computational overhead.
4Adaptability or versatility
If all fingerprint data is retained for future training batches, then model adaptability is improved, but processing costs increase
Solution Approach 1:
The patent extracts and stores only the most representative samples (exemplars) from the fingerprint data in an encoded format. This selective extraction maintains the model's ability to adapt to new device classes while dramatically reducing the amount of data that needs to be processed during incremental training, thereby lowering computational energy requirements.
Solution Approach 2:
The patent applies partial action by storing only a subset of the most informative samples rather than all fingerprint data. This partial retention of data is sufficient to maintain model adaptability for future incremental training batches while significantly reducing the processing burden compared to retaining the complete data set.
Data Source
AI summary
A disclosed example includes obtaining first data associated with a first device class; build a vocabulary including keys that map to values for an incremental training batch, the incremental training batch based on the first data and exemplars from memory, the exemplars associated with a set of device classes, the exemplars include first means closest to first overall means for ones of the set of the device classes that were stored to the at least one memory during a previous incremental training batch; train a model based on the keys as input features and an updated set of the device classes that includes the first device class; and select a set of samples from the first data and the exemplars, the set of the samples includes second means closest to second overall means for ones of the updated set of the device classes.