Externalized Embedding Layer for Event-Driven Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, particularly event-driven deep learning (EDDL) models, face significant challenges due to the exponential increase in the size of their embedding layers when processing auto-generated inputs, leading to large memory requirements and difficulties in packaging, deployment, and storage.
Innovation Solution
The embedding layer is externalized and stored separately from the machine learning models, allowing multiple models to share it, reducing the size of individual models and enabling more efficient storage and processing by using a remote network connection for data retrieval and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the embedding layer is included within the machine learning model to process auto-generated inputs, then the model can accurately transform and map inputs into embeddings, but the size of the machine learning model increases exponentially requiring hundreds of gigabytes of memory space
Solution Approach 1:
The system separates the embedding layer from the machine learning model into an independent external component. The embedding layer is deployed as a standalone service that receives input data, transforms it into embeddings, and returns the embeddings to the ML model. This segmentation allows the ML model to remain compact while the embedding layer handles the computationally intensive transformation tasks externally.
Solution Approach 2:
An intermediary embedding layer service is introduced between the input data source and the machine learning model. This intermediary handles the complex embedding transformations independently, allowing the ML model to receive pre-processed embeddings without needing to contain the large embedding layer internally. The intermediary manages the connection between raw inputs and model processing.
2Adaptability or versatility
If the embedding layer size increases to handle more auto-generated inputs, then the model can process more input types, but the challenges in packaging, deploying, storing and executing the model increase significantly
Solution Approach 1:
The external embedding layer is designed as a universal service that can handle multiple types of inputs and serve multiple machine learning models simultaneously. Instead of each model containing its own specialized embedding layer, a single shared embedding layer provides transformation capabilities for diverse input types, reducing overall system complexity and simplifying deployment.
Solution Approach 2:
The solution moves the embedding layer from the internal horizontal dimension of individual models to an external vertical dimension as a shared service infrastructure. This dimensional shift allows the embedding functionality to be accessed by multiple models without increasing any single model's complexity, enabling easier packaging and deployment.
3Reliability
If multiple machine learning models each include their own embedding layer, then each model can independently process inputs, but the same embedding transformations are stored repeatedly across models increasing total memory requirements
Solution Approach 1:
Multiple machine learning models share a common external embedding layer instead of each model having its own separate embedding layer. The shared embedding layer consolidates all embedding transformations in a single location, eliminating redundant storage across multiple models while maintaining the ability for each model to independently process inputs through the shared service.
Solution Approach 2:
Instead of copying the same embedding layer across multiple models, the system creates a single referenceable instance that multiple models can access. The embedding transformations are computed once and shared across all models, reducing total memory usage from O(n) copies to O(1) shared instance while maintaining model independence.
Data Source
AI summary
Systems, methods, and computer program products are directed to machine learning techniques that use a separate embedding layer. This can allow for continuous monitoring of a processing system based on events that are continuously generated. Various events may have corresponding feature data associated with at least one action relating to a processing system. Embedding vectors that correspond to the features are retrieved from an embedding layer that is hosted on a separate physical device or a separate computer system from a computer that hosts the machine learning system. The embedding vectors are processed though the machine learning model, which may then make a determination (e.g. whether or not a particular user action should be allowed). Generic embedding vectors additionally enable the use of a single remote embedding layer for multiple different machine learning models, such as event driven data models.