Edge Model Layer Merging for Memory-Constrained Video Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Edge servers face memory and computing resource constraints due to the increasing size and complexity of video analytics models, leading to latency and throughput issues, especially with the exponential growth in IoT devices requiring real-time data processing.
Innovation Solution
Merging models by identifying sharable layers across multiple models based on similarity and matching properties, reducing the combinational search space and retraining requirements, and using an intelligent heuristic to improve memory use and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple trained models are implemented on the edge server to perform different analytic tasks, then the functionality and versatility of the edge server is improved, but the memory requirements and resource consumption increase significantly
Solution Approach 1:
The patent merges multiple trained models into a single consolidated model by identifying and combining common layers across different models. This allows the edge server to maintain multiple analytics functionalities (object recognition, facial recognition, vehicle identification, etc.) while reducing overall memory consumption by sharing common computational components among models.
Solution Approach 2:
The consolidated model structure enables a single model to perform multiple analytics tasks by incorporating different task-specific heads or output layers while sharing common feature extraction layers. This universal architecture allows the edge server to handle diverse analytics functions within unified memory constraints.
2Measurement precision
If model size and complexity are increased to improve analytics accuracy, then the measurement precision and reliability of data analytics is improved, but the memory footprint and processing overhead increase
Solution Approach 1:
By merging common layers across multiple models, the patent reduces redundant memory storage while preserving the complex feature extraction capabilities needed for high accuracy. The shared layers maintain the analytical precision required for accurate object recognition, facial analysis, and vehicle identification without duplicating computational resources.
3Quantity of substance
If models are swapped on and off of GPU memory to manage memory constraints, then the memory usage is controlled, but latency and accuracy issues occur
Solution Approach 1:
The consolidated model architecture allows all necessary model components to reside simultaneously in GPU memory as a single unified structure, eliminating the need to swap between multiple separate models. This reduces memory access latency and maintains continuous processing capability while staying within memory constraints through efficient layer sharing.
4Quantity of substance
If models are compressed and quantized to reduce memory footprint, then the memory consumption is reduced, but accuracy is traded off
Solution Approach 1:
The patent achieves memory reduction through structural merging of common layers rather than through compression or quantization of model parameters. This approach maintains the full precision of model weights and activations while reducing redundant memory allocation, thereby preserving analytics accuracy without relying on approximation techniques.
5Power
If video is uploaded to the cloud for processing to overcome edge server limitations, then processing power and memory availability are improved, but bandwidth costs increase and latency is substantially increased
Solution Approach 1:
The consolidated model enables the edge server to maintain sophisticated analytics processing capabilities locally by efficiently packing multiple model functions into a single memory-resident structure. This preserves low-latency local processing while reducing the need for cloud uploads, as the merged model充分利用 the edge server's available resources through intelligent layer sharing.
Data Source
AI summary
Systems and methods are provided for merging models for use in an edge server under the multi-access edge computing environment. In particular, a model merger selects a layer of a model based on a level of memory consumption in the edge server and determines sharable layers based on common properties of the selected layer. The model merger generates a merged model by generating a single instantiation of a layer that corresponds to the sharable layers. A model trainer trains the merged model based on training data for the respective models to attain a level of accuracy of data analytics above a predetermined threshold. The disclosed technology further refreshes the merged model upon observing a level of data drift that exceeds a predetermined threshold. The refreshing of the merged model includes detaching and/or splitting consolidated sharable layers of sub-models in the merged model. By merging models, the disclosed technology reduces memory footprints of models used in the edge server, rectifying memory scarcity issues in the edge server.