Opportunistic Memory Pools for Edge Resource Fragmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current edge computing deployments face resource fragmentation and underutilization of memory and storage bandwidth due to varying resource footprints of different workloads, leading to significant resource losses.
Innovation Solution
The implementation of Opportunistic Memory Pools (OMPs) that dynamically adjust available memory by dividing it into virtual partitions, tracking unused memory resources, and aggregating them into a monolithic resource pool, allowing temporary utilization by the platform and applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If edge platforms are configured in a balanced and general-purpose way, then different types of workloads can achieve good performance, but memory and storage bandwidth are under-utilized when workloads are not network bound
Solution Approach 1:
The patent implements dynamic memory allocation where memory initially allocated to Smart NICs can be temporarily borrowed by the host system when the NICs are not fully utilizing it. The memory allocation changes dynamically based on workload demands, allowing the system to adapt between general-purpose usage and specialized network-bound usage.
Solution Approach 2:
The system changes the memory allocation parameter based on workload conditions. When Smart NICs are network-bound and utilizing memory fully, the memory remains dedicated. When they are not network-bound, the memory allocation parameter changes to allow host access, thus optimizing utilization without sacrificing adaptability.
2Reliability
If memory is allocated to specific devices like Smart NICs, then network-bound workloads achieve good performance, but significant memory resources are lost when these devices are not fully utilizing the allocated memory
Solution Approach 1:
The Smart NICs monitor their own memory utilization and automatically return unused memory to the host system without requiring manual intervention. The system self-regulates memory allocation based on actual workload demands, ensuring network performance is maintained while freeing up resources.
Solution Approach 2:
The system implements feedback mechanisms where memory utilization metrics from Smart NICs are continuously monitored. Based on this feedback, the system dynamically adjusts memory allocation, returning unused memory to the host pool and allocating it when needed, thus optimizing both performance and resource availability.
3Reliability
If memory is constantly allocated to devices, then device performance is ensured, but resource fragmentation occurs and unused memory cannot be utilized by other applications
Solution Approach 1:
The patent segments memory management into dedicated portions for Smart NICs and a shared pool for the host. This segmentation allows the system to maintain guaranteed memory for network operations while creating a separate pool of resources that can be dynamically allocated to other applications, reducing fragmentation and simplifying overall management.
Data Source
AI summary
Methods and apparatus for opportunistic memory pools. The memory architecture is extended with logic that divides and tracks the memory fragmentation in each of a plurality of smart devices in two virtual memory partitions: (1) the allocated-unused partition containing memory that is earmarked for (allocated to), but remained un-utilized by the actual workloads running, or, by the device itself (bit-streams, applications, etc.); and (2) the unallocated partition that collects unused memory ranges and pushes them in to an Opportunistic Memory Pool (OMP) which is exposed to the platform's memory controller and operating system. The two partitions of the OMP allow temporary utilization of otherwise unused memory. Under alternate configurations, the total amount of memory resources is presented as a monolithic resource or two monolithic memory resources (unallocated and allocated but unused) available for utilization by the devices and applications running in the platform.