Edge Deduplication via Approximation Functions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Edge computing resources face memory capacity exhaustion due to limited resources, leading to reduced performance and latency issues when handling vast amounts of data, as they are not equipped to manage the same volume of data as cloud data centers.
Innovation Solution
Implementing a deduplication system in edge resources that uses a deduplication logic unit to selectively apply data deduplication techniques, such as approximation functions and hash tables, to reduce the number of unique data instances stored in memory, thereby conserving memory resources and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If edge resources receive and store vast amounts of data from client compute devices, then data processing capability is improved, but memory capacity is exhausted
Solution Approach 1:
The patent applies deduplication technology to create a mapping between original data and compressed representations. Instead of storing complete duplicate data instances, the system stores references to a centralized data store, significantly reducing the quantity of data held in edge memory while maintaining processing capability.
Solution Approach 2:
The patent extracts duplicate data instances from edge memory by implementing a deduplication logic unit that identifies and removes redundant data. Only unique data instances are retained in edge memory, while duplicates are referenced from a centralized store, thereby conserving memory capacity.
2Loss of time
If edge resources store more data to maintain low latency, then service latency is reduced, but memory capacity exhaustion occurs
Solution Approach 1:
The patent uses deduplication to store only unique data instances in edge memory while maintaining references to duplicate data in a centralized store. This reduces the volume of data that must be kept in memory for low-latency access, allowing the system to maintain fast service response without exhausting memory capacity.
Solution Approach 2:
The patent introduces a centralized data store as an intermediary between edge resources and data storage. This mediator handles the burden of storing duplicate data instances, allowing edge resources to maintain minimal local cache for low-latency operations while offloading duplicate storage to the centralized system.
3Adaptability or versatility
If edge resources are equipped with more memory to handle vast data volumes, then data management capability is improved, but resource limitations are exceeded
Solution Approach 1:
The patent implements deduplication to reduce the amount of data that must be stored in edge memory. By storing only unique data instances and referencing duplicates from a centralized store, the system achieves effective data management capability without requiring proportionally large memory resources at each edge location.
Solution Approach 2:
The patent creates a universal data management approach where edge resources share common data storage responsibilities with a centralized system. Multiple edge locations can access the same centralized data store, allowing resource sharing and reducing the individual memory requirements at each edge node while maintaining overall data management capability.
Data Source
AI summary
Technologies for providing deduplication of data in an edge network includes a compute device having circuitry to obtain a request to write a data set. The circuitry is also to apply, to the data set, an approximation function to produce an approximated data set. Additionally, the circuitry is to determine whether the approximated data set is already present in a shared memory and write, to a translation table and in response to a determination that the approximated data set is already present in the shared memory, an association between a local memory address and a location, in the shared memory, where the approximated data set is already present. Additionally, the circuitry is to increase a reference count associated with the location in the shared memory.