Multi-Tiered Federated Deduplication for Edge Datacenter Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing support systems for hardware and application issues in edge datacenters consume significant customer processing and network bandwidth due to the large number of devices and diverse operational requirements, making it difficult to efficiently monitor and support datacenter devices worldwide.
Innovation Solution
A multi-tiered federated deduplication framework is implemented, with client modules at edge locations collecting data and performing initial deduplication, followed by CDN servers and backend servers performing additional deduplication to reduce communication footprint and bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data from all edge devices is transmitted to backend servers for support processes, then complete device monitoring and support coverage is achieved, but network bandwidth and processing resources are excessively consumed
Solution Approach 1:
The patent segments the data transmission process into multiple stages by introducing intermediate CDN servers between edge devices and backend servers. Edge devices first transmit data to local CDN servers, which perform deduplication before forwarding to backend servers. This segmentation reduces the volume of data transmitted over the network by eliminating redundant data at intermediate stages.
Solution Approach 2:
The patent applies preliminary deduplication action at CDN servers before data reaches backend servers. By performing deduplication in advance at the CDN level, the system reduces the data volume that needs to be transmitted to backend servers, thereby conserving network bandwidth and processing resources while maintaining complete monitoring coverage.
2Reliability
If data from all edge devices is transmitted to backend servers for support processes, then complete device monitoring and support coverage is achieved, but customer processing bandwidth and system resources are excessively consumed
Solution Approach 1:
The patent introduces CDN servers as intermediary components between edge devices and backend servers. These intermediaries handle the complex deduplication logic and data filtering, simplifying the overall system architecture. The CDN servers absorb the complexity of data management, allowing backend servers to focus on processing only unique data without managing the complexity of handling all device data.
3Loss of energy
If deduplication is performed at multiple levels (edge platforms, CDN servers, backend servers), then bandwidth and compute resources are significantly reduced, but system architecture complexity increases
Solution Approach 1:
The patent implements a universal deduplication framework that operates across multiple tiers (edge platforms, CDN servers, backend servers) with consistent functionality. Each tier performs the same deduplication operation, allowing the system to scale horizontally. This multi-functional approach reduces bandwidth consumption at every level while maintaining architectural consistency, making the added complexity manageable through standardized patterns.
Data Source
AI summary
A method comprises collecting data corresponding to operation of one or more devices of an edge platform, performing a first deduplication process on the data, and causing transmission of at least a first portion of the data over at least one communications network to a first server based at least in part on the first deduplication process to enable the first server to perform a second deduplication process on the first portion of the data prior to transmission of at least a second portion of the data over the at least one communications network to a second server.