Co-Clustering Compression for Edge-to-Cloud AI Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performance constraints in edge computing environments, such as unstable data transfer rates and high latency, are exacerbated by the need for efficient data processing and transmission from IoT units to cloud resources, where existing methods fail to optimize data transmission efficiency effectively.
Innovation Solution
The method employs co-clustering algorithms like DBSCAN to identify beneficial parameter pairs for compressed encoding, reducing data transmission by assigning numeric value tuples to clusters and transmitting cluster identifiers, which are then decoded in the cloud to generate surrogate values for AI training, thereby improving data transmission efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If co-clustering algorithms are used to compress data from multiple parameters, then data transmission efficiency is improved and bandwidth consumption is reduced, but computational complexity at the edge device increases
Solution Approach 1:
The patent segments the data compression process into two distinct phases: (1) edge device performs co-clustering analysis to identify parameter groups and generate compressed encodings, and (2) cloud resource performs decoding and surrogate generation. This segmentation allows complex computation to be distributed, reducing the burden on edge devices while maintaining compression benefits.
Solution Approach 2:
The patent applies preliminary action by performing co-clustering analysis and identifying parameter groups before actual data transmission occurs. The edge device pre-processes the data to determine which parameters should be co-clustered, establishing the compression scheme in advance. This preliminary computation enables more efficient real-time compression during data transmission.
2Quantity of substance
If data is compressed using co-clustering, then the amount of data transmitted is reduced, but measurement precision of individual parameter values is lost
Solution Approach 1:
The patent introduces an intermediary approach where instead of transmitting precise individual parameter values, the system transmits compressed cluster identifiers that serve as proxies. The cloud resource then acts as an intermediary by generating surrogate values that approximate the original parameters. This intermediary mechanism preserves the essential information needed for AI training while significantly reducing transmission volume.
Solution Approach 2:
The patent fundamentally changes the parameter representation from continuous numeric values to discrete cluster identifiers. By transforming the data from its original high-precision numeric form to compressed integer cluster codes, the system achieves dramatic reduction in transmission volume. The parameter change is reversible through surrogate generation, which reconstructs approximate numeric values from the compressed identifiers.
3Quantity of substance
If co-clustering is applied to multiple parameters, then data compression ratio is improved, but the time required for cluster analysis increases
Solution Approach 1:
The patent segments the cluster analysis process by focusing on identifying groups of parameters that benefit from co-clustering, rather than performing exhaustive analysis on all possible parameter combinations. This selective segmentation of the analysis process achieves high compression ratios for the most beneficial parameter groups while limiting the time investment in analysis.
Data Source
AI summary
Co-clustering of at least some parameters is employed to reduce data transferred between edge and cloud resources. Single-parameter cluster information, including cluster counts, for each of two or more parameters of interest is accessed. Each parameter may represent a time series of numeric values sent from an IoT unit to an edge device. A co-clustering ratio is determined for each unique parameter pair. The co-clustering ratio indicates whether the number of clusters produced by a co-clustering algorithm applied to a group of parameters is less than the number of clusters required to represent the parameters without co-clustering. Co-cluster groups may be identified based on the cluster ratios. For each co-cluster group, the co-clustering algorithm may be invoked to produce compressed encodings of numeric value tuples. The compressed encoding is then transmitted to a cloud computing resource and decoded into a tuple of surrogate values.