Analytical Workload Data Reduction for Accurate Low-Cost Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing devices face challenges in efficiently managing large volumes of data for computer-implemented services due to computational resource constraints and high transmission costs, leading to time delays and inefficient data processing.
Innovation Solution
Implementing a data reduction system that uses data reduction algorithms and factors to minimize data size based on type-specific criteria, dynamically updating these factors to ensure accurate inference generation and efficient data usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data is transmitted and processed in full size, then inference accuracy is maintained, but computational resource consumption and transmission costs increase
Solution Approach 1:
The system performs preliminary data reduction at the data originator before transmission. Data is reduced using algorithms and factors determined in advance, so that only essential data is transmitted and processed, maintaining inference accuracy while reducing computational resource consumption during actual service delivery
Solution Approach 2:
The system changes the parameter of data size by applying reduction factors to transform full-size data into reduced data. This parameter transformation allows the system to work with smaller data volumes while preserving the essential information needed for accurate inferences
2Loss of information
If data is transmitted in full size, then complete information is available for processing, but transmission time and costs increase
Solution Approach 1:
Data reduction is performed in advance at the originator before transmission occurs. This preliminary action ensures that only the necessary information is packaged for transmission, eliminating redundant data and thereby reducing transmission time while preserving information completeness
Solution Approach 2:
The system extracts and removes unnecessary data elements from the full dataset, keeping only the essential information needed for accurate inference. This extraction process reduces the transmitted data volume without losing critical information
3Measurement precision
If data reduction factors are updated dynamically, then inference accuracy is optimized, but system complexity increases
Solution Approach 1:
The system implements feedback mechanisms where inference results are evaluated against criteria, and this feedback is used to dynamically update reduction factors. The feedback loop continuously optimizes the reduction process, improving inference accuracy while automating the adjustment of complexity
Solution Approach 2:
The reduction factors are made dynamic rather than static, allowing the system to adapt to changing conditions and data characteristics. This dynamic adjustment enables optimization of inference accuracy while the system automatically manages the complexity through adaptive algorithms
Data Source
AI summary
Methods and systems for managing distribution of data. Distribution of data in a system may consume limited computing resources. To manage the computing resources used for data distribution, data that may be distributed may be reduced in size. The amount of reduction may be set based on criteria. The resulting distributed reduced size data may be usable for various purposes including, for example, providing computer implemented services. The computer implemented services may be any type and quantity of services.