Learning Apparatus for Distributed Inference via Confidence-Based Data Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed inference processing in deep neural networks faces challenges with high communication traffic due to the need for intermediate data exchange between edge devices, leading to reduced processing speed and performance.
Innovation Solution
A learning apparatus that divides target data into partial data, processes it through both a first and a second network model, calculates confidence values for each prediction, and updates model parameters based on the differences between prediction results and confidences, enabling ensemble processing and knowledge distillation to improve inference accuracy while reducing communication costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If distributed inference processing is implemented to distribute load across multiple edge devices, then processing stability and adaptability are improved, but communication traffic increases leading to reduced processing speed
Solution Approach 1:
The patent extracts and processes only the essential intermediate data needed for distributed inference, rather than transmitting all intermediate data. By identifying and transmitting only critical features and confidence values, the communication traffic is significantly reduced while maintaining inference accuracy and processing stability across distributed edge devices.
2Loss of energy
If patch images are used to reduce communication traffic, then traffic volume is reduced, but inference accuracy deteriorates due to loss of information
Solution Approach 1:
The patent applies local quality by processing different regions of the input data with different network models specialized for their specific characteristics. Each edge device processes local patches with specialized models that maintain high inference accuracy for their specific regions, while only essential intermediate results are communicated, thus reducing traffic without sacrificing overall accuracy.
Solution Approach 2:
The patent changes parameters by transmitting confidence values and essential features rather than raw intermediate data. This parameter transformation reduces the volume of communicated data while preserving the critical information needed for accurate distributed inference, effectively reducing communication traffic without degrading inference performance.
3Measurement precision
If intermediate data is communicated between devices to maintain inference accuracy, then inference performance is preserved, but communication overhead increases
Solution Approach 1:
The patent introduces confidence values as intermediary elements that mediate between different edge devices. Instead of directly communicating complex intermediate data, each device computes confidence values that serve as simplified intermediaries, enabling accurate distributed inference with reduced communication overhead and simplified coordination between devices.
Data Source
AI summary
According to one embodiment, a learning apparatus includes a processor. The processor divides target data into pieces of partial data. The processor inputs the pieces of partial data into a first network model to output a first prediction result and calculates a first confidence indicating a degree of contribution to the first prediction result. The processor inputs the target data into a second network model to output a second prediction result and calculates a second confidence indicating a degree of contribution to the second prediction result. The processor updates a parameter of the first network model, based on the first prediction result, the second prediction result, the first confidence and the second confidence.