Monocular Depth Estimation With Federated Training Data Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth estimation methods face challenges in generating diverse training datasets for machine learning models due to privacy concerns and the need for large-scale data collection, leading to poor generalization and accuracy issues, especially in consumer devices lacking specialized sensors.
Innovation Solution
A federated learning approach is employed to generate a training dataset on client devices using a community ML model, where images are evaluated for similarity and error metrics to determine their value for training, while maintaining privacy, and model weights are shared to update global and local models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large-scale training data collection is performed to improve model accuracy, then model accuracy is improved, but data privacy concerns worsen
Solution Approach 1:
A federated learning system acts as an intermediary between multiple client devices and the central server. Each client device trains local models using its own private data without sharing the data itself. Only model updates (gradients or weights) are transmitted to the server for aggregation, thereby improving overall model accuracy while preserving data privacy at the source.
Solution Approach 2:
The training process is segmented into multiple independent client devices, each contributing to the global model without exposing their private data. The global model is divided into local model instances that are trained independently on separate devices, allowing data to remain distributed and private while still enabling collaborative learning.
2Measurement precision
If specialized sensors are added to consumer devices for depth estimation, then depth estimation accuracy is improved, but device cost and complexity worsen
Solution Approach 1:
Physical specialized sensors (mechanical/optical systems) are replaced with a software-based machine learning model that runs on the device's existing processor. The depth estimation function is implemented through neural network computations rather than dedicated hardware sensors, reducing device complexity and cost while maintaining accuracy.
Solution Approach 2:
The machine learning model enables depth estimation functionality using the device's existing camera and processor, making the device multi-functional without adding specialized sensors. The same hardware components serve both standard imaging and depth estimation purposes.
3Adaptability or versatility
If diverse training datasets are generated from multiple client devices, then model generalization is improved, but data communication overhead worsens
Solution Approach 1:
Only the essential model update information (gradients or weight changes) is extracted and transmitted from each client device, rather than transmitting the entire training dataset. This extraction of minimal necessary data achieves diverse model training while minimizing communication overhead and energy consumption.
4Measurement precision
If continuous model training is performed on client devices, then model accuracy is improved, but device energy consumption worsens
Solution Approach 1:
Instead of continuous training, the system performs periodic training rounds where client devices train local models and submit updates to the server at intervals. The server aggregates these updates and distributes improved global models in subsequent rounds, reducing continuous energy consumption while maintaining model improvement progress.
Data Source
AI summary
Broadly speaking, this disclosure generally relates to methods, systems and apparatuses for performing monocular depth estimation, i.e. depth estimation using a single camera. In particular, this disclosure relates to a method for generating a training dataset for training a machine learning, ML, model using federated learning to perform depth estimation. Advantageously, the method to generate a training dataset enables a diverse training dataset to be generated while maintaining user data privacy. This disclosure also provides methods for training the ML model using the generated training dataset. Advantageously, the methods determine whether a community ML model that is trained by client devices needs to be retrained, and/or whether a global ML model, which is used to generate the community ML model, needs to be retrained.