Multimodal Age Estimation via Federated Learning and TEE
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for age estimation in interactive communication networks face challenges in accurately and privately determining user age, especially for minors, due to inadequate self-declaration mechanisms, privacy concerns, and high costs associated with centralized machine learning models, while also lacking effective solutions that integrate both audio and visual modalities.
Innovation Solution
A method for estimating user attributes on a client device using local processing, which captures and analyzes user-generated content from multiple modalities such as audio, video, and text, employing machine learning classifiers to dynamically update age labels during interactive sessions, and utilizing federated learning for model adaptation and improvement without sharing data with servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If centralized machine learning models are used for age estimation, then model accuracy can be improved through large datasets, but user privacy is compromised due to data collection and transmission requirements
Solution Approach 1:
A trusted execution environment (TEE) acts as an intermediary between the user's personal data and the machine learning model. The TEE securely processes biometric data locally on the user's device, extracting age-related features without exposing raw data to external systems. This mediator enables accurate age estimation while maintaining privacy compliance with GDPR and other regulations.
Solution Approach 2:
The system performs age estimation self-service on the user's own device rather than requiring data transmission to centralized servers. The local machine learning model processes biometric data (facial features, voice characteristics) directly on the user's smartphone or computer, eliminating the need for data collection and transmission while maintaining estimation accuracy.
2Measurement precision
If biometric data is collected for accurate age verification, then estimation precision improves, but compliance with privacy regulations becomes difficult
Solution Approach 1:
The system extracts only the essential age-related features from biometric data within the trusted execution environment, rather than collecting and storing complete biometric datasets. The TEE processes facial geometry, voice pitch, and other age-indicative features locally, extracting minimal necessary information for age verification while discarding or not transmitting sensitive personal data, thus achieving both accuracy and GDPR compliance.
3Productivity
If multiple signal modalities are processed locally, then processing speed increases, but device complexity increases
Solution Approach 1:
The processing system is segmented into two parts: a lightweight feature extraction component that runs locally on the user's device for real-time age estimation, and a more complex model training component that runs on remote servers. The local device performs rapid inference using pre-trained models, while the server handles computationally intensive model updates, distributing complexity appropriately.
Solution Approach 2:
The machine learning models are pre-trained on servers using large datasets before being deployed to user devices. This preliminary action on the server side prepares optimized model weights and structures that can then be efficiently executed on resource-constrained local devices, enabling real-time processing without requiring the local device to perform complex training computations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A privacy-preserving, real-time method may estimate automatically an attribute of a user of a multimedia device, such as the user age range, during a multimedia interactive session. The attribute estimation may be done locally on the user device by combining and cross-training machine learning classifiers on at least two modalities such as voice, text, haptics, video, image, sound, or other media modalities originating from the user-generated content during the interactive session. The method may further employ a federated learning architecture so that the attribute estimation and cross-training updates of the machine learning classifiers happen without the need to share user's personal data or user-generated content beyond the local device, thus ensuring compliance with privacy regulations in particular for minors. Such a method also offers a considerable cost advantage over server-side implementations.