Federated Speech Recognition Model Aggregation

The system segments the training process into local and global phases. Local models are trained independently on user devices using unlabelled data, while only model weight updates (not raw data) are transmitted to the central server for aggregation. This segmentation allows the system to achieve high recognition accuracy through distributed training while preserving user privacy by keeping sensitive audio data localized.

The patent introduces an intermediary mechanism where local acoustic model weights serve as intermediaries between user data and the global model. Instead of sharing raw audio data, the system exchanges processed model parameters that capture linguistic patterns without exposing sensitive information. This intermediary approach enables knowledge transfer while maintaining privacy boundaries.

The system performs preliminary training actions at the local level using unlabelled data before global aggregation. Local models are pre-trained on device-specific unlabelled audio data, capturing user-specific acoustic characteristics. This preliminary action reduces the need for extensive human labelling of every dataset, as the federated aggregation process combines these pre-trained local models to build the global model.

The system enables self-service training where local devices automatically train their own acoustic models using available unlabelled data without requiring manual intervention for data collection or labelling. The federated learning framework automatically aggregates these self-trained local models, eliminating the need for centralized data annotation teams and significantly reducing labor costs and training time.