Local Feedback Mechanism for Speech Recognition Privacy

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

In traditional speech recognition systems, users are hesitant to provide audio data for feedback due to privacy concerns, which hinders the developers' ability to enhance and update the systems with accurate data, especially in local application environments.

Innovation Solution

Implementing a local feedback mechanism that filters user data to address privacy concerns, allowing for the collection and submission of filtered data to system developers, while also enabling the classification of previously uncertain data as non-private, thereby incentivizing users to contribute valuable feedback for improving speech recognition models.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If users provide audio data for feedback to system developers, then the accuracy and enhancement of speech recognition models is improved, but privacy concerns and user trust deteriorate

Engineering Contradiction:
Improvespeech recognition accuracyVSAvoidprivacy risk
Core Design Contradiction:
Measurement precisionVSObject-affected harmful factors

Solution Approach 1:

The patent introduces a local feedback mechanism as an intermediary between users and system developers. This mechanism filters and processes audio data locally on the user's device before transmitting any information to the developer. The local feedback mechanism acts as a mediator that protects user privacy by ensuring sensitive audio data never leaves the local device, while still enabling the system to benefit from user feedback for model enhancement.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent extracts only the necessary feedback information from the audio data at the local level, separating the useful feedback signals from the private audio content. By extracting only metadata, feedback labels, or processed information rather than raw audio, the system maintains the ability to improve models while removing the harmful element of private audio transmission.

Inventive Principle:
Principle #2Taking out (Extraction)

2Productivity

If raw audio data is transmitted to system developers for training, then model enhancement is improved, but data privacy and user security deteriorate

Engineering Contradiction:
Improvemodel training efficiencyVSAvoiddata security
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The local feedback mechanism serves as a security intermediary that processes data locally before any transmission occurs. This ensures that even if transmission channels are compromised, no sensitive audio data can be intercepted. The mechanism mediates between the need for data transmission and the need for security by processing all sensitive operations locally.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent segments the data processing workflow into local processing (on user device) and remote processing (on developer server). Sensitive audio processing occurs locally, while only anonymized or processed feedback information is transmitted remotely. This segmentation isolates the security-critical operations from the network transmission layer.

Inventive Principle:
Principle #1Segmentation

3Measurement precision

If users are required to provide feedback data, then speech recognition system accuracy is improved, but user participation and ease of use deteriorate

Engineering Contradiction:
Improverecognition accuracyVSAvoiduser participation
Core Design Contradiction:
Measurement precisionVSEase of operation

Solution Approach 1:

The local feedback mechanism enables users to participate in model enhancement through self-service operations on their own devices. Users can provide feedback by simply interacting with the speech recognition system normally, and the local mechanism automatically captures, processes, and transmits the feedback without requiring users to manually upload audio files or provide sensitive information. This maintains ease of use while improving accuracy.

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS10157609B2Local and remote aggregation of feedback data for speech recognition
Publication Date: 2018.12.18 MICROSOFT TECHNOLOGY LICENSING LLC
  • US10157609B2 patent drawing
  • US10157609B2 patent drawing
  • US10157609B2 patent drawing

AI summary

A local feedback mechanism for customizing training models based on user data and directed user feedback is provided in speech recognition applications. The feedback data is filtered at different levels to address privacy concerns for local storage and for submittal to a system developer for enhancement of generic training models.