ASR Encoder Features for Non-Intrusive Speech Intelligibility Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Intrusive speech intelligibility estimation methods require access to a clean reference signal, are computationally complex, and labor-intensive, making them impractical for real-time or large-scale applications, especially in noisy environments or with speakers who have atypical speech patterns.
Innovation Solution
A non-intrusive method using features derived from an automatic speech recognition (ASR) encoder to estimate speech intelligibility, disentangling intelligibility-relevant information from irrelevant factors, enabling the use of large ASR models to generate accurate intelligibility scores without a clean reference signal.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If intrusive speech intelligibility estimation methods are used, then measurement precision is improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent introduces a pre-trained neural network model as an intermediary that takes ASR encoder features as input and outputs intelligibility scores. This mediator approach allows the system to achieve accurate intelligibility estimation without requiring clean reference signals or complex intrusive processing, thereby reducing computational complexity while maintaining measurement precision.
Solution Approach 2:
The patent uses features extracted from the ASR encoder as a proxy representation of the speech signal. Instead of requiring the actual clean reference signal for comparison, the system copies and processes the ASR encoder features through the pre-trained neural network to obtain intelligibility estimates, significantly reducing computational requirements while maintaining accuracy.
2Measurement precision
If intrusive speech intelligibility estimation methods are used, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent employs a pre-trained neural network model that has been trained in advance on large datasets. This preliminary training allows the model to quickly and accurately estimate speech intelligibility from ASR encoder features without requiring time-consuming real-time processing or reference signal generation, thus reducing processing time while maintaining high measurement precision.
3Measurement precision
If human speech intelligibility evaluation is used, then measurement precision is improved, but productivity decreases
Solution Approach 1:
The patent replaces the mechanical process of human listening and subjective evaluation with an automated neural network-based system. The pre-trained neural network processes ASR encoder features to generate intelligibility scores automatically, eliminating the need for human annotators and significantly improving evaluation efficiency and productivity while maintaining measurement precision through the model's training on comprehensive datasets.
Data Source
AI summary
A method, computer program product, and computing system for non-intrusive speech intelligibility estimation. A degraded audio signal is processed in a pretrained automatic speech recognition (ASR) system; ASR encoder features of the degraded audio signal are generated; the ASR encoder features are processed to identify patterns in the ASR encoder features; patterns that represent levels of intelligibility of the audio signal are recognized; and a predicted intelligibility of the audio signal based on the recognized patterns of the ASR encoder features is determined.