Flexible Language Model Schema for ASR Resource Management

Solution Overview

Problem

Current language models for automatic speech recognition (ASR) require multiple large models to be maintained in memory, which can tax resources when serving multiple domains simultaneously, making it inefficient for recognition service providers.

Innovation Solution

A flexible schema for customizing language modeling components is introduced, where a computing device provides hints to a recognition service provider to combine multiple language modeling components based on specific domains, allowing for 'on-the-fly' or offline initialization of tailored language models for improved resource management.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If multiple large language models are maintained in memory to serve different domains simultaneously, then speech recognition quality across multiple domains is improved, but resource consumption and memory usage increase

Engineering Contradiction:

Improvespeech recognition qualityVSAvoidmemory usage

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The language model is divided into multiple domain-specific components (e.g., shopping domain, games domain, music domain, movies domain). Each component covers a specific domain, and these components are combined dynamically based on the speech input domain to form a customized language model, eliminating the need to maintain all domain models in memory simultaneously

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically combines language model components based on domain identification. When speech is received, the system identifies the domain and selectively activates only the relevant language model components needed for that domain, rather than using a static full-language model for all inputs

Inventive Principle:

Principle #15Dynamics

2Adaptability or versatility

If multiple large language models are maintained in memory to serve different domains simultaneously, then coverage of multiple domains is improved, but device complexity and resource management burden increase

Engineering Contradiction:

Improvedomain coverageVSAvoidresource management complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The language model is divided into multiple domain-specific components (e.g., shopping domain, games domain, music domain, movies domain). Each component covers a specific domain, and these components are combined dynamically based on the speech input domain to form a customized language model, eliminating the need to maintain all domain models in memory simultaneously

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A single language model component can serve multiple functions by being combined with different other components. The modular architecture allows each component to be reused across different domain combinations, reducing the total number of components needed while maintaining broad domain coverage

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS10497367B2Flexible schema for language model customization

Publication Date: 2019.12.03 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

The customization of language modeling components for speech recognition is provided. A list of language modeling components may be made available by a computing device. A hint may then be sent to a recognition service provider for combining the multiple language modeling components from the list. The hint may be based on a number of different domains. A customized combination of the language modeling components based on the hint may then be received from the recognition service provider.