Filler Word Detection via NLP Tokenization and Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Speech recognition systems struggle to accurately convert speech disfluencies, particularly filler words, into text due to inadequate training and the challenges posed by hesitations and context-dependent usage.
Innovation Solution
The implementation of computer programs and techniques that utilize tokenization and labeling of transcripts through Natural Language Processing (NLP) libraries, such as spaCy, to identify filler words by applying series of rules that examine patterns, relationships, verbatim, and context within the text data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If speech recognition systems use traditional training methods, then the system is simpler to implement, but the accuracy of converting filler words into text deteriorates
Solution Approach 1:
The patent segments the filler word detection process into distinct stages: tokenization of the transcript, labeling of tokens as filler words or non-filler words, and rule-based identification. This segmentation allows each component to be optimized independently, improving overall accuracy without requiring complete system redesign.
Solution Approach 2:
The patent introduces an intermediary labeling layer between the raw transcript and the final filler word identification. By first labeling tokens as filler or non-filler using NLP libraries, the system creates an intermediate representation that simplifies the detection process and improves accuracy.
2Measurement precision
If the system applies multiple rules to examine patterns and context, then the detection accuracy improves, but the processing time increases
Solution Approach 1:
The patent performs preliminary tokenization and labeling of the transcript before applying the complex rule-based detection. This preliminary action prepares the data in advance, allowing the subsequent rule application to operate on pre-processed information, thereby reducing overall processing time while maintaining high accuracy.
3Measurement precision
If the system uses NLP libraries for tokenization and labeling, then the filler word identification improves, but the dependency on external libraries increases
Solution Approach 1:
The patent implements a self-service approach where the NLP library functions are encapsulated within the system's own modules. The tokenization and labeling operations are performed as internal system functions, reducing external dependencies while maintaining the benefits of NLP-based processing.
Data Source
AI summary
Introduced here are computer programs and associated computer-implemented techniques for discovering the presence of filler words through tokenization of a transcript derived from audio content. When audio content is obtained by a media production platform, the audio content can be converted into text content as part of a speech-to-text operation. The text content can then be tokenized and labeled using a Natural Language Processing (NLP) library. Tokenizing/labeling may be performed in accordance with a series of rules associated with filler words. At a high level, these rules may examine the text content (and associated tokens/labels) to determine whether patterns, relationships, verbatim, and context indicate that a term is a filler word. Any filler words that are discovered in the text content can be identified as such so that appropriate action(s) can be taken.