AI Phoneme-Audio Feature Fusion for Accurate Speech Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice interaction systems struggle with inaccurate alignment of phonemes to text, which affects the performance of speech scoring and oral tests.
Innovation Solution
An audio processing method using artificial intelligence that determines phoneme features and audio features, fuses them based on a learned mapping relationship, and performs phoneme classification to improve alignment accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If phoneme alignment is performed using traditional methods, then the system can operate with simple processing, but the alignment accuracy is insufficient
Solution Approach 1:
The patent merges audio features and phoneme features through feature fusion to create a unified representation that improves alignment accuracy. The fusion process combines complementary information from both feature types, allowing the system to achieve higher precision without requiring completely separate processing pipelines for each feature type.
Solution Approach 2:
The patent introduces an attention mechanism as an intermediary component that learns the mapping relationship between audio features and phoneme features. This attention mechanism acts as a mediator that dynamically weights and integrates the two feature types, enabling accurate alignment while maintaining modular system architecture.
2Measurement precision
If feature fusion is performed to improve classification accuracy, then phoneme alignment improves, but computational complexity increases
Solution Approach 1:
The attention mechanism serves as an efficient intermediary that computes feature mappings through learned attention weights rather than exhaustive feature combinations. This approach achieves high classification accuracy by focusing computational resources on the most relevant feature relationships identified by the attention mechanism.
Solution Approach 2:
The system changes the parameter representation by learning optimal feature fusion weights through the attention mechanism. Instead of using fixed fusion parameters, the system adapts parameters dynamically based on the input features, improving classification accuracy while maintaining computational efficiency through parameter sharing and learned transformations.
Data Source
AI summary
This application provides an audio processing method performed by an electronic device. The method includes: determining a phoneme feature of at least one phoneme of a given text; determining an audio feature of an audio frame in audio data corresponding to the text; for the audio frame: obtaining a weight of the phoneme feature of the at least one phoneme based on a mapping relationship between the phoneme feature of the at least one phoneme and the audio feature of the audio frame, fusing the audio feature of the audio frame and the phoneme feature of the at least one phoneme based on the weight of the phoneme feature of the at least one phoneme to obtain a fused feature of the audio frame, and determining a start time and a stop time of a phoneme in the audio data based on the fused feature of the audio frame.