Driver Speech Valence Detection via Acoustic-Phonetic Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies fail to effectively identify and mitigate negative emotions in drivers, which contribute to aggressive driving behaviors and road rage, leading to a significant number of fatal crashes.
Innovation Solution
A method using a data processing system with a processor and memory to determine the valence of a driver's speech utterance by combining acoustic and phonetic features through a multi-stage fusion model, which includes convolutional and fully-connected sub-networks, to generate a valence vector that can trigger vehicle control maneuvers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current technologies are used to identify driver emotions, then the system simplicity is maintained, but the accuracy of negative emotion detection is insufficient
Solution Approach 1:
The system segments emotion detection into multiple independent modules: acoustic feature extraction, phonetic feature extraction, and valence prediction. Each module processes specific aspects of speech independently, then their results are combined to achieve high detection accuracy while maintaining manageable system complexity through modular design
Solution Approach 2:
The system merges acoustic features (pitch, energy, duration) and phonetic features (phoneme sequences, lexical content) into a unified valence prediction model. This combination of multiple feature types enables comprehensive emotion detection that overcomes the limitations of single-feature approaches
2Measurement precision
If a multi-stage fusion model is used to combine acoustic and phonetic features, then the valence prediction accuracy is improved, but the computational complexity increases
Solution Approach 1:
The system performs preliminary feature extraction and processing before the main prediction stage. Acoustic features are extracted from audio signals and phonetic features are extracted from speech text separately in advance, then fed into the valence prediction model. This preliminary processing organizes data efficiently, reducing the computational burden during real-time prediction
Solution Approach 2:
The system introduces intermediate feature representations as mediators between raw audio/text inputs and final valence predictions. Acoustic features serve as intermediaries for audio input, phonetic features serve as intermediaries for text input, and their combination forms an intermediate representation that the prediction model processes to generate valence scores
Data Source
AI summary
A method includes receiving acoustic features and phonetic features associated with an utterance from a driver in a vehicle, providing the acoustic features and the phonetic features to a feature fusion sub-network, receiving a feature fusion utterance representation from the feature fusion sub-network, providing one of the acoustic features or the phonetic features to a non-fusion sub-network trained using supervised learning, receiving a non-fusion utterance representation from the non-fusion sub-network, generating an intermediate utterance representation based on the feature fusion utterance representation and the non-fusion utterance representation, providing at least a portion of the intermediate utterance representation to a fully-connected sub-network trained using supervised learning, receiving a valence vector from the fully-connected sub-network, and causing a vehicle control system to perform a vehicle maneuver based on the valence vector.