Multi-modal feature-based stress detection method and system

Abstract

The invention provides a multi-modal feature-based stress detection method and system. The method comprises the following steps of acquiring a voice recognition result of an original voice input by auser and alignment information of a text and the voice; outputting word-level acoustic feature abstract representation by an original voice waveform and the alignment information; coding linguistic features of words in the text, wherein the linguistic features include the positions, parts of speech and meanings of the words; modeling local prominence of the stressed words relative to the neighborwords through a convolutional network according to the word-level acoustic feature abstract representation and the linguistic features, and modeling a local context dependent relationship in a statement of the original voice; extracting a global dependent relationship between the stressed words and the statement of the whole original voice through a recurrent neural network or an attention mechanism layer according to abstract features with the local context dependent relationship; and taking abstract features with the global dependent relationship as input, and outputting stress classification. The stress highlighting position in the voice can be automatically and effectively detected, so that the satisfaction degree of a user is improved.

Description

technical field [0001] The invention relates to the technical field of accent detection, in particular to an accent detection method and system based on multimodal features. Background technique [0002] Stress is the logical or emotional focus that the speaker emphasizes in the process of voice interaction. The perception and detection of accent has a very broad application prospect in the field of human-computer interaction. For example, in a human-computer spoken dialogue system, the semantic meaning and the user's true intention can be correctly understood according to the detected accented words. [0003] The simplest method of the earliest accent detection system is to manually mark the focus word. The advantage of this method is that the accuracy of the focus mark is relatively high, and it is more in line with human auditory perception. The disadvantage is that it usually requires a lot of manpower and is different. The results marked by other people will be differe...

AI Technical Summary

Problems solved by technology

[0005] The traditional accent detection model mainly has the following problems: (1) The traditional accent detection method does not effectively use the context information of the feature sequence, and the accent is a local salient feature in speech, which is highly dependent on its context; (2) Only the information of a single modality is considered to detect the accent, that is, only the acoustic information in the speech is used; (3) The acoustic features at the frame level are directly mapped to the acoustic features at the word level by means of statistics, and the loss a lot of useful information

Due to these problems, the accuracy and recall of the current stress detection are affected.

Images