Speech recognition method and system for dysarthria based on visual facial contour movement

A technology for dysarthria and speech recognition, applied in speech recognition, speech analysis, instruments, etc., can solve problems such as confusion of vowels and consonants, difficulty in meeting speakers, and difficulty in collecting speech with dysarthria, so as to improve accuracy. Effect
CN113241065BActive Publication Date: 2022-05-24BEIJING TECHNOLOGY AND BUSINESS UNIVERSITY
6 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN Β· China
Patent Type
Patents(China)
Current Assignee / Owner
BEIJING TECHNOLOGY AND BUSINESS UNIVERSITY
Publication Date
2022-05-24

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a method and system for speech recognition of dysarthria based on visual facial contour movement. The system includes multi-modal data acquisition, multi-modal fusion feature calculation, multi-modal speech recognition calculation and language model calculation modules; multi-modal The state data acquisition calculation module is used to obtain the facial contour motion video data of the dysarthria and the voice data synchronized with the video; the multi-modal fusion feature calculation module is used to fuse the facial contour motion features and speech acoustic features; multi-modal speech recognition The calculation module is used to obtain the mapping relationship from multimodal features to phoneme characters; the language model calculation module is used to obtain the mapping relationship from phoneme characters to Chinese sentences. The present invention obtains the fused multimodal features by fusing the speech acoustic feature parameters and the pronunciation actions of the dysarthria, and utilizes the fused multimodal features to perform dysarthria speech recognition, thereby effectively improving dysarthria speech recognition Accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of speech recognition for dysarthria, in particular to a method and a system for enhancing speech recognition for dysarthria based on the movement trajectory of visual facial contours, which can be applied to assist speech rehabilitation of dysarthria. Background technique

[0002] Existing research shows that the proportion of dysarthria caused by stroke is 30%-40%, of which 15% of dysarthria caused by stroke cannot be completely recovered. The dysarthria seriously affects the speaker's ability to communicate, resulting in a decline in the quality of life, which brings both physical and psychological pain to the dysarthria. Therefore, improving the intelligibility of dysarthria speech and effectively improving the speech recognition ability of dysarthria has important social significance and practical value.

[0003] At present, the mainstream practice in the field of dysarthria speech recognition technolo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More