Multimodal automatic scoring method for college English speech

Abstract

The invention provides a multi-modal automatic scoring method, medium and equipment for college English speech. The method comprises the following steps: acquiring historical speech data; extracting text features, audio features and video features, and performing model training to obtain a language use evaluation sub-model, a speech expression evaluation sub-model and a non-language evaluation sub-model; generating a fourth data set according to the output of the three sub-models and the comprehensive score; performing model training to obtain a multi-modal fusion learning model; obtaining a speech video to be scored, extracting corresponding text features, audio features and video features, and outputting corresponding single scores through the three sub-models; inputting the single score into a multi-modal fusion learning model, and outputting a final score result corresponding to the speech video to be scored through the multi-modal fusion learning model; multi-modal scoring can be carried out on the English speech, and the scoring accuracy and the scoring efficiency are improved; and meanwhile, the cost for scoring the English speech is reduced.

Description

technical field [0001] The invention relates to the technical field of deep learning, in particular to a multimodal automatic scoring method for college English speeches. Background technique [0002] College English speech is a communicative activity characterized by multimodality. During the speech process, the speaker needs to use verbal and non-verbal modes to cooperate with each other. [0003] In related technologies, when scoring English speeches, most of them start from a single mode or directly manually score. Wherein, the unimodal scoring is, for example: extracting the speech in the speech process, and then scoring the speech according to the speech; or obtaining the speech text and scoring the speech text. However, the scoring methods of these methods are too single, resulting in inaccurate final scoring results. However, manual scoring is often subject to subjective influence, resulting in unstable scoring results; moreover, manual scoring requires a lot of t...

AI Technical Summary

Problems solved by technology

However, the scoring methods of these methods are too single, resulting in inaccurate final scoring results

However, manual scoring is often subject to subjective influence, resulting in unstable scoring results; moreover, manual scoring requires a lot of time and energy wasted, and the cost is high

Images