Training method, translation method and translation system for sign language video translation model

Abstract

The invention discloses a training method, a translation method and a translation system for a sign language video translation model, and belongs to the field of pattern recognition and intelligent systems. The method comprises the following steps: firstly, preprocessing sign language video data in an isolated word data set, extracting a character region in a sign language video, detecting and tracking a moving hand by utilizing a Mask RCNN network model, extracting hand features by utilizing a first network module, and extracting human body features by utilizing a second network module; performing feature information fusion on the extracted hand features and human body features by using a full connection layer, and performing training to obtain a weight model of isolated words; taking theobtained weight model of the isolated word as a pre-training model of a third network module, and performing feature extraction on the continuous statement video by utilizing the third network module; and inputting an extraction result into an Seq2Seq model for training to obtain a trained sign language video translation model, and further proposing a translation method and a system architecture.The invention is good in sign language video translation effect and high in accuracy.

Description

technical field [0001] The invention belongs to the field of pattern recognition and intelligent systems, and in particular relates to a training method for a sign language video translation model, a translation method and a system. Background technique [0002] The number of deaf-mute people in my country has reached more than 20 million. Sign language is the main way for deaf-mute people to communicate with the outside world. It is a set of communication tools composed of hand shapes, movements, expressions, and postures. my country has issued a general sign language standard in 2018, but there are very few hearing people who are proficient in sign language, and it is difficult for the hearing-impaired group to convey their thoughts and intentions to normal people, which makes the life and travel of the deaf-mute still face challenges. Many difficulties. Sign language recognition, a technology that relies on machine translation of sign language, can facilitate the communic...

AI Technical Summary

Problems solved by technology

[0007] Technical problem: The present invention aims at the problems of poor detection effect and low translation accuracy in sign language translation in the prior art. The present invention provides a training method, a translation method and a translation system for a sign language video translation model, which can perform sign language translation. Better detection to improve the accuracy of sign language video translation

Images