Translation model training method, translation method and system for sign language video in specific scene

Abstract

The invention discloses a translation model training method, a translation method and a translation system for a sign language video in a specific scene, and belongs to the field of video natural language generation. According to the invention, during training, a filtering network is trained through the constructed actor-double-critic deep reinforcement learning training architecture, then the keyframe sequence set of the sign language video is screened out through the trained filtering network, and finally the deep sequence self-encoding network is trained based on deep learning. The filtering network removes the space-time redundancy of the video frames, and the depth sequence self-coding network fuses the attention mechanism, so that the trained sign language video translation model has high translation accuracy, and the translation method and system of the sign language video in the specific scene are further provided. And on the basis that the trained translation model has relatively high translation accuracy, sign language video translation is carried out, so that the accuracy of sign language video translation is effectively improved.

Description

technical field [0001] The invention belongs to the field of video natural language generation, is applied to sign language video recognition, and specifically relates to a translation model training method, a translation method and a system of sign language video in a specific scene. Background technique [0002] There are about 72 million speech and hearing impaired groups in China. This group uses sign language as a tool to communicate, but sign language has not been widely popularized in the whole society. There are many inconveniences when the speech and hearing impaired groups carry out social activities. The current public environmental facilities and product design often ignore the special needs of this group. Especially in some public places such as stations, airports, and civil service places, it is very difficult for normal people who cannot understand sign language to understand the meaning of sign language. This situation hinders communication and communication...

AI Technical Summary

Problems solved by technology

[0005] Technical problem: The present invention aims at the problem of poor translation accuracy when translating existing sign language videos, and provides a translation model training method for sign language videos in specific scenes, collecting data sets of specific scenes, and effectively training through sign language video translation models, Obtain a sign language video translation model with high translation accuracy, and based on the trained sign language video translation model, provide a specific scene sign language video translation method and system, thereby effectively improving the accuracy of sign language video translation

Images