Optimization method and device for sign language recognition system based on dynamic pseudo-tag decoding

Abstract

The invention discloses a sign language recognition system optimization method and device based on dynamic pseudo-tag decoding, which is applied to the sign language recognition system. The sign language recognition system includes: a feature extractor, a time sequence modeler and a sequence decoder; the method includes: adopting continuous connection The classification is used as the optimization function, and the entire sign language recognition system is trained by the end-to-end training method to obtain a preliminary usable sign language recognition system; the parameters of the feature extractor are optimized, the parameters of the time series modeler are optimized, and the time series features extracted after optimization are used to optimize the time series. The parameters of the domain integration network are embedded back into the sign language recognition system as a temporal modeler. The present invention can better couple multiple modules in the sign language recognition system, thereby improving the accuracy of sign language recognition.

Description

technical field [0001] The invention relates to the technical field of sign language recognition systems, in particular to a method and device for optimizing a sign language recognition system based on dynamic pseudo-label decoding. Background technique [0002] With the rapid development and application of technologies such as virtual reality, somatosensory devices, and wearable devices, people's demand for multimodal human-computer interaction methods is increasing. A very important part of this is to enable the computer to accurately perceive a variety of information expressions including natural language, gesture language, and facial expressions. Sign language uses hand shape changes and arm movements, supplemented by facial expressions and lip movements to express precise semantic information, and in the long-term development, it has standardized grammar, clear semantics and a sound vocabulary system. [0003] The purpose of sign language recognition is to design a cer...

AI Technical Summary

Problems solved by technology

[0007] 1. The continuous sign language recognition system is composed of multiple modules such as feature extractor, timing modeler and sequence decoder. Traditional optimization methods only rely on end-to-end training methods, and it is difficult to optimize all modules, and the module parameters are difficult to couple with each other. The recognition rate not tall

[0008] 2. Using a single time series modeler cannot take into account both long-term and short-term sequence information. The time-domain convolutional network can model short-term phrase information, lacking the modeling of overall grammar and context; while the threshold recurrent neural network can Model long-term language information, lack of short-term visual motor modeling

Images