Continuous sign language recognition method based on spatiotemporal residual network and temporal convolutional network

Abstract

The present invention is a continuous sign language recognition method based on a spatiotemporal residual network and a time-series convolutional network. The method adopts a spatiotemporal residual network to overcome the disadvantage of using a three-dimensional convolutional neural network with a large amount of computation and the short-term use of a two-dimensional convolutional neural network. The defect of insufficient spatiotemporal feature extraction; the time-series convolutional network enhances the temporal correlation between block-level features, which can solve the long-term dependence problem in the RNN network to a certain extent, and then solve the inherent conditional independence of CTC to a certain extent. Hypothesis brings the problem of missing correlation between output words.

Description

technical field [0001] The technical solution of the present invention relates to the field of deep learning image processing and pattern recognition, and specifically relates to a continuous sign language recognition method based on a spatiotemporal residual network and a time series convolutional network. Background technique [0002] Continuous sign language recognition aims to identify each isolated word sign in a continuous sign language video. Specifically, it uses a computer to use deep learning related methods to extract sign language features in continuous sign language videos, and then perform end-to-end recognition. Sign language recognition involves many research fields such as video acquisition and processing, computer vision, human-computer interaction, pattern recognition, and natural language processing. [0003] The sign language used by the deaf-mute is composed of a large number of gestures, body movements and facial expressions. A difficult problem in co...

AI Technical Summary

Problems solved by technology

[0005] The technical problem to be solved by the present invention is to provide a continuous sign language recognition method based on spatio-temporal residual network and temporal convolutional network. Shortcomings and insufficient extraction of short-term spatio-temporal features using two-dimensional convolutional neural networks; time-series convolutional networks enhance the time correlation between block-level features, which can solve the problem of long-term dependence in RNN networks to a certain extent, and then To a certain extent, it solves the problem of lack of correlation between output words caused by the inherent conditional independence assumption of CTC

Images