Continuous sign language recognition method based on space-time residual network and time sequence convolution network

Abstract

The invention relates to a continuous sign language recognition method based on a space-time residual network and a time sequence convolutional network. According to the method, the space-time residual network is adopted, so that the defect of large calculated amount caused by completely using a three-dimensional convolutional neural network and the defect of insufficient short-term space-time feature extraction caused by completely using a two-dimensional convolutional neural network are overcome; time correlation between block-level features is enhanced through the sequential convolutional network, the problem of long-term dependence in the RNN can be solved to a certain extent, and then the problem of correlation loss between output words caused by CTC inherent conditional independencehypothesis is solved to a certain extent.

Description

technical field [0001] The technical solution of the present invention relates to the fields of deep learning image processing and pattern recognition, specifically a continuous sign language recognition method based on a spatiotemporal residual network and a temporal convolutional network. Background technique [0002] Continuous sign language recognition aims to recognize each isolated word sign language in continuous sign language videos. Specifically, it is to use computers to use deep learning-related methods to extract sign language features in continuous sign language videos, and then perform end-to-end recognition. Sign language recognition involves multiple research fields such as video acquisition and processing, computer vision, human-computer interaction, pattern recognition, and natural language processing. [0003] The sign language used by the deaf-mute consists of a large number of gestures, body movements, and facial expressions. One of the difficulties in ...

AI Technical Summary

Problems solved by technology

[0005] The technical problem to be solved by the present invention is to provide a continuous sign language recognition method based on spatio-temporal residual network and temporal convolutional network. Shortcomings and insufficient extraction of short-term spatio-temporal features using two-dimensional convolutional neural networks; time-series convolutional networks enhance the time correlation between block-level features, which can solve the problem of long-term dependence in RNN networks to a certain extent, and then To a certain extent, it solves the problem of lack of correlation between output words caused by the inherent conditional independence assumption of CTC

Images