Sign language recognition method based on space-time attention mechanism

A recognition method and attention technology, applied in character and pattern recognition, computer parts, instruments, etc., can solve problems such as lack of effectiveness, inability to highlight temporal differences, and weak temporal and spatial correlations.

Active Publication Date: 2020-05-01
CHONGQING UNIV OF POSTS & TELECOMM
View PDF10 Cites 28 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the general CNN-RNN-based network is lacking in the effectiveness of spatial feature extraction, and cannot highlight the differences

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Sign language recognition method based on space-time attention mechanism
  • Sign language recognition method based on space-time attention mechanism
  • Sign language recognition method based on space-time attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0051] The technical solutions in the embodiments of the present invention will be described clearly and in detail below with reference to the drawings in the embodiments of the present invention. The described embodiments are only some of the embodiments of the invention.

[0052] The technical scheme that the present invention solves the problems of the technologies described above is:

[0053] Such as figure 1 Shown, a kind of sign language recognition method based on space-time attention mechanism, it comprises the following steps:

[0054] S1, data preprocessing. Each sign language video is sampled into 32 frames; for videos with a frame number greater than 32, the redundant frames are evenly distributed to both ends for deletion, and the key frames in the middle are retained; for videos with a frame number less than 32, in order to ensure the timing of the data , repeat the last frame. And normalize the image scale of the sampled pictures, and uniformly cut them into...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a sign language recognition method based on a space-time attention mechanism. The method comprises the following steps: firstly, sampling a sign language video into a continuous sign language sequence with a uniform length as input of a model; and then, inputting the video frame sequence into a spatial attention network formed by 3D residual blocks, so that the network canautomatically pay attention to a salient region in a space. Analyzing the extracted convolution features through a ConvLSTM convolution long-short-term memory network, extracting long-time sequence features, and distributing time attention weights of different video frames to generate feature representation of the video; and finally, the generated feature representation passes through a Softmax classifier, and classification categories are output in a vector form. According to the invention, the interference of redundant information on recognition can be reduced, and the recognition accuracy is improved.

Description

technical field [0001] The invention belongs to the field of image processing and pattern recognition, in particular to a sign language recognition method based on a spatio-temporal attention mechanism. Background technique [0002] The purpose of sign language recognition is to translate sign language into text or voice for output, so as to achieve the purpose of communication between deaf people and normal people, and between deaf people and deaf people. Sign language contains picture information and motion information, so how to effectively extract the spatio-temporal features in sign language is the focus of sign language recognition research. [0003] The traditional sign language recognition framework includes four parts: preprocessing, gesture detection, feature extraction, and classification. Among them, feature extraction and classification are the two key parts. Therefore, researchers focus on designing complex artificial features, while using traditional machine...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06K9/62G06N3/04
CPCG06N3/049G06V40/28G06N3/045G06F18/213G06F18/241G06F18/214
Inventor 罗元李丹张毅汪杰陈顺
Owner CHONGQING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap