A Sign Language Recognition System Based on Pretrained Hand Model Perceptual Representation
A recognition system and pre-training technology, applied in the field of sign language recognition, can solve the problems of limited interpretability, sign language video error detection, low recognition performance, etc., and achieve good application prospects and performance enhancement effects.
Active Publication Date: 2022-07-15
UNIV OF SCI & TECH OF CHINA
View PDF7 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005] Existing approaches to sign language recognition employ a straightforward data-driven paradigm with supervised learning under text corresponding to sign language videos, but they often have limited interpretability
At the same time, due to the limited scale of sign language data, existing methods usually overfit
Likewise, despite the potential computational superiority of hand poses, existing pose extractors often suffer from false detections for sign language videos, leading to poor final recognition performance
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0014] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present invention.
[0015] In order to solve the technical problem of low recognition performance of mobile phones in the prior art, the embodiment of the present invention provides a sign language recognition system that can pre-train the perception representation of a hand model. The system mainly includes two important stages:
[0016] 1) In the self-supervised pre-training stage, better hand model perception representation is obtained through pre-training.
[0017] Hand ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a sign language recognition system capable of pre-training hand model perception and representation. The system includes two stages of self-supervision pre-training and downstream task fine-tuning. These two stages are carried out in sequence. The introduction of hand shape prior and combined with the mask modeling strategy can better capture the multi-level context information in sign language videos. After good pre-training, only simple fine-tuning is needed to enhance the performance of downstream tasks. Has a good application prospect.
Description
technical field [0001] The invention relates to the technical field of sign language recognition, in particular to a sign language recognition system capable of pretraining the perception representation of a hand model. Background technique [0002] Sign language is a visual language and it is the preferred mode of communication for the hearing-impaired. In order to facilitate convenient communication between hearing and deaf people, automatic sign language recognition technology came into being and has been widely studied. The technology has broad social significance and application prospects. The sign language recognition task is defined as recognizing an input sign language video as the corresponding text, which is a fine-grained classification problem. [0003] Hands play a leading role in sign language representation. It usually occupies a relatively small size and has a dynamic background. At the same time, the hand exhibits a similar appearance with self-occlusion...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06V40/20G06V40/10G06V10/764G06V10/82G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06V40/28G06V40/107G06N3/047G06N3/045G06F18/241G06F18/2415
Inventor 李厚强周文罡胡鹤臻赵伟超
Owner UNIV OF SCI & TECH OF CHINA