Continuous sign language recognition system based on multi-language cooperation

A recognition system, multi-language technology, applied in character and pattern recognition, instruments, biological neural network models, etc., can solve problems such as unfavorable sign language commonality and mining, and achieve the effect of improving sign language recognition performance

Active Publication Date: 2020-12-25
UNIV OF SCI & TECH OF CHINA
View PDF5 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This approach can achieve certain results, but it ignores the problem of similar visual patterns between different sign languages, and the practice of separate and independent training is not conducive to the model's mining of sign language commonality

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Continuous sign language recognition system based on multi-language cooperation
  • Continuous sign language recognition system based on multi-language cooperation
  • Continuous sign language recognition system based on multi-language cooperation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0022] Most of the existing sign language recognition frameworks only recognize a single sign language, and different sign languages ​​have shared visual patterns. Using different sign language data sets to train multiple independent models will ignore the commonality between different language sign languages. The present invention expresses the same visual pattern between different sign language languages ​​through a shared time sequence encoder, a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a continuous sign language recognition system based on multi-language collaboration. The system uses a public visual feature encoder to extract feature expressions, and uses different time sequence modeling networks (i.e., target sequence models) to learn corresponding sign language features for sign languages of different languages. A shared time sequence encoder (namely ashared sequence model) is used for expressing the same visual mode among different sign languages, initialization is carried out through language embedding vectors, multilingual sign language recognition under a single framework is achieved through multilingual collaborative training, the visual commonality among different sign languages is fully mined, and the sign language recognition performance is improved.

Description

technical field [0001] The invention relates to the technical field of action recognition in computer vision, in particular to a continuous sign language recognition system based on multilingual collaboration. Background technique [0002] In the continuous sign language recognition problem, each sign language video is marked by an orderly sequence of sign language words. The essence of the continuous sign language recognition problem can be regarded as the process of learning the mapping relationship between the video sequence and the labeled text sequence. Generally speaking, a continuous sign language recognition system consists of a visual feature encoder and a temporal modeling model. The feature expression of video plays a very important role in continuous sign language recognition. In the early days, manual features such as SIFT and HOG were used to represent hand shapes and trajectories. With the successful application of deep learning in computer vision, two-dimens...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06V40/28G06V20/40G06N3/045G06N3/044G06F18/24
Inventor 李厚强周文罡蒲俊福胡鹤臻
Owner UNIV OF SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap