Gesture language teaching method, device and system based on gesture action generation and recognition

A gesture movement and sign language technology, applied in the field of computer vision and pattern recognition, can solve the problems of low imitation efficiency and poor expressiveness of learners, and achieve the effect of convenient independent learning, fast construction and improvement of teaching content

Pending Publication Date: 2022-08-02
NANJING UNIV OF TECH
View PDF1 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Technical problem: The present invention is devoted to introducing the means of artificial intelligence into classroom teaching, expanding the existing sign language teaching mode, solving the problems of poor expressiveness and low imitation efficiency of learners,

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Gesture language teaching method, device and system based on gesture action generation and recognition
  • Gesture language teaching method, device and system based on gesture action generation and recognition
  • Gesture language teaching method, device and system based on gesture action generation and recognition

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0065] The present invention will be further described below in conjunction with the embodiments and the accompanying drawings.

[0066] A first aspect of the present invention provides a sign language teaching method based on gesture action generation and recognition, combining figure 1 and figure 2 As shown, in an example of the present invention, the method includes:

[0067] Step S100: Use the T-G gesture generation network to train the gesture generation model with the sign language dataset video and the sign language text label.

[0068] like image 3 As shown, the T-G attitude generation network used in the embodiment of the present invention is an adversarial attitude generation network composed of an improved Transformer generator and a conditional discriminator, and the forward Transformer network includes an encoder and a decoder. Two parts of the structure; The backward GAN network consists of two parts: generator and discriminator. Using T-G pose generation n...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a sign language teaching method, device and system based on gesture action generation and recognition, and belongs to the field of computer vision and mode recognition. The sign language teaching method comprises the steps that sign language statement texts are trained through a T-G posture generation network, a posture generation model is generated, and a ThreeDPose posture migration technology is utilized to generate a 3D virtual character action video for sign language teaching so that a learner can simulate the 3D virtual character action video; and then training the sign language data set by using a double-flow VTN network, generating a sign language translation model, performing similarity evaluation on actions of the learner and sign language teaching actions in a standard library, and feeding back a learning effect. The sign language teaching device and the sign language teaching system are used for sign language teaching according to the provided method. By utilizing the sign language teaching system, the teaching modes of the deaf-mutes are enriched, the deaf-mutes can conveniently learn sign languages by themselves, the learning interest is improved, and the sign language teaching efficiency and quality are ensured.

Description

technical field [0001] The invention belongs to the technical fields of computer vision and pattern recognition, and in particular relates to a sign language teaching method, device and system based on gesture action generation and recognition. Background technique [0002] Sign language is a tool for hearing-impaired people to communicate and express ideas. It has the same function of transmitting information as oral language, and can express some complex or abstract language concepts. Usually according to a specific grammar, finger movements are combined with body language and facial expressions to convey information, from the original simple gestures to a vivid and special language with the function of communication and communication, sign language recognition and translation technology for the hearing impaired and the sound It provides convenience for people to communicate, and its research should not only allow the able-bodied people to read sign language, but also let ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06V40/20G06V10/25G06V10/74G06V10/764G06V10/774G06V10/80G06V10/82G06V10/94G06V10/96G06N3/04G06N3/08G09B21/04G06K9/62
CPCG06N3/08G09B21/04G06N3/047G06N3/045G06F18/22G06F18/2415G06F18/253G06F18/214
Inventor 梅雪陈玉明张启航姚彦胤秦午阳
Owner NANJING UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap