Sign language translation method and sign language-voice interaction method and system based on MR

A technology of sign language translation and voice interaction, applied in the field of MR, can solve the problems of delayed diagnosis, scarcity of sign language medical staff, and inconvenience of deaf-mute patients in seeking medical treatment, so as to enhance the experience and improve the efficiency of diagnosis and treatment

Inactive Publication Date: 2020-01-24
BEIJING HUAJIE IMI TECH CO LTD
View PDF5 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

According to the latest statistics, the number of deaf-mute patients in my country has reached 20.57 million, accounting for 1.67% of the total population. However, the

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Sign language translation method and sign language-voice interaction method and system based on MR

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] The technical solution will be described in detail below through a preferred embodiment and in conjunction with the accompanying drawings.

[0037] A sign language translation method, comprising the steps of:

[0038] (1) Obtain RGB-D images of sign language actions. RGB-D images are color (RGB) + depth (Depth Map) images, which can be realized by TOF, RGB binocular, structured light and other solutions, and can be realized by depth cameras such as 3D somatosensory cameras Captured; after shooting, the color image and RGB-D image are registered and synchronized. The registration algorithm makes the depth camera and the color camera capture exactly the same scene, and the color image and depth pixel mapping can be performed. The synchronization function can ensure Synchronous output of color and depth;

[0039] (2) Extract the gesture feature information in the RGB-D image, the gesture feature information is the characteristic rotation matrix of each joint point of the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a sign language translation method and a sign language-voice interaction method and system based on MR. The RGB-D images of the sign language of a sign language user are collected and translated, the virtual character model of the sign language user in a virtual scene reads the sign language paraphrases into audio, and the virtual character model of a voice user demonstrates the sign language actions corresponding to the voice paraphrases. According to the invention, real-time translation of sign language actions is realized; the translation method can be used for interacting with a voice user; the method can be applied to various scenes, especially in doctor-patient communication scenes, doctors and patients can perform barrier-free communication in a familiar language mode, the diagnosis and treatment efficiency is improved, the experience of the two parties is enhanced, the MR technology shows the doctor-patient communication scenes after sign language translation, and the perfect effect of doctor-patient communication of the deaf-mute patients is achieved. And the sign language translation system can be more accurate by using the deep learning technologyto train the model.

Description

technical field [0001] The invention relates to MR technology, in particular to a sign language translation method, an MR-based sign language-voice interaction method and system. Background technique [0002] As a visual language, sign language mainly expresses meaning through the movement of hands and arms supplemented by appropriate facial expressions and mouth shapes. It is an important way for deaf-mute patients to communicate with the outside world. According to the latest statistics, the number of deaf-mute patients in my country has reached 20.57 million, accounting for 1.67% of the total population. However, sign language medical staff in hospitals in my country is very scarce, so it is inconvenient for deaf-mute patients to seek medical treatment and often delays diagnosis. How to improve the quality of life of deaf-mute patients and enable them to better integrate into society is a subject of national key research. [0003] With the further development of MR techn...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G10L15/22G10L15/26
CPCG10L15/22G10L15/26G06V40/28
Inventor 闫国启李骊
Owner BEIJING HUAJIE IMI TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap