A Sign Language Recognition Method Based on Wrist Motion Sensor

Abstract

The invention discloses a sign language recognition method based on a wrist motion sensor, comprising: step 1, data collection: through a wrist device with a motion sensor, continuously collect sensor data of hand motion when the hand is moving; step 2, data cutting Divide: Segment and process the sensor data collected in the data collection step into multiple pieces of sensor data, where each piece of sensor data corresponds to a complete handwritten sentence; Step 3, data preprocessing: divide the multiple pieces of sensor data obtained after data segmentation into short The time-Fourier transform process is converted into a time-frequency image, and the high-frequency part of the time-frequency image is filtered to remove the noise-removed time-frequency image; Step 4, sign language action recognition: input the noise-removed time-frequency image to the pre-trained neural network The end-to-end prediction in the network recognizes sign language actions, and outputs the corresponding text according to the recognized sign language actions. The method can recognize sign language movements accurately and in real time, and output corresponding translation results.

Description

technical field [0001] The invention relates to the fields of human-computer interaction and mobile applications, in particular to a sign language recognition method based on a wrist motion sensor. Background technique [0002] Sign language expresses meaning through a series of hand movements and positions, and is an autonomous and self-consistent natural language specially designed and used by people with hearing impairments. According to the report of the China Disabled Persons' Federation, at the end of 2010, the total number of disabled people in my country was 85.02 million, of which 20.54 million were deeply troubled by hearing impairment. However, there is a huge communication barrier between sign language users and ordinary spoken language users. In the scenes of life, such as social activities, academic conferences, etc., it brings great difficulties to the normal communication of hearing-impaired people. [0003] The existing sign language recognition methods ca...

AI Technical Summary

Problems solved by technology

It is inappropriate to directly apply these methods to the accelerometer and gyroscope data processing that only have the motion information of the wrist, and it is difficult to achieve effective recognition

Images