Sign language translation system based on computer vision

Abstract

The invention belongs to the field of sign language translation and particularly discloses a sign language translation system based on computer vision. The system comprises a semantic library module,a coordinate processing module, a data processing module, a reference system module, a cross training module and a collection and recognition module, wherein the semantic library module is used for constructing a semantic library, collecting sign language actions, analyzing key features of each action and recording video data according to the features; the coordinate processing module is connectedwith the semantic library module; the coordinate processing module is associated with an openpose model; 21 key points are respectively arranged on the left hand and the right hand in the openpose model; and the coordinate processing module processes the video data of each action by using the openpose model, obtains coordinates of the key points in a video picture, outputs point coordinates of the picture frame by frame, and obtains a plurality of corresponding json files for each action. The system has high translation efficiency, greatly improves the recognition efficiency through an extraction processing mode, and greatly improves the recognition accuracy by a multi-frame collection mode.

Description

technical field [0001] The invention relates to the field of sign language translation, in particular to a computer vision-based sign language translation system. Background technique [0002] Sign language, as a unique way of communication, has been widely used by people who cannot speak. Through the changes of gestures, images or syllables are simulated to form meanings or words equivalent to human body language. Information communication between people is of great significance. Sign language can help deaf-mute people reintegrate into society, and has a positive role in promoting the harmonious development of society. 33% of the total number of people. Sign language has become their main way of communication in daily life, but most people do not understand sign language in daily life, and communication through pen and paper is also very limited, which leads to communication barriers between deaf-mute and normal people; Therefore, the realization of sign language translat...

AI Technical Summary

Problems solved by technology

[0003] The existing sign language translation system has a small amount of data and can only translate a small number of sentences. At the same time, the processing of sentence segmentation for multiple consecutive sentences is not perfect, and the picture needs to be relatively clear.

Images