Interactive point reading system based on gesture recognition

Abstract

The invention provides an interactive point reading system based on gesture recognition, which takes an image acquired by a camera in real time as input, preprocesses the input image, performs gesture recognition on a target image containing a hand by using a gesture recognition network, and comprises the following steps of: firstly, classifying each pixel point in the image by using the gesture recognition network; determining the category of each pixel point to obtain a hand segmentation image, and performing gesture classification according to the obtained hand segmentation image to achieve gesture recognition; transmitting the preprocessed image into an image recognition module, detecting a bounding box of a to-be-recognized object, performing matching with the bounding box according to a gesture recognition result, and performing translation and voice reading processing according to a matching result, so as to achieve a point reading function. The system achieves gesture recognition through a gesture recognition network, further achieves an interactive point reading function, and gets rid of the dependence of a traditional point reading algorithm on a point reading tool and the constraint limitation of an existing interactive point reading system on a recognition object.

Description

technical field [0001] The invention relates to gesture recognition, fingertip detection, target detection, character detection, recognition, gesture segmentation, belongs to the field of artificial intelligence, and specifically relates to an interactive point-reading system based on gesture recognition. Background technique [0002] With the continuous development of science and technology, intelligent education is gradually promoted. For example, text point reading pens, point reading machines, etc. have been widely used. At present, several interactive point reading systems rely on point reading devices (such as point reading machine, point reading pen), these systems use the point reading device to directly extract the required text, and then do the necessary processing on the text, so as to realize the point reading function; some systems also use some special sensing materials worn on the fingertips, and then Recognize the fingertips; some use artificial intelligence ...

AI Technical Summary

Problems solved by technology

[0003] First, the dependence of special identification equipment;

[0004] Second, the constraints of the recognition object, that is, some interactive point-of-view systems can only recognize text, but cannot recognize non-text objects;

[0006] Fourth, the semantics of interactive point-to-point reading are not rich enough. The existing system can only realize text point-to-point reading, but cannot realize intelligent human-computer interaction, and cannot meet people's needs for intelligence

Images