Scene character interactive understanding system for visually impaired people

Abstract

The invention discloses a visual impaired crowd-oriented scene character interactive understanding system, which comprises a mobile phone mobile terminal APP and a rear-end visual interactive calculation processing platform, the mobile phone mobile terminal APP comprises a voice wake-up component, a visual scene shooting component, a voice problem acquisition component, a voice-to-text conversion component, a logic judgment component, a data transmission transceiving component and a text-to-voice synthesis component; the back-end visual interaction processing computing platform comprises an input preprocessing module and a multi-head attention mechanism model. According to the method and the device, the character information in different scene pictures can be identified, the scene information is autonomously acquired by a user, the environmental adaptability and the expandability are high, and the sensitivity and the accuracy of character identification in the scene are high. Dynamic answering can be carried out according to user questions, and practicability and real-time performance are higher; the system can be installed on a mobile phone mobile terminal, can carry out information interaction through voice, and is convenient to use, low in cost consumption and easy to use for a user.

Description

technical field [0001] The invention belongs to the field of computer vision technology and the field of Internet technology, and in particular relates to a method for assisting visually impaired people to obtain text information in images based on visual description technology and visual question answering technology through voice interaction. Background technique [0002] According to the statistics of the Ministry of Health, there are as many as 14 million blind people in China, ranking first in the world, and the scale of the visually impaired population including blind people is still expanding. The inconvenience and danger of the daily life of the visually impaired make the visually impaired have a greater demand for facilities that can assist them in their normal lives, and various visually impaired aids have also emerged as the times require. As far as text-assisted recognition is concerned, most of the existing products cannot realize intelligent recognition of text...

AI Technical Summary

Problems solved by technology

However, the limitation of traditional visual question answering and visual description lies in the general description, which cannot describe the specific text in the image, and the description effect of image text information is poor.

Images