Visual auxiliary information determination method, system and equipment based on scene and storage medium

Abstract

The embodiment of the invention provides a scene-based visual auxiliary information determination method, system and device and a storage medium, relates to the technical field of artificial intelligence, and is used for solving the technical problem that a blind assisting system in the related technology is difficult to carry out perception and decision making based on the actual environment of a user. The scene-based visual auxiliary information determination method comprises the following steps: acquiring an image of a scene where a target object is currently located; extracting the image features according to the image; obtaining a question text associated with the scene; extracting question text features according to the question text; and determining visual auxiliary information corresponding to the question text according to the correlation between the image features and the question text features. According to the embodiment of the invention, cognition and reasoning of the environment where the target object is located are realized, the purpose of replacing the target object to recognize the scene where the target object is located is achieved, and the visual assistance effect is improved.

Description

technical field [0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to a method, system, device and storage medium for determining visual auxiliary information based on a scene. Background technique [0002] At present, the interaction between the visually impaired and the blind assistance system is mostly based on the interaction of natural language modal information according to their own needs. requirements in some special scenarios. Some blind assistance systems mainly answer some structured information or natural language question and answer tasks, which only involve the processing and recognition of natural language, and do not consider the actual scene information of the visually impaired. people's questions. For example, most blind assistance systems rely on the Internet search function. In most cases, the answer to the user's question is simply the presentation of the Internet search result, which cannot meet...

AI Technical Summary

Problems solved by technology

[0002] At present, most of the interactions between the visually impaired and blind assistance systems are based on natural language modal information based on their own needs. However, since the visually impaired cannot obtain the information of the scene they are in, this interaction usually cannot meet the needs of the visually impaired in a certain environment. needs in some special scenarios

Some blind assistance systems mainly answer some structured information or natural language question-and-answer tasks, which only involve the processing and recognition of natural language, and do not consider the actual scene information of the visually impaired, so it is impossible to realize visually impaired people based on the actual scene. answering people's questions

For example, most blindness assistance systems rely more on the Internet search function, and in most cases, the answers to users' questions are simply the presentation of Internet search results, which cannot meet the actual needs of users

Or, some blindness aiding systems use target detection technology in computer vision to detect roadblocks or find objects, but this method stays on simple recognition and detection of objects, and it is difficult to perceive and make decisions about the user's environment

Images