Object and scene image understanding method based on text-object-scene relations

Abstract

The invention relates to an object and scene image understanding method based on text-object-scene relations. The method comprises steps as follows: information of three domains are integrated through a conditional random field, and the three domains include a scene, an object and a text; features of different domains are correlated through three relations which include a scene and object relation, a scene and text relation and an object and text relation; the text is related with the object and the scene in two different manners. The relation of the text and the object is represented through matching probability of noun and the object, the relation of the text and the scene is represented through occurrence probability of the noun in description of different scenes; the matching probability of the text and the object is obtained through solution of a problem about bidirectional matching optimization constrained under the weak supervision condition; the method only requires simple annotated information, the accuracy is improved, network resources can be sufficiently utilized for image understanding, and huge manual annotation cost is saved.

Description

technical field [0001] The invention relates to a method in the technical field of computer vision for signal processing, in particular to an object and scene image understanding method based on text-object-scene relationship. Background technique [0002] In the era of big data, image data on the Internet has shown explosive growth, and an intelligent vision system that can automatically extract semantic information from images is urgently needed. After unremitting efforts in the field of computer vision, significant progress has been made in many key tasks, such as object detection, scene classification, and face recognition. However, as the ultimate goal in the field of computer vision, image understanding is still a very challenging problem. This problem becomes more difficult when faced with the massive image understanding on the Internet. People can no longer rely on accurately labeled image datasets to train models as in the past, because it is expensive to accurate...

AI Technical Summary

Problems solved by technology

However, the main disadvantage of this method is that it requires a one-to-one correspondence between objects and text instances to train the model, and it is necessary to manually mark the objects corresponding to the nouns during the prediction process.

Images