Non-supervision joint visual concept learning system and method based on images and characters

Abstract

The invention discloses a non-supervision joint visual concept learning system and method based on images and characters. The system comprises a character resolving module, a cardinal example learning module and a multi-task clustering module, wherein the character resolving module is used for extracting corresponding nouns from additional sentence description of an image to serve as visual concepts and cardinal numerals to serve as additional constraint information of a next module through social media; the cardinal example learning module is used for training classifiers of the visual concepts by a multi-example learning method guided by cardinals; and the multi-task clustering module is used for processing cross-concept diversities, namely, clustering nouns indicating similar objects into large classes to serve as the visual concepts. By adopting non-supervision automatic leaning, the problem of complexity in implementation of manual calibration under large-scale data can be solved effectively.

Description

technical field [0001] The invention relates to a visual concept method in the field of computer vision, in particular to an unsupervised joint visual concept learning system and method based on images and text. Background technique [0002] In the field of computer vision, traditional image classification and object detection methods rely more or less on human annotations, such as image-level or image-instance-level labels. In recent years, with the development of computer technology and the emergence of big data, large-scale visual concept learning has become an emerging research hotspot, and it is not easy to manually label millions or even tens of millions of data. Therefore, unsupervised learning is used for large-scale Learning visual concepts at scale is exactly what is needed today. [0003] Since it is particularly difficult to learn visual concepts from the picture itself, existing methods mostly rely on supervision or weak supervision. Existing visual concept le...

AI Technical Summary

Problems solved by technology

[0006] Aiming at the defects in the prior art, the present invention provides an unsupervised joint visual concept learning system and method based on image and text, which can effectively solve the complex problem of manual calibration under large-scale data by using unsupervised automatic learning

Images