Visual-semantic composite network and method for forming the network

Abstract

The application discloses a visual semantic composite network system and a method for generating the system. The system may include: a collecting device configured to retrieve a plurality of images and a plurality of texts associated with the images according to a given query keyword; a semantic concept determining device configured to respectively determine the retrieved image and a semantic concept of the retrieved text for said retrieved image; descriptor generating means configured to generate a textual descriptor and a visual descriptor for the determined semantic concept from the retrieved image and text; and a semantic correlation means, It is configured to determine semantic relevance and visual relevance respectively from the generated textual and visual descriptors, and combine the determined semantic relevance and determined visual relevance to generate the visual-semantic composite network system.

Description

technical field [0001] The present application relates to a visual-semantic complex network system and a method for generating the system. Background technique [0002] The large and growing number of images on the web has enabled many important applications related to web image search, browsing, and clustering. The purpose of such applications is to provide users with easier access to images on the web. A key problem facing all these tasks is how to model image correlations on the web. This problem is especially challenging due to the huge diversity and complex structure of web images. Most search engines rely on textual information to retrieve web images and measure their relevancy, and such methods have some well-known drawbacks. Due to the ambiguity of textual descriptions, images retrieved by the same keyword may originate from unrelated concepts, and exhibit greater diversity in visual content. What's more, some related images indexed under different keywords such ...

AI Technical Summary

Problems solved by technology

However, such visual features and ANN algorithms are only effective for images whose visual content is very similar (i.e., approximately replicated)

It cannot find related images with the same semantic meaning but some differences in visual content

[0003] All of the above methods only allow users to interact with a large number of web images at the micro level, i.e., search for images in a very small local area in the text or visual feature space, thus limiting the effective access to web images.

Although there have been works trying to manually organize part of network images, such work is based on artificially defined linguistic ontology, which is fundamentally different from frequently changing network images.

and it's also very expensive to scale

Images