Cross-modal image-label correlation learning method for social images

Abstract

The invention belongs to the technical field of cross-media relevance learning, and particularly relates to a social image oriented cross-modality image-label relevance learning method. The invention comprises three algorithms: multi-modal feature fusion, bidirectional relevancy measuring and cross-modality relevancy fusion; a whole social image set is described by taking a hyperimage as a basic model, the image and a label are respectively mapped into hyperimage nodes for treatment, relevancy aiming at the image and the relevancy aiming at the label are obtained, and the two different relevancies are combined according to a cross-modality fusion method to obtain a better relevancy. Compared with the traditional method, the method is high in accuracy and high in adaptivity. The method has important significance in performing efficient social image retrieval by considering multi-modal semantic information based on large-scale social images with weak labels, retrieval relevancy can be improved, user experience is enhanced, and the method has application value in the field of cross-media information retrieval.

Description

technical field [0001] The invention belongs to the technical field of cross-media correlation learning, and in particular relates to a social image-oriented cross-modal image-label correlation learning method. technical background [0002] With the development of web 2.0 technology, especially the popularity of some social networking sites, such as Flickr. Ordinary users are more likely and willing to share image resources on the Internet. How to better deal with these massive social image data, effectively organize and manage its complex structure, and then promote the cross-media retrieval of these images has become an important research hotspot [1,2,3,4,5]. ,6]. Generally speaking, each social image will be marked as a series of tags, and these tags are provided by ordinary users, and these tags usually have the user's subjective views and tendencies [7]. However, due to the problem of semantic gap, there may be huge uncertainty between the visual content of the image...

AI Technical Summary

Problems solved by technology

These methods ignore the value of rich information in multimodal features to calculate the correlation between images and labels. Therefore, it is very necessary to find suitable algorithms to fully mine and construct multimodal feature representations [21,22] ,twenty three]

[0017] (2) Evaluation of multimodal correlation. Existing methods usually only consider single-modal information when calculating correlation, either ignoring the visual information of the image itself, or ignoring the semantic information contained in the image.

Instead of making full use of multimodal information to calculate the correlation between images and labels, rich multimodal information is used to calculate the correlation between images and labels

Images