Sparse coding algorithm suitable for multi-modal information and application thereof

Abstract

The invention discloses a sparse coding algorithm suitable for multi-modal information and an application thereof. The sparse coding algorithm comprises the following steps that 1 features of images and texts of a social medium are extracted; 2 a Laplacian matrix is established for same modal features; 3 a maximized average divergence matrix is established; 4 a target function based on sparse coding is established; 5 a search algorithm of a feature symbol is adopted to update the sparse coding, and feature representation of the multi-modal information is obtained; 6 by using the obtained feature representation to carry out intersection searching. The multi-modal information can be sufficiently used to carry out coding, the distributional difference of different modals is lowered, so that the robustness of the sparse coding is improved, and the accuracy of the intersection searching is improved.

Description

technical field [0001] The invention relates to multimedia information retrieval, in particular to a sparse coding algorithm for multimodal information and its application. Background technique [0002] In recent years, with the rise of social network platforms such as Weibo and Facebook, multimedia information has shown explosive growth, which puts forward new requirements for traditional information retrieval technology. Simple text retrieval can no longer meet the increasingly complex information retrieval needs of users. Users hope to obtain data in different modalities such as text, image, audio, and video. Cross-retrieval between multi-modal information, such as inputting an image and retrieving the related text, or inputting a piece of text and retrieving the image that best matches it, has become a hot topic in the academic circle. [0003] From the existing multi-modal information processing technology, it can be seen that the core problem is the modeling of differ...

AI Technical Summary

Problems solved by technology

[0005] First, an over-complete codebook and independent sparse coding will lead to the loss of similarity information between data during the coding process, causing similar features to be coded into sparse codes with large differences, resulting in the instability of sparse coding

[0006] Second, the traditional sparse coding algorithm does not take into account the coding of multi-modal features. In the research on cross-retrieval of multi-modal information, the query item and the retrieved item are represented by features of different modalities. There will be a lot of difference, which also affects the stability of sparse coding, thereby reducing the accuracy of cross retrieval

Images