Multimodal Manifold Embedding Method for Zero-Shot Learning

Abstract

A multimodal manifold embedding method for zero-shot learning, including: input image features of training samples, text vector features corresponding to images, and weight parameters; respectively calculate the diagonal matrix and edge of each type of training samples The weight matrix and the Laplacian matrix corresponding to each class of training samples; use the Laplacian matrix of each class to construct the Laplacian matrix of all classes; calculate the multimodal manifold embedding matrix. The present invention improves the current multimodal embedding method, makes full use of the manifold information between data, achieves the purpose of effectively utilizing data information and improving the classification effect, and is a method suitable for multimodal classification and retrieval correlation domain embedding methods. The method of the present invention belongs to the method based on text vector, and can map the features of different modalities to a common space, and can calculate the similarity between different modalities in this space.

Description

technical field [0001] The invention relates to a feature embedding method of zero sample learning. In particular, it concerns a multimodal manifold embedding method for zero-shot learning. Background technique [0002] With the needs of real-world applications, zero-shot learning has gained a lot of attention. Its common method is to transform the image modality and text modality of the seen category into a common embedding space, and map the image modality of the unseen category to the common space to find its corresponding text modality. In order to determine the category it belongs to. [0003] From the perspective of embedding space, zero-shot learning can be divided into three categories: attribute feature-based methods, text vector-based methods, and methods that utilize both attribute features and text vectors. [0004] Attribute-based methods: Attribute-based methods have been used in zero-shot learning for a long time. This method first establishes an attribute ...

AI Technical Summary

Problems solved by technology

[0006] The method of combining attribute features and text vector features: attribute features and text vector features can complement each other in zero-shot learning. In order to mine more semantic information, many current studies combine attribute features and text features to obtain better classification effect, but this method also has the disadvantages of attribute-based methods and cannot be applied to large-scale zero-shot learning

Images