Multi-modal canonical correlation analysis method for zero sample classification

Abstract

Provided is a multi-modal canonical correlation analysis method for zero sample classification. The method comprises steps of: solving a mapping matrix by using the visual feature of training data and semantic features of corresponding classes; mapping the visual feature of test data and the semantic features of unseen classes onto a public space by using the mapping matrix; and classifying the test data. The method may seek for the public space between the visual features of images and the semantic features of multiple modals, and may acquire higher accuracy in zero sample classification. Thus, the method is an effective zero sample image classification method. The method is simple and practicable and excellent in effects, can be suitable for other multi-modal classification and retrieval problems besides the zero sample image classification.

Description

technical field [0001] The invention relates to a method for realizing zero-sample image classification. In particular, it relates to a multimodal canonical correlation analysis method for zero-sample classification, which establishes the connection between the visual space of images and the semantic space of image categories through multimodal canonical correlation analysis, so as to realize zero-sample image classification . Background technique [0002] For traditional image classification systems, in order to accurately identify a certain type of image, corresponding labeled training data must be given. However, the labels of training data are often difficult to obtain. Zero-shot image classification is an effective means to solve the problem of missing category labels. Its purpose is to imitate the ability of humans to recognize new categories without seeing actual visual samples. Zero-shot image classification systems use labeled training data, that is, categories th...

AI Technical Summary

Problems solved by technology

However, the original space formed by the semantic features of category names often cannot describe the category structure of the dataset well.

Images