Joint embedded model for zero sample learning

Abstract

A joint embedded model for zero sample learning comprises the steps of: (1) inputting an image feature X of a training sample, a text feature Y corresponding to an image, and weight parameters alpha, beta and lambda; calculating the sum of all text feature vectors (FORMULA as shown in the description), then calculating a feature conversion matrix M (FORMULA as shown in the description), wherein I is a unit matrix; outputting the feature conversion matrix M. According to the joint embedded model for zero sample learning, a target function is constructed by using the correlativity between different modals so that the correlation of two modal features belonging to an identical category in public space is maximized and meanwhile the correlation of two modals belonging to different categories in public space is minimized. According to the joint embedded model for zero sample learning, a transition matrix is learned by using a training sample so that the similarities between features belonging to different modals can be compared with each other.

Description

technical field [0001] The present invention relates to a joint embedding model. In particular, it concerns a joint embedding model for zero-shot learning. Background technique [0002] With the development of the Internet, multi-modal data on the network continues to grow, and multi-modal learning has gradually become a research hotspot in machine learning and data mining. Multimodal learning is to establish relationships between data features of different modalities. Ideally, multimodal learning can integrate feature information of different modalities into a common representation space to achieve comparison and retrieval at the same semantic level. In the era of big data where multimodal data is constantly growing, with the increasing demand for mining technology for multimodal data, the traditional data mining technology for unimodal models can no longer meet people's requirements. How to mine effective information between different modalities is an important task in t...

AI Technical Summary

Problems solved by technology

The disadvantages of this type of method are: long training time, high complexity, and easy to fall into local optimum

Most of the shallow multimodal models are models composed of one or two layers of structure. The advantage of this method is that it is relatively simple and low in complexity. However, shallow models are used to correlate modal features on some complex data sets. there are still some difficulties

Images