Convolutional neural network-based unsupervised multi-modal subspace clustering method

Abstract

The invention provides a convolutional neural network-based unsupervised multi-modal subspace clustering method. The method mainly comprises a multi-modal encoder, a self-expression layer and a multi-modal decoder. The process of the method is as follows: firstly, for input multi-modal data, the spatial fusion is realized through an encoder and the input data are fused into a potential space expression through a space fusion network. Secondly, a fused result is input into the self-expression layer, and the combined expression is coded in the potential space through self-expression. Finally, the combined expression generated by the output of the self-expression layer is input into the multi-modal decoder, so that different modalities are reconstructed. In this way, a final clustering resultis obtained. According to the invention, the problem that the existing subspace clustering method depends on the spatial correspondence between modals and the data dimension can be increased is solved. The combined expression can be obtained by utilizing the self-expression of the modal, and the accuracy of subspace clustering is improved.

Description

technical field [0001] The invention relates to the field of computer vision, in particular to an unsupervised multi-modal subspace clustering method based on a convolutional neural network. Background technique [0002] Any practical application in image processing, image recognition, and speech processing needs to deal with very high-dimensional data, however, these data are usually located in low-dimensional subspaces, and subspace clustering methods use different subspaces in a data set By finding clusters in the space, high-dimensional data can be processed. The subspace clustering method can be applied to image processing, which can effectively improve the efficiency of data processing; in terms of image recognition, the subspace clustering method can process different forms of images of the same object to improve recognition accuracy; also in speech processing , applying the subspace clustering method can more effectively deal with human sentences in various tones. ...

AI Technical Summary

Problems solved by technology

[0004] In view of the problem that the previous subspace clustering method relies on the spatial correspondence between modalities and increases the data dimension when outputting, the purpose of the present invention is to provide an unsupervised multimodal subspace based on convolutional neural network For the clustering method, for the input multimodal data, the encoder is used to realize the spatial fusion first, and then it is fused into the latent space representation through the space fusion network, and then the fusion result is input into the self-expression layer, and the self-expression is used in the latent space representation. The space encodes the joint representation, and finally, the joint representation generated by the output of the self-expression layer is input into the multimodal decoder, and the different modalities are reconstructed to obtain the final clustering result

Images