Multi-modal fusion saliency detection method based on spatial pyramid pool

Abstract

The invention discloses a multi-modal fusion saliency detection method based on a spatial pyramid pool. A convolutional neural network is constructed in a training stage, the convolutional neural network comprises an input layer, a hidden layer and an output layer, the input layer comprises two sub-input layers, the hidden layer comprises 10 neural network blocks, 2 spatial pyramid pool multi-modal fusion layers, 4 convolutional layers, 3 deconvolution layers and 3 convolutional layers for transition, and the output layer comprises 3 sub-output layers; inputting respective three-channel components of each left viewpoint image and each depth image in the training set into a convolutional neural network for training to obtain three saliency detection images corresponding to each left viewpoint image; obtaining a convolutional neural network training model by calculating a loss function value between a set formed by the saliency detection graph and a set formed by the real human eye gazegraph; in the test stage, a convolutional neural network training model is used for prediction; the method has the advantages of high detection accuracy and high detection efficiency.

Description

technical field [0001] The invention relates to a visual saliency detection technology, in particular to a multi-modal fusion saliency detection method based on a spatial pyramid pool. Background technique [0002] In recent years, saliency detection has become a very attractive research topic in computer vision. Visual saliency detection is a method of identifying the most obvious objects or regions in an image, and it has been used as a preprocessing step in computer vision with great success in vision applications such as object redirection, scene classification, visual tracking, image Retrieval, Semantic Segmentation, etc. Inspired by the human visual attention mechanism, many early visual saliency detection methods exploit low-level visual features (such as color, texture, and contrast) and heuristic priors to simulate and approximate human saliency. These traditional techniques are considered useful because of maintaining good image structure and reducing computation...

AI Technical Summary

Problems solved by technology

[0004] Most of the existing saliency detection methods use the method of deep learning, and there are many models that use the combination of convolution layer and pooling layer. However, the feature map obtained by simply using convolution operation and pooling operation is single and not It is representative, which will lead to less feature information of the obtained image, and eventually lead to poor effect of the obtained saliency prediction map and low prediction accuracy

Images