A Multimodal Fusion Saliency Detection Method Based on Spatial Pyramid Pooling

Abstract

The invention discloses a multimodal fusion saliency detection method based on a spatial pyramid pool, which constructs a convolutional neural network in the training stage, including an input layer, a hidden layer and an output layer, the input layer includes two sub-input layers, and the hidden layer Including 10 neural network blocks, 2 spatial pyramid pool multimodal fusion layers, 4 convolutional layers, 3 deconvolutional layers, 3 transitional convolutional layers, the output layer includes 3 sub-output layers; the training is concentrated The respective three-channel components of each left-viewpoint image and depth image are input to the convolutional neural network for training, and three saliency detection maps corresponding to each left-viewpoint image are obtained; the set formed by calculating the saliency detection map is compared with the real The convolutional neural network training model is obtained by the loss function value between the sets formed by the human eye gaze; in the test phase, the convolutional neural network training model is used for prediction; the advantage is that the detection accuracy is high and the detection efficiency is high.

Description

technical field [0001] The invention relates to a visual saliency detection technology, in particular to a multi-modal fusion saliency detection method based on a spatial pyramid pool. Background technique [0002] In recent years, saliency detection has become a very attractive research topic in computer vision. Visual saliency detection is a method of identifying the most obvious objects or regions in an image, and it has been used as a preprocessing step in computer vision with great success in vision applications such as object redirection, scene classification, visual tracking, image Retrieval, Semantic Segmentation, etc. Inspired by the human visual attention mechanism, many early visual saliency detection methods exploit low-level visual features (such as color, texture, and contrast) and heuristic priors to simulate and approximate human saliency. These traditional techniques are considered useful because of maintaining good image structure and reducing computation...

AI Technical Summary

Problems solved by technology

[0004] Most of the existing saliency detection methods use the method of deep learning, and there are many models that use the combination of convolution layer and pooling layer. However, the feature map obtained by simply using convolution operation and pooling operation is single and not It is representative, which will lead to less feature information of the obtained image, and eventually lead to poor effect of the obtained saliency prediction map and low prediction accuracy

Images