Multi-modal fusion saliency detection method based on convolution block attention module

Abstract

The invention discloses a multi-modal fusion saliency detection method based on a convolution attention module. The method comprises: in a training stage, constructing a convolutional neural network;inputting the left viewpoint image and the depth image of the original image into a convolutional neural network for training to obtain a corresponding saliency detection image; calculating a loss function between a set formed by saliency detection images generated by the model and a set formed by corresponding real human eye gaze images to obtain an optimal weight vector and an offset term of theconvolutional neural network classification training model; and in a test stage, inputting the three-dimensional image in the selected data set into the trained convolutional neural network model toobtain a saliency detection image. According to the visual saliency detection method, extraction of image features is optimized by applying a novel module, multi-scale and multi-mode feature fusion iscarried out, and finally the detection efficiency and the detection accuracy of visual saliency detection are improved.

Description

technical field [0001] The invention relates to a visual saliency detection method based on deep learning, in particular to a multimodal fusion saliency detection method based on a convolution block attention module. Background technique [0002] Recognizing salient stimuli in the visual field is an important attention mechanism in humans, that is, when looking freely, our eyes tend to pay attention to areas of the scene that have unique changes in visual stimuli, such as: bright colors, special Textural or more complex semantic aspects, this mechanism guides our eyes to salient informative, rich regions of the scene. The mechanism of human vision was first studied by neuroscientists, and its most widely used is imaging examination in the field of medical treatment. Medical imaging examination is the basis for effective follow-up diagnosis and treatment. In recent years, computer vision has also carried out research on this, and in the field of computer vision, this researc...

AI Technical Summary

Problems solved by technology

[0004] Most of the existing saliency detection methods have adopted the method of deep learning, using the method of combining the convolution layer and the pooling layer to extract image features, but the image features obtained by simply using the convolution operation and the pooling operation are not representative. characteristics, especially the pooling operation will lose the feature information of the image, which will lead to poor effect of the obtained saliency prediction map and low prediction accuracy

Images