RGBD saliency detection method based on feature aggregation

Abstract

The invention provides an RGBD saliency detection method based on feature aggregation. The RGBD saliency detection method comprises the steps that firstly, an input image is preprocessed; then constructing a significance detection network; the saliency detection network comprises a feature extraction network and a feature aggregation network. The feature extraction network is a pair of asymmetricdouble-flow backbone networks constructed based on ResNet50, and is divided into an RGB image feature extraction branch and a depth image feature extraction branch. The feature aggregation network comprises K nearest neighbors GNNs, a region enhancement module, a hierarchical fusion module and a block Non-local module. And finally, training a saliency detection network, and performing saliency detection through the trained saliency detection network. According to the method, 2D appearance and 3D geometric information are efficiently combined and reasoned, information from two different modes of an RGB image and a depth image is fully fused, and the multi-scale expression ability of the model is further improved through the hierarchical fusion module, so that rough-level features and fine-level features are well fused together.

Description

technical field [0001] The RGBD saliency detection method based on feature aggregation belongs to the field of computer vision, especially using convolutional neural networks to aggregate feature information of different modalities contained in RGB images and depth images. Background technique [0002] With the rapid development of computer vision and the wave of artificial intelligence, deep learning technology has been widely used. Using computer technology to simulate the attention mechanism of the human eye has become a new and challenging research hotspot. Visual salience means that when humans observe a certain area, there is a local area in the visual field that can attract human visual attention, and this local area is called a salient area. Saliency detection is mainly used to highlight salient regions in images or videos. Generally speaking, saliency detection is widely used in image segmentation, object recognition, video coding and other fields, and it is of gr...

AI Technical Summary

Problems solved by technology

2) There are inherent differences in the two different modalities of RGB image and depth image, so it is difficult to effectively fuse these two modalities through simple concatenation

Specifically, RGB images mainly reflect appearance information such as color, texture, and brightness, while depth images express more spatial and geometric information in the scene. The two are quite different. If only a simple fusion strategy is adopted, it may lead to Incompatibilities

Images