A rgbd saliency detection method based on multi-scale feature fusion

Abstract

The invention provides an RGBD saliency detection method based on multi-scale feature fusion. The invention first constructs a saliency detection model, extracts multi-level RGB image features and depth image features through a dual-stream feature extraction module; further extracts deep-level features through a multi-scale feature pooling module and a multi-scale feature aggregation module. At the same time, the deep fusion module is used to fuse the features from the feature extraction branch, the multi-scale feature pooling module and the multi-scale feature aggregation module step by step. The salient boundary refinement module performs boundary constraints through the shallow features from the RGB image feature extraction branch and the depth image feature extraction branch to achieve the purpose of refining the boundary; at the same time, it uses the output features of the deep fusion module to perform global constraints to achieve global optimization the goal of. The invention realizes end-to-end saliency prediction, introduces edge information into saliency detection, and can fully and effectively use RGB image information and depth image information to predict saliency regions.

Description

technical field [0001] The invention belongs to the field of computer vision, and in particular uses a deep convolutional neural network to fuse feature information contained in RGB images and depth images through a multi-scale method. Background technique [0002] Saliency detection aims to distinguish the most visually distinct objects or regions in a scene, and has a wide range of applications in the fields of visual tracking, image segmentation, and object detection. At the same time, with the rapid development of deep learning technology, convolutional neural network has become the mainstream method for processing saliency detection tasks. However, most of the existing saliency detection methods based on deep learning are aimed at 2D image saliency detection tasks, that is, only relying on RGB images and ignoring the corresponding depth information, which greatly limits the accuracy and efficiency of saliency detection. , especially when salient objects are indistingui...

AI Technical Summary

Problems solved by technology

Generally speaking, there are two main factors hindering the further development of RGBD image saliency detection tasks: First, although the emergence of devices such as Kinect and light field cameras has greatly facilitated the acquisition of depth information, it still inevitably introduces a large number of Noise, to a certain extent, interferes with the effective expression of depth information. At the same time, the existing depth image datasets available are extremely limited, lacking large-scale datasets such as the RGB image dataset ImageNet, and it is difficult to fit network models with complex structures. ; Second, how to effectively fuse the information of two different modalities, RGB information and depth information, is challenging. RGB images contain a lot of semantic information such as color and texture, while depth images contain rich edges and Geometric information such as shape, the two complement each other, which is conducive to more accurate highlighting of salient regions

Images