RGBD salient object detection method based on twin network

Abstract

The invention discloses an RGBD salient object detection method based on a twin network in the technical field of image processing and computer vision. The RGBD salient object detection method comprises the steps of 1, obtaining an RGB image and a depth image of a to-be-detected picture; 2, inputting the RGB image and the depth image into a 'twin network-decoder' neural network, outputting an RGBDsaliency detection result, and performing joint training on the 'twin network-decoder' neural network in advance, including the twin network and the decoder; s2, inputting the RGB image and the depthimage into a twin network, and outputting RGB and depth hierarchical features of a twin network side path; and inputting the RGB and depth hierarchical features into a decoder, and outputting an RGBDsaliency detection result. According to the method, a twin network is combined with a decoder network structure with a fusion function, and hierarchical features are subjected to feature fusion and then decoded, so that RGB information and depth information supplement each other, the detection performance is improved, and a refined RGBD detection result is obtained.

Description

technical field [0001] The invention relates to the technical fields of image processing and computer vision, in particular to an RGBD salient object detection method based on a twin network. Background technique [0002] Salient object detection intends to automatically detect areas or objects that human eyes focus on in images or scenes, and the detection results are called saliency maps, which can be used in various computer vision applications such as target detection and recognition, image compression, image retrieval, Content-based image editing. Although there are many salient object detection models and algorithms for RGB (that is, the input picture is a single RGB color image), for RGBD—that is, the input is a single RGB color image and its corresponding scene depth (Depth) map. Sexual object detection methods have been relatively lacking. With the increasing popularity of depth cameras such as Microsoft Kinect, Intel RealSense, and mobile phone depth cameras, the...

AI Technical Summary

Problems solved by technology

Chen et al. proposed "Progressively complementarity-aware fusion network for rgb-dsalient object detection" in 2018. This method uses two-way parallel neural networks (two-way parallel neural network structures are inconsistent, and parameters are not shared) for RGB and depth information. The features are extracted separately and then fused. Although the detection effect is better than the traditional method of manually designing features, the two-way parallel network structure increases the model parameters while making the whole model more complex, difficult to train, and not conducive to mining RGB. The commonality of the salient features between the map and the depth map, so the detection effect is limited in complex scenes

Images