Salient target detection system and method fused with three-mode image

Abstract

The invention provides a saliency target detection system and method fused with a three-mode image, and belongs to the technical field of image saliency detection. According to the system, more detailed fusion of advanced features of three modals is realized through global attention weighted fusion, and large-scale features are obtained through dilated convolution operation; for large-scale features, a feature matrix multiplication mode is used to store relevance of overall information of feature maps; more sufficient fusion of cross-modal information is realized through bimodal attention fusion, a depth image is taken as a main guide, other two modal special images are respectively taken as auxiliary supplementation, the two modals supplement each other, and characteristics are processed by combining and using modes of cavity convolution, matrix multiplication, matrix addition and the like, so that better cross-modal fusion is realized; complementary fusion of three-mode information is realized through three-mode interactive weighting, and meanwhile, on the basis of inheriting decoding features of the previous layer, detailed features of the current layer are supplemented, so that information is continuously enriched in the whole decoding process.

Description

technical field [0001] The invention belongs to the technical field of image saliency detection, and in particular relates to a saliency target detection system and method for fusing three-mode images. Background technique [0002] Salient object detection is mainly used to detect the most important and useful objects or regions in an image. Salient object detection is used as a preprocessing step to replace the original image with the detected object area, and enter the next stage of processing and analysis, such as image segmentation, object tracking, object retrieval and recognition, etc. [0003] In the past ten years, most of the research has focused on the salient object detection (SOD for short) of visible light RGB (RGB is the color of the three channels of red, green, and blue), that is, RGB-SOD. RGB salient object detection utilizes rich color and texture information in visible light images, and achieves good detection results. However, in some complex scenes suc...

AI Technical Summary

Problems solved by technology

However, the existing salient target detection methods based on deep learning have the following disadvantages: ① In the face of more complex scenes, the RGB single-modal salient target detection method can no longer achieve satisfactory results

②The existing RGB-D-based salient target detection method can only be used as auxiliary information, and does not solve the problem of poor prediction results under complex conditions such as rainy days, heavy fog, and darkness

③The existing RGB-T salient target detection method, under the condition of clear RGB image, the prediction result is easily affected by the T image, resulting in inaccurate prediction results

Images