Cross-modal image saliency detection method

Abstract

The invention discloses a cross-modal image saliency detection method. The paired multi-modal images are inputted and different modes are divided by a super-pixel segmentation algorithm so as to obtain uniform super-pixel regions with approximate size; a multimodal image saliency detection model based on the graph manifold sorting algorithm is designed, and the cross-modal soft consistency constraint and the manifold sorting fitting term sparsity constraint are introduced; the super-pixels on four sides of the image are used as seed nodes and the similarity between other nodes and seed nodes is calculated so as to obtain a preliminary saliency map; referring to the foreground points obtained in the previous stage as the seed nodes and the similarity of other nodes to the node is calculated, and the final saliency map is obtained. A method for complementarily fusing the multimodal images based on the graph manifold sorting algorithm is provided, and l1 norm is introduced to realize cross-modal soft consistency constraint and the manifold sorting function fitting term sparsity constraint, i.e. partial inconsistency is allowed on the basis of coordination of multiple modes, and the robustness of the fitting term is increased.

Description

technical field [0001] The invention relates to a computer vision technology, in particular to a cross-modal image saliency detection method. Background technique [0002] Saliency detection based on manifold sorting is based on background nodes and foreground nodes, and an image saliency detection algorithm is proposed. Firstly, the salient image based on the background node set is calculated by using the background node as prior knowledge, combined with the image color features and positional relationship, because the background node is determined, according to the contrast relationship between the image superpixel and the background node, the foreground object can be superimposed The pixels are well highlighted, but the suppression of background noise is insufficient, and then the foreground is used as prior knowledge, and the saliency image based on the foreground node set calculated by using the image color features and position relationship can suppress the background ...

AI Technical Summary

Problems solved by technology

[0005] The technical problem to be solved by the present invention is: the existing technology is easy to introduce modal noise or error, so that the multi-modal information cannot be well fused together, and a cross-modal image saliency detection method is provided

Images