Video saliency object detection model and system based on cross attention mechanism

Abstract

The invention relates to a video saliency object detection method and system based on a cross attention mechanism. The method comprises the following steps: A, inputting an input adjacent frame imageinto a similar network structure sharing parameters, and extracting high-level and low-level features; b, performing feature re-registration and alignment on the saliency features in the single-frameimage by using a self-attention module; c, utilizing an inter-frame cross attention mechanism to obtain the relationship dependence on the position of the salient object on the inter-frame space-timerelationship, acting on the advanced feature as a weight, and capturing the consistency of salient object detection on the space-time relationship; d, fusing the extracted intra-frame advanced features and low-level features of the adjacent frames and the space-time features with the inter-frame dependency relationship; e, performing feature dimension reduction on the input features, and outputting a pixel-level classification result by using a classifier; and F, establishing a depth video saliency object detection model based on a cross attention mechanism, and accelerating the training of the model by using GPU parallel computing.

Description

technical field [0001] The invention belongs to the field of video salient object detection and video segmentation, and in particular relates to a video salient object detection model and system based on a cross-attention mechanism. The model and system use the short-term memory function of the cross-attention mechanism to keep progress While improving the accuracy of intra-saliency detection, capture the saliency correlation and consistency information between consecutive adjacent frames, so as to complete the pixel-level classification task of video salient object detection. Background technique [0002] The human visual system can quickly and accurately locate highly discriminative objects or scene areas (also known as salient objects) in the field of vision, which has led to the simulation, research and exploration of human visual perception in the visual field. Studies have shown that the human visual attention mechanism analyzes and integrates part of the information i...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to propose a video salient object detection model and system based on a cross-attention mechanism, aiming to solve the problems of model accuracy and temporal feature modeling in the prior art

Images