A new media material matching method based on cross-modal retrieval

Abstract

This invention discloses a new media material matching method based on cross-modal retrieval, relating to the fields of video retrieval and new media material processing technology. It addresses the problem that when candidate video segments cross a cut point, the initial state of the action before the cut point and the final state of the action after the cut point are used together for video feature formation, making it impossible to determine whether the two states change continuously within the same continuous shot. The method obtains retrieval text containing state-change actions and candidate video segments, identifies the target object, the initial state of the action, the final state of the action, and the cut point, and forms a corresponding similarity range based on the cross-modal similarity of multiple frames before and after the cut point. In continuous shots that do not cross the cut point, the analysis starting point is determined; when a reverse similarity change occurs, the starting point is re-determined until the similarity range after the cut point is entered, thus obtaining the target video segment and re-performing cross-modal matching.

Images