Method and system for video instance retrieval combined with specific characters and scenes

Abstract

The invention relates to a video instance retrieval method and system for a specific person and a scene, including the instance retrieval of a specific person in a video, the specific scene retrieval based on local and global combination optimization, and the video instance retrieval based on high score retention. Neighbor-extended video instance retrieval, fusing specific person retrieval and specific scene retrieval results, including for each shot, fusing the initial scene retrieval results with the person retrieval results after neighbor expansion, and then fusing the initial person retrieval results with the neighbor-extended For the scene retrieval result, take the maximum value of the two fusion results to obtain the shot sorting result of the video instance retrieval. The video instance retrieval and sorting results provided by the invention are more reliable, and have strong expansibility and applicability.

Description

technical field [0001] The invention belongs to the technical field of video retrieval, and relates to a video instance retrieval technical solution, in particular to a video instance retrieval method and system combining specific characters and scenes. Background technique [0002] In the evaluation of video analysis and retrieval technology, video instance retrieval refers to a given query sample (a large number of video clips or image data) and a video library, and retrieves all video clips (shots) in which a given query sample appears from the video library. , sorted according to their similarity to the given query examples. The query samples can be several images containing specific objects such as specific people, cars, and objects in different scenes, and sometimes video clips containing the object are also given. Video instance retrieval combined with specific characters and scenes refers to retrieving the clips of a specific character appearing in a specific scene ...

AI Technical Summary

Problems solved by technology

[0004] Chinese patent document number CN105678250A, published (announcement) date 2016.06.15, discloses a face recognition method and device in a video, the face recognition method and device in the video described in the invention adopts a dynamic recognition method, using video The relevant features of each frame image in the time dimension complement the information of each frame image, thereby improving the accuracy of face recognition. Although this method belongs to the field of video retrieval technology, this method only performs the retrieval of people and does not There is no scene retrieval, which is different from the research perspective of video instance retrieval that combines specific characters and scenes

[0005] Chinese Patent Document No. CN106022313A, published (announcement date) 2016.10.12, discloses a face recognition method that can automatically adapt to the scene. The face recognition method that automatically adapts to the scene described in the method refers to the use of convolutional neural networks. Compared with the traditional manual operation, it is more automatic and does not involve the retrieval of scenes. It is different from the research angle of video instance retrieval that combines specific characters and scenes.

Images