Video query method based on visual large model, electronic device, and medium

Abstract

This invention discloses a video query method, electronic device, and medium based on a large visual model. The method includes: evaluating the value of each video frame in the original video stream, setting a value score threshold, classifying video frames with a value score greater than the threshold as high-value video frames, and vice versa; storing the preprocessed high-value and low-value video frames into a structured video database; responding to a user's natural language query, the large language model compiles it into a structured query canonical representation; converting the structured query canonical representation into a directed acyclic graph (DAG), and determining the query execution order of each operator based on the DAG; obtaining several candidate video segments; and scheduling the query plan of each operator to be executed on its corresponding candidate video segments based on a context-based multi-armed slot machine algorithm, thereby obtaining the video query result.

Images