Source-Side Video Tagging for Selective VLM Frame Processing

The patent applies preliminary action by tagging frames with relevant information markers during video capture or encoding, before the machine learning model processes them. This allows the system to identify and flag important frames in advance, so that when frames are sampled for processing, the model receives both the sampled frames and the pre-identified relevant frames, ensuring no relevant information is lost while maintaining reduced computational load through selective processing.

The patent extracts only the relevant frames from the entire video stream by using relevance markers embedded during encoding or capture. Instead of processing all frames, the system identifies and extracts specifically those frames marked as relevant (through motion detection, object detection, or other tagging mechanisms during encoding), then processes only these extracted frames through the machine learning model, significantly reducing computational resource consumption while capturing all relevant information.

The patent applies preliminary action by pre-tagging frames with relevance information during the video encoding or capture phase. This preliminary classification of frames based on their content (using motion vectors, object detection, or other analysis during encoding) creates a metadata layer that guides subsequent processing. The machine learning model then processes frames in a simplified manner by following these pre-established relevance markers, achieving both processing simplicity and capture of critical details without requiring complex real-time analysis during model execution.