Text-Guided Video Summaries for Dense Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video summary technologies face challenges in identifying specific objects within compressed video data due to increased object density and the need for multiple trained models for various actions, making it difficult to detect and analyze events efficiently.
Innovation Solution
A video summary system that utilizes an image detection module to convert text into a text embedding vector and an image into an image embedding vector, calculating similarity to identify objects matching the input text, with training data enhanced by Korean captions generated using a large natural language model to improve performance for Korean text.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If video summary technology compresses original video by selecting frames with dynamic objects, then video length is reduced and key events are preserved, but the number of objects appearing simultaneously increases making identification difficult
Solution Approach 1:
The patent introduces text as an intermediary element that mediates between the user's search intent and the visual content in the summary video. Users input text descriptions of objects or events they want to find, and the system uses this text to guide the detection and filtering process, making it easier to locate specific objects amidst the increased object density in compressed videos
Solution Approach 2:
The patent segments the video analysis process into distinct modules: a text encoder that processes user input, an image encoder that processes video frames, and a similarity calculation module that matches them. This segmentation allows each component to specialize in one task, improving overall detection accuracy despite the increased complexity of analyzing compressed videos with multiple objects
2Measurement precision
If traditional object detection uses multiple trained models for various actions, then detection accuracy for specific actions improves, but system complexity and training requirements increase significantly
Solution Approach 1:
The patent creates a universal image detection module that can handle multiple types of objects and actions simultaneously through a single unified model. This model uses dual-encoder architecture with text and image encoders that work together to detect various objects and actions without requiring separate trained models for each type, significantly reducing system complexity while maintaining detection accuracy
Solution Approach 2:
The patent changes the detection approach from using multiple specialized models with different trained parameters to a single model that uses adjustable similarity thresholds and text embeddings. By changing the text input and similarity calculation parameters, the same model can detect different objects and actions, eliminating the need for extensive retraining for each new detection task
Data Source
AI summary
The video summary system according to an embodiment of the present disclosure identifies objects by transforming text descriptions into text embedding vectors via a text encoder and images into image embedding vectors using an image encoder. By assessing the similarity between these vectors, users can input a description of an object or its actions to detect the said object. This approach mitigates the challenge of monitoring when the object count in the summarized video surpasses that of the original.