Video Character Recognition Using Occlusion-Aware Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI-based character recognition systems are inefficient, inaccurate, and not scalable for identifying characters and objects in video clips, particularly when facial features are occluded, and require significant human intervention.
Innovation Solution
A real-time intelligent character recognition system that uses a processor, object detector, image recognizer, and cognitive tracker to identify and track characters and objects in video clips, even when facial features are occluded, by implementing cognitive learning operations and deep learning techniques to generate tagged face identity models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional face detection tools are used, then recognition accuracy is maintained for forward-facing stills, but the system becomes inefficient and not scalable for video clips with occluded faces
Solution Approach 1:
The system dynamically adapts its recognition approach based on whether faces are visible or occluded. When faces are occluded, it switches from face-based recognition to object-based recognition using associated objects as identifiers, enabling continuous tracking throughout the video clip without manual intervention
Solution Approach 2:
Associated objects serve as intermediaries between the occluded face and the recognition system. When facial features are not visible, the system detects and tracks objects associated with the character (such as clothing, accessories, or items they are holding) to maintain identification
2Measurement precision
If manual face identity tests are applied across video clips, then identification accuracy improves, but the process becomes labor intensive and not scalable
Solution Approach 1:
The system performs automatic character identification and tracking without requiring human intervention. It autonomously detects faces, generates identity models, tracks characters across frames, and applies face identity tests programmatically, eliminating the need for manual analysis of thousands of hours of video content
Solution Approach 2:
The system pre-generates face identity models and associated object databases before processing video clips. This preliminary preparation enables rapid automated recognition during video analysis, eliminating the need for manual face identity tests during the actual processing phase
3Ease of operation
If face-based recognition is used, then identification is straightforward for visible faces, but the system fails when facial features are occluded or lighting conditions vary
Solution Approach 1:
The system implements multiple recognition functions: face-based recognition when faces are visible, object-based recognition when faces are occluded, and associated object tracking as a supplementary method. This multi-functional approach ensures reliable character identification across all viewing conditions throughout the video clip
Solution Approach 2:
The system prepares alternative recognition methods in advance. When face detection fails due to occlusion or poor lighting, pre-established object associations and character models serve as backup identification mechanisms, ensuring continuous reliable recognition without interruption
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Examples of a character recognition system are provided. In an example, the system may receive an object detection requirement pertaining to a video clip. The system may identify a visual media feature map from visual media data to process the object detection requirement. The system may implement an artificial intelligence component to segment the visual media feature map into a plurality of regions, and identify a plurality of image proposals therein. The system may implement a first cognitive learning operation to allocate a human face identity for a human face and an object name for an object present in the video clip. The system may determine a face identity model for the human face present in the plurality of image proposals and generate a tagged face identity model. The system may implement a second cognitive learning operation to assemble the plurality of frames with an appurtenant tagged face identity model.