Video Frame Cropping Using Multi-Frame Subject Identity Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video image processing methods fail to accurately track and display the main subject in complex environments, leading to inaccurate cropping and scaling, resulting in non-consecutive presentation of the main subject during video calls or surveillance.
Innovation Solution
A video image processing method that determines main subjects using identity information across multiple frames, crops and scales the image based on the main subject's location, ensuring accurate and consecutive display of the main subject.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If subject sensing technology is applied to perform human body detection and tracking in real-time, then the effect of 'a picture moves with a subject' is achieved, but erroneous detection and missing detection occur in complex device environments
Solution Approach 1:
The system performs preliminary subject detection and identity assignment in historical frames before the current frame. By pre-establishing subject identities and tracking their locations across multiple frames, the system prepares reference information that helps maintain reliable subject tracking even when detection fails in complex environments.
Solution Approach 2:
The system uses detection results from historical frames as feedback to improve current frame detection. By comparing subject locations and identities across multiple frames, the system can correct erroneous detections and fill in missing detections, thereby improving overall detection accuracy and reliability in complex environments.
2Adaptability or versatility
If cropping and scaling is performed based on real-time subject location, then the displayed picture adapts to subject movement, but the cropped and scaled image cannot completely display the subject when detection is inaccurate
Solution Approach 1:
The system performs preliminary cropping and scaling operations based on subject locations from historical frames before the current frame is processed. By pre-establishing the cropping region based on reliable historical subject positions, the system ensures that the subject is completely displayed even when current frame detection is inaccurate.
Solution Approach 2:
The system uses subject location information from historical frames as feedback to guide the cropping and scaling process. By continuously refining the cropping region based on tracked subject positions across multiple frames, the system maintains complete subject display while adapting to subject movement, preventing truncation even during detection fluctuations.
3Productivity
If only current frame subject detection is used, then the processing is simple and fast, but the subject location positioning is inaccurate leading to non-consecutive pictures
Solution Approach 1:
The system performs preliminary subject detection and identity assignment in historical frames, storing subject identity information and location data for future reference. This preliminary action creates a tracking buffer that maintains subject continuity across frames, ensuring consecutive picture display without requiring complex real-time re-detection for each frame.
Solution Approach 2:
The system uses subject tracking information from historical frames as feedback to maintain continuous subject identification in the current frame. By carrying forward subject identities and locations from previous frames, the system ensures that the same subject is consistently tracked across multiple frames, achieving consecutive picture display while maintaining processing efficiency.
Data Source
AI summary
Embodiments of this application disclose a video image processing method and apparatus. A specific solution is as follows: obtaining identity information and location information of each subject in an ith video image frame; determining M main subjects from the ith video image frame based on identity information of subjects in N video image frames before the ith video image frame, where the identity information of the subjects in the N video image frames includes identity information of the M main subjects; cropping the ith video image frame based on location information of the main subjects, where a cropped ith video image frame includes the M main subjects; and scaling down or scaling up the cropped ith video image frame, so that a display displays the cropped ith video image frame based on a preset display specification.