Speaker Recognition Across Scene Switches in Multi-Person Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speaker recognition methods in multi-person conversation videos suffer from poor accuracy due to frequent scene switching and speaker movement, leading to poor continuity and integrity of the video, as they treat the entire video as a whole rather than segmenting it effectively.
Innovation Solution
The method involves scene detection to divide the video into segments, extracting facial and audio features, and using cross-scene distance features from face depth and histogram of oriented gradient features to track faces across scenes, improving recognition accuracy by recognizing speakers segment by segment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If the entire video is treated as a whole for speaker recognition, then the process is simple, but the recognition accuracy deteriorates due to scene switching and speaker movement
Solution Approach 1:
The video is divided into multiple video segments based on scene detection results. Each segment is processed independently for speaker recognition, which maintains processing simplicity while improving accuracy by avoiding the negative effects of scene switching and speaker movement that occur when treating the entire video as a whole.
2Measurement precision
If scene detection is performed to divide the video into segments, then speaker recognition accuracy is improved, but the processing complexity increases
Solution Approach 1:
The video is divided into multiple video segments based on scene detection results. Each segment is processed independently for speaker recognition, which maintains processing simplicity while improving accuracy by avoiding the negative effects of scene switching and speaker movement that occur when treating the entire video as a whole.
Solution Approach 2:
Scene detection is performed in advance to identify scene switching points before speaker recognition is conducted. This preliminary action allows the system to proactively segment the video into appropriate segments, ensuring that speaker recognition is performed on continuous portions of the video where speakers do not frequently switch or move, thereby improving accuracy without significantly increasing overall processing complexity.
3Measurement precision
If face depth features and cross-scene distance features are extracted to track faces across scenes, then face tracking accuracy is improved, but the computational requirements increase
Solution Approach 1:
Face depth features are extracted as intermediary representations to bridge the gap between adjacent video frames. These depth features, combined with histogram of oriented gradient features, serve as mediators for tracking faces across scene transitions. The system calculates distance between these intermediary features to maintain face tracking accuracy while managing computational energy through efficient feature matching.
Data Source
AI summary
A speaker recognition method and apparatus, an electronic device, a medium, and a program product, where the speaker recognition method includes: performing scene detection on a video to be recognized, and dividing the video to be recognized into a plurality of video segments based on a result of the scene detection; separating the video segment to obtain audio data and video frames in the video segment; extracting facial features of the video frames and extracting an audio feature of the audio data; for a plurality of video frames with scene switching, extracting face depth features of the plurality of video frames, and calculating a distance between the face depth features of adjacent video frames in the plurality of video frames, to obtain a cross-scene distance feature; and recognizing a speaker from faces included in the video segments based on the cross-scene distance feature, the facial features and the audio feature.