Multi-Camera People Matching for Smooth Front-Face Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video conferencing systems struggle with displaying all participants clearly when multiple cameras are used, and participants may unintentionally turn away from the camera, leading to loss of visibility of their front face.
Innovation Solution
A multi-camera people matching and selection method using machine learning to analyze images, synchronize member features and images, adjust preference scores, and display the best appearance based on spatial relationships and temporal proximities, ensuring smooth display of front faces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple cameras are used to capture all participants, then the coverage of participants is improved, but the difficulty of identifying and selecting the best appearance for each participant increases
Solution Approach 1:
The system divides the complex task of processing multiple camera feeds into separate modules: feature extraction for each camera, synchronization module, and selection module. Each camera's footage is processed independently to extract features, then these features are synchronized and compared to identify the best appearance, making the overall complex task manageable and systematic.
Solution Approach 2:
The patent introduces feature embeddings as an intermediary representation that bridges the raw camera images and the final selection decision. Instead of directly comparing images, the system extracts features, creates embeddings as intermediate representations, and uses these embeddings for synchronization and selection, simplifying the detection and measurement process.
2Stability of the object's composition
If the system automatically tracks and selects participant appearances, then the smoothness of display is improved, but the computational complexity and processing time increase
Solution Approach 1:
The system performs preliminary feature extraction and embedding creation for each camera feed in advance, rather than processing complete images in real-time during the selection process. This preliminary action reduces the computational burden during the critical selection and display phases, enabling smooth transitions while managing complexity.
Solution Approach 2:
The patent transforms the selection problem from comparing raw images to comparing feature embeddings with different dimensional parameters. By changing the representation parameter from pixel data to feature vectors, the system achieves more efficient computational processing while maintaining the ability to identify the best appearance for smooth display transitions.
3Measurement precision
If the system uses machine learning models to analyze images, then the accuracy of member identification is improved, but the processing time and computational resources increase
Solution Approach 1:
The system extracts only the essential features from the full images using machine learning models, rather than processing the complete image data. By taking out and focusing on specific salient features (facial features, body posture, etc.), the system achieves high identification accuracy while significantly reducing the processing time and computational resources required compared to analyzing entire images.
Data Source
AI summary
A multi-camera people matching and selection method includes capturing a plurality of images by a plurality of cameras in a space, analyzing the plurality of images captured by the plurality of cameras to generate a plurality of member features by using a machine learning model, transmitting the plurality of member features and corresponding cropped or full images to a plurality of corresponding buffers, synchronizing the plurality of member features and the corresponding cropped or full images according to temporal proximities in the corresponding buffers, generating a matching result according to differences of corresponding feature embeddings and spatial relationships realized by object detection or direction of audio to identify a member, adjusting preference scores of members based on member features for displaying the appearance of the member smoothly, adjusting display scores of members according to presences of the members, and displaying images of best appearance.