Panoramic Video Face Normalization for Standard Conferencing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Panoramic images with 360° coverage are not compatible with standard video conferencing systems, leading to issues such as faces appearing too small when resized or a significant portion of the display being unused when cropped, resulting in an unsatisfactory user experience.

Innovation Solution

The panoramic image is normalized to ensure face sizes appear consistent, and then resized and embedded in a standard resolution video stream, with features like automatic speaker detection and user-controlled highlighting to optimize display usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the panoramic image is resized to fit in a standard video stream, then the compatibility with standard video conferencing systems is improved, but the face sizes become too small to be practically useful

Engineering Contradiction:

Improvecompatibility with standard video conferencing systemsVSAvoidface size in the video stream

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The panoramic image is divided into multiple standard-resolution video frames by identifying and extracting multiple face regions. Each face is detected, cropped, and placed into separate video frames, allowing multiple faces to be displayed at appropriate sizes within the standard video stream format.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The solution transitions from displaying a single panoramic view to creating multiple video frames that can be arranged in different spatial configurations (e.g., grid layout). This dimensional transformation allows multiple faces from the panorama to be presented at optimized sizes across multiple frames rather than forcing all faces into a single small frame.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Manufacturing precision

If the panoramic image is cropped to fit in a standard video window, then the face sizes remain recognizable, but a majority of the display is unused

Engineering Contradiction:

Improveface size in the video streamVSAvoiddisplay area utilization

Core Design Contradiction:

Manufacturing precisionVSArea of stationary object

Solution Approach 1:

Instead of cropping a single portion of the panorama, the image is segmented into multiple face regions that are then distributed across multiple video frames. This segmentation allows the entire display area to be utilized by arranging multiple frames in a grid or other spatial configuration, eliminating unused display space while maintaining recognizable face sizes.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Multiple cropped face regions from the panoramic image are merged into a composite video stream by placing them in separate video frames that are then combined. This merging approach fills the entire display area with useful content (multiple faces at proper sizes) rather than leaving large portions unused.

Inventive Principle:

Principle #5Merging (Combining)

3Adaptability or versatility

If the panoramic image is transmitted in standard resolution, then the compatibility is improved, but the user experience deteriorates due to loss of detail

Engineering Contradiction:

Improvecompatibility with standard video conferencing systemsVSAvoidimage detail in panoramic view

Core Design Contradiction:

Adaptability or versatilityVSLoss of information

Solution Approach 1:

Instead of transmitting the entire high-resolution panoramic image in standard resolution (which would lose detail), the system creates multiple standard-resolution copies of specific face regions. Each face is copied from the panorama at its original high resolution, then placed into a standard-resolution video frame, preserving face detail while maintaining compatibility.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The solution extracts only the relevant face regions from the panoramic image rather than transmitting the entire panorama in reduced resolution. By taking out and isolating the important face areas, the system preserves their detail quality while fitting them into standard video frames, avoiding the information loss that would occur with full-panorama downsampling.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS7768544B2Embedding a panoramic image in a video stream

Publication Date: 2010.08.03 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Techniques are described for embedding a panoramic image in standard resolution. A panoramic image is received from a panoramic camera and a location of a speaker is determined, either automatically or manually. A portion of the panoramic image is cropped to isolate the speaker and the cropped portion of the image is enlarged to standard video resolution. The panoramic image is combined with the cropped portion and transmitted via a standard video streaming format. In at least one implementation, the panoramic image is normalized so that face sizes of persons in the image appear approximately equal regardless of the distance each person is situated relative to the camera.