Face Thumbnail Extraction for Moving Image Scene Navigation

Solution Overview

Problem

Existing imaging devices struggle to effectively allow users to identify and navigate through large collections of moving image data, as the content of scenes is often unclear from thumbnail images, leading to difficulties in finding specific interesting parts within long scenes.

Innovation Solution

The solution involves detecting and extracting face images from each scene, generating thumbnail images from these faces, and displaying them in a time-series order, allowing users to easily identify scene content and start playing specific scenes from designated face thumbnail images.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If head images are displayed as thumbnails for each scene, then the display structure is simple, but the user cannot easily understand the content of the image data

Engineering Contradiction:

Improvedisplay structureVSAvoidscene content understanding

Core Design Contradiction:

Device complexityVSLoss of information

Solution Approach 1:

The patent extracts face images from within the scene thumbnails, isolating the most informative elements (faces) from the complete scene images. This allows the thumbnail to display only the critical content (faces) rather than the entire scene, improving content understanding while maintaining simple display structure.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies local quality by enhancing specific regions (face areas) within the thumbnail while suppressing or removing non-face regions. This creates thumbnails with varying local characteristics where face-containing areas are prominently displayed, enabling users to quickly identify and understand scene content through face detection.

Inventive Principle:

Principle #3Local quality

2Loss of information

If the entire scene is displayed in the thumbnail, then the complete content is visible, but it is difficult to identify specific interesting parts within long scenes

Engineering Contradiction:

Improvescene content visibilityVSAvoidnavigation to interesting parts

Core Design Contradiction:

Loss of informationVSEase of operation

Solution Approach 1:

The patent extracts face images from within the scene thumbnails, isolating the most informative elements (faces) from the complete scene images. This allows the thumbnail to display only the critical content (faces) rather than the entire scene, improving content understanding while maintaining simple display structure.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent performs preliminary face detection and extraction during the thumbnail generation phase, preparing the most relevant content in advance. This preliminary action enables users to quickly identify interesting parts (faces) without needing to navigate through or analyze the entire scene, facilitating faster access to desired content.

Inventive Principle:

Principle #10Preliminary action

3Ease of operation

If index images are generated and recorded on the recording medium, then image search is enabled, but memory capacity is reduced and time is consumed

Engineering Contradiction:

Improveimage search capabilityVSAvoidmemory capacity

Core Design Contradiction:

Ease of operationVSQuantity of substance

Solution Approach 1:

The patent extracts only the essential face information from complete scenes to create compact thumbnails. This extraction process creates highly condensed search indices that occupy minimal storage space while retaining the most valuable searchable content (faces), thus enabling efficient image search without consuming significant memory capacity.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent performs preliminary face detection and extraction during the thumbnail generation phase, preparing the most relevant content in advance. This preliminary action enables users to quickly identify interesting parts (faces) without needing to navigate through or analyze the entire scene, facilitating faster access to desired content.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9143691B2Apparatus, method, and computer-readable storage medium for displaying a first image and a second image corresponding to the first image

Publication Date: 2015.09.22 SONY GROUP CORP

AI summary

An apparatus for processing image, includes an input unit for inputting user operation information, a recording medium for recording moving image data, a data processor for retrieving data recorded on the recording medium and generating display data in response to an input to the input unit, and a display unit for displaying the display data. The data processor selects a frame containing an image of a person's face from a scene as a moving image recording unit recorded on the recording medium, generates a thumbnail image of a face region extracted from the selected frame and displays on the display unit a list of generated thumbnail images arranged in a time-series order.