Virtual Viewpoint Video Search via Metadata Association
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image search systems face difficulties in efficiently retrieving desired virtual viewpoint video images from a multitude of virtual viewpoint video images, even when date of capture and scene information are specified, due to the variability in camera viewpoints.
Innovation Solution
An image search system that accumulates virtual viewpoint video image data along with associated metadata, including orientation and position parameters, allows users to input search conditions via a user interface, and extracts and presents relevant data based on these parameters, using techniques such as quaternion representation for orientation and three-dimensional coordinates for position.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple virtual viewpoint video images are generated from the same scene with different camera viewpoints, then the user experience and viewing angles are improved, but it becomes difficult to specify and retrieve a desired virtual viewpoint video image from the plurality of generated images
Solution Approach 1:
The patent applies preliminary action by pre-associating metadata (camera position, orientation, time, scene information) with each virtual viewpoint video image at the time of generation and storage. This preliminary organization of data enables efficient retrieval later without requiring complex search operations when users need to find specific viewpoint images from multiple generated variants.
Solution Approach 2:
The patent introduces metadata as an intermediary between the virtual viewpoint video images and the search function. This metadata layer (containing camera position, orientation, timestamp, and scene information) mediates the search process by providing structured indices that enable users to retrieve specific images based on viewpoint parameters without directly analyzing the video content itself.
2Productivity
If conventional search elements (date of image capturing and scene) are used to search for virtual viewpoint video images, then basic retrieval is possible, but it remains difficult to specify a desired image when multiple images exist for the same scene
Solution Approach 1:
The patent adds another dimension to the search capability by incorporating camera position and orientation information alongside the conventional time and scene dimensions. This creates a multi-dimensional search space where users can specify not just when and what scene, but also from which viewpoint angle, thereby distinguishing between multiple images of the same scene captured from different perspectives.
Solution Approach 2:
The patent changes the search parameters by introducing camera position coordinates and orientation angles as additional search criteria. This transforms the search system from handling only temporal and categorical parameters (time, scene) to also handling spatial and angular parameters (position, orientation), enabling precise differentiation and retrieval of specific viewpoint images.
Data Source
AI summary
The image search system according to the present invention accumulates virtual viewpoint video image data generated based on image data obtained by capturing an object from a plurality of directions by a plurality of cameras and a virtual viewpoint parameter used for generation of the virtual viewpoint video image data in association with each other. Then, the image search system extracts, in a case where a search condition is input via an input unit, virtual viewpoint video image data associated with a virtual viewpoint parameter corresponding to the search condition from the accumulated virtual viewpoint video image data. Further, the image search system presents information of the extracted virtual viewpoint video image data as results of the search. Due to this, convenience relating to a search for a virtual viewpoint video image improves.