Virtual Viewpoint Image Switching for Subject Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
When viewing virtual viewpoint images generated from multiple camera perspectives, subjects such as players or balls can quickly move out of the frame, especially when the viewpoint is adjusted closer to the subject, leading to a loss of realistic content.
Innovation Solution
An image processing apparatus that includes an obtaining unit for virtual viewpoint information and a display controller to switch to a second virtual viewpoint image with a larger angle of view when a user instructs a change in viewpoint, ensuring the subject remains within the frame by calculating the second virtual viewpoint based on the first virtual viewpoint's position and direction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the viewpoint is set closer to the subject (player or ball), then more realistic content is obtained, but the subject may move out of the frame
Solution Approach 1:
The system dynamically switches between a first virtual viewpoint (closer to subject, more realistic) and a second virtual viewpoint (larger angle of view, subject guaranteed in frame) based on real-time detection of subject position. This dynamic adaptation resolves the contradiction by allowing the system to optimize for realism when the subject is visible, and automatically switch to a safer viewpoint when the subject risks moving out of frame.
Solution Approach 2:
The system continuously monitors the subject's position in the frame and uses this feedback to determine when to switch viewpoints. When the subject is detected to be at the edge of the frame or moving out, the system switches from the first viewpoint to the second viewpoint, ensuring the subject remains visible while maintaining realistic viewing experience when possible.
2Adaptability or versatility
If the viewpoint is shifted in accordance with user operation, then user freedom is improved, but the subject may move out of the frame
Solution Approach 1:
The system provides feedback-based viewpoint control where user operations are monitored and the system responds by switching viewpoints when the subject risks moving out of frame. This maintains user freedom to initiate viewpoint changes while ensuring the subject remains visible through automatic system intervention when needed.
Solution Approach 2:
The system takes preliminary action by detecting when a viewpoint change might cause the subject to move out of frame, and switches to the second viewpoint in advance to prevent the subject from leaving the frame. This anticipatory measure preserves user freedom while preventing the harmful outcome.
3Manufacturing precision
If a second virtual viewpoint image with larger angle of view is displayed, then the subject remains in frame, but the viewing experience may become less realistic
Solution Approach 1:
The system dynamically selects between the first virtual viewpoint (more realistic, smaller angle of view) and the second virtual viewpoint (less realistic, larger angle of view) based on real-time subject position detection. This dynamic switching resolves the contradiction by using the larger angle of view only when necessary to keep the subject in frame, while preferring the more realistic viewpoint when the subject is safely within the frame.
Data Source
AI summary
An image processing apparatus includes an obtaining unit configured to obtain virtual viewpoint information indicating a position and/or a direction of a virtual viewpoint corresponding to a virtual viewpoint image generated based on images captured by a plurality of cameras which capture images of a subject from different directions, and a display controller configured to display, while a first virtual viewpoint image corresponding to the virtual viewpoint information obtained by the obtaining unit is displayed on a display unit, a second virtual viewpoint image having an angle of view larger than an angle of view of the first virtual viewpoint image being displayed on the display unit, when an instruction for switching to a virtual viewpoint having a position and/or a direction which are/is not set by a user is received.