Virtual Viewpoint Image Privacy Masking via User Operation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual viewpoint image generation techniques fail to effectively switch between display and non-display of subjects based on user operation, leading to potential inappropriate virtual viewpoint images, particularly in scenarios where privacy protection is necessary, such as monitoring spectators instead of players during events like soccer games.
Innovation Solution
An image processing apparatus determines whether to display a privacy mask in a virtual viewpoint image based on virtual viewpoint information, hiding specific regions only when the user's operation indicates a need for privacy protection, thereby allowing users to operate the virtual viewpoint without compromising realism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If privacy mask is always displayed on subjects in virtual viewpoint image, then privacy protection is enhanced, but realism of the virtual viewpoint image is reduced
Solution Approach 1:
The patent applies dynamics by making the privacy mask display state changeable based on user operation. The system transitions from a static always-display state to a dynamic state that adjusts according to user actions. When the user operates the virtual viewpoint to focus on specific subjects, the system dynamically switches between displaying and hiding privacy masks, thereby maintaining privacy protection while preserving image realism during normal viewing conditions.
Solution Approach 2:
The patent applies local quality by selectively applying privacy masks to specific regions or subjects based on local conditions. Instead of uniformly applying privacy masks to all subjects, the system identifies specific subjects that require privacy protection based on their spatial position, visual characteristics, or contextual information, and applies masks only to those local regions. This selective approach maintains privacy where necessary while preserving realism in other areas.
2Shape
If invisible-making processing is stopped to display subjects, then realism is improved, but privacy protection is compromised when users operate virtual viewpoint to monitor spectators
Solution Approach 1:
The patent applies feedback by implementing a control mechanism that monitors user operations and adjusts privacy mask display accordingly. The system receives feedback from user actions (such as virtual viewpoint movement or zoom operations) and uses this information to determine when to display or hide privacy masks. This feedback loop ensures that privacy protection is activated only when necessary based on actual user behavior, preventing unnecessary disruption to image realism during normal viewing.
Solution Approach 2:
The patent applies preliminary action by pre-configuring the system with privacy protection mechanisms that are activated only when specific conditions are met. The system is prepared with the capability to display privacy masks, but they are not automatically displayed during all operations. Instead, the system waits for specific user actions (such as focusing on potential privacy-sensitive areas) before activating the privacy mask, thereby maintaining realism during normal operations while preparing privacy protection when needed.
3Object-affected harmful factors
If hiding processing is applied to all regions, then privacy protection is enhanced, but appropriate virtual viewpoint images cannot be generated for normal viewing
Solution Approach 1:
The patent applies parameter changes by adjusting the visibility parameter of privacy masks based on operational parameters. The system changes the display state of privacy masks from a fixed state to a variable state that depends on operational parameters such as virtual viewpoint position, zoom level, and user action type. By dynamically changing these parameters, the system can selectively apply hiding processing only when necessary, ensuring that appropriate virtual viewpoint images are generated for normal viewing conditions while maintaining privacy protection when required.
Data Source
AI summary
An image processing apparatus obtains virtual viewpoint information indicating at least a position of a virtual viewpoint and a view direction from the virtual viewpoint and generates a virtual viewpoint image based on the virtual viewpoint information and a plurality of images captured from a plurality of viewpoints by hiding processing a specific region of the virtual viewpoint image, the specific region being determined based on that the obtained virtual viewpoint information satisfies a condition.