Image Processing Apparatus Viewpoint Rendering

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current methods for converting three-dimensional virtual objects into two-dimensional display images for applications like comics and augmented reality are inefficient due to high processing loads, particularly when rendering from multiple viewpoints, leading to monotonous images and increased processing demands.

Innovation Solution

An image processing program and apparatus that superimpose and display virtual objects onto photographed images by rendering three-dimensional virtual objects from a fixed virtual viewpoint, reducing processing load by limiting viewpoints and using filters and temporary erasure settings to manage display areas effectively.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If images are rendered from multiple viewpoint directions, then image variety and interest are improved, but processing load increases significantly

Engineering Contradiction:
Improveimage varietyVSAvoidprocessing load
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent segments the virtual space into multiple display areas (first display area, second display area, etc.) that can be independently rendered. Each display area corresponds to a specific viewpoint direction, allowing the system to render only the necessary portions of the virtual space rather than the entire scene from all viewpoints simultaneously. This segmentation reduces the overall processing load while maintaining the ability to present diverse images from different viewpoints.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements selective rendering by determining whether to render images from multiple viewpoint directions based on predetermined conditions. The system performs partial rendering actions - rendering from multiple viewpoints only when necessary (e.g., when objects are positioned to benefit from multi-viewpoint display), and single-viewpoint rendering when sufficient. This partial action approach maintains image variety when needed while reducing processing load during routine operations.

Inventive Principle:
Principle #16Partial or excessive action

2Reliability

If rendering is always executed for each image from multiple viewpoint directions, then complete virtual space representation is achieved, but processing time and computational resources increase

Engineering Contradiction:
Improvevirtual space representationVSAvoidprocessing time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent introduces dynamic rendering control where the system adaptively determines the number of viewpoint directions to render based on real-time conditions. The rendering process transitions between static single-viewpoint mode and dynamic multi-viewpoint mode depending on object positions, display area configurations, and predetermined conditions. This dynamic approach ensures complete virtual space representation is maintained when necessary while reducing processing time during standard operations.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent changes the parameter of viewpoint direction count dynamically. Instead of always rendering from a fixed number of viewpoints, the system adjusts this parameter based on predetermined conditions such as object positioning, display area availability, and scene complexity. When conditions warrant, the system increases the number of viewpoint directions for comprehensive representation; otherwise, it reduces to a single viewpoint to minimize processing time.

Inventive Principle:
Principle #35Parameter changes

3Device complexity

If the number of viewpoint directions is reduced to one, then processing load is reduced, but image monotony increases and趣味性 decreases

Engineering Contradiction:
Improveprocessing loadVSAvoidimage variety
Core Design Contradiction:
Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent implements periodic multi-viewpoint rendering where the system alternates between single-viewpoint and multi-viewpoint rendering modes. During normal operation, the system uses a single viewpoint to maintain low processing load. Periodically, when predetermined conditions are met (such as specific object positions or user interactions), the system switches to multi-viewpoint rendering to provide image variety and maintain user interest. This periodic action ensures processing load remains manageable while preventing image monotony.

Inventive Principle:
Principle #19Periodic action

Solution Approach 2:

The patent creates a universal rendering system that can function in multiple modes - single-viewpoint mode for routine operations and multi-viewpoint mode for special cases. The same rendering engine handles both modes, making the system multi-functional. This universality allows the system to maintain low processing load during standard operations while retaining the capability to provide diverse, interesting images when needed, without requiring separate dedicated systems for each function.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS11087526B2Image processing program, image processing apparatus, and image processing method
Publication Date: 2021.08.10 SQUARE ENIX HLDG CO LTD
  • US11087526B2 patent drawing
  • US11087526B2 patent drawing
  • US11087526B2 patent drawing

AI summary

A non-transitory computer-readable medium including an image processing program is provided. The image processing program causes a terminal device to: obtain a photographed image obtained by photographing a real space; display a display frame having one or more display areas in a display area of the photographed image in a case where a predetermined condition is satisfied; obtain virtual space information at least containing three-dimensional virtual object information and positional relationship information, the three-dimensional virtual object information regarding one or more three-dimensional virtual objects arranged in a virtual space in advance; and render a certain three-dimensional virtual object viewed behind the display area when the display frame is photographed from the virtual viewpoint in a case where the display frame in the virtual space is virtually photographed from a virtual viewpoint, and display the certain three-dimensional virtual object in the display area of the display frame.