Ray Tracing Video Rendering via Region Segmentation and Synthesis

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

The ray tracing method requires simulation of thousands to tens of thousands of rays per pixel, leading to extremely long processing times, which is a significant bottleneck in movie production and other applications where speed is crucial.

Innovation Solution

An information processing apparatus that renders model data using both ray tracing and non-ray tracing methods, synthesizing the results to create a high-quality video at high speed by combining low-quality ray traced videos with high-quality additional videos, and temporally or spatially thinning out the rendering process to reduce the number of ray tracing operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If ray tracing method is used to render model data, then light source component representations and realistic images are achieved, but processing time becomes extremely long

Engineering Contradiction:
Improveimage qualityVSAvoidprocessing time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent divides the image into multiple regions with different rendering priorities. High-priority regions (containing important visual information) are rendered with high quality using ray tracing, while low-priority regions are rendered with lower quality using non-ray tracing methods. This segmentation allows the system to achieve overall high image quality while significantly reducing total processing time by not applying computationally expensive ray tracing uniformly across the entire image.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies different rendering qualities to different spatial locations within the image. Instead of uniform high-quality ray tracing everywhere, the system concentrates computational resources on specific local regions that contribute most to perceived image quality, while using faster rendering methods in other areas. This local quality approach resolves the contradiction by making image quality high where it matters most while maintaining acceptable quality elsewhere, thereby reducing overall processing time.

Inventive Principle:
Principle #3Local quality

2Manufacturing precision

If ray tracing method is used for all frames, then consistent high image quality is achieved, but rendering speed becomes too slow for practical use

Engineering Contradiction:
Improveimage qualityVSAvoidrendering speed
Core Design Contradiction:
Manufacturing precisionVSProductivity

Solution Approach 1:

The patent dynamically adjusts the rendering method and quality level based on frame-specific conditions such as scene complexity, motion magnitude, and region importance. Rather than statically applying ray tracing to all frames, the system makes dynamic decisions about which frames and regions require high-quality rendering, allowing it to maintain high image quality when necessary while achieving high rendering speeds when possible, thus resolving the contradiction between quality and productivity.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent applies ray tracing selectively to only the necessary portions of frames rather than universally. By identifying and rendering only the critical regions that require high quality (partial action), the system achieves acceptable overall image quality while dramatically reducing the computational burden and increasing rendering speed, thereby resolving the contradiction between maintaining image quality and achieving practical rendering speeds.

Inventive Principle:
Principle #16Partial or excessive action

3Measurement precision

If high number of rays per pixel is used in ray tracing, then accurate light simulation is achieved, but processing time increases significantly

Engineering Contradiction:
Improvelight simulation accuracyVSAvoidsimulation time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies high-ray-count ray tracing only to specific local regions where accurate light simulation is critical (such as regions with complex lighting effects, reflections, or refractions), while using lower-ray-count or non-ray-tracing methods in regions where simple lighting suffices. This localized approach to high precision rendering maintains light simulation accuracy where needed while significantly reducing the total simulation time across the entire image.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent segments the image into regions requiring high-fidelity light simulation and regions where approximate rendering is acceptable. By applying different ray counts to different segments based on their lighting complexity and visual importance, the system achieves accurate light simulation in critical areas while maintaining overall rendering efficiency, thereby resolving the contradiction between measurement precision and time consumption.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS20240169654A1Information processing apparatus, information processing method, and information processing program
Publication Date: 2024.05.23 SONY GROUP CORP
  • US20240169654A1 patent drawing
  • US20240169654A1 patent drawing
  • US20240169654A1 patent drawing

AI summary

There is provided an information processing apparatus to create a rendered video by a ray tracing method at higher speed. The information processing apparatus includes: a renderer that renders model data by a ray tracing method and creates a ray traced video, and renders the model data by a method different from the ray tracing method and creates an additional video; and a video synthesis unit that synthesizes the additional video and the ray traced video and creates a synthesized video.