Strobe Image Generation Using 3D Model Relevance

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing technologies for generating strobe images from two-dimensional moving images are limited in their ability to effectively generate and edit such images, particularly in representing the movement and trajectory of subjects over time.

Innovation Solution

An image processing apparatus and method that generates 3D models from multiple viewpoint images captured over time, disposing these models in a three-dimensional space based on their relevance to a key object, allowing for the creation of a strobe model that accurately represents the movement and trajectory of objects.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If 3D models of objects at multiple times are disposed in a three-dimensional space to generate a strobe model, then the representation of object movement and trajectory is improved, but the data complexity increases

Engineering Contradiction:
Improverepresentation precision of object movementVSAvoiddata complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent extracts and prioritizes only the most relevant objects (key objects) from the complex scene at each time point, rather than processing all objects equally. This selective extraction reduces the effective data volume while maintaining accurate representation of critical movements and trajectories.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies different processing qualities to different objects based on their relevance. Key objects receive high-quality detailed processing and prominent display, while non-key objects receive simplified processing. This local differentiation maintains representation precision for important movements while reducing overall data complexity.

Inventive Principle:
Principle #3Local quality

2Reliability

If all objects in the scene are processed to create a comprehensive strobe model, then the completeness of movement representation is improved, but the processing time increases

Engineering Contradiction:
Improvecompleteness of movement representationVSAvoidprocessing time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The system extracts and focuses processing on key objects that are most relevant to the movement analysis, rather than uniformly processing all objects in the scene. This selective approach maintains reliable movement representation for critical elements while significantly reducing processing time.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies partial processing to non-key objects, processing them to a lesser extent than key objects. This partial action approach ensures that processing time is not excessively consumed by less important objects while still maintaining adequate representation of overall movement patterns.

Inventive Principle:
Principle #16Partial or excessive action

3Ease of operation

If 3D models are disposed according to their relevance to the key object, then the naturalness of the strobe image is improved, but the algorithm complexity increases

Engineering Contradiction:
Improvenaturalness of strobe imageVSAvoidalgorithm complexity
Core Design Contradiction:
Ease of operationVSDevice complexity

Solution Approach 1:

The system incorporates feedback mechanisms to automatically determine object relevance and disposition. By analyzing spatial relationships and movement patterns, the algorithm adapts its processing to create natural-looking strobe images, reducing the need for complex manual configuration while maintaining visual naturalness.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS11557087B2Image processing apparatus and image processing method for generating a strobe image using a three-dimensional model of an object
Publication Date: 2023.01.17 SONY GROUP CORP
  • US11557087B2 patent drawing
  • US11557087B2 patent drawing
  • US11557087B2 patent drawing

AI summary

Provided are an image processing apparatus and an image processing method that enable a strobe image using a 3D model to be generated. A strobe model in which 3D models of an object at a plurality of times generated from a plurality of viewpoint images captured from a plurality of viewpoints are disposed in a three-dimensional space is generated. When the strobe model is generated, a target object that is a target in which the 3D model is disposed in the strobe model is set according to a degree of object relevance indicating relevance with a key object serving as a reference for disposition of the 3D model in the strobe model.