3D Model Generation via Two-Phase Mesh Refinement

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing 3D scanning technologies face limitations in real-time processing capabilities, resulting in low-quality 3D models and provide limited feedback to users during data collection, while offline processing solutions lack immediate feedback and require extensive computational resources.

Innovation Solution

A method that generates a first 3D model in real-time for feedback during image capture and allows for a secondary processing phase to improve model quality using intermediate computational results, combining real-time and post-processing techniques to enhance accuracy and efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If real-time 3D model generation is performed with limited processing capabilities, then processing speed is improved, but manufacturing precision deteriorates

Engineering Contradiction:
Improveprocessing speedVSAvoid3D model quality
Core Design Contradiction:
SpeedVSManufacturing precision

Solution Approach 1:

The patent divides the 3D model generation process into two distinct phases: a first process that generates an initial 3D mesh representation during or after image capture, and a second process that performs detailed post-processing to generate a refined 3D model. This segmentation allows the system to balance real-time responsiveness with high-quality output by handling different processing depths at different stages.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The first process performs preliminary 3D model generation during or immediately after image capture, providing an initial structure that can be used for real-time feedback. This preliminary action prepares the foundation for subsequent detailed processing, enabling the system to provide immediate results while maintaining the option for higher quality refinement later.

Inventive Principle:
Principle #10Preliminary action

2Manufacturing precision

If offline processing is performed with extensive computational resources, then manufacturing precision is improved, but loss of time increases

Engineering Contradiction:
Improve3D model qualityVSAvoidprocessing time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The system performs preliminary 3D model generation during or immediately after image capture in the first process, providing an initial result that can be used while the object is still being scanned or shortly after. This eliminates the need to wait for extensive offline processing while still maintaining the option for quality refinement.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements feedback mechanisms where the system provides 3D model information during the scanning process, allowing users to monitor progress and quality in real-time. This feedback enables users to determine when sufficient data has been captured, reducing unnecessary processing time while maintaining quality standards.

Inventive Principle:
Principle #23Feedback

3Manufacturing precision

If more pictures are captured from multiple angles to cover detailed regions, then manufacturing precision is improved, but loss of time increases

Engineering Contradiction:
Improve3D model qualityVSAvoiddata collection time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The system provides feedback during the image capture process, allowing users to monitor which regions have been adequately covered and which require additional images. This real-time feedback enables users to efficiently determine when sufficient data has been captured, reducing unnecessary photographing time while ensuring comprehensive coverage of detailed regions.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The patent allows for progressive processing where a complete 3D model can be generated from a subset of captured images when time is constrained, while still providing quality results. The system can process available data immediately rather than requiring all possible images to be captured and processed, balancing completeness with time efficiency.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS11315313B2Methods, devices and computer program products for generating 3D models
Publication Date: 2022.04.26 SONY GROUP CORP
  • US11315313B2 patent drawing
  • US11315313B2 patent drawing
  • US11315313B2 patent drawing

AI summary

A method of generating a 3D model may include receiving a plurality of 2D images of a physical object captured from a respective plurality of viewpoints in a 3D scan of the physical object in a first process. The method may include receiving a first process 3D mesh representation of the physical object and calculating respective second process estimated position and/or orientation information for each one of the respective plurality of viewpoints of the plurality of 2D images. The method may include generating a second process 3D mesh representation of the physical object using the plurality of 2D images, the second process estimated position and/or orientation information, and the first process 3D mesh representation of the physical object. The method may include generating a 3D model of the physical object by applying surface texture information from the plurality of 2D images to the second process 3D mesh representation of the physical object.