Neural Network Material Property Extraction from Single Image

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional digital image processing systems face challenges in accurately extracting material properties from images due to complexity and require multiple images, expensive hardware, and simplifying assumptions, limiting flexibility and accuracy.

Innovation Solution

A deep-learning based framework using a neural network encoder, material classifier, and decoders to extract spatially varying material properties from a single digital image captured with flash illumination, eliminating the need for expensive equipment and simplifying assumptions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If multiple digital images and camera/lighting calibrations are used to accurately capture material properties, then measurement precision improves, but device complexity and ease of operation worsen

Engineering Contradiction:

Improvematerial property extraction accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent uses a pre-trained neural network model (copy of learned knowledge) to extract material properties from a single image, eliminating the need for multiple images and complex calibration systems while maintaining accurate material property extraction

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces the mechanical/optical calibration system with a deep learning-based computational system that automatically learns material properties from image data, substituting physical calibration procedures with algorithmic processing

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If expensive and powerful hardware systems are used for material capture, then measurement precision improves, but cost increases

Engineering Contradiction:

Improvematerial property extraction accuracyVSAvoidsystem cost

Core Design Contradiction:

Measurement precisionVSEase of manufacture

Solution Approach 1:

The patent employs standard mobile device cameras and consumer-grade computing hardware with a software-based neural network solution, replacing expensive specialized hardware while achieving comparable or superior material property extraction accuracy

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Solution Approach 2:

The patent substitutes expensive specialized material capture hardware with a software-based deep learning system that runs on conventional computing devices, dramatically reducing system cost while maintaining precision

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Ease of operation

If simplifying assumptions are made about geometry, material properties, or lighting conditions, then ease of operation improves, but measurement precision worsens

Engineering Contradiction:

Improvesystem flexibilityVSAvoidmaterial property extraction accuracy

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The patent changes the approach from assuming fixed lighting and geometry conditions to using a neural network that learns to handle varied lighting conditions, material types, and geometric configurations, thereby improving both flexibility and accuracy

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent uses a pre-trained neural network model that has learned from diverse training data encompassing various lighting conditions, material types, and geometries, enabling accurate extraction without simplifying assumptions

Inventive Principle:

Principle #26Copying

4Ease of operation

If conventional machine learning methods are used that handle only diffuse materials or constant material properties, then ease of operation improves, but measurement precision and adaptability worsen

Engineering Contradiction:

Improvesystem simplicityVSAvoidmaterial type coverage

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The patent extends the capability from handling only diffuse or constant material properties to handling spatially-varying material properties including both diffuse and specular materials, achieving comprehensive material coverage while maintaining ease of use through automated processing

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent creates a universal neural network-based system that can extract material properties from various material types (diffuse, specular, spatially-varying) using a single unified approach, replacing multiple specialized methods

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS10810469B2Extracting material properties from a single image

Publication Date: 2020.10.20 ADOBE INC

AI summary

Systems, methods, and non-transitory computer-readable media are disclosed for extracting material properties from a single digital image portraying one or more materials by utilizing a neural network encoder, a neural network material classifier, and one or more neural network material property decoders. In particular, in one or more embodiments, the disclosed systems and methods train the neural network encoder, the neural network material classifier, and one or more neural network material property decoders to accurately extract material properties from a single digital image portraying one or more materials. Furthermore, in one or more embodiments, the disclosed systems and methods train and utilize a rendering layer to generate model images from the extracted material properties.