Neural Network Line Drawing to Rendered Image Conversion

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for converting line drawings to rendered images often result in unwanted irregularities due to treating edges as absolute truths, which are undesirable in the final rendered image.

Innovation Solution

A system and method that utilizes a neural network, specifically a conditional Generative Adversarial Network (cGAN), trained with noise data to account for irregularities in line drawings, allowing it to generate rendered images by learning the line drawing as a guideline rather than an absolute truth, thereby ignoring and correcting irregularities.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If edge detection techniques treat edges of an object as an absolute truth to generate a rendered image identical to the line drawing, then the conversion accuracy is improved, but unwanted irregularities appear in the rendered image

Engineering Contradiction:

Improveconversion accuracyVSAvoidunwanted irregularities

Core Design Contradiction:

Measurement precisionVSObject-generated harmful factors

Solution Approach 1:

The neural network is trained in advance with noise-added line drawings to learn the statistical characteristics of hand-drawn irregularities. This preliminary training enables the network to anticipate and correct irregularities before they appear in the final rendered image, resolving the contradiction between accuracy and irregularity removal

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

A neural network is introduced as an intermediary between the line drawing and the rendered image. The network learns to distinguish between intentional design features and unwanted irregularities, acting as a mediator that filters out harmful irregularities while preserving the essential structure and boundaries of the original drawing

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If traditional image processing methods are used to convert line drawings to rendered images, then the process is simple and fast, but the output contains irregularities from hand-drawn variations

Engineering Contradiction:

Improveprocessing speedVSAvoidhand-drawn irregularities

Core Design Contradiction:

ProductivityVSObject-generated harmful factors

Solution Approach 1:

Traditional mechanical image processing methods are replaced with a neural network-based system. The neural network automatically learns to correct hand-drawn irregularities through training, maintaining fast processing speeds while eliminating the need for complex manual correction steps

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS10521700B2Methods and systems for converting a line drawing to a rendered image

Publication Date: 2019.12.31 HONDA MOTOR CO LTD

AI summary

The system includes a memory that stores instructions for executing processes converting line drawings to rendered images. The system also includes a processor configured to execute the instructions. The instructions cause the processor to: train a neural network to account for irregularities in the line drawings by introducing noise data into training data of the neural network; receive a first line drawing from an input device; generate a first rendered image based on features identified in the first line drawing; and display the first rendered image on an output device.