Neural Network Shading for Real-Time Digital Impasto Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer-based impasto painting systems face challenges in replicating the realistic lighting effects and real-time interaction of traditional painting, as existing solutions either lack complex global light transport simulations or provide instant feedback, failing to render visually indistinguishable digital artworks from real paintings.
Innovation Solution
A system that generates a local shading function on-the-fly for each pixel in a paint stroke using a neural network, which takes a neighborhood patch of pixels and their geometries and material properties as input to output a shading function representing global light transport effects, allowing for real-time rendering of complicated light transport effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If physically-based light transport simulation method is used to render completed artwork, then rendering quality is improved, but rendering time increases perceptibly long
Solution Approach 1:
The system pre-computes and stores lighting information in a radiance cache during an offline phase, capturing global light transport effects in advance. This preliminary action allows the rendering system to retrieve pre-computed lighting data during interactive painting, eliminating the need to perform complex light transport simulations in real-time while maintaining high rendering quality
Solution Approach 2:
The rendering process is divided into two distinct phases: an offline pre-computation phase that calculates and stores radiance cache data, and an online interactive phase that retrieves and applies pre-computed lighting information. This segmentation allows computationally intensive simulations to be performed separately from the interactive painting process, resolving the contradiction between quality and speed
2Productivity
If local shading effects are rendered with zero pixel thickness assumption, then rendering performance is acceptable, but lighting effect quality falls short of real painting
Solution Approach 1:
The radiance cache acts as an intermediary data structure that stores pre-computed global light transport information. Instead of directly simulating complex light interactions during rendering, the system uses this intermediate cache to provide accurate lighting effects efficiently, bridging the gap between simple local shading and full global illumination
3Reliability
If every brush stroke causes re-rendering of artwork, then rendering robustness is improved, but real-time interaction capability deteriorates
Solution Approach 1:
The system performs preliminary computation of lighting effects in advance and stores them in a radiance cache. During interactive painting, artists can apply brush strokes and see immediate results by retrieving pre-computed lighting data, eliminating the need to re-render the entire artwork for each stroke while maintaining rendering robustness through the pre-established lighting model
Data Source
AI summary
According to one general aspect, systems and techniques for rendering a painting stroke of a three-dimensional digital painting include receiving a painting stroke input on a canvas, where the painting stroke includes a plurality of pixels. For each of the pixels in the plurality of pixels, a neighborhood patch of pixels is selected and input into a neural network and a shading function is output from the neural network. The painting stroke is rendered on the canvas using the shading function.