Layered material rendering method and device based on meta-learning

Abstract

The present invention discloses a layered material rendering method and device based on meta-learning, the method comprising: (1) obtaining a sample data set, each sample comprising a layered material parameter, an incident light direction, an outgoing light direction, and a reflectance as a label; (2) constructing a reflectance prediction network model, the reflectance prediction network model comprising a meta-network, a coordinate transformation module, and a BSDF network; (3) training the reflectance prediction network model using the sample data set; (4) inputting the layered material parameters to be rendered into the trained meta-network to obtain adaptive weights, and then extracting the shared weights in the trained BSDF network to form a weight θ. F ;(5) Renderer reads weight Θ F , in the rendering process, the weight Θ is used according to the direction of the incident light and the direction of the outgoing light F The reflectance is calculated and layered material rendering is performed based on the reflectance. The present invention supports material editing and has low performance overhead.

Description

Technical Field

[0001] The present invention relates to the technical field of computer graphics, and in particular to a meta-learning-based layered material rendering method and device. Background Art

[0002] Photorealistic rendering is a key research area in graphics and the gaming industry. Most real-world materials are layered, such as coated metals, varnished wood, glazed ceramics, and metallic paint. Due to the complexity of layered materials, developing efficient and accurate modeling methods for them has always been a challenging topic in the field of photorealistic rendering.

[0003] Typically, layered material models rely on either per-material pre-calculation or use randomized algorithms. While pre-calculated models successfully describe the entire inter-layer scattering of layered materials, they require time-consuming pre-calculation of each material and incur significant storage overhead, making them inapplicable to texture scenarios. Randomized algorithm-based models, while not requiring time-consuming pre-calculation, suffer from low efficiency and high variance due to the use of Monte Carlo sampling when estimating inter-layer light scattering.

[0004] Thanks to the emergence of deep neural networks, learning-based methods have gradually become an option for modeling complex materials. Deep neural networks excel at learning robust prior knowledge from large datasets, enabling unprecedented realism in the visual representation of materials. However, these methods often face a trade-off between powerful expressive power (large network models with many parameters) and efficient runtime performance (small network models with few parameters). Furthermore, they often lack physically meaningful parameters, making it impossible to achieve easy-to-use, intuitive, and responsive material editing, which is crucial for many graphics applications. Summary of the Invention

[0005] Purpose of the invention: To address the problems existing in the prior art, the present invention provides a meta-learning-based layered material rendering method and device that supports material editing and has low performance overhead.

[0006] Technical solution: The meta-learning-based layered material rendering method of the present invention includes:

[0007] (1) obtaining a sample data set, wherein each sample in the sample data set includes a layer of material parameters, an incident light direction, an outgoing light direction, and a reflectance as a label;

[0008] (2) Constructing a reflectance prediction network model, the reflectance prediction network model includes a meta-network, a coordinate transformation module, and a BSDF network, the meta-network is used to predict the adaptive weight of the BSDF network based on layered material parameters, the coordinate transformation module is used to perform coordinate transformation and encoding on the incident light direction and the outgoing light direction to obtain light direction features, and the BSDF network is used to predict the reflectance based on the light direction features using the adaptive weights predicted by the meta-network and the shared weights, wherein the shared weights are obtained through training;

[0009] (3) using a sample data set to train the reflectance prediction network model;

[0010] (4) Input the layered material parameters to be rendered into the trained meta-network to obtain the adaptive weights Then extract the shared weights in the trained BSDF network Together they form the weight Θ F ;

[0011] (5) The renderer reads the weight Θ F , in the rendering process, the weight Θ is used according to the direction of the incident light and the direction of the outgoing light F Calculates reflectance and renders layered materials based on reflectance.

[0012] Furthermore, the meta-network is specifically a multi-layer perceptron network, comprising a plurality of sequentially connected fully connected layers. The coordinate transformation module includes: a Rusinkiewicz coordinate transformation unit, for performing Rusinkiewicz coordinate transformation on the incident light direction and the outgoing light direction; a spherical harmonic encoder, for performing spherical harmonic encoding on the output of the Rusinkiewicz coordinate transformation unit; a Cartesian coordinate transformation unit, for performing Cartesian coordinate transformation on the output of the Rusinkiewicz coordinate transformation unit; and a splicer, for splicing the outputs of the spherical harmonic encoder and the Cartesian coordinate transformation unit to obtain light direction features. The BSDF network is specifically a multi-layer perceptron network, comprising a plurality of sequentially connected fully connected layers.

[0013] Furthermore, the use of the sample data set to train the reflectance prediction network model specifically includes:

[0014] (3.1) First stage training: Each sample in the sample dataset is input into the reflectance prediction network model, the loss function is calculated, and the weights of the meta-network and the BSDF network are updated in real time based on the loss function;

[0015] (3.2) Second stage training: used to randomly divide the sample dataset into two subsets; alternately select samples from the two subsets to train the reflectance prediction network model trained in the first stage, calculate the loss function for the samples in the first subset, and update the weights of the BSDF network based on the loss function; calculate the loss function for the samples in the second subset, and update the weights of the meta-network based on the loss function.

[0016] The meta-learning-based layered material rendering device of the present invention includes:

[0017] A data set acquisition module is used to acquire a sample data set, wherein each sample in the sample data set includes a layer of material parameters, an incident light direction, an outgoing light direction, and a reflectance as a label;

[0018] A reflectance prediction network model construction module, wherein the reflectance prediction network model includes a meta-network, a coordinate transformation module, and a BSDF network. The meta-network is used to predict the adaptive weights of the BSDF network based on layered material parameters. The coordinate transformation module is used to perform coordinate transformation and encoding on the incident light direction and the outgoing light direction to obtain light direction features. The BSDF network is used to predict reflectance based on the light direction features using the adaptive weights predicted by the meta-network and shared weights, wherein the shared weights are obtained through training.

[0019] A model training module, configured to train the reflectance prediction network model using a sample data set;

[0020] The weight extraction module is used to input the layered material parameters to be rendered into the trained meta-network to obtain adaptive weights Then extract the shared weights in the trained BSDF network Together they form the weight Θ F ;

[0021] Rendering module, used to read weights Θ using a renderer F , in the rendering process, the weight Θ is used according to the direction of the incident light and the direction of the outgoing light F Calculates reflectance and renders layered materials based on reflectance.

[0022] Furthermore, the meta-network is specifically a multi-layer perceptron network, including several fully connected layers connected in sequence. The coordinate transformation module includes: a Rusinkiewicz coordinate transformation unit, used to perform Rusinkiewicz coordinate transformation on the incident light direction and the outgoing light direction; a spherical harmonic encoder, used to perform spherical harmonic encoding on the output of the Rusinkiewicz coordinate transformation unit; a Cartesian coordinate transformation unit, used to perform Cartesian coordinate transformation on the output of the Rusinkiewicz coordinate transformation unit; and a splicer, used to splice the outputs of the spherical harmonic encoder and the Cartesian coordinate transformation unit to obtain light direction features. Furthermore, the BSDF network is specifically a multi-layer perceptron network, including several fully connected layers connected in sequence.

[0023] Furthermore, the model training module specifically includes:

[0024] The first stage training unit is used to input each sample of the sample data set into the reflectance prediction network model, calculate the loss function, and update the weight of the meta-network and the weight of the BSDF network in real time based on the loss function;

[0025] The second stage training unit is used to randomly divide the sample data set into two subsets; alternately select samples from the two subsets to train the reflectance prediction network model trained in the first stage, calculate the loss function for the samples of the first subset, and update the weights of the BSDF network based on the loss function; calculate the loss function for the samples of the second subset, and update the weights of the meta-network based on the loss function.

[0026] Beneficial effects: Compared with the existing technology, the present invention has the following significant advantages: the present invention has low performance overhead, low noise in rendering results, and supports material editing. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the process of the layered material rendering method based on meta-learning provided by the present invention;

[0028] Figure 2 It is a schematic diagram of the network structure of the present invention;

[0029] Figure 3 is a schematic diagram illustrating a coordinate transformation module in the present invention;

[0030] Figure 4 This is a comparison chart of the rendering effects of the present invention and the existing method;

[0031] Figure 5 This is a comparison chart of the rendering effects of the present invention and the existing method in the texture situation;

[0032] Figure 6 This is a rendering effect diagram that supports material editing. DETAILED DESCRIPTION

[0033] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0034] Example 1

[0035] This embodiment provides a layered material rendering method based on meta-learning, such as Figure 1 As shown, the following steps are included:

[0036] S1. Obtain a sample data set, where each sample in the sample data set includes a layered material parameter, an incident light direction, an outgoing light direction, and a reflectance as a label.

[0037] In specific implementation, the material parameters may include the roughness α1 of the upper surface, the relative refractive index η1 of the upper surface, the absorption coefficient σ of the medium t , the albedo ρ of the medium, the roughness α2 of the lower surface, the relative refractive index η2 of the lower surface, and the 6 material parameter values ​​form a material parameter vector Γ={α1,η1,σ t ,ρ,α2,η2}. The incident light direction is ω i , the direction of the emitted light is ω o , the reflectance of a layered material parameter in a specific incident light direction and outgoing light direction is used as a label.

[0038] S2. Construct a reflectance prediction network model, which includes a meta-network, a coordinate transformation module and a BSDF network.

[0039] like Figure 2 As shown, the meta-network is used to predict the adaptive weights of the BSDF network based on the layered material parameters Meta-network M(Γ,Θ M ), specifically a multi-layer perceptron network, including several fully connected layers connected in sequence, and the input layer dimension is the material parameter vector Γ, Θ M is the weight of the meta-network, obtained through training, and the meta-network output is The dimension is 293; in a specific implementation, the number of fully connected layers can be 16 layers, with 256 neurons in each layer, and the activation function of the network is set to ReLU.

[0040] The weights of the BSDF network used in the present invention are based on the changes in the layered material parameters. Therefore, the calculated reflectance can be updated according to the layered material parameters, resulting in better rendering effects. The BSDF network of the present invention uses adaptive weights predicted by the meta-network based on the light direction characteristics. and shared weights Predicted reflectance, shared weights Obtained through training. The BSDF network is defined as F(ω i ,ω o ,Θ F ), where Θ F is the weight parameter of the BSDF network, Represents the concat operation. The BSDF network is specifically a multi-layer perceptron network, including several fully connected layers connected in sequence. Figure 2 As shown, the input layer dimension is 168, and the adaptive weight Indicates the concat operation, W * are the weights predicted by the meta-network (each layer has a neuron whose weights and biases are predicted by the meta-network), b * is the bias predicted by the meta-network, v * Supplementary inputs for each layer of the BSDF network predicted by the meta-network (32 per layer), shared weights is the weight shared by all layered materials (regardless of the material, fixed after training). In implementation, the number of fully connected layers can be 5, with 32 neurons per layer, and the activation function of the network is set to ReLU.

[0041] like Figure 3 As shown, the coordinate transformation module is used to transform and encode the incident light direction and the outgoing light direction to obtain the light direction feature, specifically including: a Rusinkiewicz coordinate transformation unit, a spherical harmonic encoder, a Cartesian coordinate transformation unit and a splicer. The Rusinkiewicz coordinate transformation unit is used to transform the incident light direction and the outgoing light direction into Rusinkiewicz coordinates. The Rusinkiewicz coordinate transformation refers to Szymon M.Rusinkiewicz.1998.ANewChange ofVariablesforEfficientBRDFRepresentation.InRenderingTechniques'98,GeorgeDrettakisandNelsonMax(Eds.).Springer Vienna, Vienna, 11–22; The spherical harmonic encoder is used to perform spherical harmonic encoding on the output of the Rusinkiewicz coordinate transformation unit with an order of 9 to obtain a 162-dimensional vector; The Cartesian coordinate transformation unit is used to perform Cartesian coordinate transformation on the output of the Rusinkiewicz coordinate transformation unit to obtain a 6-dimensional Cartesian coordinate; The splicer is used to splice the outputs of the spherical harmonic encoder and the Cartesian coordinate transformation unit to obtain a 168-dimensional light direction feature vector.

[0042] S3. Using a sample data set to train the reflectance prediction network model.

[0043] In practice, the training method consists of two phases. This two-phase strategy makes training easier to converge. Specifically, it includes:

[0044] S31, first stage training: input each sample of the sample data set into the reflectance prediction network model, calculate the loss function, and update the weight of the meta-network and the weight of the BSDF network in real time based on the loss function;

[0045] S32, Second Stage Training: Randomly split the sample dataset into two subsets. Alternately select samples from the two subsets to train the reflectance prediction network model trained in the first stage. A loss function is calculated for the samples in the first subset and the BSDF network weights are updated based on the loss function. A loss function is calculated for the samples in the second subset and the meta-network weights are updated based on the loss function. The reverse-Huber-loss loss is used throughout training.

[0046] S4. Input the layered material parameters to be rendered into the trained meta-network to obtain the adaptive weights Then extract the shared weights in the trained BSDF network Together they form the weight Θ F .

[0047] S5. Renderer reads weight Θ F , in the rendering process, the weight Θ is used according to the direction of the incident light and the direction of the outgoing light F Calculates reflectance and renders layered materials based on reflectance.

[0048] Example 2

[0049] This embodiment provides a meta-learning-based layered material rendering device, including:

[0050] A data set acquisition module is used to acquire a sample data set, wherein each sample in the sample data set includes a layer of material parameters, an incident light direction, an outgoing light direction, and a reflectance as a label;

[0051] A reflectance prediction network model construction module, wherein the reflectance prediction network model includes a meta-network, a coordinate transformation module, and a BSDF network. The meta-network is used to predict the adaptive weights of the BSDF network based on layered material parameters. The coordinate transformation module is used to perform coordinate transformation and encoding on the incident light direction and the outgoing light direction to obtain light direction features. The BSDF network is used to predict reflectance based on the light direction features using the adaptive weights predicted by the meta-network and shared weights, wherein the shared weights are obtained through training.

[0052] A model training module, configured to train the reflectance prediction network model using a sample data set;

[0053] The weight extraction module is used to input the layered material parameters to be rendered into the trained meta-network to obtain adaptive weights Then extract the shared weights in the trained BSDF network Together they form the weight Θ F ;

[0054] Rendering module, used to read weights Θ using a renderer F , in the rendering process, the weight Θ is used based on the set incident light direction and outgoing light direction F Calculates reflectance and renders layered materials based on reflectance.

[0055] Furthermore, the meta-network is specifically a multi-layer perceptron network, including several fully connected layers connected in sequence. The coordinate transformation module includes: a Rusinkiewicz coordinate transformation unit, used to perform Rusinkiewicz coordinate transformation on the incident light direction and the outgoing light direction; a spherical harmonic encoder, used to perform spherical harmonic encoding on the output of the Rusinkiewicz coordinate transformation unit; a Cartesian coordinate transformation unit, used to perform Cartesian coordinate transformation on the output of the Rusinkiewicz coordinate transformation unit; and a splicer, used to splice the outputs of the spherical harmonic encoder and the Cartesian coordinate transformation unit to obtain light direction features. Furthermore, the BSDF network is specifically a multi-layer perceptron network, including several fully connected layers connected in sequence.

[0056] Furthermore, the model training module specifically includes:

[0057] The first stage training unit is used to input each sample of the sample data set into the reflectance prediction network model, calculate the loss function, and update the weight of the meta-network and the weight of the BSDF network in real time based on the loss function;

[0058] The second stage training unit is used to randomly divide the sample data set into two subsets; alternately select samples from the two subsets to train the reflectance prediction network model trained in the first stage, calculate the loss function for the samples of the first subset, and update the weights of the BSDF network based on the loss function; calculate the loss function for the samples of the second subset, and update the weights of the meta-network based on the loss function.

[0059] The method of this embodiment corresponds to that of embodiment 1 one by one. For any incomplete details, please refer to embodiment 1 and will not be repeated here.

[0060] The present invention is simulated and verified, and the results are as follows Figure 4 、 5 , 6, Figure 5The rendering effects of the present invention and the existing method on a teapot scene are compared, where the teapot is a layered material. It can be seen that the present invention completes the same number of samples in a shorter time and the rendering result has less noise, proving that the present invention has better performance and less noise. Figure 6 The rendering effects of the present invention and the existing method on layered materials in texture situations are compared, where the earth is a layered material. It can be seen that the present invention completes more sampling times in the same amount of time, and the rendering results with the same number of sampling times have less noise, proving that the present invention has better performance, less noise, and supports layered materials in texture situations. Figure 6 This example demonstrates the rendering effect of the present invention supporting material editing, with the kettle body material being edited as a layered material. The texture resolution of the material on the left is 512*1024, and the meta-network processing time is 27 seconds. The texture resolution of the material on the right is 1024*1024, and the meta-network processing time is 53 seconds. Because the parameters read into the meta-network of the present invention have physical meaning, it is possible to perform targeted editing of specific material properties, such as surface roughness.

[0061] The above disclosure is only a preferred embodiment of the present invention and cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A meta-learning-based layered material rendering method, characterized by include: (1) obtaining a sample data set, wherein each sample in the sample data set includes a layer of material parameters, an incident light direction, an outgoing light direction, and a reflectance as a label; (2) Constructing a reflectance prediction network model, the reflectance prediction network model includes a meta-network, a coordinate transformation module, and a BSDF network, the meta-network is used to predict the adaptive weight of the BSDF network based on layered material parameters, the coordinate transformation module is used to perform coordinate transformation and encoding on the incident light direction and the outgoing light direction to obtain light direction features, and the BSDF network is used to predict the reflectance based on the light direction features using the adaptive weights predicted by the meta-network and the shared weights, wherein the shared weights are obtained through training; The coordinate transformation module includes: A Rusinkiewicz coordinate transformation unit, used for performing Rusinkiewicz coordinate transformation on the incident light direction and the outgoing light direction; A spherical harmonic encoder, used for performing spherical harmonic encoding on the output of the Rusinkiewicz coordinate transformation unit; a Cartesian coordinate transformation unit, configured to perform Cartesian coordinate transformation on the output of the Rusinkiewicz coordinate transformation unit; A splicer, used for splicing the outputs of the spherical harmonic encoder and the Cartesian coordinate transformation unit to obtain light direction features; (3) using a sample data set to train the reflectance prediction network model; (4) Input the layered material parameters to be rendered into the trained meta-network to obtain the adaptive weights Then extract the shared weights in the trained BSDF network Together they form the weight Θ F ; (5) Renderer reads weight Θ F , in the rendering process, the weight Θ is used according to the direction of the incident light and the direction of the outgoing light F Calculates reflectance and renders layered materials based on reflectance.

2. The meta-learning-based layered material rendering method according to claim 1, characterized in that: The meta-network is specifically a multi-layer perceptron network, including a plurality of fully connected layers connected in sequence.

3. The meta-learning-based layered material rendering method according to claim 1, wherein: The BSDF network is specifically a multi-layer perceptron network, including several fully connected layers connected in sequence.

4. The meta-learning-based layered material rendering method according to claim 1, wherein: The adopting of the sample data set to train the reflectance prediction network model specifically includes: (3.1) First stage training: Each sample in the sample dataset is input into the reflectance prediction network model, the loss function is calculated, and the weights of the meta-network and the BSDF network are updated in real time based on the loss function; (3.2) Second stage training: The sample dataset is randomly divided into two subsets; samples from the two subsets are alternately selected to train the reflectance prediction network model trained in the first stage, the loss function is calculated for the samples in the first subset, and the weights of the BSDF network are updated based on the loss function; the loss function is calculated for the samples in the second subset, and the weights of the meta-network are updated based on the loss function.

5. A layered material rendering device based on meta-learning, characterized in that include: A data set acquisition module is used to acquire a sample data set, wherein each sample in the sample data set includes a layer of material parameters, an incident light direction, an outgoing light direction, and a reflectance as a label; A reflectance prediction network model construction module, wherein the reflectance prediction network model includes a meta-network, a coordinate transformation module, and a BSDF network. The meta-network is used to predict the adaptive weights of the BSDF network based on layered material parameters. The coordinate transformation module is used to perform coordinate transformation and encoding on the incident light direction and the outgoing light direction to obtain light direction features. The BSDF network is used to predict reflectance based on the light direction features using the adaptive weights predicted by the meta-network and shared weights, wherein the shared weights are obtained through training. The coordinate transformation module includes: A Rusinkiewicz coordinate transformation unit, used for performing Rusinkiewicz coordinate transformation on the incident light direction and the outgoing light direction; A spherical harmonic encoder, used for performing spherical harmonic encoding on the output of the Rusinkiewicz coordinate transformation unit; a Cartesian coordinate transformation unit, configured to perform Cartesian coordinate transformation on the output of the Rusinkiewicz coordinate transformation unit; A splicer, used for splicing the outputs of the spherical harmonic encoder and the Cartesian coordinate transformation unit to obtain light direction features; A model training module, configured to train the reflectance prediction network model using a sample data set; The weight extraction module is used to input the layered material parameters to be rendered into the trained meta-network to obtain adaptive weights Then extract the shared weights in the trained BSDF network Together they form the weight Θ F ; Rendering module, used to read weights Θ using a renderer F , in the rendering process, the weight Θ is used according to the direction of the incident light and the direction of the outgoing light F Calculates reflectance and renders layered materials based on reflectance.

6. The meta-learning-based layered material rendering device according to claim 5, characterized in that: The meta-network is specifically a multi-layer perceptron network, including a plurality of fully connected layers connected in sequence.

7. The meta-learning-based layered material rendering device according to claim 5, characterized in that: The BSDF network is specifically a multi-layer perceptron network, including several fully connected layers connected in sequence.

8. The meta-learning-based layered material rendering device according to claim 5, characterized in that: The model training module specifically includes: The first stage training unit is used to input each sample of the sample data set into the reflectance prediction network model, calculate the loss function, and update the weight of the meta-network and the weight of the BSDF network in real time based on the loss function; The second stage training unit is used to randomly divide the sample data set into two subsets; alternately select samples from the two subsets to train the reflectance prediction network model trained in the first stage, calculate the loss function for the samples of the first subset, and update the weights of the BSDF network based on the loss function; calculate the loss function for the samples of the second subset, and update the weights of the meta-network based on the loss function.

Images