A dynamic PBR material generation method and system based on a state space model

Abstract

This invention provides a method and system for generating dynamic PBR materials based on a state-space model, relating to the field of dynamic PBR material generation. The method includes: acquiring historical sample data and a variational autoencoder (VAE), the VAE comprising an encoder and a decoder; the encoder encodes the PBR texture at each time step to obtain a sequence of hidden state vectors; the VAE is trained using the PBR textures at all time steps until convergence; the state-space model is trained by combining the hidden state vector sequence and its corresponding external condition vector sequence to obtain a trained state-space model; the initial PBR texture to be evolved is acquired and input into the converged VAE to obtain an initial hidden state vector, which is then input into the trained state-space model to iteratively obtain a sequence of hidden state vectors for the generation stage, which is then decoded to construct a PBR texture sequence to obtain a dynamically evolving PBR material. The PBR material of this invention can evolve dynamically.

Description

Technical Field

[0001] This invention relates to the field of dynamic PBR material generation technology, and in particular to a dynamic PBR material generation method and system based on a state-space model. Background Technology

[0002] Physically Based Rendering (PBR) materials are widely used in modern computer graphics, capable of simulating real-world lighting and material interactions.

[0003] PBR maps include: a base color map, which describes the inherent diffuse color of the material; a roughness map, which controls the smoothness of the surface's micro-geometry; a metallicity map, which distinguishes between metallic and non-metallic areas; a normal map, which encodes the surface's micro-bumps and dents in tangent space; and an ambient occlusion map, which simulates indirect light attenuation caused by geometric occlusion.

[0004] However, most PBR materials in existing technologies are static and cannot evolve dynamically over time, making it difficult to simulate physical processes such as aging, corrosion, and weathering of materials under external environments. Summary of the Invention

[0005] Based on this, the purpose of this invention is to provide a dynamic PBR material generation method and system based on a state-space model, which solves the technical problem that static PBR materials cannot evolve dynamically in the prior art.

[0006] This invention provides a method for generating dynamic PBR materials based on a state-space model, comprising: A variational autoencoder is constructed by acquiring historical sample data of material temporal evolution. The sample data includes PBR maps at continuous time points and the external condition vectors corresponding to the PBR maps. The variational autoencoder includes an encoder and a decoder. The encoder compresses the PBR map at each time point into the hidden state space to obtain a hidden state vector with reduced dimension, thereby obtaining a sequence of hidden state vectors. The variational autoencoder is trained using the PBR maps at all time points until convergence, thereby fixing the encoder parameters and decoder parameters to obtain the converged variational autoencoder. The space in which the real number vectors output by the encoder are located is denoted as the hidden state space. A state-space model is constructed and trained by combining the hidden state vector sequence and its corresponding external condition vector sequence to obtain a trained state-space model; the initial PBR texture to be evolved is obtained and input into the converged variational autoencoder to obtain the initial hidden state vector, and the preset external condition vector sequence corresponding to the initial hidden state vector is determined. The initial hidden state vector and the corresponding preset external condition vector sequence are input into the trained state space model for iteration to generate hidden state vectors at multiple subsequent time steps, thereby constructing a hidden state vector sequence for the generation stage. The hidden state vector sequence for the generation stage is then restored by the decoder to obtain multiple PBR maps, thereby constructing a PBR map sequence. The PBR map has the same dimension as the input initial PBR map. The PBR map sequence is input into the PBR rendering engine to obtain a PBR material that dynamically evolves over time.

[0007] The aforementioned dynamic PBR material generation method based on the state-space model collects temporal PBR texture sample data of materials under controlled external conditions, trains a variational autoencoder to obtain the latent state-space representation, and constructs a state-space model to learn the state transition rules. Secondly, the initial PBR texture is encoded into a latent state vector, combined with a preset external condition vector sequence, and the latent state vector sequence of the generation stage at subsequent time steps is predicted iteratively through the state-space model. The decoder restores the PBR texture sequence, and finally sends it to the rendering engine to generate a dynamic material effect that evolves smoothly over time.

[0008] In addition, the dynamic PBR material generation method based on the state-space model according to the present invention may also have the following additional technical features: Furthermore, in the step of training the variational autoencoder, the total loss function of the variational autoencoder is: ; In the formula, L VAE E is the total loss function of the variational autoencoder; p(Y) To p (Y) The mathematical expectation; p (Y) The data generation distribution for the PBR texture; z is the hidden state vector; Y is the PBR texture; D ψ For decoder, ψ These are the network parameters that the decoder needs to learn; For encoder, These are the network parameters that the encoder needs to learn; β KL The weighting coefficients for the KL divergence are used to balance the relative importance of reconstruction error and hidden state space regularization. Let KL divergence be the KL divergence. The posterior distribution parameterized by the encoder, It is a standard normal prior.

[0009] Furthermore, in the state-space model, the evolution of the hidden state vector is described using discrete-time state-space equations, where the functional expression of the discrete-time state-space equations is: ; In the formula: k is the index variable for subsequent time steps, representing the specific position in the sequence, k=1,2,…,K, where K represents the sequence length; This represents the hidden state vector calculated at time k-1 in step k; when k=1, Right now , represents the hidden state vector at the initial time of the input calculated in step 1, and its value is the true hidden state vector extracted from the initial PBR map by the encoder. When k>1, Based on the training strategy or , This is the true hidden state vector extracted by the encoder from the true PBR texture at the (k-1)th subsequent time step. This is the hidden state vector predicted by the model at the (k-1)th subsequent time step; This is the hidden state vector predicted by the model for the k-th subsequent time step; Let be the discretized state transition matrix at the k-th subsequent time step. ,in, A The system matrix is ​​learnable; Let be the learnable step size parameter for the k-th subsequent time step. , For step size prediction networks, For the first The external condition vector at each subsequent time step; Given the discretized input matrix at the k-th subsequent time step, it is calculated using the numerically stable zero-order preservation method. ,in, The input matrix is ​​learnable; For the first The mapped external condition vector at each subsequent time step. ,in, W c The conditional mapping matrix, ; b c For bias vectors, ; d represents the real number field; d is the dimension of the hidden state vector; is the dimension of the external condition vector.

[0010] Furthermore, the learnable system matrix A adopts a structured parameter form, which is derived from a low-dimensional core matrix. Indirectly implemented through projection and reconstruction operations arrive Then The state transition, wherein the method for indirectly implementing the state transition specifically includes: Will Hidden state vectors in Through projection matrix P Mapped to Obtain the intermediate state vector at the (k-1)th subsequent time step. , ; The intermediate state vector at the (k-1)th subsequent time step The discretized HiPPO state transition matrix at the k-th subsequent time step Perform state transitions to obtain the state evolution term, and then... The mapped external condition vector at each subsequent time step Discretized HiPPO input matrix at the k-th subsequent time step A linear transformation is performed to obtain the input transformation term. The state evolution term is then added to the input transformation term to obtain the intermediate state vector at the k-th subsequent time step. , ; Then through the output matrix C The intermediate state vector at the k-th subsequent time step Reconstruction Obtain the hidden state vector predicted by the model at the k-th subsequent time step. , ; in: Represents a d-dimensional real vector space; express 3D real vector space; low-dimensional core matrix ,in, d is the dimension of the hidden state vector. Represented as an intermediate state vector The dimension; Represents the real number field; low-dimensional core matrix Initialize the HiPPO matrix structure to a negative definite form based on Legendre polynomials to capture long-range dependencies; P Let be the projection matrix. ; C For the output matrix, The projection matrix and output matrix are randomly initialized, and the projection matrix will... Projecting the hidden state vector in the middle to The output matrix will Reconstructing the intermediate state vector in While maintaining the long-range memory capability of the HiPPO matrix, the computational complexity is reduced. k is an index variable for subsequent time steps, representing the specific position in the sequence, k=1,2,…,K, where K represents the sequence length; Let be the intermediate state vector at the k-th subsequent time step. When k=1, Right now , which is the intermediate state vector at the initial moment; when k>1, Let be the intermediate state vector at the (k-1)th subsequent time step. ; This represents the discretized HiPPO state transition matrix at the k-th subsequent time step. Let be the discretized HiPPO input matrix at the k-th subsequent time step.

[0011] Furthermore, in the step of training the state-space model, the total loss in the training state-space model stage is: L=L recon + α L state + β L smooth ; In the formula, L represents the total loss during the training of the state-space model; L recon For texture reconstruction loss during the training of the state-space model; L state The hidden state prediction loss during the training of the state-space model; L smooth Temporal smoothness loss during the training of the state-space model; α The weighting coefficients for the hidden state prediction loss are... β These are the weighting coefficients for the temporal smoothness loss; in: ; In the formula, This is the PBR map of the predicted k-th subsequent time step from the decoder output; The PBR texture is the actual texture for the k-th subsequent time step; The perceptual loss is based on the VGG network; K represents the sequence length; k is the index variable at subsequent time steps, representing the specific position in the sequence; The weight coefficients for L1 reconstruction. The weighting coefficients for perceived loss; in: ; In the formula, This is the hidden state vector predicted by the model for the k-th subsequent time step; This is the true hidden state vector extracted by the encoder from the true PBR texture at the k-th subsequent time step; K represents the sequence length; k is the index variable for subsequent time steps, representing the specific position in the sequence; in:

[0012] In the formula: Represents the L1 norm; For feature extractors of pre-trained VGG networks; For image deformation operations based on optical flow fields; for arrive The optical flow field estimation is used to describe the motion displacement of pixels between adjacent PBR maps; The PBR texture is the actual texture for the k-th subsequent time step; This is the PBR map of the predicted k-th subsequent time step from the decoder output; The weighting coefficients for feature continuity. , which is the weighting coefficient for the deformation of the optical flow field; The total loss during the training of the state-space model; K represents the sequence length; k is the index variable at subsequent time steps, representing the specific position in the sequence; This represents the PBR map of the (k-1)th subsequent time step of the decoder output prediction.

[0013] Furthermore, in the step of constructing a PBR map sequence by reconstructing the hidden state vector sequence in the generation stage using the decoder, the method further includes: The obtained PBR map sequence was subjected to adjacent time step difference analysis to calculate the perceptual difference between adjacent time step PBR maps. The methods for analyzing the differences in PBR textures at adjacent time points include: If the decoder outputs the predicted PBR map at the k-th subsequent time step during the generation phase... The PBR map of the prediction at the (k-1)th subsequent time step output by the decoder during the generation phase. If the perceived difference does not exceed the preset perceived difference threshold δ, then it means Good visual continuity; if the decoder outputs the predicted PBR map at the k-th subsequent time step during the generation stage. The PBR map of the prediction at the (k-1)th subsequent time step output by the decoder during the generation phase. If the perceived difference exceeds the preset perceived difference threshold δ, then it means Poor visual continuity triggers a feedback adjustment mechanism; this mechanism employs smooth interpolation based on optical flow field constraints to adjust... .

[0014] Another aspect of the present invention provides a dynamic PBR material generation system based on a state-space model, the system comprising: The acquisition module is used to acquire historical sample data of material time-series evolution and construct a variational autoencoder. The sample data includes PBR maps at continuous time points and the external condition vectors corresponding to the PBR maps. The variational autoencoder includes an encoder and a decoder. The encoder compresses the PBR map at each time step into the hidden state space to obtain a hidden state vector with reduced dimension, thereby obtaining a sequence of hidden state vectors. The variational autoencoder is trained using the PBR maps at all time steps until convergence, thereby fixing the encoder parameters and decoder parameters to obtain the converged variational autoencoder. The space in which the real vectors output by the encoder are located is denoted as the hidden state space. The training module is used to construct a state-space model and train it by combining the hidden state vector sequence and its corresponding external condition vector sequence to obtain a trained state-space model; the initial PBR texture to be evolved is obtained and input into the converged variational autoencoder to obtain the initial hidden state vector, and the preset external condition vector sequence corresponding to the initial hidden state vector is determined. The restoration module is used to input the initial hidden state vector and the corresponding preset external condition vector sequence into the trained state space model for iteration to generate hidden state vectors at multiple subsequent time steps, thereby constructing a hidden state vector sequence for the generation stage. The hidden state vector sequence for the generation stage is restored by the decoder to obtain multiple PBR maps, thereby constructing a PBR map sequence. The PBR map has the same dimension as the input initial PBR map. The PBR map sequence is input into the PBR rendering engine to obtain a PBR material that dynamically evolves over time.

[0015] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dynamic PBR material generation method based on the state-space model as described above.

[0016] In another aspect, the present invention provides a data processing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the dynamic PBR material generation method based on the state-space model as described above. Attached Figure Description

[0017] Figure 1 This is a flowchart of the dynamic PBR material generation method based on a state-space model in the first embodiment of the present invention; The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0018] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0020] To facilitate understanding of the present invention, several embodiments are given below. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the present invention will be more thorough and complete.

[0021] Example 1 Please see Figure 1 The figure shows a dynamic PBR material generation method based on a state-space model in the first embodiment of the present invention, the method including steps S101 to S104: S101. Obtain historical sample data of material time-series evolution and construct a variational autoencoder. The variational autoencoder includes an encoder and a decoder. The encoder compresses the PBR map at each time step into the hidden state space to obtain a hidden state vector with reduced dimension, and then obtains a sequence of hidden state vectors.

[0022] The space containing the real-valued vectors output by the encoder is denoted as the hidden state space. As a concrete example, the sample data comes from both real-world data collection and physical simulation generation. Physical simulation generation involves using materials science software or physics-based material aging models to simulate processes such as corrosion, drying, and cracking in a computer. Different sequences of external condition vectors are set to generate PBR map data at continuous time points. Specifically, the sample data includes PBR maps at continuous time points and the corresponding external condition vectors. The PBR maps include a base color map, roughness map, metallicity map, normal map, and ambient occlusion map. The external condition vectors include environmental parameters such as time, temperature, humidity, and light intensity, used to drive the material evolution process.

[0023] Secondly, the acquired raw PBR texture data was organized into a time series sample set. Where n=0,1,...,N-1 are global sample indices. , Indicates the first The PBR texture at each sampling time point, where H is the height of the PBR texture and W is the width of the PBR texture. C In this embodiment, the number of channels is [number]. 9 corresponds to the nine channels in the five types of PBR textures. Specifically, the nine channels include 3 channels for base color, 1 channel for roughness, 1 channel for metallicity, 3 channels for normals, and 1 channel for ambient occlusion. Let N be the real number field, and N be the total number of samples.

[0024] Furthermore, a variational autoencoder includes an encoder and a decoder. The encoder converts the PBR texture... Compressed into hidden state vectors The hidden state vectors are then reconstructed into a PBR map using a decoder. Here, H is the height of the PBR map, and W is the width of the PBR map. C In this embodiment, the number of channels is [number]. 9 corresponds to the nine channels in the five types of PBR textures. For the real number field, This is the PBR texture at the nth sampling time. This represents the true hidden state vector extracted by the encoder from the real PBR texture at the nth sampling time, where d is the dimension of the hidden state vector.

[0025] Specifically, the encoder consists of multiple convolutional layers and fully connected layers, while the decoder consists of fully connected layers and transposed convolutional layers. Specifically, the encoder contains four convolutional layers with a stride of 2, and their channel numbers are 32, 64, 128, and 256 respectively. These layers are used to progressively downsample the PBR maps of the nine channels. Each layer is followed by batch normalization and a ReLU activation function, ultimately yielding a 256×4×4 feature map. This map is then passed through a fully connected layer to output the mean parameters of the hidden state vector distribution. And log-variance parameter The dimension of the hidden state vector The decoder consists of a fully connected layer and four transposed convolutional layers, which reconstruct the hidden state vector into a 9-channel PBR map. The output layer uses the Tanh activation function to make the output value range [-1, 1].

[0026] S102. Train the variational autoencoder using PBR maps at all times until convergence, thereby fixing the encoder and decoder parameters and obtaining the converged variational autoencoder.

[0027] In the training step of the variational autoencoder, the total loss function of the variational autoencoder is: ; In the formula, L VAE E is the total loss function of the variational autoencoder; p(Y) To p (Y) The mathematical expectation; p (Y) The data generation distribution for the PBR texture; z is the hidden state vector; Y is the PBR texture; D ψ For decoder, ψ These are the network parameters that the decoder needs to learn; For encoder, These are the network parameters that the encoder needs to learn; β KL The weighting coefficients for the KL divergence are used to balance the relative importance of reconstruction error and hidden state space regularization. Let KL divergence be the KL divergence. The posterior distribution parameterized by the encoder, It is a standard normal prior.

[0028] During training, the variational autoencoder was trained using all PBR texture data. The specific settings for training the variational autoencoder were: batch size of 32, Adam (Adaptive Moment Estimation) optimizer, learning rate of 0.001, and 10 iterations. After training, the encoder and decoder parameters were fixed for subsequent hidden state vector extraction.

[0029] After training converges, each of the original time-series samples is... Input encoder (where n is the global sample index, n=0,1,…,N-1, N is the total number of samples, corresponding to the nth sampling time). , Indicates the first (PBR map at each sampling time), taking the mean parameter of the encoder output hidden state vector distribution. As Hidden state vector Construct a dataset of hidden state vector sequences and its corresponding sequence of external condition vectors The hidden state vector sequence dataset and its corresponding external condition vector sequence are divided into a training set, a validation set, and a test set, with a ratio of 8:1:1. The training set, validation set, and test set are all divided into several sequence samples according to a fixed sequence length K. In this embodiment, K is taken as 8.

[0030] in, This represents the true hidden state vector extracted by the encoder from the true PBR texture at the nth sampling time. Indicates the first PBR texture at each sampling time; Let be the external condition vector at the nth sampling time.

[0031] It should be further explained that during the training phase of the variational autoencoder, a global sample index *n* is used to mark the actual sampling time in the dataset, with each sample encoded independently, and *N* being the total number of samples. However, the state-space model deals with sequence prediction tasks relative to the initial time step. Therefore, after entering the state-space model training and generation phase, the indexing system will be adjusted to the index variable *k* of subsequent time steps, where *k* represents the sequence length. Let be the external condition vector at the nth sampling time. For the first The external condition vector at each subsequent time step.

[0032] S103. Construct a state-space model and train it by combining the hidden state vector sequence and its corresponding external condition vector sequence to obtain a trained state-space model; obtain the initial PBR texture to be evolved and input it into the converged variational autoencoder to obtain the initial hidden state vector, and determine the preset external condition vector sequence corresponding to the initial hidden state vector.

[0033] As a concrete example, to ensure the model can accurately learn the evolution of materials and generate smooth and continuous dynamic effects, a state-space model is trained. This state-space model includes... , as well as ; The texture reconstruction loss is used during the training of the state-space model. The hidden state prediction loss is used during the training of the state-space model. This is the temporal smoothness loss during the training of the state-space model.

[0034] In the state-space model, the evolution of the hidden state vector is described by discrete-time state-space equations, where the functional expression of the discrete-time state-space equations is: ; In the formula: k is the index variable for subsequent time steps, representing the specific position in the sequence, k=1,2,…,K, where K represents the sequence length; This represents the hidden state vector calculated at time k-1 in step k; when k=1, Right now This represents the hidden state vector at the initial time step of the first step's input calculation, with values ​​taken as the true hidden state vector extracted from the initial PBR map by the encoder. When k>1, Based on the training strategy or , This is the true hidden state vector extracted by the encoder from the true PBR texture at the (k-1)th subsequent time step. This is the hidden state vector predicted by the model at the (k-1)th subsequent time step; This is the hidden state vector predicted by the model for the k-th subsequent time step; Let be the discretized state transition matrix at the k-th subsequent time step. , A The system matrix is ​​learnable; Let be the learnable step size parameter for the k-th subsequent time step. ; This is a step size prediction network, where the input to the step size prediction network is... and splicing; For the first The external condition vector at each subsequent time step; Given the discretized input matrix at the k-th subsequent time step, it is calculated using the numerically stable zero-order preservation method. ,in, For the first The mapped external condition vector at each subsequent time step. ,in, W c The conditional mapping matrix, ; b c For bias vectors, ; d represents the real number field; d is the dimension of the hidden state vector; The dimension of the external condition vector; The input matrix is ​​learnable.

[0035] Considering the long-range dependency of material evolution, meaning the current state is influenced by the cumulative effects of multiple historical moments, if we directly... Learning the system matrix A is not only computationally complex, but random initialization also makes it difficult to maintain the long-term transmission of historical information. Therefore, a learnable system matrix A adopts a structured parametric form to support efficient computation, which is indirectly achieved through a low-dimensional core matrix, projection, and reconstruction operations. arrive Then State transitions, specifically, indirect methods for achieving state transitions include: Will Hidden state vectors in Through projection matrix P Mapped to Obtain the intermediate state vector at the (k-1)th subsequent time step. , ; The intermediate state vector at the (k-1)th subsequent time step The discretized HiPPO state transition matrix at the k-th subsequent time step Perform state transitions to obtain the state evolution term, and then... The mapped external condition vector at each subsequent time step Discretized HiPPO input matrix at the k-th subsequent time step A linear transformation is performed to obtain the input transformation term. The state evolution term is then added to the input transformation term to obtain the intermediate state vector at the k-th subsequent time step. , ; Then through the output matrix C The intermediate state vector at the k-th subsequent time step Reconstruction Obtain the hidden state vector predicted by the model at the k-th subsequent time step. , ; in: Represents a d-dimensional real vector space; express 3D real vector space; low-dimensional core matrix ,in, d is the dimension of the hidden state vector. Represented as an intermediate state vector The dimension; Represents the real number field; low-dimensional core matrix Initialize the HiPPO matrix structure to a negative definite form based on Legendre polynomials to capture long-range dependencies; P Let be the projection matrix. ; C For the output matrix, The projection matrix and output matrix are randomly initialized. The projection matrix will... Projecting the hidden state vector in the middle to The output matrix will Reconstructing the intermediate state vector in While maintaining the long-range memory capability of the HiPPO matrix, the computational complexity is reduced. k is an index variable for subsequent time steps, representing the specific position in the sequence, k = 1, 2, ..., K, where K represents the sequence length; where... Projection , , Let be the intermediate state vector at the k-th subsequent time step. When k=1, Right now Let be the intermediate state vector at the initial moment. When k>1, Let be the intermediate state vector at the (k-1)th subsequent time step. ; This represents the hidden state vector calculated at time k-1 in step k; when k=1, Right now This represents the hidden state vector at the initial time step of the first step's input calculation, with values ​​taken as the true hidden state vector extracted from the initial PBR map by the encoder. When k>1, Based on the training strategy or , This is the true hidden state vector extracted by the encoder from the true PBR texture at the (k-1)th subsequent time step. This is the hidden state vector predicted by the model at the (k-1)th subsequent time step; , It is a low-dimensional core matrix. ; The learnable step size parameter for the k-th subsequent time step. ， This represents the discretized HiPPO state transition matrix at the k-th subsequent time step. The discretized HiPPO input matrix for the k-th subsequent time step is calculated using a numerically stable zero-order preservation method. , For a learnable low-dimensional input matrix, This is the hidden state vector predicted by the model for the k-th subsequent time step; For the first The mapped external condition vector at each subsequent time step.

[0036] Furthermore, to support adaptive temporal evolution, the learnable step size parameter for the k-th subsequent time step... The evolution rate is dynamically generated by a small neural network based on the external condition vector and the previous time-step input hidden state vector. This allows the model to adaptively adjust its evolution rate according to the context, for example, accelerating evolution when external conditions change drastically and slowing it down when conditions are stable. Specifically, the learnable step size parameter for the k-th subsequent time step... The dynamically generated expression is: ; In the formula, To obtain the external condition vector at the k-th subsequent time step, we first map it to the dimension of the hidden state vector through a linear layer, thus obtaining the mapped external condition vector at the k-th subsequent time step. , ,in, The conditional mapping matrix, ; For bias vectors, ; d represents the real number field; d is the dimension of the hidden state vector; The dimension of the external condition vector; Used for state updates. And step size prediction network. The input is and The splicing is used to achieve effective integration of conditional information and state information.

[0037] Through the above decomposition, the computational complexity is significantly reduced while maintaining the long-range memory capability of the HiPPO matrix.

[0038] To further enhance the state-space model's ability to capture multi-scale temporal features during material evolution, the hidden state vector at the (k-1)th time step is calculated in the k-th step. Decomposed into the rapidly changing hidden state component vector of the input at time k-1, calculated in step k. And the slowly changing hidden state component vector of the input at time k-1, calculated in step k. , and Multi-scale time evolution generation is achieved by modeling state transition equations at different time scales. The specific implementation process is as follows: First, the decomposition is achieved through a component extraction matrix: ,

[0039] In the formula, To extract matrices for rapidly changing components, Extracting the slowly changing components matrix; k is the index variable for subsequent time steps, representing the specific position in the sequence, k=1,2,…,K, where K represents the sequence length; when k=1, Right now , represents the hidden state vector at the initial time of the input calculated in step 1, and its value is the true hidden state vector extracted from the initial PBR map by the encoder. After decomposition, it was obtained , When k>1 and a teacher-forced strategy is adopted, the true hidden state vector extracted by the encoder from the true PBR map at the (k-1)th subsequent time step will be used. As After decomposition, it was obtained , When k>1 and a planned sampling strategy is used, the hidden state vector predicted by the model at the (k-1)th subsequent time step is... As After decomposition, it was obtained , ; Next, regarding Evolution:

[0040]

[0041]

[0042] right Evolution:

[0043]

[0044]

[0045] Finally, and The result is obtained by fusion using vector addition. :

[0046] In the formula, k is the index variable for subsequent time steps, representing the specific position in the sequence, k=1,2,…,K, where K represents the sequence length; Let be the rapidly changing intermediate state vector at the k-th subsequent time step. When k=1, Right now This represents the rapidly changing intermediate state vector at the initial moment. When k > 1, Let be the rapidly changing intermediate state vector at the (k-1)th subsequent time step. ; Let be the slowly changing intermediate state vector at the k-th subsequent time step. When k=1, Right now This represents the slowly changing intermediate state vector at the initial moment. When k > 1, This represents the slowly changing intermediate state vector at the (k-1)th subsequent time step. ; To rapidly change the projection matrix, To enable rapid changes in the output matrix; For a slowly changing projection matrix, Output matrix that changes slowly; This represents the rapidly changing hidden state component vector of the input at time k-1, calculated in step k. This represents the slowly changing hidden state component vector of the input at time k-1, calculated in step k; Let be the hidden state vector predicted by the model for the k-th subsequent time step. This represents the rapidly changing hidden state vector predicted by the model for the k-th subsequent time step. This is the slowly changing hidden state vector predicted by the model for the k-th subsequent time step; ; ; = ; = ; For the learnable, rapidly changing step size parameter at the k-th subsequent time step; The learnable, slowly changing step size parameter is used for the k-th subsequent time step. Represented as a rapidly changing low-dimensional core matrix; Represented as a slowly changing low-dimensional core matrix; It is represented as a learnable, rapidly changing low-dimensional input matrix; It is represented as a learnable, slowly changing low-dimensional input matrix; It is represented as the discretized, rapidly changing HiPPO state transition matrix at the k-th subsequent time step; It is represented as the discretized, slowly changing HiPPO state transition matrix at the k-th subsequent time step; It is represented as the discretized, rapidly changing HiPPO input matrix at the k-th subsequent time step; It is represented as the discretized, slowly changing HiPPO input matrix at the k-th subsequent time step; For the first The mapped external condition vector at each subsequent time step; Let represent the real number field, and d be the dimension of the hidden state vector. Represented as an intermediate state vector The dimension, and .

[0047] Furthermore, to improve the long-term stability and accuracy of the model, the training process of the state-space model employs a strategy combining teacher-mandated training and planned sampling. Teacher-mandated training forces the model to quickly learn state transition rules in the early stages of training, avoiding training instability caused by accumulated errors. Planned sampling probabilities linearly increase from 0 to a certain value, or are dynamically adjusted based on validation set performance, allowing the model to gradually adapt to free-running.

[0048] As a specific example, the strategies of teacher-mandated and planned sampling include: The first two iterations use a teacher-forced strategy to obtain the true hidden state vector extracted by the encoder from the true PBR map at the (k-1)th subsequent time step. As input; starting from the 3rd iteration, the hidden state vector for the (k-1)th subsequent time step is gradually introduced to predict the model. As input, the planned sampling probability increases linearly from 0 to 0.5, allowing the model to gradually adapt to its own prediction error and improve the robustness of long sequence generation. The specific settings for training the state-space model are: batch size of 8, Adam optimizer, learning rate of 0.00005, and 15 iterations. After each iteration, the loss is calculated on the validation set, and the model parameters with the lowest validation loss are saved as the optimal model. Gradient clipping is used during training to ensure that the global L2 norm of the gradients of all trainable parameters in the state-space model does not exceed 1.0, preventing gradient explosion.

[0049] Specifically, the total loss during the training of the state-space model is expressed as follows: L=L recon + α L state + β L smooth ; In the formula, L represents the total loss during the training of the state-space model; L recon For texture reconstruction loss during the training of the state-space model; L state The hidden state prediction loss during the training of the state-space model; L smooth Temporal smoothness loss during the training of the state-space model; α The weighting coefficients for the hidden state prediction loss are... β These are the weighting coefficients for the temporal smoothness loss; in: ; In the formula, This is the PBR map of the predicted k-th subsequent time step from the decoder output; The PBR texture is the actual texture for the k-th subsequent time step; The perceptual loss is based on the VGG (Visual Geometry Group) network; K represents the sequence length; k is the index variable at subsequent time steps, representing the specific position in the sequence; The weight coefficients for L1 reconstruction. The weighting coefficients for perceived loss; in: ; In the formula, This is the hidden state vector predicted by the model for the k-th subsequent time step; This is the true hidden state vector extracted by the encoder from the true PBR texture at the k-th subsequent time step; K represents the sequence length; k is the index variable for subsequent time steps, representing the specific position in the sequence; in:

[0050] In the formula: This represents the L1 norm, which is the sum of the absolute values ​​of all elements. For feature extractors of pre-trained VGG networks; For image deformation operations based on optical flow fields; for arrive The optical flow field estimation is used to describe the motion displacement of pixels between adjacent PBR maps; The PBR texture is the actual texture for the k-th subsequent time step; This is the PBR map of the predicted k-th subsequent time step from the decoder output; The weighting coefficients for feature continuity. , which is the weighting coefficient for the deformation of the optical flow field; The texture reconstruction loss is used during the training of the state-space model. The hidden state prediction loss is used during the training of the state-space model. Temporal smoothness loss during the training of the state-space model; The total loss during the training of the state-space model; K represents the sequence length; k is the index variable at subsequent time steps, representing the specific position in the sequence; This represents the PBR map of the (k-1)th subsequent time step of the decoder output prediction.

[0051] S104. Input the initial hidden state vector and the corresponding preset external condition vector sequence into the trained state space model for iteration to generate hidden state vectors at multiple subsequent time steps, thereby constructing the hidden state vector sequence of the generation stage; the hidden state vector sequence of the generation stage is restored by the decoder to obtain multiple PBR maps, thereby constructing a PBR map sequence. The PBR map has the same dimension as the input initial PBR map. The PBR map sequence is input into the PBR rendering engine to obtain a PBR material that dynamically evolves over time.

[0052] In the step of constructing a PBR map sequence by decoding the hidden state vector sequence during the generation phase, the method further includes: The obtained PBR map sequence was subjected to adjacent time step difference analysis to calculate the perceptual difference between adjacent time step PBR maps. The methods for analyzing the differences in PBR textures at adjacent time points include: If the decoder outputs the predicted PBR map at the k-th subsequent time step during the generation phase... The PBR map of the prediction at the (k-1)th subsequent time step output by the decoder during the generation phase. If the perceived difference does not exceed the preset perceived difference threshold δ, then it means Good visual continuity; if the decoder outputs the predicted PBR map at the k-th subsequent time step during the generation stage. The PBR map of the prediction at the (k-1)th subsequent time step output by the decoder during the generation phase. If the perceived difference exceeds the preset perceived difference threshold δ, then it means Poor visual continuity triggers a feedback adjustment mechanism; this mechanism employs smooth interpolation based on optical flow field constraints to adjust... .

[0053] Specifically, as a concrete example, the dynamically evolving PBR material generation method includes: First, input The initial hidden state vector is obtained through the encoder. Set a preset external condition vector sequence. ,in, This is the initial PBR texture to be evolved during the generation phase. This represents the true hidden state vector extracted by the encoder from the initial PBR texture to be evolved during the generation phase. Let k be the preset external condition vector for the k-th subsequent time step, where k = 1, 2, ..., K. Then, iterative prediction is performed using the trained state-space model to construct the hidden state vector sequence for the generation stage. The specific iterative process includes: when k = 1, the hidden state vector of the initial time step calculated in step 1 is used as the input. Values , combined The input is fed into the state-space model to obtain the hidden state vector predicted by the model at the first subsequent time step during the generation phase. When k>1, the hidden state vector at time k-1 of the input will be calculated in step k. The value is obtained from the previous prediction. , combined The input is fed into the state-space model to obtain the hidden state vector predicted by the model at the k-th subsequent time step during the generation phase. Iterate sequentially until k=K to construct the hidden state vector sequence of the generation stage. .

[0054] Secondly, the decoder restores the hidden state vector sequence from the generation phase into a PBR texture sequence. ,in, for The reconstruction result after encoder and decoder This is the PBR map of the prediction at the k-th subsequent time step output by the decoder during the generation phase. , where k=1,2,..,K.

[0055] As a concrete example, even with the temporal smoothness loss used in the state-space model training stage, long-term free generation can still accumulate errors, leading to abrupt changes between adjacent PBR textures. Therefore, perceptual differences are calculated for adjacent PBR textures, using the LPIPS distance metric. If the perceptual difference exceeds a preset perceptual difference threshold... In this embodiment, If the value is 0.15, the feedback adjustment mechanism is triggered. As a specific example, the feedback adjustment mechanism employs smooth interpolation based on optical flow field constraints, and the specific methods include: The RAFT model is used to estimate the PBR map of the decoder's output prediction at the (k-1)th subsequent time step in the generation phase. The PBR map of the prediction at the k-th subsequent time step output by the decoder during the generation phase. Optical flow field ,right conduct The operation yields the interpolated PBR texture during the generation stage. This forces the generated results to conform to the laws of motion in the physical world.

[0056] As a concrete example, relying entirely on smooth interpolation based on optical flow field constraints might mask new material evolution features or detailed textures that should be present at the current moment. Therefore, and By fusion coefficient Linear mixing is performed to further suppress jumps and avoid distortion caused by over-correction. The specific mixing method is as follows: ， ， in, For the generation stage via optical flow field right conduct The interpolated PBR texture generated after the operation; The perceived difference in actual calculations; Preset the perceived difference threshold; To achieve the maximum fusion coefficient, in this embodiment, Take 0.6; The fusion coefficient is... , Adjust dynamically according to the size of the difference. The maximum value is 0.6. Update according to the above mixed method. This effectively suppresses abrupt changes between adjacent PBR textures, ensuring the visual continuity of the generated sequence. After generation, the PBR texture sequence is output as PNG format for each channel. To facilitate direct import into the PBR rendering engine for real-time rendering, metallicity and roughness are merged into a single RGBA image, where the R channel stores metallicity and the Alpha channel stores smoothness, i.e., smoothness = 1 - roughness.

[0057] In summary, the dynamic PBR material generation method based on the state-space model in the above embodiments of the present invention collects temporal PBR texture sample data of materials under controlled external conditions, trains a variational autoencoder to obtain a hidden state-space representation, and constructs a state-space model to learn the state transition rules. Secondly, the initial PBR texture to be evolved is encoded into a hidden state vector, combined with a preset external condition vector sequence, and the hidden state vector sequence of the generation stage at subsequent time steps is iteratively predicted through the state-space model. The sequence is then decoded back into a PBR texture sequence and finally sent to the rendering engine to generate a dynamic material effect that evolves smoothly over time.

[0058] Example 2 The dynamic PBR material generation system based on a state-space model in the second embodiment of the present invention includes: The acquisition module is used to acquire material time-series evolution sample data and construct a variational autoencoder. The sample data includes PBR maps at continuous time points and the external condition vectors corresponding to the PBR maps. The variational autoencoder includes an encoder and a decoder. The encoder compresses the PBR map at each time point into the hidden state space to obtain a dimension-reduced hidden state vector, thereby obtaining a sequence of hidden state vectors. The variational autoencoder is trained using the PBR maps at all time points until convergence, thereby fixing the encoder parameters and decoder parameters to obtain the converged variational autoencoder. The space in which the real number vectors output by the encoder are located is denoted as the hidden state space. The training module is used to construct a state-space model and train it by combining the hidden state vector sequence and its corresponding external condition vector sequence to obtain a trained state-space model; the initial PBR texture to be evolved is obtained and input into the converged variational autoencoder to obtain the initial hidden state vector, and the preset external condition vector sequence corresponding to the initial hidden state vector is determined. The restoration module is used to input the initial hidden state vector and the corresponding preset external condition vector sequence into the trained state space model for iteration to generate hidden state vectors at multiple subsequent time steps, thereby constructing a hidden state vector sequence for the generation stage. The hidden state vector sequence for the generation stage is restored by the decoder to obtain multiple PBR maps, thereby constructing a PBR map sequence. The PBR map has the same dimension as the input initial PBR map. The PBR map sequence is input into the PBR rendering engine to obtain a PBR material that dynamically evolves over time.

[0059] In summary, the dynamic PBR material generation system based on the state-space model in the above embodiments of the present invention collects temporal PBR texture sample data of materials under controlled external conditions, trains a variational autoencoder to obtain a hidden state-space representation, and constructs a state-space model to learn the state transition rules. Secondly, the initial PBR texture to be evolved is encoded into a hidden state vector, combined with a preset external condition vector sequence, and the hidden state vector sequence of the generation stage at subsequent time steps is iteratively predicted through the state-space model. The sequence is then decoded back into a PBR texture sequence and finally sent to the rendering engine to generate a dynamic material effect that evolves smoothly over time.

[0060] Furthermore, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the methods described above.

[0061] Furthermore, embodiments of the present invention also propose a data processing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the methods described above.

[0062] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0063] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0064] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0065] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0066] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for generating dynamic PBR materials based on a state-space model, characterized in that, include: A variational autoencoder is constructed by acquiring historical sample data of material temporal evolution. The sample data includes PBR maps at continuous time points and the external condition vectors corresponding to the PBR maps. The variational autoencoder includes an encoder and a decoder. The encoder compresses the PBR map at each time point into the hidden state space to obtain a hidden state vector with reduced dimension, thereby obtaining a sequence of hidden state vectors. The variational autoencoder is trained using the PBR maps at all time points until convergence, thereby fixing the encoder parameters and decoder parameters to obtain the converged variational autoencoder. The space in which the real number vectors output by the encoder are located is denoted as the hidden state space. A state-space model is constructed and trained by combining the hidden state vector sequence and its corresponding external condition vector sequence to obtain a trained state-space model; the initial PBR texture to be evolved is obtained and input into the converged variational autoencoder to obtain the initial hidden state vector, and the preset external condition vector sequence corresponding to the initial hidden state vector is determined. The initial hidden state vector and the corresponding preset external condition vector sequence are input into the trained state space model for iteration to generate hidden state vectors at multiple subsequent time steps, thereby constructing a hidden state vector sequence for the generation stage. The hidden state vector sequence for the generation stage is then restored by the decoder to obtain multiple PBR maps, thereby constructing a PBR map sequence. The PBR map has the same dimension as the input initial PBR map. The PBR map sequence is input into the PBR rendering engine to obtain a PBR material that dynamically evolves over time.

2. The dynamic PBR material generation method based on a state-space model according to claim 1, characterized in that, In the step of training the variational autoencoder, the total loss function of the variational autoencoder is: ； In the formula, L VAE E is the total loss function of the variational autoencoder; p(Y) To p (Y) The mathematical expectation; p (Y) The data generation distribution for the PBR texture; z is the hidden state vector; Y is the PBR texture; D ψ For decoder, ψ These are the network parameters that the decoder needs to learn; For encoder, These are the network parameters that the encoder needs to learn; β KL The weighting coefficients for the KL divergence are used to balance the relative importance of reconstruction error and hidden state space regularization. Let KL divergence be the KL divergence. The posterior distribution parameterized by the encoder, It is a standard normal prior.

3. The dynamic PBR material generation method based on a state-space model according to claim 1, characterized in that, In the state-space model, the evolution of the hidden state vector is described by discrete-time state-space equations, where the functional expression of the discrete-time state-space equations is: ； In the formula: k is the index variable for subsequent time steps, representing the specific position in the sequence, k=1,2,…,K, where K represents the sequence length; This represents the hidden state vector calculated at time k-1 in step k; when k=1, Right now , represents the hidden state vector at the initial time of the input calculated in step 1, and its value is the true hidden state vector extracted from the initial PBR map by the encoder. When k>1, Based on the training strategy or , This is the true hidden state vector extracted by the encoder from the true PBR texture at the (k-1)th subsequent time step. This is the hidden state vector predicted by the model at the (k-1)th subsequent time step; This is the hidden state vector predicted by the model for the k-th subsequent time step; Let be the discretized state transition matrix at the k-th subsequent time step. ,in, A The system matrix is ​​learnable; Let be the learnable step size parameter for the k-th subsequent time step. , For step size prediction networks, For the first The external condition vector at each subsequent time step; Given the discretized input matrix at the k-th subsequent time step, it is calculated using the numerically stable zero-order preservation method. ,in, The input matrix is ​​learnable; For the first The mapped external condition vector at each subsequent time step. ,in, W c The conditional mapping matrix, ; b c For bias vectors, ; d represents the real number field; d is the dimension of the hidden state vector; is the dimension of the external condition vector.

4. The dynamic PBR material generation method based on a state-space model according to claim 3, characterized in that, The learnable system matrix A takes the form of structured parameters, which is derived from a low-dimensional core matrix. Indirectly implemented through projection and reconstruction operations arrive Then The state transition, wherein the method for indirectly implementing the state transition specifically includes: Will Hidden state vectors in Through projection matrix P Mapped to Obtain the intermediate state vector at the (k-1)th subsequent time step. , ; The intermediate state vector at the (k-1)th subsequent time step Discretized HiPPO state transition matrix at the k-th subsequent time step Perform state transitions to obtain the state evolution term, and then... The mapped external condition vector at each subsequent time step Discretized HiPPO input matrix at the k-th subsequent time step A linear transformation is performed to obtain the input transformation term. The state evolution term is then added to the input transformation term to obtain the intermediate state vector at the k-th subsequent time step. , ; Then through the output matrix C The intermediate state vector at the k-th subsequent time step Reconstruction Obtain the hidden state vector predicted by the model at the k-th subsequent time step. , ; in: Represents a d-dimensional real vector space; express 3D real vector space; low-dimensional core matrix ,in, d is the dimension of the hidden state vector. Represented as an intermediate state vector The dimension; Represents the real number field; low-dimensional core matrix Initialize the HiPPO matrix structure to a negative definite form based on Legendre polynomials to capture long-range dependencies; P For the projection matrix, ; C For the output matrix, The projection matrix and output matrix are randomly initialized, and the projection matrix will... Projecting the hidden state vector in the middle to The output matrix will Reconstructing the intermediate state vector in While maintaining the long-range memory capability of the HiPPO matrix, the computational complexity is reduced. k is an index variable for subsequent time steps, representing the specific position in the sequence, k=1,2,…,K, where K represents the sequence length; Let be the intermediate state vector at the k-th subsequent time step. When k=1, Right now , which is the intermediate state vector at the initial moment; when k>1, Let be the intermediate state vector at the (k-1)th subsequent time step. ; This represents the discretized HiPPO state transition matrix at the k-th subsequent time step. Let be the discretized HiPPO input matrix at the k-th subsequent time step.

5. The dynamic PBR material generation method based on a state-space model according to claim 1, characterized in that, In the step of training the state-space model, the total loss in the training stage of the state-space model is: L=L recon + α L state + β L smooth ; In the formula, L represents the total loss during the training of the state-space model; L recon For texture reconstruction loss during the training of the state-space model; L state The hidden state prediction loss during the training of the state-space model; L smooth Temporal smoothness loss during the training of the state-space model; α The weighting coefficients for the hidden state prediction loss are... β These are the weighting coefficients for the temporal smoothness loss; in: ； In the formula, This is the PBR map of the predicted k-th subsequent time step from the decoder output; The PBR texture is the actual texture for the k-th subsequent time step; The perceptual loss is based on the VGG network; K represents the sequence length; k is the index variable at subsequent time steps, representing the specific position in the sequence; The weight coefficients for L1 reconstruction. The weighting coefficients for perceived loss; in: ； In the formula, This is the hidden state vector predicted by the model for the k-th subsequent time step; This is the true hidden state vector extracted by the encoder from the true PBR texture at the k-th subsequent time step; K represents the sequence length; k is the index variable for subsequent time steps, representing the specific position in the sequence; in: In the formula: Represents the L1 norm; For feature extractors of pre-trained VGG networks; For image deformation operations based on optical flow fields; for arrive The optical flow field estimation is used to describe the motion displacement of pixels between adjacent PBR maps; The PBR texture is the actual texture for the k-th subsequent time step; This is the PBR map of the predicted k-th subsequent time step from the decoder output; The weighting coefficients for feature continuity. , which is the weighting coefficient for the deformation of the optical flow field; The total loss during the training of the state-space model; K represents the sequence length; k is the index variable at subsequent time steps, representing the specific position in the sequence; This represents the PBR map of the (k-1)th subsequent time step of the decoder output prediction.

6. The dynamic PBR material generation method based on a state-space model according to claim 1, characterized in that, In the step of constructing a PBR map sequence by reconstructing the hidden state vector sequence in the generation stage using the decoder, the method further includes: The obtained PBR map sequence was subjected to adjacent time step difference analysis to calculate the perceptual difference between adjacent time step PBR maps. The methods for analyzing the differences in PBR textures at adjacent time points include: If the decoder outputs the predicted PBR map at the k-th subsequent time step during the generation phase... The PBR map of the prediction at the (k-1)th subsequent time step output by the decoder during the generation phase. If the perceived difference does not exceed the preset perceived difference threshold δ, then it means Good visual continuity; if the decoder outputs the predicted PBR map at the k-th subsequent time step during the generation stage. The PBR map of the prediction at the (k-1)th subsequent time step output by the decoder during the generation phase. If the perceived difference exceeds the preset perceived difference threshold δ, then it means Poor visual continuity triggers a feedback adjustment mechanism; this mechanism employs smooth interpolation based on optical flow field constraints to adjust... .

7. A dynamic PBR material generation system based on a state-space model, characterized in that, The system includes: The acquisition module is used to acquire historical sample data of material time-series evolution and construct a variational autoencoder. The sample data includes PBR maps at continuous time points and the external condition vectors corresponding to the PBR maps. The variational autoencoder includes an encoder and a decoder. The encoder compresses the PBR map at each time step into the hidden state space to obtain a hidden state vector with reduced dimension, thereby obtaining a sequence of hidden state vectors. The variational autoencoder is trained using the PBR maps at all time steps until convergence, thereby fixing the encoder parameters and decoder parameters to obtain the converged variational autoencoder. The space in which the real vectors output by the encoder are located is denoted as the hidden state space. The training module is used to construct a state-space model and train it by combining the hidden state vector sequence and its corresponding external condition vector sequence to obtain a trained state-space model; the initial PBR texture to be evolved is obtained and input into the converged variational autoencoder to obtain the initial hidden state vector, and the preset external condition vector sequence corresponding to the initial hidden state vector is determined. The restoration module is used to input the initial hidden state vector and the corresponding preset external condition vector sequence into the trained state space model for iteration to generate hidden state vectors at multiple subsequent time steps, thereby constructing a hidden state vector sequence for the generation stage. The hidden state vector sequence for the generation stage is restored by the decoder to obtain multiple PBR maps, thereby constructing a PBR map sequence. The PBR map has the same dimension as the input initial PBR map. The PBR map sequence is input into the PBR rendering engine to obtain a PBR material that dynamically evolves over time.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the dynamic PBR material generation method based on the state-space model as described in any one of claims 1-6.

9. A data processing apparatus, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the dynamic PBR material generation method based on the state-space model as described in any one of claims 1-6.

Images