A digital human image reproduction method and system based on NeRF

Abstract

The present invention discloses a method and system for replicating digital human images based on NeRF, and relates to the technical field of digital human image replicating. The method comprises: collecting digital human images to obtain training data sets; expanding and generating training samples; reconstructing a model using the expanded digital human image data training data; extracting features from the results of digital human image replicating output by the autoencoder, and inputting the feature vector after feature extraction into the classifier model to perform digital human image data screening. The present invention improves the details and authenticity of the generated samples, makes the expanded data closer to the real collected images in terms of fidelity, enhances the adaptability and stability when screening digital human image data, makes the screening model more adaptable when facing diversified replica data, improves the classification effect, and finally ensures the high quality and consistency of the replica data.

Description

Technical Field

[0001] The present invention relates to the field of digital human technology, and in particular to a method and system for reproducing a digital human image based on NeRF. Background Art

[0002] In recent years, with the rapid development of virtual reality, augmented reality, film and television special effects, and digital entertainment, the demand for digital human images has continued to increase. Digital human image reproduction, that is, using 3D modeling and image synthesis technology to digitally reproduce the appearance, movements, and expressions of real people, has become an important direction in the industry. However, existing technologies face many challenges in detail reproduction, noise adaptability, and model generalization, and it is difficult to meet the application requirements of high realism and high precision.

[0003] The existing technology still has problems such as unclear details and rough textures that make it difficult to achieve natural effects. Traditional autoencoder algorithms have poor adaptability to noise in practical applications. When the data is interfered by noise, the generated digital human image will be distorted or blurred. Gradients may disappear or explode during the training process. Especially when processing complex high-dimensional data, model training is unstable and it is difficult to handle data diversity, resulting in unsatisfactory screening effects, which ultimately affects the quality of the replicated data. Summary of the invention

[0004] In order to solve the technical problems existing in the digital human image replication in the prior art, the present invention provides a digital human image replication method and system based on NeRF.

[0005] The present invention is achieved through the following technical solutions:

[0006] A method for replicating a digital human image based on NeRF, characterized by comprising:

[0007] S1: A training dataset is obtained by capturing digital human images from multiple angles using a high-resolution camera and a specially configured lighting system.

[0008] S2: Generate training samples based on the generative adversarial network algorithm of quantum super-resolution synthesis;

[0009] S3: An autoencoder algorithm based on high-dimensional matrix decomposition uses the expanded digital human image data training data to reconstruct the model;

[0010] S4: A neural network algorithm based on dynamic population evolution optimization is used to extract features from the digital human image reproduction results output by the autoencoder;

[0011] S5: The extreme learning machine algorithm based on the low-rank matrix inputs the feature vector after feature extraction into the classifier model to screen the digital human image data.

[0012] Furthermore, the generative adversarial network algorithm based on quantum super-resolution synthesis includes the following steps:

[0013] S21: Initialize the network parameters of the generator and discriminator of the generative adversarial network;

[0014] S22: The training process includes multiple iteration cycles. In each cycle, the generator and the discriminator are trained alternately according to the adversarial strategy. The generator generates digital human image data samples, and the discriminator distinguishes the real samples from the generated samples.

[0015] S23: In each training cycle, the generator and discriminator are optimized, and the parameters of the generator and discriminator are updated using the gradient descent method;

[0016] S24: During the training of the generator, the quantum gates are dynamically adjusted to achieve high-dimensional feature transformation of the simulated data;

[0017] S25: Repeat the above steps until the preset stop iteration condition is met, indicating that the model training is completed.

[0018] Furthermore, the autoencoder algorithm based on high-dimensional matrix decomposition includes the following steps:

[0019] S31: Initialize the network parameters of the autoencoder;

[0020] S32: During the training process of the autoencoder, each input digital human image data is converted into a representation of the latent space through the encoder;

[0021] S33: The encoded digital human image data is then fed into a decoder, and the decoder reconstructs the original input digital human image data from the compressed latent representation;

[0022] S34: In each iteration, adding a small amount of random noise to the input digital human image data;

[0023] S35: Update the parameters of the decoder through the back propagation algorithm;

[0024] S36: Repeat the above steps until the preset stop iteration condition is met, indicating that the model training is completed.

[0025] Furthermore, the neural network algorithm based on dynamic population evolution optimization includes the following steps:

[0026] S41: According to the initialization method of the swarm bionic algorithm, in the initialization stage, an initial population is generated, and each individual represents a configuration of network weights;

[0027] S42: for each individual in the population, use its corresponding neural network configuration to process the input training data of the digital human image, calculate the output of the model, and evaluate its performance according to a predetermined loss function;

[0028] S43: According to the fitness of the individuals, the best performing individuals are selected from the current population and retained as candidate solutions for the next generation;

[0029] S44: Generate new individuals through crossover and mutation operations. The crossover operation allows two excellent individuals to exchange some genes to generate new offspring.

[0030] S45: Repeat the above steps until the preset stop iteration condition is met, indicating that the model training is completed.

[0031] Furthermore, the extreme learning machine algorithm based on the low-rank matrix includes the following steps:

[0032] S51: Initialize the parameters of the extreme learning machine;

[0033] S52: before the extreme learning machine training, the weight matrix of the hidden layer is subjected to low-rank perturbation;

[0034] S53: using the adjusted weights and biases, calculating the hidden layer output of the training data of the digital human image through a single forward propagation, and calculating the output weight;

[0035] S54: Calculate the parameter statistical characteristics of the hidden layer neurons through the output of the hidden layer, which are used in the loss function calculation of the extreme learning machine to achieve parameter sparseness of the hidden layer neurons;

[0036] S55: The classification performance of the extreme learning machine is evaluated by the loss function, and the model parameters are adjusted based on the feedback.

[0037] Furthermore, the calculation method of the loss function in the adversarial network is expressed as:

[0038]

[0039] In the formula, Represents a sample of real digital human image data, The pseudo digital human image data output by the generator, and are the outputs of the discriminator for the real digital human image data and the generated digital human image data respectively; represents the discriminator function; represents a generator function; represents the loss of the generative adversarial network; represents the loss function of the generative adversarial network; express expectations; represents the quantum state distribution of noise; Represents the distribution of real digital human image data; Indicates compliance with a specific distribution; is the nonlinear loss term.

[0040] Further, the addition of a small amount of random noise is expressed as follows:

[0041]

[0042]

[0043] In the formula, The digital human image data after adding noise, It is the parameter update operation; is from a mean of 0 and a variance of Noise drawn from a normal distribution; For the The noise level of the iteration; The mean is 0 and the variance is Normal distribution of is the variance of the noise; is a normal distribution.

[0044] Furthermore, the crossover and mutation operations are expressed as:

[0045]

[0046]

[0047] In the formula, is the crossover rate, is the weight of the neural network corresponding to the first individual selected, is the bias of the neural network corresponding to the first individual selected, is the weight of the neural network corresponding to the second individual selected, is the bias of the neural network corresponding to the second individual selected, is the weight of the neural network corresponding to the individual after the crossover operation, is the bias of the neural network corresponding to the individual after the crossover operation;

[0048] Moreover, the mutation operation performs a small random perturbation on the newly generated individual weights, which can be expressed as:

[0049]

[0050]

[0051] In the formula, represents the variance of the variation, is the weight of the neural network corresponding to the individual after the mutation operation, is the bias of the neural network corresponding to the individual after the mutation operation.

[0052] The present invention also provides a digital human image reproduction system based on NeRF, based on the digital human image reproduction method based on NeRF as described above, which comprises:

[0053] The digital human image acquisition module is used to acquire training data sets. Specifically, it uses a high-resolution camera and a specially configured lighting system to capture digital human images from multiple angles to obtain the three-dimensional geometric information and surface texture details of the target human.

[0054] A training sample expansion generation module, which is used to generate samples based on a generative adversarial network algorithm for quantum super-resolution synthesis, wherein the generator part of the generative adversarial network simulates high-dimensional feature mapping of digital human image data through quantum gate operations;

[0055] A digital human image data training data reconstruction module, which is used for an autoencoder algorithm based on high-dimensional matrix decomposition to learn and reproduce the high-dimensional geometry and surface features of the character from the expanded digital human image training data. The autoencoder structure has two parts: an encoder and a decoder. The encoder is responsible for compressing the input high-dimensional digital human image data into low-dimensional potential representations, and the decoder reconstructs the original input digital human image data from these potential representations;

[0056] The result feature extraction module of the digital human image replication is used to adopt the neural network algorithm based on dynamic population evolution optimization as the feature extraction model, and use the self-correction mechanism to automatically adjust the evolution rules according to the characteristics of the current digital human image training data, and adjust the probability of crossover and mutation according to the change trend of the loss function;

[0057] The digital human image data screening module is used to use the extreme learning machine algorithm based on a low-rank matrix as a digital human image data screening model to input the feature vector after feature extraction into the classifier model for digital human image data screening.

[0058] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which program instructions of a digital human image replication method based on NeRF are stored. The program instructions of the digital human image replication method based on NeRF can be executed by one or more processors to implement the steps of the digital human image replication method based on NeRF as described above.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] (1) The present invention adopts a generative adversarial network algorithm for quantum super-resolution synthesis, which captures the high-dimensional features of data through quantum gate operations, thereby improving the details and authenticity of the generated samples. Without increasing the computational complexity, it effectively improves the resolution of the generated data and the detail performance of local areas, improves the adaptability of lighting, shadows and textures, and makes the expanded data closer to the real collected images in terms of realism.

[0061] (2) In the data reconstruction process, the present invention adopts an autoencoder algorithm based on high-dimensional matrix decomposition, and enhances the robustness of the model in a strong noise environment by adopting a noise adaptation training mechanism, thereby improving the reconstruction accuracy and stability of the model.

[0062] (3) The present invention improves the model generalization ability and training efficiency through a dynamic population evolution optimization algorithm, while enhancing the effectiveness and accuracy of feature extraction.

[0063] (4) The present invention adopts a low-rank matrix perturbation strategy to optimize the extreme learning machine model, thereby enhancing its adaptability and stability in screening digital human image data, making the screening model more adaptable when facing diversified replica data, improving the classification effect, and ultimately ensuring the high quality and consistency of the replica data. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0065] Figure 1 This is a schematic diagram of a method flow of a digital human image replication method based on NeRF according to an embodiment of the present application;

[0066] Figure 2 is a training flow chart of an autoencoder algorithm based on high-dimensional matrix decomposition according to an embodiment of the present application;

[0067] Figure 3 It is a training flow chart of a neural network algorithm based on dynamic population evolution optimization according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0069] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0070] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The illustrations only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the form, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0071] The digital human image replica data is constructed through NeRF (Neural Radiance Fields) technology. The core of NeRF is a neural network that receives spatial coordinates and viewing directions as input, and outputs color values ​​(RGB) and density values; the density value represents the "opacity" of the point and is used to generate depth information of the image; during the implementation of NeRF, the neural network converts the replicated digital human image data into an implicit representation of the three-dimensional scene through back-propagation learning, and can generate realistic images from any angle, and achieve light and shadow effects through density and color information, ultimately presenting a complete three-dimensional model.

[0072] See also Figure 1 , a method for replicating a digital human image based on NeRF, comprising the following steps:

[0073] S1: Digital human image collection to obtain training data set;

[0074] Capturing digital humanoid images from multiple angles using a high-resolution camera and a specially configured lighting system, using a device equipped with a high-precision depth sensor, including a depth camera or a 3D scanning device, to obtain the three-dimensional geometric information and surface texture details of the target humanoid;

[0075] The collected digital human image data is encoded and stored on a high-performance server, using a lossless compression format to preserve the details of the image and depth information, such as using EXR or TIFF format.

[0076] S2: training sample expansion generation;

[0077] The collection, acquisition, labeling and preprocessing of training data for digital human images are time-consuming and labor-intensive, and insufficient training samples can easily lead to poor generalization ability of the model, while affecting the accuracy of the model. Based on this, the present invention adopts a generative adversarial network algorithm based on quantum super-resolution synthesis to generate samples. The generator part of the generative adversarial network simulates the high-dimensional feature mapping of digital human image data through quantum gate operations, thereby effectively increasing the resolution and detail richness of the digital human image data without increasing the computational complexity, optimizing the local area details of the generated image, and having better adaptability and expressiveness to changes in lighting, shadows and textures.

[0078] Specifically, the training process of the generative adversarial network algorithm based on quantum super-resolution synthesis is as follows:

[0079] S21: Initialize the network parameters of the generator and the discriminator of the generative adversarial network. Optionally, the network parameters of the generator and the discriminator are initialized randomly.

[0080] The input of the generator uses a quantum initialization strategy when initialized to simulate the superposition and entanglement characteristics of quantum states, which helps to capture the high-dimensional complexity of digital human image data at the very beginning, expressed as:

[0081]

[0082] In the formula, represents the quantum state vector, is the number of quantum bits, For the The initialization phase of the qubits, is the quantum ground state; Is an imaginary unit.

[0083] S22: The training process includes multiple iterations. In each cycle, the generator and the discriminator are trained alternately according to the adversarial strategy. The generator tries to create digital human image data samples that are as realistic as possible, and the discriminator tries to distinguish which are generated samples and which are real samples. The calculation method of the loss function in the adversarial process is expressed as follows:

[0084]

[0085] In the formula, Represents a sample of real digital human image data, The pseudo digital human image data output by the generator, and are the outputs of the discriminator for the real digital human image data and the generated digital human image data respectively; represents the discriminator function; represents a generator function; represents the loss of the generative adversarial network; represents the loss function of the generative adversarial network; express expectations; represents the quantum state distribution of noise; Represents the distribution of real digital human image data; ~ indicates compliance with a specific distribution; is the nonlinear loss term.

[0086] Furthermore, in order to handle nonlinear and complex digital human image data distribution, the calculation method of the nonlinear loss term is expressed as:

[0087]

[0088] in, is the weight vector of the discriminator; is the transpose of the discriminator’s weight vector; is the bias term of the discriminator, is the feature mapping function applied to the output of the generator.

[0089] In one embodiment, the feature mapping function applied to the generator output is defined as a deep feature extractor, and the calculation method is expressed as:

[0090]

[0091] in and are the weights and biases of the feature extraction layer, is the ReLU activation function.

[0092] S23: In each training cycle, the generator and discriminator are optimized. The parameters of the generator and discriminator are updated using the gradient descent method. The update method is expressed as:

[0093]

[0094]

[0095] In the formula, Indicates parameter update operation; and are the parameters of the generator and the discriminator respectively, is the learning rate of the generative adversarial network, and are the gradients corresponding to the loss functions respectively. Preferably, Set to 0.01.

[0096] S24: During the training of the generator, the quantum gate is dynamically adjusted to achieve high-dimensional feature transformation of the simulated data to meet the high fidelity requirements in the replication process. The dynamic adjustment of the quantum gate is expressed as:

[0097]

[0098]

[0099] In the formula, For the The quantum gate operation of the iteration is used as the quantum state vector of the next iteration; represents the quantum state vector of the tth iteration; is the Hamiltonian, which represents the interaction between quantum bits and single quantum bit operations; It is the iteration interval, the default setting is 1; For the The local magnetic field of a quantum bit, For the The qubit and The coupling strength between the qubits is is the first Pauli matrix of the qubit, is the second Pauli matrix of the qubit, A third Pauli matrix divided into qubits; Is an imaginary unit.

[0100] S25: Repeat the above steps until the preset stop iteration condition is met, indicating that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000 times.

[0101] After the training of the digital human image data expansion model is completed, the trained digital human image data expansion model is used to increase the number of samples. For example, if the original collected samples are 800, and the digital human image data expansion model generates 200 samples, then the expanded digital human image data set contains 1000 samples.

[0102] S3: Reconstruct the model using the expanded digital human image training data;

[0103] The function of the digital human image data reconstruction model is to achieve digital human image reproduction by reconstructing the digital human image data;

[0104] The reconstruction model is an autoencoder algorithm based on high-dimensional matrix decomposition, which learns and reproduces the high-dimensional geometry and surface features of the character from the training data of the expanded digital human image. The autoencoder structure has two parts: an encoder and a decoder. The encoder is responsible for compressing the input high-dimensional digital human image data into low-dimensional potential representations, and the decoder reconstructs the original input digital human image data from these potential representations.

[0105] In order to enhance the robustness of the model in a strong noise environment, the present invention adopts a noise adaptation training mechanism in the training process of the autoencoder, which can not only process the digital human image data collected from an ideal environment, but also effectively deal with various noise interferences that may be encountered in practical applications, thereby ensuring the quality and reliability of the reconstructed digital human image data.

[0106] Specifically, Figure 2 As shown in the figure, the training process of the autoencoder algorithm based on high-dimensional matrix decomposition is as follows:

[0107] S31: Initialize the network parameters of the autoencoder. Optionally, the weights and biases of the autoencoder are initialized randomly.

[0108] S32: During the training process of the autoencoder, each input digital human image data is first converted into a representation of the latent space through the encoder. These representations capture the key features of the input digital human image data and are effectively compressed in the latent space, expressed as:

[0109]

[0110] In the formula, represents the latent space representation; is the input digital human image data of the autoencoder, and are the encoder weights and biases of the autoencoder, respectively. is the Sigmoid activation function.

[0111] In one embodiment, the encoder uses a regularization term to update the weight parameters, taking into account the reconstruction error and model complexity, optimizing the learning process of the encoder, and ensuring the generalization ability of the model during the training process. The update method is expressed as:

[0112]

[0113] In the formula, is the update amount of the encoder weight; is the learning rate of the autoencoder, is the reconstruction loss function; is the L2 regularization term of the encoder weights to prevent overfitting; is the weight coefficient of the regularization term; is the gradient of the L2 regularization term of the encoder's weights with respect to the encoder's weights; is the gradient of the encoder weights. Preferably, Set to 0.01.

[0114] S33: The encoded digital human image data is then fed into a decoder, which attempts to reconstruct the original input digital human image data from these compressed latent representations. The decoder reconstruction process is represented as:

[0115]

[0116] In the formula, The reconstructed digital human image data of the decoder, and are the weights and biases of the decoder of the autoencoder, respectively.

[0117] S34: In each iteration, a small amount of random noise is added to the input digital human image data to simulate different interferences that may be encountered in the real environment, which helps to improve the adaptability and robustness of the model when facing real application scenarios, expressed as:

[0118]

[0119]

[0120] In the formula, The digital human image data after adding noise, It is the parameter update operation; is from a mean of 0 and a variance of Noise drawn from a normal distribution; For the The noise level of the iteration; The mean is 0 and the variance is Normal distribution of is the variance of the noise; is a normal distribution.

[0121] In one embodiment, in order to adapt the model to different levels of noise, a strategy of dynamically adjusting the noise level, that is, a noise level adjustment strategy, is adopted to dynamically adjust the noise mode, which is expressed as:

[0122]

[0123] In the formula, and In the The iteration and The noise level of the iteration; is the noise-adjusted learning rate; It is The rate of change of the reconstruction error at each iteration is used to guide the adaptive adjustment of the noise level.

[0124] S35: Update the decoder parameters through the back propagation algorithm so that the reconstruction error of the model is gradually reduced to ensure the balance between learning efficiency and model performance during the entire training process. The update method is expressed as:

[0125]

[0126] In the formula, and Respectively expressed in The iteration and The decoder weights for the iteration, is the learning rate of the autoencoder.

[0127] S36: Repeat the above steps until the preset stop iteration condition is met, indicating that the model training is completed. In one embodiment, the preset stop iteration condition is to reach a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000 times.

[0128] S4: Extract features from the digital human image reproduction result output by the autoencoder.

[0129] The present invention uses a 6-layer fully connected neural network for feature extraction. In the prior art, some solutions use neural networks for feature extraction. In some neural network structures, they may encounter problems such as gradient vanishing, gradient explosion, or falling into local optimal solutions, which affects the stability of training and the performance of the model. Therefore, the present invention uses a neural network algorithm based on dynamic population evolution optimization as a feature extraction model. In traditional population evolution algorithms, the evolution of all individuals is based on fixed rules. The present invention uses a self-correction mechanism to automatically adjust the evolution rules according to the characteristics of the current digital human image training data, and adjusts the probability of crossover and mutation according to the changing trend of the loss function, so that the algorithm can more flexibly adapt to different digital human image data distributions, and improve the generalization ability and training efficiency of the model.

[0130] Specifically, the training process of the neural network algorithm based on dynamic population evolution optimization is as follows:

[0131] S41: According to the initialization method of the swarm bionic algorithm, in the initialization stage, an initial population is generated, and each individual represents a configuration of network weights. Specifically, the population size is , for The individual initialization weights and biases are expressed as:

[0132]

[0133]

[0134] In the formula, For the The weight of the neural network corresponding to each individual, For the The bias of the neural network corresponding to each individual, Indicates The weight matrix of each individual in the initial state; Indicates The bias of each individual in the initial state; represents the initialization variance; It means the mean is 0 and the variance is Normal distribution of is a normal distribution. Preferably, Set to 0.01.

[0135] S42: For each individual in the population, use its corresponding neural network configuration to process the input digital human image training data, calculate the output of the model, and evaluate its performance according to a predetermined loss function. Specifically, for the i-th individual, use its weight and bias to calculate the loss on the digital human image training data set, expressed as:

[0136]

[0137] In the formula, For the The loss of the neural network corresponding to each individual; Indicates the number of samples input in the current batch; represents the composite loss function, Represents a neural network model function; Represents the characteristics of the jth sample; Indicates The labels of samples.

[0138] In one embodiment, the composite loss function includes a regularization term, which can increase the generalization ability of the model. The calculation method is expressed as:

[0139]

[0140] In the formula, is the mean square error function, is the regularization term, is the regularization parameter.

[0141] Furthermore, the calculation method of the regularization term is expressed as:

[0142]

[0143] In the formula, For the first The kth weight of the neural network corresponding to each individual.

[0144] S43: According to the fitness of the individual, the best performing individual is selected from the current population and retained as the candidate solution for the next generation. Specifically, based on the fitness of the individual, the excellent individual has a higher probability of being selected. The calculation method of the probability of an individual being selected is expressed as:

[0145]

[0146] In the formula, Indicates The probability of an individual being selected; is the parameter that controls the selection pressure; is the loss of the neural network corresponding to the i-th individual; For the The loss of the neural network corresponding to each individual. Preferably, Set to 2.

[0147] S44: Generate new individuals through crossover and mutation operations. The crossover operation allows two excellent individuals to exchange some genes to generate new offspring; the mutation operation randomly changes some genes in an individual to increase the diversity of the population. Specifically, the crossover operation randomly selects two individuals for gene exchange, which is expressed as:

[0148]

[0149]

[0150] In the formula, is the crossover rate, is the weight of the neural network corresponding to the first individual selected, is the bias of the neural network corresponding to the first individual selected, is the weight of the neural network corresponding to the second individual selected, is the bias of the neural network corresponding to the second individual selected, is the weight of the neural network corresponding to the individual after the crossover operation, is the bias of the neural network corresponding to the individual after the crossover operation. Preferably, Set to 0.3.

[0151] Moreover, the mutation operation performs a small random perturbation on the newly generated individual weights, which can be expressed as:

[0152]

[0153]

[0154] In the formula, represents the variance of the variation, is the weight of the neural network corresponding to the individual after the mutation operation, is the bias of the neural network corresponding to the individual after the mutation operation. Preferably, Set to 0.04.

[0155] S45: Repeat the above steps until the preset stop iteration condition is met, indicating that the model training is completed. In one embodiment, the preset stop iteration condition is to reach a preset maximum number of iterations, preferably, the preset maximum number of iterations is set to 1000 times.

[0156] S5: Input the feature vector after feature extraction into the classifier model to screen the digital human image data;

[0157] The present invention adopts an extreme learning machine algorithm based on a low-rank matrix as a digital human image data screening model, and utilizes a low-rank matrix perturbation strategy to fine-tune the weight matrix of the hidden layer to increase the adaptability and stability of the model when screening diverse digital human image data.

[0158] Specifically, the training process of the extreme learning machine algorithm based on the low-rank matrix is ​​as follows:

[0159] S51: Initialize the parameters of the extreme learning machine. In one embodiment, the initialization method is expressed as:

[0160]

[0161]

[0162]

[0163] In the formula, and are the initial values ​​of the hidden layer weights and biases of the extreme learning machine, is the dimension of the input digital human image data of the extreme learning machine, is the number of nodes in the hidden layer of the extreme learning machine; is the initialization range value of the extreme learning machine, which is used to ensure random initialization of weights and biases; is a random function.

[0164] S52: Before the extreme learning machine training, the weight matrix of the hidden layer is perturbed with low rank. By adding a small, low rank matrix to the original weight, it is used to simulate the potential disturbance and variation of the digital human image data and improve the robustness of the model, which is expressed as:

[0165]

[0166] In the formula, Generate a The rank of the matrix is , represents the strength of low-rank perturbation; is the disturbance coefficient, which controls the magnitude of the disturbance; is the weight of the extreme learning machine after adding low-rank perturbation.

[0167] In one embodiment, the low-rank matrix is ​​generated as follows:

[0168]

[0169] In the formula, and is a randomly generated matrix, is the first randomly generated matrix, is the second randomly generated matrix; is a The diagonal matrix of , whose elements on the diagonal decrease from high to low, represents different singular values, simulating the contribution of the principal components; is the transpose of the second randomly generated matrix; Represents a dimensional space.

[0170] Furthermore, the calculation method of the perturbation coefficient is expressed as:

[0171]

[0172] In the formula, represents the Frobenius norm; is a preset scaling factor that adjusts the relative strength of the perturbation to ensure that the perturbation does not excessively affect the characteristics of the original weights.

[0173] S53: Using the adjusted weights and biases, the hidden layer output of the training data of the digital human image is calculated through a single forward propagation, and the output weight is calculated using the least squares method or a similar technique. The method of performing forward propagation and calculating the output weight is expressed as:

[0174]

[0175]

[0176] In the formula, is the hidden layer output of the extreme learning machine; is the transpose of the hidden layer output of the extreme learning machine; is the training data of the digital human image of the extreme learning machine, is the target output of the extreme learning machine; is the output layer weight of the extreme learning machine, which is calculated by regularized pseudo-inverse; I is the unit matrix; is the regularization coefficient of the extreme learning machine, which is used to control the stability of the solution.

[0177] S54: After obtaining the hidden layer output, the parameter statistical characteristics of the hidden layer neurons are calculated through the hidden layer output, which are used to calculate the loss function of the extreme learning machine. That is, the parameter statistical characteristics of the hidden layer neurons are considered through the loss function of the extreme learning machine to achieve parameter sparsification of the hidden layer neurons, which is expressed as:

[0178]

[0179]

[0180]

[0181] In the formula, is the number of neurons in the hidden layer of the extreme learning machine, is the hidden layer mean pass statistic of the extreme learning machine; is the hidden layer standard deviation statistic of the extreme learning machine; is the hidden layer average difference statistic of the extreme learning machine; is the hidden layer output of the extreme learning machine corresponding to the The output value of a neuron, and are the hidden layer weights and biases of the extreme learning machine, respectively.

[0182] S55: The classification performance of the extreme learning machine is evaluated by the loss function, and the model parameters are adjusted according to the feedback. The calculation method is expressed as:

[0183]

[0184] In the formula, It is the loss function of the extreme learning machine, which is used to measure the error between the predicted output and the actual output; is the L2 norm; is the first statistic weight factor; is the second statistic weight factor; is the weight factor of the third statistic.

[0185] If the loss function of the extreme learning machine is less than the preset threshold, it means that the training of the extreme learning machine is completed. Otherwise, the extreme learning machine is retrained until the loss function of the extreme learning machine is less than the preset threshold.

[0186] After the classifier model training is completed, the classifier is used to screen the replicated digital human image data, which is specifically achieved through the classification categories of the classifier, that is, the classification categories of the extreme learning machine are two categories, corresponding to passing the screening and failing the screening;

[0187] The digital human image data that passes the screening is used as the final replicated digital human image data.

[0188] In this embodiment, the present invention adopts a generative adversarial network algorithm synthesized by quantum super-resolution, and captures the high-dimensional features of the data through quantum gate operations, thereby improving the details and authenticity of the generated samples, making the expanded data closer to the real collected images in terms of fidelity. In the data reconstruction process, the robustness of the model in a strong noise environment is enhanced by adopting a noise adaptation training mechanism, and the reconstruction accuracy and stability of the model are improved. Through the dynamic population evolution optimization algorithm, the generalization ability and training efficiency of the model are improved, and the effectiveness and accuracy of feature extraction are enhanced. The extreme learning machine model is optimized by using a low-rank matrix perturbation strategy to enhance its adaptability and stability when screening digital human image data, so that the screening model is more adaptable when facing diversified replica data, and the classification effect is improved, and finally the high quality and consistency of the replica data are guaranteed.

[0189] The embodiment of the present invention further proposes a digital human image reproduction system based on NeRF, based on the digital human image reproduction method based on NeRF as described above, comprising:

[0190] The digital human image acquisition module is used to acquire training data sets. Specifically, it uses a high-resolution camera and a specially configured lighting system to capture digital human images from multiple angles to obtain the three-dimensional geometric information and surface texture details of the target human.

[0191] A training sample expansion generation module, which is used to generate samples based on a generative adversarial network algorithm for quantum super-resolution synthesis, wherein the generator part of the generative adversarial network simulates high-dimensional feature mapping of digital human image data through quantum gate operations;

[0192] A digital human image data training data reconstruction module, which is used for an autoencoder algorithm based on high-dimensional matrix decomposition to learn and reproduce the high-dimensional geometry and surface features of the character from the expanded digital human image training data. The autoencoder structure has two parts: an encoder and a decoder. The encoder is responsible for compressing the input high-dimensional digital human image data into low-dimensional potential representations, and the decoder reconstructs the original input digital human image data from these potential representations;

[0193] The result feature extraction module of the digital human image replication is used to adopt the neural network algorithm based on dynamic population evolution optimization as the feature extraction model, and use the self-correction mechanism to automatically adjust the evolution rules according to the characteristics of the current digital human image training data, and adjust the probability of crossover and mutation according to the change trend of the loss function;

[0194] The digital human image data screening module is used to use the extreme learning machine algorithm based on a low-rank matrix as a digital human image data screening model to input the feature vector after feature extraction into the classifier model for digital human image data screening.

[0195] In addition, an embodiment of the present invention also proposes a computer-readable storage medium, on which program instructions of a digital human image replication method based on NeRF are stored. The program instructions of the digital human image replication method based on NeRF can be executed by one or more processors to implement the steps of the digital human image replication method based on NeRF as described above.

[0196] The embodiments described above are only descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for replicating a digital human image based on NeRF, characterized in that: include: S1: A training data set is obtained by collecting digital humanoid images from multiple angles using a high-resolution camera and a specially configured lighting system; specifically, a depth camera or 3D scanning device is used to obtain the three-dimensional geometric information and surface texture of the target humanoid; S2: Generate training samples based on the generative adversarial network algorithm of quantum super-resolution synthesis; S3: An autoencoder algorithm based on high-dimensional matrix decomposition uses the expanded digital humanoid image data training data to reconstruct the model; S4: A neural network algorithm based on dynamic population evolution optimization is used to extract features from the digital human image reproduction results output by the autoencoder; S5: The feature vector extracted by the extreme learning machine algorithm based on the low-rank matrix is ​​input into the classifier model to screen the digital human image data; Specifically, the training process of the generative adversarial network algorithm based on quantum super-resolution synthesis includes: The input of the generator uses the quantum initialization strategy when initialized, which is expressed as: ; In the formula, is the quantum state vector, is the number of quantum bits, For the The initialization phase of the qubits, is the quantum ground state; is an imaginary unit; Specifically, the training process of the autoencoder algorithm based on high-dimensional matrix decomposition includes: in each iteration, a small amount of random noise is added to the input digital human image data, expressed as: , ; In the formula, The digital human image data after adding noise. is the digital human image data before adding noise, It is the parameter update operation; is from a mean of 0 and a variance of Noise drawn from a normal distribution; For the The noise level of the iteration; The mean is 0 and the variance is Normal distribution of is the variance of the noise; is a normal distribution; The noise level adjustment strategy is used to dynamically adjust the noise mode, which is expressed as: ; In the formula, and In the The iteration and The noise level of the iteration; is the noise-adjusted learning rate; It is The rate of change of the reconstruction error at each iteration is used to guide the adaptive adjustment of the noise level.

2. The method for replicating a digital human image based on NeRF according to claim 1, characterized in that: The generative adversarial network algorithm based on quantum super-resolution synthesis includes the following steps: S21: Initialize the network parameters of the generator and discriminator of the generative adversarial network; S22: The training process includes multiple iteration cycles. In each cycle, the generator and the discriminator are alternately trained according to the adversarial strategy. The generator generates digital human image data samples, and the discriminator distinguishes the real samples from the generated samples. S23: In each training cycle, the generator and discriminator are optimized, and the parameters of the generator and discriminator are updated using the gradient descent method; S24: During the training of the generator, the quantum gates are dynamically adjusted to achieve high-dimensional feature transformation of the simulated data; S25: Repeat the above steps until the preset stop iteration condition is met, which means that the training of the generative adversarial network model is completed.

3. The method for replicating a digital human image based on NeRF according to claim 1, characterized in that: The autoencoder algorithm based on high-dimensional matrix decomposition comprises the following steps: S31: Initialize the network parameters of the autoencoder; S32: During the training process of the autoencoder, each input digital human image data is converted into a representation of the latent space through the encoder; S33: The encoded digital human image data is then fed into a decoder, which reconstructs the original input digital human image data from the compressed latent representation; S34: in each iteration, adding a small amount of random noise to the input digital human image data; S35: Update the parameters of the decoder through the back propagation algorithm; S36: Repeat the above steps until the preset stop iteration condition is met, indicating that the autoencoder model training is completed.

4. The method for replicating a digital human image based on NeRF according to claim 1, characterized in that: The neural network algorithm based on dynamic population evolution optimization includes the following steps: S41: According to the initialization method of the swarm bionic algorithm, in the initialization stage, an initial population is generated, and each individual represents a configuration of network weights; S42: for each individual in the population, use its corresponding neural network configuration to process the input training data of the digital human image, calculate the output of the model, and evaluate its performance according to a predetermined loss function; S43: According to the fitness of the individuals, the best performing individuals are selected from the current population and retained as candidate solutions for the next generation; S44: Generate new individuals through crossover and mutation operations. The crossover operation allows two excellent individuals to exchange some genes to generate new offspring. S45: Repeat the above steps until the preset stop iteration condition is met, which means that the neural network model training is completed.

5. The method for replicating a digital human image based on NeRF according to claim 1, characterized in that: The extreme learning machine algorithm based on low-rank matrix includes the following steps: S51: Initialize the parameters of the extreme learning machine; S52: before the extreme learning machine training, the weight matrix of the hidden layer is subjected to low-rank perturbation; S53: using the adjusted weights and biases, calculating the hidden layer output of the training data of the digital human image through a single forward propagation, and calculating the output weight; S54: Calculate the parameter statistical characteristics of the hidden layer neurons through the output of the hidden layer, which are used in the loss function calculation of the extreme learning machine to achieve parameter sparseness of the hidden layer neurons; S55: The classification performance of the extreme learning machine is evaluated by the loss function, and the model parameters are adjusted based on the feedback.

6. The method for replicating a digital human image based on NeRF according to claim 2, characterized in that: The calculation method of the loss function in the generative adversarial network is expressed as: , where Represents real digital human image data samples, Pseudo digital humanoid image data output by the generator, and are the outputs of the discriminator for real digital human image data and generated digital human image data respectively; represents the discriminator function; represents a generator function; represents the loss of the generative adversarial network; represents the loss function of the generative adversarial network; express expectations; represents the quantum state distribution of noise; Represents the distribution of real digital humanoid image data; Indicates compliance with a specific distribution; is the nonlinear loss term.

7. The method for replicating a digital human image based on NeRF according to claim 4, characterized in that: The crossover and mutation operations are expressed as: , , where is the crossover rate, is the weight of the neural network corresponding to the first individual selected, is the bias of the neural network corresponding to the first individual selected, is the weight of the neural network corresponding to the second individual selected, is the bias of the neural network corresponding to the second individual selected, is the weight of the neural network corresponding to the individual after the crossover operation, is the bias of the neural network corresponding to the individual after the crossover operation; Moreover, the mutation operation performs a small random perturbation on the newly generated individual weights, which can be expressed as: , , where represents the variance of the variation, is the weight of the neural network corresponding to the individual after the mutation operation, is the bias of the neural network corresponding to the individual after the mutation operation.

8. A NeRF-based digital human image reproduction system, based on the NeRF-based digital human image reproduction method according to any one of claims 1 to 7, comprising: A digital humanoid image acquisition module, which is used to acquire a training data set; Specifically, a high-resolution camera and a specially configured lighting system are used to capture digital humanoid images from multiple angles to obtain the three-dimensional geometric information and surface texture details of the target humanoid. A training sample expansion generation module, which is used to generate samples based on a generative adversarial network algorithm for quantum super-resolution synthesis, wherein the generator part of the generative adversarial network simulates high-dimensional feature mapping of digital human image data through quantum gate operations; A digital humanoid image data training data reconstruction module, which is used for an autoencoder algorithm based on high-dimensional matrix decomposition to learn and reproduce the high-dimensional geometry and surface features of the character from the expanded digital humanoid image training data, wherein the autoencoder structure has two parts: an encoder and a decoder, wherein the encoder is responsible for compressing the input high-dimensional digital humanoid image data into low-dimensional potential representations, and the decoder reconstructs the original input digital humanoid image data from these potential representations; The result feature extraction module of the digital humanoid image replication is used to adopt a neural network algorithm based on dynamic population evolution optimization as a feature extraction model, and to automatically adjust the evolution rules according to the characteristics of the current digital humanoid image training data using a self-correction mechanism, and to adjust the probability of crossover and mutation according to the change trend of the loss function; The digital human image data screening module is used to use the extreme learning machine algorithm based on a low-rank matrix as a digital human image data screening model to input the feature vector after feature extraction into the classifier model for digital human image data screening.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions of a digital human image replication method based on NeRF, and the program instructions of the digital human image replication method based on NeRF can be executed by one or more processors to implement the steps of the digital human image replication method based on NeRF as described in any one of claims 1 to 7.

Images