Image enhancement generation method based on face key point adaptive deformation

Abstract

The invention relates to the technical field of image enhancement, in particular to an image enhancement generation method based on face key point self-adaptive deformation. Comprising the steps of receiving a popular science museum face image data stream, performing topology reconstruction through a three-dimensional parameterized model, outputting a source face grid, mapping the source face grid to a UV parameter space, and decoupling and outputting an albedo texture tensor and a geometric texture tensor; calculating vertex displacement through a preset topology corresponding relation, constructing a differential homeomorphic deformation field, and constructing a continuous manifold coordinate field through nonlinear space adaptation; constructing an implicit neural texture representation network, performing overfitting training, inputting coordinate points of the continuous manifold coordinate field into the implicit neural texture representation network point by point, performing frequency modulation, calculating texture color values, and generating an enhanced facial image. According to the method, the continuous manifold coordinate field, the implicit neural texture representation network and the frequency modulation mechanism are deeply coupled, so that reconstruction from a low-resolution feature to a high-resolution visual signal is realized.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, specifically to an image enhancement generation method based on adaptive deformation of facial key points. Background Technology

[0002] In immersive virtual reality experiences (such as astronomical science museums or historical reconstruction scenes), real-time mapping of collected surface texture data onto a pre-defined 3D geometric model surface is an important current application of computer graphics rendering. Existing technologies generally employ texture synthesis methods based on 2D planar rasterization or discrete texture sampling. This involves extracting the region of interest in the texture and analyzing the color channel distribution, or calculating the pixel mean and gradient information of the texture map, to perform global or local resampling of the texture coordinates and color details of the surface material. This is currently the mainstream technique for improving the rendering quality of model surfaces.

[0003] However, existing image restoration and enhancement schemes have significant shortcomings in terms of structural consistency and high-frequency detail preservation. Their image optimization processing logic is primarily based on pixel statistics in a two-dimensional plane, failing to address image artifacts caused by the mismatch between edge features of the source image and structural features of the target region, leading to feature distortion in the synthesized image. Furthermore, existing technologies only perform linear or nonlinear transformations of color values ​​within a fixed texture coordinate system, unable to handle the drastic changes in texture sampling density caused by complex three-dimensional surface deformation. When nonlinear vertex displacement or mesh reconstruction is required on the geometric model surface, severe noise enhancement, edge blurring, and block effects occur. Additionally, during the immersive virtual character fusion process, the nonlinear spatial adaptation of the user's original facial geometric features with the target virtual character's skeleton can easily lead to insufficient resolution and a loss of geometric details.

[0004] To address this, an image enhancement and generation method based on adaptive deformation of facial key points is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide an image enhancement generation method based on adaptive deformation of facial key points to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an image enhancement and generation method based on adaptive deformation of facial key points, comprising: The system receives facial image data streams from the science museum and skeletal data of the virtual character. Based on the facial image data streams from the science museum, it performs topological reconstruction through a 3D parametric model and outputs the source facial mesh. The source facial mesh is then mapped to the UV parameter space, and the albedo texture tensor and geometric texture tensor are decoupled and output. Based on the virtual character's skeletal data and source facial mesh, vertex displacement is calculated through a preset topological correspondence to construct a differential homeomorphic deformation field; the differential homeomorphic deformation field is applied to the UV parameter space, and a continuous manifold coordinate field is constructed through nonlinear spatial adaptation. An implicit neural texture representation network is constructed, and overfitting training is performed based on the albedo texture tensor to fit the network weight parameters to a continuous space color function; based on the geometric texture tensor, a geometric detail embedding vector is output through Fourier feature transformation. The coordinates of the continuous manifold coordinate field are input point by point into the implicit neural texture representation network to generate a query index. Based on the query index, the geometric detail embedding vector is injected into the implicit neural texture representation network, frequency modulation is performed, texture color values ​​are calculated, and an enhanced facial image is generated.

[0007] Preferably, the specific process of outputting the source facial mesh includes: inputting the facial image data stream from the science museum into a depth parameter regression model, mapping it to a low-dimensional semantic manifold space for feature decoupling, and outputting an identity shape vector controlling the mesh geometry, an expression vector controlling facial micro-expressions, and six-degree-of-freedom pose parameters defining the head's spatial position; retrieving statistical basis data stored within the three-dimensional parameterized model, the statistical basis data including an average facial geometry template, orthogonal shape principal component basis tensors, and orthogonal deformation principal component basis tensors; using the identity shape vector and the expression vector as weight coefficients, performing a linear weighted recombination operation on the orthogonal shape principal component basis tensors and the orthogonal deformation principal component basis tensors, and superimposing the operation result onto the average facial geometry template to generate an instance topological mesh under standard pose; constructing a rigid body transformation matrix based on the six-degree-of-freedom pose parameters, performing rotation and translation operations on the instance topological mesh, aligning it with the pixel coordinate system of the facial image data stream from the science museum, and outputting the source facial mesh.

[0008] Preferably, the specific generation process of the albedo texture tensor and the geometric texture tensor includes: calling the UV topology layout data pre-placed in the three-dimensional parameterized model, establishing the index relationship between the three-dimensional vertex coordinates of the source face mesh and the two-dimensional texture coordinates in the UV parameter space, and calculating the barycentric coordinate interpolation coefficients within the triangular face patches in the source face mesh; based on the index relationship and the barycentric coordinate interpolation coefficients, performing reverse texture resampling on the facial image data stream of the science museum, projecting the RGB values ​​of the image pixels onto the UV parameter space, and generating a texture tensor containing illumination information and fine texture details. The original coupled texture tensor of the section is obtained; a preset intrinsic texture decomposition network is invoked to receive the original coupled texture tensor, calculate the global ambient lighting parameters through the built-in spherical harmonic illumination regression layer, and perform inverse division on the original coupled texture tensor to remove illumination using the global ambient lighting parameters, outputting an albedo texture tensor that preserves the inherent color of the skin; the pixel-level difference residual between the original coupled texture tensor and the albedo texture tensor is calculated, and high-pass filtering and gray-scale quantization are performed on the pixel-level difference residual to extract the geometric texture tensor that characterizes the micro-geometric features of the skin surface.

[0009] Preferably, the specific construction process of the differential homeomorphic deformation field includes: traversing the vertices of the source face mesh to generate vertex indices; extracting a sparse source feature control point set based on preset anatomical semantic labels; and parsing a sparse target bone key point set with topological isomorphism from the virtual character bone data; calculating the Euclidean space difference between the sparse source feature control point set and the sparse target bone key point set to generate a sparse displacement vector set representing the local deformation trend; constructing a spatial interpolation equation based on the radial basis kernel function; using the sparse displacement vector set as a weighted control term; performing regularized interpolation operations on all coordinate points in the UV parameter space to generate an initial dense deformation potential field; and performing Jacobian determinant constraint optimization on the initial dense deformation potential field to output the differential homeomorphic deformation field.

[0010] Preferably, the specific construction process of the continuous manifold coordinate field includes: performing discretization sampling on the UV parameter space to initialize and generate a regular coordinate grid tensor representing the physical position of the standard texture space; performing vector superposition operation on the differential homeomorphic deformation field and the regular coordinate grid tensor to calculate the spatial drift of each grid point under nonlinear deformation, generating a drift coordinate tensor containing geometric distortion information; performing boundary topology constraint verification on the drift coordinate tensor, using a truncation function to restrict out-of-bounds coordinates within a preset parameter domain; performing texture coordinate normalization mapping on the verified drift coordinate tensor, linearly scaling the numerical range to a preset activation interval of the implicit neural texture representation network, and outputting a continuous manifold coordinate field.

[0011] Preferably, the specific training process of the implicit neural texture representation network includes: initializing and constructing a fully connected perceptron topology, configuring a frequency domain position encoding input layer and a sinusoidal activation hidden layer with periodic response characteristics, and establishing a nonlinear mapping path from low-dimensional coordinate space to high-dimensional color space; performing discrete grid sampling on the albedo texture tensor, extracting the two-dimensional spatial coordinate index of the pixel and its corresponding RGB color ground truth value, and constructing a set of self-supervised training data pairs; performing forward propagation inference, mapping the two-dimensional spatial coordinate index to a high-dimensional harmonic feature tensor through the frequency domain position encoding input layer, and inputting it into the sinusoidal activation hidden layer, and regressing to output the predicted color signal; constructing a photometric reconstruction loss function, calculating the signal intensity difference between the predicted color signal and the RGB color ground truth value, and driving the network weight parameters to iteratively update in the direction of minimizing photometric error based on the backpropagation gradient descent algorithm until the network converges, thus solidifying the discrete pixel information of the albedo texture tensor into a continuous spatial color function.

[0012] Preferably, the specific generation process of the enhanced facial image includes: constructing a blank raster tensor matching the target display resolution; traversing the pixel indices of the blank raster tensor; indexing the corresponding two-dimensional floating-point coordinate values ​​in parallel from the continuous manifold coordinate field to form a query coordinate batch vector; processing the geometric detail embedding vector using a mapping perceptron; regressing and outputting frequency modulation parameters for each hidden layer of the implicit neural texture representation network, wherein the frequency modulation parameters include a feature scaling factor and a feature bias factor; inputting the query coordinate batch vector into the implicit neural texture representation network; during signal forward propagation, performing a channel-by-channel affine transformation modulation operation on the linear transformation output of each hidden layer using the feature scaling factor and the feature bias factor, and processing it through a sinusoidal activation function to output a sinusoidal activation feature tensor; performing color space synthesis mapping on the sinusoidal activation feature tensor to output texture color values; and filling the texture color values ​​with pixel buffers according to the spatial arrangement order of the pixel indices to generate the enhanced facial image.

[0013] Preferably, the specific construction of the implicit neural texture representation network includes: the frequency domain position encoding input layer is configured as a harmonic mapper, used to receive the two-dimensional coordinate vector input from the continuous manifold coordinate field, and project it onto a Fourier feature space of a preset dimension, outputting a high-dimensional harmonic feature tensor; the cascaded modulated sinusoidal sensing unit sequence is composed of several structurally isomorphic hidden layer units connected in series, constituting the sinusoidal activation hidden layer of the implicit neural texture representation network, each of the hidden layer units containing a linear transformation branch and a frequency modulation interface; the linear transformation branch performs a fully connected weighted multiplication operation on the input signal; the frequency modulation interface receives the scaling factor and bias factor transformed from the geometric detail embedding vector, performs an element-wise affine transformation on the output of the linear transformation branch, and inputs the transformation result into a periodic sinusoidal activation function, outputting a sinusoidal activation feature tensor; the color regression mapping layer receives the final-level features output by the cascaded modulated sinusoidal sensing unit sequence, maps the feature space to a three-channel RGB color space through color space synthesis mapping, and outputs texture color values.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. When dealing with low-resolution facial image data streams from science museums, instead of directly stretching discrete pixels, the UV parameter space is first discretized to generate a drift coordinate tensor containing geometric distortion information, which is then transformed into continuous manifold coordinates. This processing method maps discrete low-resolution coordinates into a continuous field with sub-pixel precision. This allows for parallel indexing of corresponding two-dimensional floating-point coordinate values ​​from the continuous manifold coordinate field when generating high-resolution images. The deformation mapping relationship remains within the continuous domain, reducing the mosaic effect caused by limited grid precision and laying a spatial coordinate foundation for improving the final output resolution.

[0015] 2. By constructing an implicit neural texture representation network and overfitting training based on the albedo texture tensor, the network weight parameters are fitted to a continuous spatial color function. This decouples image sharpness from the number of pixels, eliminating reliance on a limited number of pixels for information storage. Instead, a frequency-domain position encoding input layer maps low-dimensional coordinates to a high-dimensional space, driving the network to learn the mathematical distribution patterns of texture. By solidifying the discrete pixel information of the albedo texture tensor into a continuously parameterized function expression within the network, and then using frequency modulation of the geometric texture tensor, a resolution-independent storage format is achieved. During the generation stage, inference can be performed based on a blank raster tensor matching any high-specification target display resolution. Even with extremely low source image resolution, the network can infer smooth color transitions based on continuous functions, achieving better resolution reconstruction.

[0016] 3. By using a geometric texture tensor and Fourier feature transformation, a geometric detail embedding vector is output, and signal processing techniques are employed to specifically extract and enhance the frequency domain information representing high-resolution features. To address the lack of detail caused by low resolution, this application employs a frequency modulation strategy, specifically using the feature scaling factor and the feature bias factor to perform channel-by-channel affine transformation modulation on the sinusoidal activation feature maps of each hidden layer. This process is equivalent to performing super-resolution compensation within the feature space of the neural network, forcibly injecting high-frequency geometric features into the texture representation manifold as signal gain. In this way, when calculating texture color values, the implicit network can supplement the microscopic details lost in the low-resolution source image, enabling the generated enhanced facial image to possess the rich detail characteristic of high-resolution images in visual perception, thus solving the problems of image flatness and blurring caused by insufficient resolution.

[0017] 4. By deeply coupling a continuous manifold coordinate field, an implicit neural texture representation network, and a frequency modulation mechanism, an end-to-end rendering architecture independent of input resolution is constructed. Differential homeomorphic deformation fields are used to transform discrete low-resolution inputs into a continuous coordinate stream with sub-pixel accuracy, which is then used as an index input to a neural network fitted with a continuous parameterized function expression. During this process, arbitrary high-resolution targets defined by a blank raster tensor are frequency-modulated by geometric detail embedding vectors. This coupling mechanism enables the network to dynamically predict and synthesize pixel data that is not present in the source image within a continuous parameter space. Through this "discrete-continuous-rediscrete" overall mapping strategy, this application achieves the reconstruction from low-resolution features to high-resolution visual signals, ensuring that the generated enhanced facial images exhibit more realistic visual effects. Attached Figure Description

[0018] Figure 1 This is a flowchart of an image enhancement and generation method based on adaptive deformation of facial key points proposed in an embodiment of this invention application; Figure 2 This is a flowchart illustrating the geometric deformation and manifold space construction process proposed in an embodiment of this invention. Figure 3 This is a flowchart illustrating the neural texture representation and image enhancement generation process proposed in an embodiment of this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figures 1-3 The image enhancement and generation method based on adaptive deformation of facial key points provided by this invention has the following specific steps: The system receives facial image data streams from the science museum and skeletal data of the virtual character. Based on the facial image data streams from the science museum, it performs topological reconstruction through a 3D parametric model and outputs the source facial mesh. The source facial mesh is then mapped to the UV parameter space, and the albedo texture tensor and geometric texture tensor are decoupled and output. Based on the virtual character's skeletal data and source facial mesh, vertex displacement is calculated through a preset topological correspondence to construct a differential homeomorphic deformation field; the differential homeomorphic deformation field is applied to the UV parameter space, and a continuous manifold coordinate field is constructed through nonlinear spatial adaptation. An implicit neural texture representation network is constructed, and overfitting training is performed based on the albedo texture tensor to fit the network weight parameters to a continuous space color function; based on the geometric texture tensor, a geometric detail embedding vector is output through Fourier feature transformation. The coordinates of the continuous manifold coordinate field are input point by point into the implicit neural texture representation network to generate a query index. Based on the query index, the geometric detail embedding vector is injected into the implicit neural texture representation network, frequency modulation is performed, texture color values ​​are calculated, and an enhanced facial image is generated.

[0021] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.

[0022] Example 1

[0023] This application discloses an image enhancement and generation method based on adaptive deformation of facial key points, see embodiments. Figure 1 The specific steps proposed in this invention include: S1. Receiving facial image data streams from a science museum and skeletal data of a virtual character; performing topological reconstruction based on the facial image data streams from the science museum using a three-dimensional parametric model to output a source facial mesh; S2. Mapping the source facial mesh to the UV parameter space and decoupling the output albedo texture tensor and geometric texture tensor; S3. Calculating vertex displacements based on the virtual character skeletal data and the source facial mesh using a preset topological correspondence to construct a differential homeomorphic deformation field; applying the differential homeomorphic deformation field to the UV parameter space and constructing a continuous manifold coordinate field through nonlinear spatial adaptation; S4. Constructing an implicit neural texture representation network; performing overfit training based on the albedo texture tensor to fit the network weight parameters to a continuous spatial color function; outputting a geometric detail embedding vector based on the geometric texture tensor through Fourier feature transformation; S5. Inputting the coordinate points of the continuous manifold coordinate field point by point into the implicit neural texture representation network to generate a query index; injecting the geometric detail embedding vector into the implicit neural texture representation network based on the query index, performing frequency modulation, calculating texture color values, and generating an enhanced facial image.

[0024] Further, the system receives facial image data streams from the science museum and skeletal data of the virtual character. Based on the facial image data streams from the science museum, it performs topological reconstruction using a 3D parametric model and outputs the source facial mesh; this corresponds to step S1 above. The specific implementation process includes: The facial image data stream from the science museum is input into a deep parametric regression model, mapped to a low-dimensional semantic manifold space for feature decoupling, and outputs an identity shape vector controlling the mesh geometry, an expression vector controlling facial micro-expressions, and six-degree-of-freedom pose parameters defining the head's spatial position. The statistical basis data stored within the three-dimensional parametric model is retrieved, including an average facial geometry template, orthogonal shape principal component basis tensors, and orthogonal deformation principal component basis tensors. The identity shape vector and the expression vector are used as weighting coefficients to perform a linear weighted recombination operation on the orthogonal shape principal component basis tensors and the orthogonal deformation principal component basis tensors, and the result is superimposed on the average facial geometry template to generate an instance topological mesh under standard pose. A rigid body transformation matrix is ​​constructed based on the six-degree-of-freedom pose parameters, and rotation and translation operations are performed on the instance topological mesh to align it with the pixel coordinate system of the science museum's facial image data stream, outputting the source facial mesh.

[0025] Specifically, the facial image data stream from the science museum is input into a depth parameter regression model, mapped to a low-dimensional semantic manifold space for feature decoupling, and outputs an identity shape vector controlling the mesh geometry, an expression vector controlling facial micro-expressions, and six-degree-of-freedom pose parameters defining the head's spatial position. In this embodiment, the implementation environment is set in the digital human interactive exhibition area of ​​the science museum. The front-end acquisition device uses an Intel RealSense D435 depth camera, set to RGB mode with a resolution of 1920×1080 pixels. High-dimensional facial semantics are extracted from uncontrolled lighting and background. In the specific implementation, a face detection algorithm is first used to sample the video stream at 30 frames per second. The facial region is cropped using a five-point alignment method and scaled to a 224×224 pixel three-channel tensor using bicubic interpolation. The pixel values ​​are normalized to a floating-point range of 0 to 1 as network input. The deep parametric regression model uses a ResNet-50 pre-trained on the ImageNet dataset as its backbone network and performs fine-tuning training on a face dataset with 3DMM parameter annotations. The original fully connected classification layer is removed and replaced with a regression layer with an output dimension of 257. During the model's forward inference, the input tensor sequentially passes through convolutional layers, pooling layers, and four residual modules for feature extraction, ultimately mapping to a low-dimensional semantic manifold space. The output 257-dimensional vector undergoes a slicing and decoupling operation: dimensions 1 to 80 are truncated as identity shape vectors controlling the mesh geometry; these vectors follow a standard normal distribution and primarily encode rigid anatomical features such as face width and nose bridge height. Dimensions 81 to 144 are truncated as expression vectors controlling facial micro-expressions, corresponding to the intensity of action units in the facial motion coding system. Dimensions 145 to 224 are truncated as coarse texture coefficients, and dimensions 225 to 251 are truncated as spherical harmonic illumination coefficients to aid texture decomposition. Dimensions 252 to 257 are truncated as six-DOF pose parameters defining the head's spatial position, including three radian Euler angle parameters and three millimeter-level spatial translation parameters. The depth parametric regression model employs a supervised training mechanism with a hybrid loss function. Specifically, a total loss function is constructed, which is a weighted average of keypoint reprojection loss, photometric reconstruction loss, and parameter regularization loss. To balance geometric alignment accuracy and texture detail reproduction, the specific values ​​of the weight coefficients are set as follows: keypoint weight is set to 1.6, photometric reconstruction weight is set to 1.92, and regularization weight is set to 2.0 × 10⁻⁶. -4The keypoint reprojection loss constrains geometric alignment accuracy: using the shape vector, expression vector, and pose parameters predicted by the network, the vertices of the 3D mesh are projected onto the 2D image plane through weak perspective projection transformation. The weighted Euclidean distance between the coordinates of the 68 feature points obtained from the projection and the coordinates of the real keypoints in the training dataset is calculated. The photometric reconstruction loss constrains the consistency of texture and lighting: based on the predicted albedo coefficient and spherical harmonic illumination coefficient, a reconstructed face image is generated using a differentiable rendering layer. The difference between the reconstructed image and the original input image at the pixel level L2 norm is calculated, forcing the network to learn the correct texture distribution and lighting environment. The parameter regularization loss prevents ill-conditioned values ​​in shape and expression parameters: L2 regularization constraints are applied to the identity shape vector and expression vector to ensure that the predicted parameter distribution conforms to the standard normal distribution prior of the 3DMM statistical model. A small weight coefficient is set here to prevent excessive regularization from smoothing out facial expression details (such as slightly closed eyes or a slight smile). During the training phase, a public face dataset containing 300W-LP and AFLW2000-3D (which provides images and their corresponding ground truth 3DMM parameters) was used, and the Adam optimizer was employed at 10... -4 The initial learning rate is minimized to minimize the total loss function until the network converges, thus completing the parameter fine-tuning.

[0026] In this embodiment, a temporal feature aggregation unit is introduced at the model output to establish a sliding window to cache the semantic feature vectors of historical frames. Attention weights are assigned by calculating the cosine similarity between the current frame and historical frames, vector weighted smoothing is performed, and stable parameters fused with temporal information are output. In interactive scenarios in science museums, ambient light flicker often causes numerical jumps in the geometric parameters of single-frame predictions. Instead of directly outputting the single-frame prediction results of the regression model, the 257-dimensional feature vectors of the most recent 5 frames are stored in a FIFO queue. A lightweight attention mechanism is introduced to calculate the Euclidean distance between the current frame vector and each vector in the queue in real time, generating a normalized confidence score. Specifically, a soft weighting mechanism based on a Gaussian kernel function is adopted: first, the Euclidean distance between the current frame feature vector and each historical frame vector in the queue is calculated; then, a Gaussian function is used to map this distance into non-negative weights, so that the weights of frames with larger distances decay exponentially, thereby quickly suppressing the influence of abnormally jumping frames; finally, the weights of all frames in the queue are normalized so that their sum is 1, and a weighted average is performed accordingly. By using this time-weighted summation, a smooth feature trajectory is reconstructed, and the smoothed vector is sliced ​​and decoupled. This process effectively filters out random disturbances, ensuring that the generated source facial mesh is consistent in the time dimension, without any sudden changes in facial size or subtle tremors in facial features. It also eliminates inter-frame geometric jitter in the video stream, guaranteeing the clarity of the geometric structure of the enhanced image.

[0027] Specifically, the statistical basis data stored within the 3D parametric model is retrieved. This statistical basis data includes an average facial geometry template, orthogonal shape principal component basis tensors, and orthogonal deformation principal component basis tensors. Using the identity shape vector and the expression vector as weighting coefficients, a linear weighted recombination operation is performed on the orthogonal shape principal component basis tensors and the orthogonal deformation principal component basis tensors. The result is then superimposed onto the average facial geometry template to generate an instance topology mesh under standard pose. In this embodiment, this step relies on a pre-built Basel Face within the 3D parametric model. The Model Clipping Database, serving as statistical basis data, statically resides three core tensors: First, the average facial geometry template, containing coordinate data of N 3D vertices, where N is 35709, representing a neutral face topology unaffected by any identity or expression; second, the orthogonal shape principal component basis tensor, with dimensions of 35709×3×80, storing geometric variation principal components extracted from thousands of scanned subjects; and third, the orthogonal deformation principal component basis tensor, with dimensions of 35709×3×64, built based on the FaceWarehouse dataset, covering linear variation basis for common expressions such as opening the mouth, closing the eyes, and puffing out the cheeks. In the specific operation of linear weighted reorganization, the aforementioned 80-dimensional identity shape vector is first treated as a set of weight coefficients. A tensor product operation is then performed on the third dimension with the orthogonal shape principal component basis tensor. Specifically, each of the 80 scalar coefficients is multiplied by its corresponding 35709×3 basis matrix, and the resulting 80 matrices are summed element-wise to generate an identity geometric offset representing the tourist's unique facial features. Similarly, a weighted sum is performed on the 64-dimensional expression vector and the orthogonal deformable principal component basis tensor to generate an expression geometric offset representing the current instantaneous expression. Finally, the average facial geometric template, identity geometric offset, and expression geometric offset are vector-wise summed at their corresponding vertex indices to generate an instance topological mesh in the standard model coordinate system. Taking the capture of a tourist's "surprised" expression as an example, the coefficients controlling the downward movement of the mandible and the upward movement of the brow bone in the expression vector significantly increase. This drives the offset generated by the corresponding basis, causing the vertex of the mouth region in the mesh to shift downwards by approximately 15 millimeters, thus geometrically realizing the mouth-opening action.

[0028] Specifically, a rigid body transformation matrix is ​​constructed based on the six-degree-of-freedom attitude parameters. Rotation and translation operations are performed on the instance topology mesh, aligned with the pixel coordinate system of the science museum's facial image data stream, and the source facial mesh is output. In this embodiment, the three Euler angle components of the six-degree-of-freedom attitude parameters, namely pitch, yaw, and roll, are read and converted into a 3×3 rigid body transformation matrix using the Rodrigues rotation formula. Specifically, the magnitude of the rotation vector is first calculated as the rotation angle. The cosine value is multiplied by the identity matrix, and the sine value is multiplied by the antisymmetric matrix of the rotation axis. Then, a product matrix of the minus cosine value and the rotation axis is added to construct the rotation transformation operator. Simultaneously, the three translation parameters are directly constructed into a 3×1 spatial displacement vector. Next, an affine transformation is performed on each of the 35709 vertices in the aforementioned instance topology mesh, that is, the rigid body transformation matrix is ​​multiplied by the vertex coordinate vector on the left and then added to the spatial displacement vector. This operation rotates and translates the face mesh, originally located at the origin of the model coordinate system, to a spatial pose completely consistent with the face in the image captured at the science museum. To verify the alignment accuracy, the transformed 3D vertices are projected onto the 2D image plane using a weak perspective projection formula. The average pixel Euclidean distance between the projected points and 68 key points in the image is calculated. When this distance is less than 1.5 pixels, the alignment is considered successful. The final output is the source face mesh after rigid body transformation, specifically a 3D coordinate dataset containing 35,709 vertices.

[0029] By combining deep parametric regression with statistical basis models, a geometric topological foundation is provided for subsequent image enhancement. Accurate geometric alignment is a prerequisite when dealing with resolution and detail issues. By using identity, expression, and pose vectors to linearly weight and reconstruct orthogonal principal component basis vectors, the 3D geometric structure of the user's face can be reconstructed in a low-dimensional manifold space. This statistical prior-based reconstruction method not only filters out noise interference in the input data stream but also ensures strict alignment between the source facial mesh and the input image in the pixel-level coordinate system. This ensures the accuracy of subsequent texture sampling positions and avoids texture sampling deviations and detail smearing caused by geometric model misalignment, thus guaranteeing the geometric clarity of the enhanced image from the source and reducing artifact generation.

[0030] Furthermore, the source face mesh is mapped to the UV parameter space, decoupling the output albedo texture tensor and geometric texture tensor; corresponding to step S2 above; the specific implementation process includes: The system calls the UV topology layout data pre-installed in the 3D parametric model to establish an index relationship between the 3D vertex coordinates of the source face mesh and the 2D texture coordinates in the UV parameter space, and calculates the barycentric coordinate interpolation coefficients within the triangular face patches of the source face mesh. Based on the index relationship and the barycentric coordinate interpolation coefficients, it performs inverse texture resampling on the facial image data stream of the science museum, projecting the RGB values ​​of the image pixels onto the UV parameter space to generate an original coupled texture tensor containing illumination information and texture details. The system calls a pre-installed intrinsic texture decomposition network to receive the original coupled texture tensor, calculates global ambient illumination parameters through a built-in spherical harmonic illumination regression layer, and performs inverse division to remove illumination from the original coupled texture tensor using the global ambient illumination parameters, outputting an albedo texture tensor that preserves the inherent color of the skin. The system calculates the pixel-level difference residual between the original coupled texture tensor and the albedo texture tensor, and performs high-pass filtering and gray-scale quantization on the pixel-level difference residual to extract a geometric texture tensor that characterizes the microscopic geometric features of the skin surface.

[0031] Specifically, the UV topology layout data pre-installed in the 3D parametric model is invoked to establish an index relationship between the 3D vertex coordinates of the source face mesh and the 2D texture coordinates of the UV parameter space, and the interpolation coefficients of the centroid coordinates within the triangular face patches in the source face mesh are calculated. Based on the index relationship and the centroid coordinate interpolation coefficients, reverse texture resampling is performed on the facial image data stream of the science museum, projecting the RGB values ​​of the image pixels onto the UV parameter space to generate an original coupled texture tensor containing lighting information and texture details. In this embodiment, the predefined UV topology layout data of the 3D parametric model is first loaded, including a UV mapping table, which maps 35709 3D vertices one-to-one to a normalized 2D plane coordinate system and establishes an index relationship; and a floating-point UV texture buffer with a resolution of 1024×1024 pixels and a bit depth of 32 bits. For each pixel coordinate within the buffer, an inverse rasterization lookup is performed: first, the index of the triangular facet in the UV-plane unfolding is retrieved; then, based on the pixel's center coordinates and the UV coordinates of the three vertices of the triangular facet, the centroid coordinate interpolation coefficients, denoted as α, β, and γ, are calculated, with their sum strictly equal to one. Using these coefficients, the coordinates of the three 3D vertices of the corresponding triangular facet in the source face mesh are weighted and summed to obtain the 3D surface point coordinates corresponding to the texture pixel. Next, using the camera intrinsic matrix and rigid body transformation matrix, the 3D surface point is projected back to the original 1920×1080 resolution image plane of the science museum, obtaining non-integer sampling coordinates. At these coordinates, a bilinear interpolation algorithm is used to read the RGB color value, and this color value is filled into the corresponding position in the UV texture buffer. After traversing all 1,048,576 texture pixels, the original coupled texture tensor is generated. This tensor, while preserving the original color of the facial skin, also fully records the highlights, shadows, and environmental reflections caused by the complex lighting conditions.

[0032] Specifically, a pre-built intrinsic texture decomposition network is invoked to receive the original coupled texture tensor. Global ambient lighting parameters are calculated using a built-in spherical harmonic illumination regression layer. These global ambient lighting parameters are then used to perform an inverse division operation on the original coupled texture tensor to remove illumination, outputting an albedo texture tensor that preserves the skin's inherent color. In this embodiment, the core of this step is to extract the illumination component from the coupled texture. A convolutional neural network improved with a deep convolutional encoder architecture (such as the ResNet encoder) is used as the intrinsic texture decomposition network, with the aforementioned 1024×1024×3 original coupled texture tensor as its input. To ensure the network can accurately regress the illumination parameters, supervised training is performed during the training phase using a synthetic face dataset containing ground truth illumination values ​​(such as SfSNet data). The network weights are optimized by minimizing the L2 distance between the predicted coefficients and the ground truth coefficients. The network bottleneck layer is connected to a spherical harmonic illumination regression layer, which consists of a global average pooling layer and a fully connected layer. The output is a 27-dimensional vector corresponding to the coefficients of the second-order spherical harmonic illumination model (9 coefficients for each RGB channel). An illumination reconstruction equation is constructed based on the Lambertian diffuse reflection assumption: Using the pre-computed normal map of the source face mesh, the normal vector corresponding to each pixel is substituted into nine spherical harmonic basis functions, and then linearly weighted and summed with 27 predicted coefficients to synthesize an irradiance map with the same resolution as the texture tensor, i.e., the global ambient lighting parameters. Subsequently, an inverse division illumination removal operation is performed: the RGB values ​​of the original coupled texture tensor are divided pixel by pixel by the corresponding values ​​of the global ambient lighting parameters. To prevent division by zero anomalies and numerical divergence, a smoothing constant of 1 multiplied by 10 to the power of -6 is added to the denominator. The output of the operation is the albedo texture tensor.

[0033] Specifically, the pixel-level difference residual between the original coupled texture tensor and the albedo texture tensor is calculated, and high-pass filtering and gray-quantization are performed on the pixel-level difference residual to extract the geometric texture tensor representing the micro-geometric features of the skin surface. In this embodiment, the pixel-level difference residual between the original coupled texture tensor and the albedo texture tensor in the logarithmic domain is first calculated to obtain an initial residual map containing high-frequency light and shadow variations (such as wrinkle shadows and micro-shadows caused by pore depressions) and specular reflection components. Since the micro-geometric features of the skin are mainly concentrated in the high-frequency band, Gaussian difference filtering is performed on the initial residual map: the standard deviation of the first Gaussian kernel is set to 1.0 pixel, and the standard deviation of the second Gaussian kernel is set to 3.0 pixels. The difference between the two blurred images is calculated to filter out low-frequency facial structural light and shadow variations, retaining only texture details in the frequency range of 0.5 to 5 cycles per millimeter. Finally, gray-quantization is performed on the filtering result to linearly map the numerical range to an 8-bit integer range of 0 to 255, and the single-channel intensity value is taken to generate the geometric texture tensor. This tensor encodes the surface texture information of the skin in the form of gray values. Although the tensor may contain imaging noise due to the limitations of the input image resolution, it encodes key light and shadow distribution features of the skin surface, such as dark gray lines appearing at crow's feet.

[0034] In the nonlinear adaptation process, if the lighting and shadows of the original image are not removed, they will produce incorrect visual cues due to deformation, obscuring geometric details. By using spherical harmonic illumination regression to remove the influence of ambient light, a pure albedo texture is extracted, ensuring that the stored material information is the inherent material information of the object. More importantly, by calculating the residual and performing high-pass filtering, the geometric texture tensor representing the micro-bumps and depressions of the skin is specifically extracted. This strategy of processing "color" and "high-frequency geometric details" in layers allows for the separate encoding and preservation of easily lost geometric details, preventing them from being smoothed out as noise during subsequent neural network overfitting, thus improving the texture and detail richness of the generated image.

[0035] Furthermore, based on the virtual character's skeletal data and the source facial mesh, vertex displacements are calculated through a preset topological correspondence to construct a differential homeomorphic deformation field; corresponding to step S3 above; the specific implementation process includes: The vertices of the source facial mesh are traversed to generate vertex indices. A sparse source feature control point set is extracted based on preset anatomical semantic labels. A sparse target bone key point set with topological isomorphism is parsed from the virtual character bone data. The Euclidean space difference between the sparse source feature control point set and the sparse target bone key point set is calculated to generate a sparse displacement vector set representing the local deformation trend. A spatial interpolation equation based on the radial basis kernel function is constructed. Using the sparse displacement vector set as a weighted control term, regularized interpolation is performed on all coordinate points in the UV parameter space to generate an initial dense deformation potential field. Jacobi determinant constraint optimization is performed on the initial dense deformation potential field to output a differential homeomorphic deformation field.

[0036] Specifically, the vertices of the source facial mesh are traversed to generate vertex indices. A sparse source feature control point set is extracted based on preset anatomical semantic labels. A sparse target bone key point set with topological isomorphism is then parsed from the virtual character skeleton data. In this embodiment, semantic traversal is performed on the source facial mesh. Based on preset anatomical semantic labels (defined by the facial motion coding system), 68 highly saliency feature vertex indices are automatically locked, such as the left corner of the mouth vertex (index number 8156) and the nose tip vertex (index number 2045), thereby extracting the sparse source feature control point set. Simultaneously, virtual character skeleton data is received. This skeleton data follows standard skeleton specifications and contains a total of 135 body bones and fine facial bones. Through a preset "bone-mesh" skinning weight mapping table, a sparse target bone key point set with topological isomorphism to the aforementioned 68 source feature points is parsed. For example, the left corner of the eye vertex of the source mesh is mapped to the origin of the local coordinate system in the virtual character skeleton, where the eye bone is located. Specifically, the "bone-mesh" skinning weight mapping table is stored using a key-value pair structure, where the key is the feature vertex index of the source facial mesh, and the value is the name and binding weight of the corresponding virtual character's master bone. For example, the mapping table stores: vertex with index number 8156 corresponds to the semantics of the left corner of the mouth, which is mapped to the bone "J_Bip_C_Mouth_L" with a binding weight of 1.0; vertex with index number 2045 corresponds to the semantics of the tip of the nose, which is mapped to the bone "J_Bip_C_Nose" with a binding weight of 1.0; vertex with index number 1024 corresponds to the semantics of the chin, which, although affected by the neck bones, is mapped to the master bone "J_Bip_C_Jaw" with the largest weight in this table, with a recorded weight of 0.8. In this embodiment, for vertices affected by multiple bones (such as index 1024), the bone with the largest weight value (i.e., J_Bip_C_Jaw) is selected as the positioning reference for the target key point, and its local coordinate origin is used as the target position.

[0037] Specifically, the Euclidean space difference between the sparse source feature control point set and the sparse target bone key point set is calculated to generate a sparse displacement vector set representing the local deformation trend. A spatial interpolation equation based on the radial basis function kernel is constructed, and the sparse displacement vector set is used as a weighted control term to perform regularized interpolation operations on all coordinate points in the UV parameter space to generate an initial dense deformation potential field. In this embodiment, the core of this step is to construct a smooth and elastic deformation transfer mechanism. First, in the normalized UV texture parameter space (with values ​​ranging from 0 to 1), the two-dimensional Euclidean space difference between each source feature control point and the corresponding target bone key point mapping coordinates is calculated to generate a sparse displacement vector set containing 68 two-dimensional vectors. Subsequently, a spatial interpolation equation based on the radial basis function is constructed. In this embodiment, a Gaussian kernel function with infinite-order smoothness is selected as the basis, and its mathematical expression is: the basis function value is equal to the negative exponent of the natural logarithm base e, where the exponent is the shape parameter ε multiplied by the square of the Euclidean distance from the control point to the interpolation point. The shape parameter ε is set to 0.5 to control the local attenuation range of deformation effects. A 68×68 linear equation system is established, using the sparse displacement vector set as known constraints, to solve for the coefficients of 68 weighted control terms. Using the solved weight coefficients, regularized interpolation is performed on all 1024×1024 pixel coordinates in the UV parameter space: for any pixel coordinate, the distance to all 68 control points is calculated, substituted into a Gaussian kernel function to obtain the response value, and then summed with the weight coefficients to calculate the estimated displacement of that pixel. After traversing the UV parameter space, an initial dense deformation potential field is generated. This field is stored in tensor form with dimensions of 1024×1024×2, recording the initial mapping relationship from the real-person texture space to the virtual character texture space.

[0038] Specifically, Jacobian determinant constraint optimization is performed on the initial dense deformation potential field to output a differential homeomorphic deformation field. In this embodiment, this step aims to eliminate texture folding and tearing that may occur during deformation (i.e., ensuring the topological homeomorphism of the mapping). The first-order partial derivatives of the initial dense deformation potential field in the U and V directions are calculated to construct a 2×2 Jacobian matrix at each pixel. The condition for differential homeomorphism requires that the determinant value of this Jacobian matrix must always be greater than zero. The Jacobian determinant constraint optimization process includes: traversing the Jacobian determinant values ​​of all pixels and detecting whether there are singular regions less than or equal to a threshold (set to 0.1). If a singular point is detected (e.g., excessive compression of the upper and lower lip textures during a mouth-closing action, resulting in a negative determinant), an iterative optimization process based on the L-BFGS algorithm is initiated: the total energy function is defined as a weighted sum of the data fitting term and the topological regularization term. Specifically, the data fitting term calculates the sum of the squares of the Euclidean distances between the displacement vector of the current mesh vertex and the displacement vector generated by the initial interpolation, used to constrain the deformation field from deviating from the control range of the original feature points. The topological regularization term employs a truncated square penalty mechanism, that is, for each sampling point in the deformation field, when its Jacobian determinant value is less than a preset threshold (e.g., 0.1), the square of the difference between the determinant value and the threshold is calculated as a penalty value, while no penalty is calculated for regions greater than or equal to the threshold. The gradient of the energy function with respect to the pixel displacement is calculated through backpropagation, guiding the deformation potential field to be updated along the gradient descent direction, and a temporal hot-start strategy is adopted, that is, the deformation field calculated in the previous frame is used as the initial estimate of the current frame, until the Jacobian determinant value of all pixels in the entire field is strictly greater than 0.1. After about 3 to 5 iterations of convergence, the final differential homeomorphic deformation field is output. This field ensures that when mapping the aged skin texture of the tourist to the young skeletal structure of the virtual character, the facial texture remains continuous, smooth, and without overlap, even under extreme facial stretching.

[0039] In this embodiment, an anatomically rigid constraint mask is introduced. Based on the distribution of facial bones and muscles, differentiated regularization weights are set for different regions. High penalties are applied to rigid regions such as the bridge of the nose, while constraints are reduced for flexible regions such as the cheeks, guiding the optimization process to conform to biomechanical characteristics. A grayscale rigid mask image aligned with the UV layout is preloaded, where the grayscale value for the nasal and brow bone regions is 1 (high rigidity), and the grayscale value for the cheek and mouth regions is 0.2 (high elasticity). During Jacobian determinant constraint optimization, this mask is multiplied into the energy function as a penalty term coefficient. When the algorithm detects non-uniform stretching of the mesh in the bridge of the nose region, a large loss value is generated, forcing the deformation field to maintain a rigid transformation in that region; while in the corners of the mouth region, larger nonlinear distortions are allowed to match exaggerated expressions. This anatomically based differentiated constraint simulates the skeletal support under the skin, ensuring that the final mapping maintains topological smoothness while preserving reasonable proportions of the facial features, preventing unnatural distortions in rigid facial regions.

[0040] By constructing a differential homeomorphic deformation field based on radial basis kernel functions and Jacobian constraints, the topological continuity of textures during spatial adaptation is guaranteed, avoiding tearing and distortion of details. When mapping user faces to virtual skeletons, drastic nonlinear deformations can easily destroy the local structure of the image. By using sparse keypoints to drive the generation of a dense potential field and applying Jacobian determinant constraints, the deformation mapping is ensured to be reversible and smooth (i.e., differential homeomorphic). This allows the relative neighborhood relationships between pixels to be maintained in areas of texture stretching or compression, preventing patch flipping or texture overlap. This mathematical smoothness constraint provides a stable spatial reference for the construction of continuous manifold coordinate fields, ensuring that texture details maintain a natural transition even after significant deformation, avoiding the introduction of artificial artifacts.

[0041] Furthermore, the differential homeomorphic deformation field is applied to the UV parameter space, and a continuous manifold coordinate field is constructed through nonlinear spatial adaptation; this corresponds to step S3 above; see [link to relevant documentation]. Figure 2 The specific implementation process includes: Discretization sampling is performed on the UV parameter space to initialize and generate a regular coordinate grid tensor representing the physical position of the standard texture space; the differential homeomorphic deformation field and the regular coordinate grid tensor are subjected to vector superposition operation to calculate the spatial drift of each grid point under nonlinear deformation, generating a drift coordinate tensor containing geometric distortion information; boundary topological constraint verification is performed on the drift coordinate tensor, and the out-of-bounds coordinates are restricted to the preset parameter domain using a truncation function; texture coordinate normalization mapping is performed on the verified drift coordinate tensor to linearly scale the numerical range to the preset activation interval of the implicit neural texture representation network, and a continuous manifold coordinate field is output.

[0042] Specifically, discretization sampling is performed on the UV parameter space to initialize and generate a regular coordinate grid tensor representing the physical location in the standard texture space. In this embodiment, this step aims to construct a physical reference coordinate system for implicit neural texture queries. First, in a tensor space with dimensions of 1×1024×1024×2, a grid generation function is called to perform discretization sampling, generating equally spaced floating-point number sequences from zero to one on the horizontal U-axis and vertical V-axis, respectively. The sampling step size is set to 1 divided by 1024, which is approximately 0.0009765625. By combining these two orthogonal sequences through a Cartesian product, a regular coordinate grid tensor is initialized and generated. Each element in this tensor represents an absolute physical anchor point on the facial texture map in the standard state without any changes in appearance. For example, the coordinate point (0.5, 0.5) corresponds to the texture center of the nose tip region, while (0.2, 0.3) corresponds to a specific skin region on the right cheek.

[0043] Specifically, the differential homeomorphic deformation field and the regular coordinate grid tensor are subjected to vector superposition to calculate the spatial drift of each grid point under nonlinear deformation, generating a drift coordinate tensor containing geometric distortion information. In this embodiment, the differential homeomorphic deformation field is received. This deformation field is essentially a displacement vector map of the same resolution, recording the offset of each point in the UV space. Parallel vector addition is performed, adding the regular coordinate grid tensor and the differential homeomorphic deformation field element by element. This operation simulates elastic deformation in the physical world: when a tourist makes a laughing expression, causing the muscles at the corners of the mouth to stretch outward and upward, the texture coordinates of the corners of the mouth, originally located at (0.6, 0.6), may be added with a spatial drift of (0.02, -0.01), drifting to (0.62, 0.59). The generated drift coordinate tensor thus encodes the nonlinear geometric distortion information of the facial skin at the current moment.

[0044] Specifically, boundary topology constraint verification is performed on the drift coordinate tensor. A truncation function is used to restrict out-of-bounds coordinates to within a preset parameter domain. In this embodiment, this step aims to eliminate the risk of sampling overflow caused by numerical calculation errors or extreme deformations. A hardware-accelerated truncation function is invoked to perform boundary topology constraint verification on all values ​​in the drift coordinate tensor: any coordinate value less than zero is corrected to zero, and any coordinate value greater than one is corrected to one. This ensures that all subsequent coordinate query requests input into the neural network fall within the valid texture parameter space, preventing rendering black spots or random noise artifacts caused by index out-of-bounds errors.

[0045] Specifically, a texture coordinate normalization mapping is performed on the verified drift coordinate tensor, linearly scaling the numerical range to a preset activation interval of the implicit neural texture representation network, outputting a continuous manifold coordinate field. In this embodiment, this step is to adapt to the input sensitivity characteristics of the implicit neural texture representation network. A linear scaling transformation is performed on the truncated drift coordinate tensor: all coordinate values ​​are multiplied by two and then subtracted by one. Through this affine transformation, the original texture coordinates within the range are remapped to the preset activation interval ([-1, 1] interval), typically defined within the centered interval [-1, 1] to obtain the best gradient propagation efficiency. For example, the original center point (0.5, 0.5) is mapped to (0, 0), while the top left corner (0, 0) becomes (-1, -1). The final output is defined as a continuous manifold coordinate field.

[0046] By constructing a continuous manifold coordinate field and performing normalized mapping, efficient adaptation between the physical deformation space and the neural network input space is achieved. To address the insufficient resolution, the continuous query capability of the neural network must be relied upon, and the continuous manifold coordinate field acts as a bridge connecting discrete mesh deformation and continuous function query. Spatial drift is calculated through vector superposition operations, and complex nonlinear deformations are transformed into standardized coordinate signals using truncation functions and normalization. This ensures that every coordinate point input into the implicit network is valid and uniformly distributed, avoiding neural network inference distortion caused by out-of-bounds coordinate values ​​or sparse distribution, allowing the network to focus on fitting high-frequency texture functions.

[0047] Furthermore, an implicit neural texture representation network is constructed, and overfitting training is performed based on the albedo texture tensor to fit the network weight parameters to a continuous space color function; based on the geometric texture tensor, a geometric detail embedding vector is output through Fourier feature transformation; corresponding to step S4 above; the specific implementation process includes: A fully connected perceptron topology is initialized, and a frequency-domain position encoding input layer and a sinusoidal activation hidden layer with periodic response characteristics are configured to establish a nonlinear mapping path from low-dimensional coordinate space to high-dimensional color space. Discrete grid sampling is performed on the albedo texture tensor to extract the two-dimensional spatial coordinate index of the pixel and its corresponding RGB color ground truth value, and a self-supervised training data pair set is constructed. Forward propagation inference is performed to map the two-dimensional spatial coordinate index to a high-dimensional harmonic feature tensor through the frequency-domain position encoding input layer, and input it into the sinusoidal activation hidden layer to regress and output the predicted color signal. A photometric reconstruction loss function is constructed to calculate the signal intensity difference between the predicted color signal and the RGB color ground truth value, and based on the backpropagation gradient descent algorithm, the network weight parameters are driven to iteratively update in the direction of minimizing photometric error until the network converges, thus solidifying the discrete pixel information of the albedo texture tensor into a continuous spatial color function.

[0048] Specifically, a fully connected perceptron topology is initialized, a frequency-domain position encoding input layer and a sinusoidal activation hidden layer with periodic response characteristics are configured, and a nonlinear mapping path from low-dimensional coordinate space to high-dimensional color space is established. Discrete grid sampling is performed on the albedo texture tensor to extract the two-dimensional spatial coordinate index of the pixel and its corresponding RGB color ground truth value, and a self-supervised training data pair set is constructed. Forward propagation inference is performed to map the two-dimensional spatial coordinate index to a high-dimensional harmonic feature tensor through the frequency-domain position encoding input layer, and input it into the sinusoidal activation hidden layer to regress and output the predicted color signal. Based on the geometric texture tensor, a geometric detail embedding vector is output through Fourier feature transformation. In this embodiment, an implicit neural texture representation network architecture based on a multilayer perceptron is constructed. The network input layer is configured as a frequency-domain position encoding input layer, which receives the two-dimensional coordinate vector of the continuous manifold coordinate field and projects it to the high-dimensional harmonic space through a Fourier feature mapping composed of 256 sets of sine and cosine functions to alleviate the "spectral bias" problem of the neural network and capture high-frequency texture changes. The backbone network contains five fully connected hidden layers, also known as sinusoidal activation hidden layers, each with 512 neurons, using a periodic sinusoidal function as the activation function. To ensure effective gradient propagation of the sinusoidal activation function in deep networks, the network weight parameters are initialized using a Sitzmann uniform distribution scheme. For each hidden layer, the elements in its weight matrix range from the interval [- , Uniform sampling is performed within the layer, where n=512 is the number of input neurons. The frequency factor of the first hidden layer is set to 30 to stimulate the network's ability to respond to subtle texture structures, and subsequent layers are set to 1. Simultaneously, the geometric texture tensor (containing microscopic grayscale information such as wrinkles and pores) is processed: based on the geometric texture tensor, a two-dimensional discrete Fourier transform (DFT) is performed to extract its amplitude spectrum. Specifically, the geometric texture tensor (1024×1024) is subjected to a two-dimensional DFT to obtain a frequency domain feature matrix. This frequency domain feature matrix is ​​then centered, divided into frequency intervals by concentric rings, and the mean and variance of the spectral energy within each frequency band are calculated to construct a one-dimensional spectral feature sequence; this sequence is then mapped and compressed into a 512-dimensional geometric detail embedding vector through an independent fully connected coding layer. This embedding vector primarily encodes the microstructure and high-frequency concavity / convexity information of the skin.

[0049] In this embodiment, a gradient-guided adaptive sampling strategy is implemented. The gradient magnitude map of the albedo texture is calculated and transformed into a sampling probability distribution. The sampling point density is increased in high-frequency regions with significant gradients and decreased in smooth regions, constructing a non-uniformly distributed training set. Before constructing self-supervised training data pairs, the gradient map of the albedo texture tensor is calculated using the Sobel operator. The gradient values ​​are normalized and mapped to a sampling probability density function: for regions with gradients greater than 0.8, such as wrinkles around the eyes and pupil edges, the sampling probability is set to 5 times the baseline value; for regions with gradients less than 0.1, such as the cheeks, only 20% of the baseline sampling rate is retained. During each iteration of data generation, pixels are extracted according to this probability distribution, forcing the implicit network to prioritize the fitting of high-frequency details with its limited weight parameters. Based on this strategy, the reconstruction error of the network for small textures is reduced within the same training time. When the generated image is magnified, the direction of skin texture is clearly presented, rather than as blurry color blocks. Furthermore, the network's efficiency in capturing and restoring high-frequency texture details is improved, solving the problems of blurring and oversmoothing in complex texture regions.

[0050] Specifically, a photometric reconstruction loss function is constructed to calculate the signal intensity difference between the predicted color signal and the true RGB color value. Based on the backpropagation gradient descent algorithm, the network weight parameters are iteratively updated in the direction of minimizing the photometric error until the network converges, thus solidifying the discrete pixel information of the albedo texture tensor into a continuous spatial color function. In this embodiment, this step essentially compresses discrete image data into continuous function parameters. The training target is the albedo texture tensor, i.e., the clean skin texture after removing illumination. In each training iteration, a batch of pixels (e.g., a batch size of 262,144 points) is randomly sampled from this 1024×1024 tensor, and their normalized coordinates (u, v) are extracted as input, while their RGB color values ​​are extracted as true value labels. During the backpropagation stage, a photometric reconstruction loss function is constructed to calculate the signal intensity difference between the network's predicted color and the true color, i.e., the L1 distance. The Adam optimizer is used, with an initial learning rate of 1×10⁻⁶. -4 This process drives iterative updates of the network weights. Due to the powerful fitting ability of the implicit neural texture representation network, this process typically converges within 500 iterations. The converged network weights are then treated as a "continuous space color function," locking the weight parameters of the implicit neural texture representation network. During the inference phase, these fixed network weights are no longer updated with gradients. As long as arbitrary precision coordinates are input (such as a sampling grid corresponding to 4K resolution), the network can calculate and output the corresponding smooth color value through a function.

[0051] By introducing a latent neural texture representation network with frequency-domain positional encoding and a sinusoidal activation mechanism, the model's ability to fit and reproduce high-frequency texture details is improved. Utilizing the periodic response characteristics of the sinusoidal activation function, combined with frequency-domain positional encoding, high-frequency changes in texture data can be captured and memorized. Driving the network to overfit a single facial texture through photometric reconstruction loss effectively upscales and solidifies discrete, low-resolution texture images into a high-capacity continuous function model, thus solving the jagged detail problem caused by the resolution limitations of the original image and achieving super-resolution texture reconstruction.

[0052] Further, the coordinates of the continuous manifold coordinate field are input point by point into the implicit neural texture representation network to generate a query index; based on the query index, the geometric detail embedding vector is injected into the implicit neural texture representation network, frequency modulation is performed, texture color values ​​are calculated, and an enhanced facial image is generated; this corresponds to step S5 above; see [link to relevant documentation]. Figure 3 The specific implementation process includes: A blank raster tensor matching the target display resolution is constructed. The pixel indices of the blank raster tensor are traversed, and the corresponding two-dimensional floating-point coordinate values ​​are indexed in parallel from the continuous manifold coordinate field to form a query coordinate batch vector. The geometric detail embedding vector is processed using a mapping perceptron, and the frequency modulation parameters for each hidden layer of the implicit neural texture representation network are regressed and output. The frequency modulation parameters include a feature scaling factor and a feature bias factor. The query coordinate batch vector is input into the implicit neural texture representation network. During signal forward propagation, the linear transformation output of each hidden layer is subjected to a channel-wise affine transformation modulation operation using the feature scaling factor and the feature bias factor, and then processed by a sinusoidal activation function to output a sinusoidal activation feature tensor. The sinusoidal activation feature tensor is subjected to color space synthesis mapping to output texture color values. The texture color values ​​are filled with pixel buffers according to the spatial arrangement order of the pixel indices to generate an enhanced facial image.

[0053] The frequency domain position encoding input layer is configured as a harmonic mapper, used to receive the two-dimensional coordinate vector input from the continuous manifold coordinate field and project it onto a Fourier feature space of a preset dimension, outputting a high-dimensional harmonic feature tensor; the cascaded modulated sinusoidal sensing unit sequence is composed of several structurally isomorphic hidden layer units connected in series, forming a sinusoidal activation hidden layer of the implicit neural texture representation network, each of the hidden layer units containing a linear transformation branch and a frequency modulation interface; the linear transformation branch performs a fully connected weighted multiplication operation on the input signal; the frequency modulation interface receives the scaling factor and bias factor transformed from the geometric detail embedding vector, performs an element-wise affine transformation on the output of the linear transformation branch, and inputs the transformation result into a periodic sinusoidal activation function, outputting a sinusoidal activation feature tensor; the color regression mapping layer receives the final-level features output from the cascaded modulated sinusoidal sensing unit sequence, maps the feature space to a three-channel RGB color space through color space synthesis mapping, and outputs texture color values.

[0054] Specifically, a blank raster tensor matching the target display resolution is constructed. The pixel indices of this blank raster tensor are traversed, and the corresponding two-dimensional floating-point coordinate values ​​are extracted in parallel from the continuous manifold coordinate field to form a query coordinate batch vector. In this embodiment, this step essentially utilizes the continuity characteristic of implicit neural networks to perform texture resampling. To ensure that the skin texture of the virtual character remains sharp in close-up shots, a blank raster tensor with dimensions of 4096×4096 (i.e., 4K standard) is first constructed as the target texture container. Based on the continuity definition of the continuous manifold coordinate field, high-precision resampling is performed at the blank raster tensor of the target 4K resolution to generate a high-density coordinate field. Subsequently, each pixel index in this high-density field is traversed, and its corresponding two-dimensional floating-point coordinate value after differential homeomorphic transformation is extracted. These two-dimensional floating-point coordinate values ​​are flattened into a query coordinate batch vector with a length of 16,777,216 (i.e., 4096 squared).

[0055] Specifically, the geometric detail embedding vector is processed using a mapping perceptron, and the regression output is the frequency modulation parameters for each hidden layer of the implicit neural texture representation network. These frequency modulation parameters include feature scaling factors and feature bias factors. In this embodiment, this step aims to transform microscopic geometric features into instructions for controlling texture generation. The mapping perceptron network adopts a "backbone-multi-head" composite structure: a backbone network consisting of three fully connected layers (512 neurons per layer) receives the geometric detail embedding vector, which encodes high-frequency geometric information such as pore depressions and wrinkles, extracting a 512-dimensional shared latent code containing global geometric semantics. Subsequently, this shared latent code is distributed to the corresponding linear projection heads of each layer of the implicit neural texture representation network. These linear projection heads, as the output terminals of the mapping perceptron, map and expand the 512-dimensional shared latent code into the feature scaling factors and feature bias factors required by each hidden layer. For example, when processing regions containing deep wrinkles, the mapping network outputs a larger scaling factor, which dynamically amplifies the amplitude and frequency of the sinusoidal activation function in subsequent steps, forcing the network to generate more dramatic color changes in that region.

[0056] Specifically, the query coordinate batch vector is input into the implicit neural texture representation network. During signal forward propagation, the feature scaling factor and the feature bias factor are used to perform channel-by-channel affine transformation modulation on the linear transformation output of each hidden layer, and then processed by a sinusoidal activation function to output a sinusoidal activation feature tensor. The sinusoidal activation feature tensor is then subjected to color space synthesis mapping to output texture color values. In this embodiment, the input signal first passes through a frequency domain position encoding input layer, which is configured as a harmonic mapper to address the spectral bias problem of the neural network. Its specific operational logic is as follows: the input two-dimensional coordinate vector is projected onto a Fourier feature space containing 10 sets of logarithmically linearly distributed frequencies. Specifically, this is a geometric series expansion sequence with base 2, meaning the set of frequency coefficients covers a range from 2... 0 Up to 2 9For each input coordinate value, the sine and cosine values ​​of its product with pi (π) and the frequency coefficient are calculated. This expands the original low-dimensional coordinate signal into a harmonic feature tensor containing 40 high-dimensional components. This 40-dimensional harmonic feature tensor is input into the hidden layer of the implicit neural texture representation network. A fully connected weight multiplication operation is performed through the linear transformation branch configured in this layer. Specifically, a matrix multiplication is performed using a weight matrix with 40 input channels and 512 output channels. This linearly projects the low-dimensional frequency features onto a 512-dimensional high-dimensional feature space that matches the width of the hidden layer of the implicit network, achieving feature dimension alignment and dimensionality increase. The output is a linearly transformed feature, thus giving the network the ability to capture details from low-frequency contours to high-frequency hair-like details. Subsequently, the signal enters a cascaded sequence of sinusoidal sensing units (5 layers, 512 neurons per layer). In the signal propagation of each layer, a fully connected linear transformation is first performed; then, an element-wise affine transformation (i.e., multiplying and adding corresponding elements) is performed using the aforementioned feature scaling factor and feature bias factor; finally, it is processed by a periodic sinusoidal activation function with a frequency parameter of 30, outputting a sinusoidal activation feature tensor. After 5 layers of processing, the final-level features enter the color regression mapping layer, where a linear projection layer compresses the high-dimensional features into three-channel RGB values, which are the texture color values.

[0057] Specifically, the texture color values ​​are filled with pixel buffers according to the spatial arrangement order of the pixel indices to generate an enhanced facial image. In this embodiment, this step transforms the discrete inference results into visualized image data. Since the inference is performed in batches (e.g., inferring 65536 points each time to avoid memory overflow), the output color vectors of all batches are first collected and concatenated to restore a total vector of length 16777216. Subsequently, based on the spatial arrangement order of the pixel indices in the initially constructed 4096×4096 blank raster tensor, the total vector is reshaped into a two-dimensional image tensor. The image generated at this time is the enhanced facial image, which not only has 4K ultra-high resolution, but also, due to the correction of the differential homeomorphic field, its texture topology fits the skeletal pose of the current virtual character; at the same time, thanks to the continuous fitting characteristics of the implicit network, even when magnified, the skin edges are smooth and jagged, and the details of pores and wrinkles are clearly distinguishable.

[0058] By constructing a frequency-modulated implicit neural network architecture, geometric detail embedding vectors are used to generate feature scaling and bias factors. The response frequency and amplitude of the sinusoidal perception unit are dynamically adjusted through the frequency modulation interface, enabling the implicit network to not only memorize colors but also synthesize details in real time based on the input geometric texture tensor. Even in extremely stretched regions, the network can infer the corresponding texture changes in continuous coordinate space based on the injected high-frequency geometric information, thus visually compensating for the resolution loss caused by deformation and stretching. This ensures cross-scale consistency from macroscopic structure to microscopic details, and the final generated image preserves the skeletal shape of the virtual character while enhancing the user's facial high-frequency features.

[0059] This invention provides an image enhancement generation method based on adaptive deformation of facial key points. By constructing a generation architecture that combines an implicit neural texture representation network with a differential homeomorphic deformation field, it solves the resolution degradation problem when nonlinearly mapping user facial features to virtual skeletons. This method transforms facial texture from a discrete pixel array into a continuous spatial color function, achieving a clearer resolution representation of the texture. Simultaneously, by decoupling albedo and geometric texture, and injecting geometric details into the network through frequency modulation, the generated image not only possesses the correct topological structure but also adaptively enhances microscopic geometric textures that may be lost due to deformation. This continuous representation method based on manifold fields effectively overcomes the resolution bottleneck of traditional texture mapping under complex deformations, ensuring the fidelity and clarity of the enhanced image in high-frequency details.

[0060] Example 2

[0061] This second embodiment, building upon the first embodiment, further demonstrates an example of adapting the user's original facial geometric features to the target virtual character's skeleton during the immersive virtual character fusion process in a science museum. The specific implementation process is as follows: First, an RGB video stream with a resolution of 1920×1080 pixels is captured using a depth camera. During the depth parameter regression stage, the input image is scaled to 224×224 pixels and normalized using a bicubic interpolation algorithm before being input into an improved ResNet-50 network. This network outputs a 257-dimensional feature vector with a precision of 32-bit floating-point numbers. The first 80 dimensions strictly correspond to the principal component coefficients of the identity shape in the BFM09 model, the next 64 dimensions correspond to facial expression coefficients, the following 80 dimensions are used for coarse texture reconstruction, the subsequent 27 dimensions are second-order spherical harmonic illumination coefficients, and the last 6 dimensions are head pose parameters. The first 80-dimensional identity vector is then multiplied by a pre-stored 35709×3×80 identity basis tensor to generate a personalized geometric model with an average vertex displacement of approximately 2.4 mm.

[0062] In the texture space processing stage, inverse rasterization is performed in the UV space at a resolution of 1024×1024 using the aforementioned geometric model. For each UV coordinate, its barycentric coordinate weight within the corresponding triangular facet is calculated; for example, for a cheek pixel, its barycentric coordinate coefficients might be 0.3, 0.5, and 0.2. These weights are used to sample the original 1080P image, generating the original coupled texture tensor. Subsequently, an ambient lighting map is constructed using the output 27 spherical harmonic coefficients. The first nine values ​​of these coefficients define the ambient light distribution in the red channel, and the irradiance map is calculated through a linear combination of spherical harmonic basis functions. Pixel-wise division is performed, dividing the pixel values ​​of the original texture by the irradiance values ​​to obtain an albedo texture tensor with an average brightness of 128 and a standard deviation of 45. Simultaneously, by calculating the logarithmic difference and applying a Gaussian difference filter with a standard deviation of 2.0, the geometric texture tensor with grayscale values ​​concentrated in the range of 10 to 50 is extracted.

[0063] To construct the differential homeomorphic deformation field, 68 Dlib standard feature points on the source mesh were identified and mapped to key nodes at the corresponding bone level in the virtual character skeleton. For example, the left corner of the mouth vertex with index 3456 in the source mesh was mapped to the joint position of the virtual skeleton. The displacement vectors of both in UV space were calculated, and a radial basis function interpolation model containing 68 control points was constructed. A Gaussian function with a shape parameter of 0.5 was selected as the kernel function. After solving for the 68 weight coefficients, displacement interpolation was performed on all 1,048,576 pixels in the image, and Jacobi constraint optimization was initiated. The optimizer set the lower limit threshold of the Jacobi determinant to 0.1. After approximately 45 L-BFGS iterations, approximately 3.5% of the negative determinant region (i.e., the texture folding region) in the initial field was corrected to positive values, and the differential homeomorphic deformation field was output.

[0064] In the enhanced image generation stage, an implicit neural texture representation network was constructed. Its input layer was expanded using a geometric series with a frequency coefficient of 2, mapping two-dimensional coordinates to a 40-dimensional Fourier feature vector (containing sine and cosine components of ten frequency bands). The main body of the network consists of five sinusoidally activated fully connected layers, each with 512 neurons. Using a 1024×1024 albedo texture as the ground truth and a batch size of 262144, overfitting training was performed using the Adam optimizer with a learning rate of 0.01%. At the 600th iteration, the loss function value decreased to the order of 10 to the power of -5, achieving convergence, and the network weights were locked. During inference, a 4096×4096 sampling grid was generated, and a scaling factor (ranging from 0.8 to 1.2) dynamically calculated from the geometric texture tensor was used to frequency modulate the network, outputting the enhanced facial image.

[0065] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An image enhancement and generation method based on adaptive deformation of facial key points, characterized in that, include: Receive facial image data streams from the science museum and skeletal data of the virtual character. Based on the facial image data streams from the science museum, perform topology reconstruction through a 3D parametric model and output the source facial mesh. Map the source face mesh to the UV parameter space to decouple the output albedo texture tensor and geometric texture tensor; Based on the virtual character's skeletal data and source facial mesh, vertex displacement is calculated through a preset topological correspondence to construct a differential homeomorphic deformation field; By applying the differential homeomorphic deformation field to the UV parameter space and constructing a continuous manifold coordinate field through nonlinear space adaptation; An implicit neural texture representation network is constructed, and overfitting training is performed based on the albedo texture tensor to fit the network weight parameters to a continuous space color function; based on the geometric texture tensor, a geometric detail embedding vector is output through Fourier feature transformation. The coordinates of the continuous manifold coordinate field are input point by point into the implicit neural texture representation network to generate a query index; Based on the query index, geometric details are embedded into a vector and injected into a hidden neural texture representation network for frequency modulation, texture color values ​​are calculated, and an enhanced facial image is generated.

2. The image enhancement and generation method based on adaptive deformation of facial key points according to claim 1, characterized in that, The specific process of outputting the source facial mesh includes: inputting the facial image data stream from the science museum into a depth parameter regression model, mapping it to a low-dimensional semantic manifold space for feature decoupling, and outputting an identity shape vector controlling the mesh geometry, an expression vector controlling facial micro-expressions, and six-degree-of-freedom pose parameters defining the head's spatial position; retrieving statistical basis data stored within the three-dimensional parameterized model, which includes an average facial geometry template, orthogonal shape principal component basis tensors, and orthogonal deformation principal component basis tensors; using the identity shape vector and the expression vector as weight coefficients, performing a linear weighted recombination operation on the orthogonal shape principal component basis tensors and the orthogonal deformation principal component basis tensors, and superimposing the operation result onto the average facial geometry template to generate an instance topological mesh under standard pose; constructing a rigid body transformation matrix based on the six-degree-of-freedom pose parameters, performing rotation and translation operations on the instance topological mesh, aligning it with the pixel coordinate system of the facial image data stream from the science museum, and outputting the source facial mesh.

3. The image enhancement and generation method based on adaptive deformation of facial key points according to claim 1, characterized in that, The specific generation process of the albedo texture tensor and geometric texture tensor includes: calling the UV topology layout data pre-placed in the 3D parametric model, establishing the index relationship between the 3D vertex coordinates of the source face mesh and the 2D texture coordinates in the UV parameter space, and calculating the barycentric coordinate interpolation coefficients within the triangular face patches of the source face mesh; based on the index relationship and the barycentric coordinate interpolation coefficients, performing reverse texture resampling on the facial image data stream of the science museum, projecting the RGB values ​​of the image pixels onto the UV parameter space, and generating a texture tensor containing illumination information and texture details. The original coupled texture tensor is used; a preset intrinsic texture decomposition network is invoked to receive the original coupled texture tensor, and global ambient lighting parameters are calculated through a built-in spherical harmonic illumination regression layer. The global ambient lighting parameters are then used to perform an inverse division operation on the original coupled texture tensor to remove illumination, and an albedo texture tensor that preserves the inherent color of the skin is output. The pixel-level difference residual between the original coupled texture tensor and the albedo texture tensor is calculated, and high-pass filtering and gray-scale quantization are performed on the pixel-level difference residual to extract a geometric texture tensor that represents the micro-geometric features of the skin surface.

4. The image enhancement and generation method based on adaptive deformation of facial key points according to claim 1, characterized in that, The specific construction process of the differential homeomorphic deformation field includes: traversing the vertices of the source face mesh to generate vertex indices; extracting a sparse source feature control point set based on preset anatomical semantic labels; and parsing a sparse target bone key point set with topological isomorphism from the virtual character bone data; calculating the Euclidean space difference between the sparse source feature control point set and the sparse target bone key point set to generate a sparse displacement vector set representing the local deformation trend; constructing a spatial interpolation equation based on the radial basis kernel function; using the sparse displacement vector set as a weighted control term; performing regularized interpolation operations on all coordinate points in the UV parameter space to generate an initial dense deformation potential field; and performing Jacobian determinant constraint optimization on the initial dense deformation potential field to output the differential homeomorphic deformation field.

5. The image enhancement and generation method based on adaptive deformation of facial key points according to claim 1, characterized in that, The specific construction process of the continuous manifold coordinate field includes: performing discretization sampling on the UV parameter space to initialize and generate a regular coordinate grid tensor representing the physical position of the standard texture space; performing vector superposition operation on the differential homeomorphic deformation field and the regular coordinate grid tensor to calculate the spatial drift of each grid point under nonlinear deformation, generating a drift coordinate tensor containing geometric distortion information; performing boundary topology constraint verification on the drift coordinate tensor, using a truncation function to restrict out-of-bounds coordinates within a preset parameter domain; performing texture coordinate normalization mapping on the verified drift coordinate tensor, linearly scaling the numerical range to a preset activation interval of the implicit neural texture representation network, and outputting a continuous manifold coordinate field.

6. The image enhancement and generation method based on adaptive deformation of facial key points according to claim 1, characterized in that, The specific training process of the implicit neural texture representation network includes: initializing and constructing a fully connected perceptron topology, configuring a frequency domain position encoding input layer and a sinusoidal activation hidden layer with periodic response characteristics, and establishing a nonlinear mapping path from low-dimensional coordinate space to high-dimensional color space; performing discrete grid sampling on the albedo texture tensor, extracting the two-dimensional spatial coordinate index of the pixel and its corresponding RGB color ground truth value, and constructing a self-supervised training data pair set; performing forward propagation inference, mapping the two-dimensional spatial coordinate index to a high-dimensional harmonic feature tensor through the frequency domain position encoding input layer, and inputting it into the sinusoidal activation hidden layer, and regressing to output the predicted color signal; constructing a photometric reconstruction loss function, calculating the signal intensity difference between the predicted color signal and the RGB color ground truth value, and driving the network weight parameters to iteratively update in the direction of minimizing photometric error based on the backpropagation gradient descent algorithm until the network converges, thus solidifying the discrete pixel information of the albedo texture tensor into a continuous spatial color function.

7. The image enhancement and generation method based on adaptive deformation of facial key points according to claim 1, characterized in that, The specific process for generating the enhanced facial image includes: constructing a blank raster tensor matching the target display resolution; traversing the pixel indices of the blank raster tensor; indexing the corresponding two-dimensional floating-point coordinate values ​​in parallel from the continuous manifold coordinate field to form a query coordinate batch vector; processing the geometric detail embedding vector using a mapping perceptron; regressing and outputting frequency modulation parameters for each hidden layer of the implicit neural texture representation network, wherein the frequency modulation parameters include a feature scaling factor and a feature bias factor; inputting the query coordinate batch vector into the implicit neural texture representation network; during signal forward propagation, performing a channel-by-channel affine transformation modulation operation on the linear transformation output of each hidden layer using the feature scaling factor and the feature bias factor, and processing it through a sinusoidal activation function to output a sinusoidal activation feature tensor; performing color space synthesis mapping on the sinusoidal activation feature tensor to output texture color values; and filling the texture color values ​​with pixel buffers according to the spatial arrangement order of the pixel indices to generate the enhanced facial image.

8. The image enhancement and generation method based on adaptive deformation of facial key points according to claim 1, characterized in that, The specific construction of the implicit neural texture representation network includes: a frequency domain position encoding input layer configured as a harmonic mapper, used to receive the two-dimensional coordinate vector input from the continuous manifold coordinate field and project it onto a Fourier feature space of a preset dimension, outputting a high-dimensional harmonic feature tensor; a cascaded modulated sinusoidal sensing unit sequence composed of several structurally isomorphic hidden layer units connected in series, forming the sinusoidal activation hidden layer of the implicit neural texture representation network, each of the hidden layer units containing a linear transformation branch and a frequency modulation interface; the linear transformation branch performs a fully connected weighted multiplication operation on the input signal; the frequency modulation interface receives scaling factors and bias factors transformed from the geometric detail embedding vector, performs an element-wise affine transformation on the output of the linear transformation branch, and inputs the transformation result into a periodic sinusoidal activation function, outputting a sinusoidal activation feature tensor; a color regression mapping layer receives the final-level features output by the cascaded modulated sinusoidal sensing unit sequence, maps the feature space to a three-channel RGB color space through color space synthesis mapping, and outputs texture color values.

Images