A voice-driven editable face reenactment method and device and a storage medium
By constructing an editable dynamic neural radiation field model and a cross-modal mapping from audio to facial expressions, the problem of the inability to personalize editing in existing technologies is solved, and high-fidelity video generation and shape and appearance editing functions of voice-driven face reenactment are realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI UNIV
- Filing Date
- 2023-02-24
- Publication Date
- 2026-05-15
AI Technical Summary
Existing voice-driven face reconstruction technology cannot perform personalized shape and texture editing.
An editable dynamic neural radiation field model is constructed, and geometric latent codes and texture latent codes are anchored in the vertices to achieve separate decoupling of shape and appearance. A cross-modal mapping from audio to facial expressions is constructed using a long short-term memory network. Combined with volume rendering, high-fidelity replay videos are generated, and users are allowed to edit the shape and appearance.
It enables personalized editing of voice-driven face replay. Users can generate high-fidelity replay videos by inputting short videos to train the model, and perform shape and appearance editing, such as face slimming, face swapping, and makeup.
Smart Images

Figure CN116228979B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and computer graphics, and in particular to a voice-driven, editable face replay method, apparatus, and storage medium. Background Technology
[0002] Face reenactment refers to synthesizing a reenactment video of a person's conversation based on their original identity information and facial expressions and postures provided by the target audience. Traditionally, this task requires artists to manually create detailed 3D facial models, followed by steps such as face rigging, motion capture, animation correction, and material adjustments, before finally rendering the image in the graphics pipeline. This process is labor-intensive and time-consuming for the production team. In recent years, with the intervention of deep learning, the traditional face reenactment process has been simplified by neural network algorithms. Users without relevant professional knowledge can now quickly create face reenactment videos end-to-end using only a single monocular RGB video.
[0003] After years of research, video-driven face replay technology has become increasingly mature. However, video-driven face replay struggles with facial occlusion, and the final replay effect heavily relies on the performer's skill level. Compared to video media, audio signals are easier to acquire and use, and can be personalized to match and replay a person's speaking style using information such as syllables and intonation. Therefore, audio-driven face replay technology has gradually become a research hotspot in computer graphics, computer vision, and cross-modal fields in recent years. This technology can be applied to areas such as virtual anchors, lip-syncing for movie dubbing, and personalized avatar customization in the metaverse, possessing broad research significance and application value.
[0004] With the rise of short videos and self-media platforms, users have placed new demands on video creation. Users often want to edit portrait videos, such as slimming faces and adding facial effects. Therefore, user-friendly, high-fidelity, and editable voice-driven face reconstruction technology has naturally become a new application requirement in the current era of self-media.
[0005] Liu et al. proposed a sketch-based facial video editing method called DeepFaceVideoEdit. This method can represent editing operations in the latent space and generate high-quality video editing results based on StyleGAN3 through specific propagation and fusion modules, allowing users to edit faces in videos using sketches and masks.
[0006] Suwajanakorn et al. proposed Synthesizing Obama, a method that uses dozens of hours of Obama's speech videos as training material. It learns the mapping from audio to lip movements through a recurrent neural network (RNN) and uses a model based on principal component analysis (PCA) to synthesize mouth textures, thus achieving speech-driven face reenactment.
[0007] Guo et al. were the first to apply neural radiation field technology to a speech-driven face replay method, using two neural radiation fields to directly construct a mapping from input audio to replay video. Compared to two-dimensional image warping and generative methods based on Generative Adversarial Networks (GANs), this method can generate more accurate dynamic facial details, demonstrating the excellent performance of neural radiation fields in the field of dynamic face generation.
[0008] Yuan et al. proposed NeRF-Editing, which establishes a correspondence between explicit mesh representation and implicit neural representation of the target scene, and uses tetrahedral mesh as a proxy to distort camera rays, thereby realizing shape editing of implicit objects.
[0009] Yang et al. proposed NeuMesh, a method that inserts geometric and texture latent codes into vertices and uses two multilayer perceptrons (MLPs) to decouple the shape and appearance attributes of static objects, constructing an editable neural radiation field. This method implicitly reconstructs 3D objects from a set of multi-view images. By changing the model mesh, geometric editing of the model can be achieved; and by replacing and modifying the latent codes in the vertices, texture editing of the model can be achieved.
[0010] However, existing voice-driven face reconstruction methods cannot personalize the shape and texture of the reconstructed face. Summary of the Invention
[0011] The purpose of this invention is to overcome the defects of the prior art by providing a voice-driven editable face replay method, apparatus and storage medium.
[0012] The objective of this invention can be achieved through the following technical solutions:
[0013] A voice-driven, editable face replay method includes the following steps:
[0014] S1. Input video materials constitute a video dataset;
[0015] S2. Perform 3D facial reconstruction on the input video material, synthesize a 3D facial mesh model, extract the identity coefficient, expression coefficient, and head pose coefficient of the source 3D deformable facial model, generate a mask, and extract the facial region image as the ground truth for training.
[0016] S3. Extract the expression coefficients of the source face 3D deformable model in the video dataset, train a long short-term memory network using aligned audio features and facial expression coefficient data, and construct a cross-modal mapping from audio to facial expression.
[0017] S4. Construct an editable dynamic neural radiation field, use a face model as a proxy to calculate the offset value of the camera light sampling point, and query the k nearest vertices of the offset sampling point in the normal space. Obtain the texture latent code and geometric latent code corresponding to the sampling point through interpolation, input the texture decoder and geometric decoder to regress the color and density of the sampling point, and generate a character replay video through volume rendering.
[0018] S5. Accept audio input, regress facial expression coefficients through the long short-term memory network, synthesize the replayed face mesh model by combining the source face identity coefficients, input the editable dynamic neural radiation field framework to regress sampling point density and color values, and synthesize the replayed video frame through volume rendering.
[0019] S6. Modify the shape of the replayed face mesh model to achieve shape editing of the target person; achieve appearance editing of the target person by changing and replacing texture latent code and geometric latent code.
[0020] Furthermore, step S2 specifically includes:
[0021] The Deep3DFace algorithm is used as the 3D face reconstruction algorithm. It fits the face shape and appearance using a convolutional neural network, as shown below:
[0022]
[0023] Where α, β, τ, ο, and ρ represent the face's identity, expression, material, lighting, and pose coefficients, respectively; once the face's identity and expression coefficients are obtained, the shape of the face is represented as:
[0024]
[0025] in, B represents the average shape of a human face. id B exp These are the bases for the shape and expression of the 3DMM model, respectively.
[0026] Furthermore, step S3 specifically includes:
[0027] Extract Mel-cepstrum features for each audio file in the dataset;
[0028] The mapping from audio to facial expression coefficients is constructed using a Long Short-Term Memory (LSTM) network, represented as follows:
[0029]
[0030] Where E is the Mel cepstral feature s (t) The encoder; h (t-1) c (t-1) These are the hidden layers and cell states of the LSTM, respectively. It is the facial expression coefficient predicted by the network in frame t; once the facial expression coefficient of a certain frame predicted by the Long Short-Term Memory Network is obtained, it can be combined with the identity coefficient of the original person to generate a three-dimensional mesh model of the replayed face.
[0031] Furthermore, step S4 specifically includes:
[0032] S401. Define a face template model with a neutral expression and anchor the geometric latent code and texture latent code at the vertices;
[0033] S402. Based on the head pose, transform the sampling points to the normal space, use the model vertices as proxies, and obtain the offset value Δx of the camera light sampling point transformation;
[0034] S403. Query the k nearest vertices of the sampling point, and obtain the texture latent code and geometric latent code corresponding to the sampling point through interpolation; regress the color and density of the sampling point through the texture decoder and geometric decoder of the neural radiation field, and render the face image.
[0035] S404. Generate replay results through volume rendering.
[0036] Furthermore, step S402 specifically includes:
[0037] In the canonical space, the face template model and the driving face mesh model are transformed into tetrahedral representations;
[0038] Transform the sampling points of the camera light to a normalized space, and find the tetrahedron corresponding to each sampling point in the driving face mesh model;
[0039] By performing centroid interpolation on the vertex displacements of the face template model and the driving face mesh model, the offset value Δx from the sampling point x in the driving face mesh model space to the sampling point x1 in the face template model space is obtained, thus realizing ray deformation.
[0040] Furthermore, step S403 specifically includes:
[0041] For a sample point x1 offset in the normal space, query the vertices of the k nearest face template models;
[0042] Obtain the geometric latent code, texture latent code, and distance indicator from the vertices of the face template model;
[0043] Input MLP-based geometry decoder F G and texture decoder F T In the regression symbol, distance field s and color c.
[0044] Furthermore, step S404 specifically includes:
[0045] Based on the equations drawn from the volume, for any starting point o, with direction d, and near and far ends m respectively. n and m f The camera ray r(m) = o + md, and its color is represented as:
[0046]
[0047] Where σ(r(m)) and c(r(m),d) are the density and color values of the sampling point, and T(m) represents the ray from m n and m f Cumulative transmittance;
[0048]
[0049] Geometry and actual color values are calculated using the signed distance field. It is represented as:
[0050]
[0051] Where T is the cumulative transmittance, Φ is the cumulative distribution of the logical distribution, s is the SDF value, and α is the opacity derived from the distance field of adjacent symbols.
[0052] Furthermore, the overall loss function L of the model is expressed as:
[0053]
[0054] A voice-driven editable face replay device includes a memory and a processor. The memory stores a computer program, and the processor can execute a voice-driven editable face replay method as described above by calling the program instructions.
[0055] A computer-readable storage medium includes a computer program that can be executed by a processor to implement a voice-driven, editable face replay method as described above.
[0056] Compared with the prior art, the present invention has the following beneficial effects:
[0057] To address the limitation of existing methods in personalizing voice-driven face replay, this invention constructs an editable dynamic neural radiation field model. This transforms the dynamic face generation problem into a sampling problem of static template faces in a normalized space. By anchoring geometric and texture latent codes to vertices, shape and appearance are decoupled, enabling editing of both the geometry and texture of the face. Users only need to input a 3-5 minute solo speech video for training. After training, the model can accept any speech as input to generate high-fidelity replay videos. Users can then edit the shape and appearance of faces in the videos, such as slimming, face swapping, and makeup. Attached Figure Description
[0058] Figure 1 This is a flowchart of the present invention;
[0059] Figure 2 Flowchart for LSTM training for cross-modal mapping from audio to facial expression coefficients;
[0060] Figure 3 This is a flowchart for training an editable dynamic neural radiation field framework.
[0061] Figure 4 This is a flowchart illustrating the geometric editing (face slimming, expression changing) process in an embodiment of the present invention.
[0062] Figure 5 This is a schematic diagram of the texture editing (face swapping) process in an embodiment of the present invention;
[0063] Figure 6 This is a schematic diagram of the texture editing (makeup) process in an embodiment of the present invention. Detailed Implementation
[0064] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.
[0065] To address the limitation of existing methods in personalizing voice-driven face replay, this invention constructs an editable dynamic neural radiation field model. This transforms the dynamic face generation problem into a sampling problem of static template faces in a normalized space. By anchoring geometric and texture latent codes to vertices, shape and appearance are decoupled, enabling editing of both the geometry and texture of the face. Users only need to input a 3-5 minute solo speech video for training. After training, the model can accept any speech as input to generate high-fidelity replay videos. Users can then edit the shape and appearance of faces in the videos, such as slimming, face swapping, and makeup.
[0066] A voice-driven, editable face replay method, with the following specific steps:
[0067] S1: Data Preprocessing. First, a video dataset is constructed from input single-person speech video footage. 3D facial reconstruction is performed on the input single-person speech video, a 3D facial mesh model is synthesized, and the identity coefficients, expression coefficients, and head pose coefficients of the source 3D Morphable Model (3DMM) are extracted. A mask is then generated, and the facial region image is extracted as the ground truth (GT) for training.
[0068] S2: Constructing a mapping from speech to facial expressions. First, using a single-person speech video dataset, extract the 3DMM facial expression coefficients from the dataset. Then, train a Long Short Term Memory (LSTM) network using aligned audio features and facial expression coefficient data to construct a cross-modal mapping from audio to facial expressions.
[0069] S3: Construct an editable dynamic neural radiation field. First, the offset values of camera light sampling points are calculated using a face model as a proxy. Then, the k nearest vertices of the offset sampling points are queried in the normal space. The texture latent code and geometric latent code corresponding to the sampling points are obtained through interpolation. These are then input into the texture decoder and geometric decoder to regress the color and density of the sampling points. Finally, the replay result is generated through volume rendering.
[0070] S4: Target Person Reenactment and Shape Editing. After the overall training of the model in this invention is completed, this method first accepts any segment of speech as input, regresses facial expression coefficients through LSTM, then combines them with the source face identity coefficients to synthesize a reenacted face mesh model. Next, it inputs an editable dynamic neural radiation field framework to regress sampling point density and color values, and finally synthesizes reenacted video frames through volume rendering. By modifying the shape of the reenactment model, the shape of the target person can be edited; by changing and replacing the texture latent code of the template model, the appearance of the target person can be edited.
[0071] The specific steps of step S1 are as follows:
[0072] The Deep3DFace algorithm is used as a 3D face reconstruction algorithm. A convolutional neural network (CNN) is used to fit the face shape and appearance. This process is represented as follows:
[0073]
[0074] Where α, β, τ, ο, and ρ represent the face's identity, expression, material, lighting, and pose coefficients, respectively. Once the face's identity and expression coefficients are obtained, the shape of the face can be represented as... in, B represents the average shape of a human face. id B exp These are the bases for the shape and expression of the 3DMM model, respectively.
[0075] The specific steps of step S2 are as follows:
[0076] First, Mel-spectral features are extracted for each audio in the training set; then, an LSTM mapping is constructed to the audio and expression coefficients, which is represented as:
[0077]
[0078] Where E is the Mel cepstral feature s (t) The encoder; h (t-1) c (t-1) These are the hidden layers and cell states of the LSTM, respectively. These are the facial expression coefficients predicted by the network in frame t. Once the facial expression coefficients are obtained, they can be combined with the original person's identity coefficients to generate a 3D mesh model of the recreated face.
[0079] Furthermore, the specific steps of step S3 are as follows:
[0080] First, a template model is defined, and geometric and texture latent codes are anchored at the vertices. Then, the sampling points are transformed to the normal space according to the head pose, and the offset value Δx of the camera light sampling points is obtained using the model vertices as proxies. Next, the k nearest vertices of the sampling points are queried, and their corresponding geometric and texture latent codes are obtained through interpolation. Finally, the density and color values are regressed through the geometric and texture decoder of the neural radiation field to render the face image.
[0081] S3-1: Light Deformation Module
[0082] First, the template mesh and driving mesh are transformed into tetrahedral representations in the normalized space. Then, the sampling points of the camera rays are transformed into the normalized space, and the tetrahedron corresponding to each sampling point is found in the driving model. Next, by performing centroid interpolation on the vertex displacements of the template model and the driving model, the offset value Δx from the sampling point x in the driving model space to the sampling point x1 in the template model space is obtained, thus realizing ray deformation.
[0083] S3-2: Texture Decoder and Geometry Decoder
[0084] For a sample point x1 offset in the canonical space, first query the vertices of the k nearest template models; then, obtain the geometric latent code, texture latent code, and distance indicator from these vertices; next, input the MLP-based geometric decoder F. G and texture decoder F T In the regression, the Signed Distance Field (SDF) s and the color c are regressed.
[0085] S3-3: Volume Drawing
[0086] Based on the equations drawn from the volume, for any starting point o, with direction d, and near and far ends m respectively. n and m f The camera ray r(m) = o + md, and its color is represented as:
[0087]
[0088] Where σ(r(m)) and c(r(m),d) are the density and color values of the sampling point, and T(m) represents the ray from m n and m f The cumulative transmittance.
[0089]
[0090] This invention uses SDF to calculate the geometry and actual color value. It is represented as:
[0091]
[0092] Where T is the cumulative transmittance, Φ is the cumulative distribution of the logical distribution, s is the SDF value, and α is the opacity derived from the adjacent SDFs.
[0093] In this invention, the overall loss function L of the model is expressed as:
[0094]
[0095] For the overall process, please refer to Figure 1 For input speech, audio features are first extracted and input into LSTM regression to recreate facial expression coefficients, which are then combined with source face identity coefficients to synthesize a face mesh model. Then, using the vertex displacement between the recreated model and the template model as a proxy, an editable dynamic neural radiation field model is constructed. Finally, RGB video frames are rendered through volume rendering.
[0096] The training process for the audio module mapping part of this invention is described in [reference needed]. Figure 2First, a dataset is constructed using single-person speech videos. Mel frequency cepstral coefficients (MFCCs) are extracted from each audio in the training set. The analysis window length is set to 0.25 milliseconds, and the interval between consecutive windows is set to 10 milliseconds. Then, 3D facial reconstruction is performed on the video faces to obtain the corresponding audio features and facial expression coefficients. Finally, an LSTM mapping from audio to facial expression is constructed.
[0097] The editable dynamic neural radiation field training process of this invention is described in detail below. Figure 3 First, a face-neutral template model is defined, and a face mesh model is reconstructed from the training video. Then, the offset value Δx of the sampling point is obtained by calculating the vertex displacement values of the face-neutral template model and the synthesized face-driven model. Next, the k vertices around the sampling point are queried in the normal space and the shape latent code and texture latent code are obtained by interpolation, and then input into the shape decoder and texture decoder. Finally, the video is replayed by volume rendering.
[0098] The geometric editing method of this invention is as follows: Figure 4 By changing the mesh model driving the face, the driving expression can be customized. By modifying the mesh shape of the neutral template model of the face, global geometric editing such as face slimming and head reduction can be achieved. The texture editing (face swapping) implementation method of this invention is as follows: Figure 5 By training editable dynamic neural radiation field models for different characters and replacing the texture latent codes in the vertices of the face template model, face-swapping can be achieved; the texture editing (makeup) implementation method of this invention is as follows: Figure 6 After the user edits the replay video frames, the vertices affected by the pixel modification are determined by the reverse of the camera light. The latent codes in these vertices are then fine-tuned using the edited image to achieve the makeup effect.
[0099] This invention also proposes a voice-driven editable face replay device and storage medium. The device includes a memory and a processor. The memory stores a computer program, and the processor can execute the voice-driven editable face replay method described above by calling the program instructions. At the hardware level, the voice-driven editable face replay device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the above-described editable face replay method. Of course, besides the software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc.
[0100] The storage medium includes a computer program that can be executed by a processor to implement a voice-driven, editable face replay method as described above. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.
[0101] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0102] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.
Claims
1. A voice-driven, editable face replay method, characterized in that, Includes the following steps: S1. Input video materials constitute a video dataset; S2. Perform 3D facial reconstruction on the input video material, synthesize a 3D facial mesh model, extract the identity coefficient, expression coefficient, and head pose coefficient of the source 3D deformable facial model, generate a mask, and extract the facial region image as the ground truth for training. S3. Extract the expression coefficients of the source face 3D deformable model in the video dataset, train a long short-term memory network using aligned audio features and facial expression coefficient data, and construct a cross-modal mapping from audio to facial expression. S4. Construct an editable dynamic neural radiation field, use a face model as a proxy to calculate the offset value of the camera light sampling point, and query the k nearest vertices of the offset sampling point in the normal space. Obtain the texture latent code and geometric latent code corresponding to the sampling point through interpolation, input the texture decoder and geometric decoder to regress the color and density of the sampling point, and generate a character replay video through volume rendering. S5. Accept audio input, regress facial expression coefficients through the long short-term memory network, synthesize the replayed face mesh model by combining the source face identity coefficients, input the editable dynamic neural radiation field framework to regress sampling point density and color values, and synthesize the replayed video frame through volume rendering. S6. Modify the shape of the replayed face mesh model to achieve shape editing of the target person; achieve appearance editing of the target person by changing and replacing texture latent code and geometric latent code.
2. The voice-driven editable face replay method according to claim 1, characterized in that, Step S2 specifically includes: The Deep3DFace algorithm is used as the 3D face reconstruction algorithm. It fits the face shape and appearance using a convolutional neural network, as shown below: Where α, β, τ, ο, and ρ represent the face's identity, expression, material, lighting, and pose coefficients, respectively; once the face's identity and expression coefficients are obtained, the shape of the face is represented as: in, B represents the average shape of a human face. id B exp These are the bases for the shape and expression of the 3DMM model, respectively.
3. The voice-driven editable face replay method according to claim 1, characterized in that, Step S3 specifically includes: Extract Mel-cepstrum features for each audio file in the dataset; The mapping from audio to facial expression coefficients is constructed using a Long Short-Term Memory (LSTM) network, represented as follows: Where E is the Mel cepstral feature s (t) The encoder; h (t-1) c (t-1) These are the hidden layers and cell states of the LSTM, respectively. It is the facial expression coefficient predicted by the network in frame t; once the facial expression coefficient of a certain frame predicted by the Long Short-Term Memory Network is obtained, it can be combined with the identity coefficient of the original person to generate a three-dimensional mesh model of the replayed face.
4. The voice-driven editable face replay method according to claim 1, characterized in that, Step S4 specifically includes: S401. Define a face template model with a neutral expression and anchor the geometric latent code and texture latent code at the vertices; S402. Based on the head pose, transform the sampling points to the normal space, use the model vertices as proxies, and obtain the offset value Δx of the camera light sampling point transformation; S403. Query the k nearest vertices of the sampling point, and obtain the texture latent code and geometric latent code corresponding to the sampling point through interpolation; regress the color and density of the sampling point through the texture decoder and geometric decoder of the neural radiation field, and render the face image. S404. Generate replay results through volume rendering.
5. The voice-driven editable face replay method according to claim 4, characterized in that, Step S402 specifically includes: In the canonical space, the face template model and the driving face mesh model are transformed into tetrahedral representations; Transform the sampling points of the camera light to a normalized space, and find the tetrahedron corresponding to each sampling point in the driving face mesh model; By performing centroid interpolation on the vertex displacements of the face template model and the driving face mesh model, the offset value Δx from the sampling point x in the driving face mesh model space to the sampling point x1 in the face template model space is obtained, thus realizing ray deformation.
6. The voice-driven editable face replay method according to claim 5, characterized in that, Step S403 specifically includes: For a sample point x1 offset in the normal space, query the vertices of the k nearest face template models; Obtain the geometric latent code, texture latent code, and distance indicator from the vertices of the face template model; Input MLP-based geometry decoder F G and texture decoder F T In the equation, the regression sign distance field s and color c are given.
7. The voice-driven editable face replay method according to claim 6, characterized in that, Step S404 specifically includes: Based on the equations drawn from the volume, for any starting point o, with direction d, and near and far ends m respectively. n and m f The camera ray r(m) = o + md, and its color is represented as: Where σ(r(m)) and c(r(m),d) are the density and color values of the sampling point, and T(m) represents the ray from m n and m f Cumulative transmittance; Geometry and actual color values are calculated using the signed distance field. It is represented as: Where T is the cumulative transmittance, Φ is the cumulative distribution of the logical distribution, s is the SDF value, and α is the opacity derived from the distance field of adjacent symbols.
8. The voice-driven editable face replay method according to claim 7, characterized in that, The overall loss function L of the model is expressed as:
9. A voice-driven, editable face replay device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor calling the program instructions to execute a voice-driven editable face replay method as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that, It includes a computer program that can be executed by a processor to implement a voice-driven editable face replay method as described in any one of claims 1-8.