Three-dimensional dynamic portrait editing method and system based on fine-tuning three-dimensional generative adversarial network

Abstract

The invention provides a three-dimensional dynamic portrait editing method and system based on a fine-tuning three-dimensional generative adversarial network, and the method comprises the steps: completing the inverse mapping of a combined three-dimensional generative adversarial network, the fine-tuning of the three-dimensional generative adversarial network based on an aligned image, the fine-tuning of the three-dimensional generative adversarial network based on an original image, and rendering an identity nerve radiation field. Using a diffusion model to edit the rendered image and then finely adjusting the three-dimensional generative adversarial network Identity, motion and camera information are effectively decoupled by combining three-dimensional generative adversarial network inverse mapping; the three-dimensional generative adversarial network is finely adjusted based on the aligned image, so that the mapping accuracy of the three-dimensional generative adversarial network to the identity information of the input video is improved; a three-dimensional generative adversarial network is finely adjusted based on an original image, and the three-dimensional generative adversarial network based on an aligned head image is expanded to half-body image rendering; by rendering an identity radiation field, using a diffusion model to edit a rendered image and finely adjusting the three-dimensional generative adversarial network, stylized editing is effectively realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular, to a three-dimensional dynamic portrait editing method and system based on a fine-tuned three-dimensional generative adversarial network, and more particularly, to a three-dimensional dynamic portrait editing method and system based on a fine-tuned three-dimensional generative adversarial network using a diffusion model. Background Art

[0002] With the rapid progress of digital technology, the technology of generating 3D stylized Avatars from video data can provide users with an unprecedented immersive experience. Generative adversarial networks (GANs), especially their variants such as StyleGAN, have set a new benchmark in the field of image and video synthesis with their excellent generation capabilities. StyleGAN can finely control the details of the generated image through its unique hierarchical style control mechanism, making it possible to extract and generate high-fidelity 3D portrait avatars from a single video in real time. This technological breakthrough not only retains the identity features and expression details in the original video, but also greatly enriches the possibility of personalized expression through model improvements based on this technology. In addition to the application of GANs, optimization-based methods also provide another path for the generation of 3D stylized Avatars. These methods first use advanced 3D reconstruction technology to restore a real 3D face model from a video or image. Subsequently, through complex mesh geometric deformation and texture migration algorithms, the final generated virtual avatar can integrate the characteristics of the target style while maintaining the identity features of the original face. Although this method has high computational complexity, it can achieve a more refined and controllable stylization effect.

[0003] However, although the existing technology has made significant progress, there are still many challenges in the stylized editing of three-dimensional dynamic portraits. For example, the stylization process is often difficult to control accurately, resulting in inconsistency in the generated results; at the same time, how to achieve efficient and natural dynamic changes while maintaining the consistency of three-dimensional data is also a difficult problem in current research. In response to the above problems, the present invention proposes an innovative solution-a three-dimensional dynamic portrait editing method based on a fine-tuned three-dimensional generative adversarial network using a diffusion model. This method combines the flexibility of the diffusion model in data generation and the advantages of 3D GANs in image quality. By fine-tuning the 3D GAN network, precise control of the stylization of three-dimensional dynamic portraits is achieved. Summary of the invention

[0004] In view of the defects in the prior art, the object of the present invention is to provide a three-dimensional dynamic portrait editing method and system based on a fine-tuned three-dimensional generative adversarial network.

[0005] According to the present invention, a 3D dynamic portrait editing method based on fine-tuning a 3D generative adversarial network is provided, comprising:

[0006] Step S1: obtaining each frame of a portrait image in a portrait video, and preprocessing the obtained portrait image, using a pre-trained generative adversarial network, based on the preprocessed portrait image and the corresponding loss function, to obtain an inverse mapping parameter corresponding to the portrait image;

[0007] Step S2: obtaining a portrait facial image based on the preprocessed portrait image, constructing a personalized three-dimensional generative adversarial network, and fine-tuning the personalized three-dimensional generative adversarial network based on the portrait facial image and the corresponding inverse mapping parameters to obtain a fine-tuned personalized three-dimensional generative adversarial network;

[0008] Step S3: performing a second fine-tuning on the fine-tuned personalized three-dimensional generative adversarial network based on each frame of the portrait image in the portrait video and the corresponding inverse mapping parameters to obtain a personalized three-dimensional generative adversarial network after the second fine-tuning;

[0009] Step S4: Rendering a multi-view image of the identity portrait through the personalized 3D generative adversarial network after the second fine-tuning based on the inverse mapping identity information parameters, and editing the obtained portrait image using a diffusion model and corresponding text prompts to obtain an edited portrait image, and training the personalized 3D generative adversarial network after the second fine-tuning based on the edited portrait image and the corresponding inverse mapping parameters to obtain an optimized stylized 3D generative adversarial network.

[0010] Preferably, the step S1 comprises: preprocessing the acquired portrait image, including: cropping and aligning the face of the acquired portrait image, and extracting camera parameters.

[0011] Preferably, the loss function in step S1 includes:

[0012]

[0013] Among them, w id Indicates the identity information of the portrait; o n Indicates the action information of the portrait; c′ n Represents the camera parameters corresponding to the preprocessed image, G raw (w id +o n ,c n ) represents the generated sample image, I′ n represents the preprocessed input image, L lpips and λ lpips Represents the learnable perceptual image block similarity loss and the corresponding weight, L L2 represents the Euclidean distance loss, λ L2 and regare the weights corresponding to balancing the Euclidean distance loss and the action information penalty loss, respectively.

[0014] Preferably, step S2 comprises: fine-tuning the personalized three-dimensional generative adversarial network based on the portrait facial image and the corresponding inverse mapping parameters, and the loss function comprises:

[0015]

[0016] Among them, G′ raw Representation Personalization for 3D Generative Adversarial Networks.

[0017] Preferably, step S3 includes: performing a second fine-tuning on the fine-tuned personalized three-dimensional generative adversarial network based on each portrait image frame in the portrait video and the corresponding inverse mapping parameters, and the loss function includes:

[0018]

[0019] Among them, c n The camera parameters corresponding to the image without preprocessing, I n Represents each portrait image frame in the portrait video, G′ raw Representing personalized half-body 3D generative adversarial networks.

[0020] Preferably, in step S4, based on the inverse mapped identity information parameters, a personalized three-dimensional generative adversarial network is used to render the identity portrait multi-view image, and the obtained portrait image is edited using a diffusion model and corresponding text prompts to obtain an edited portrait image, including:

[0021] Step S4.1: Based on the acquired portrait identity information parameter w id The personalized 3D generative adversarial network after the second fine-tuning is used to render the identity neural radiation field to obtain a multi-view portrait image;

[0022] Step S4.2: Edit the frontal portrait using the diffusion model, and edit the multi-view portrait images using the video stylization method;

[0023]

[0024] Where Diff represents the pre-trained text-guided image editing diffusion model, G edit represents the 3D generative adversarial network to be edited and fine-tuned, c m_front Indicates the camera parameters corresponding to the frontal view; Represents a frontal perspective The corresponding unedited half-length portrait image, represents an edited frontal perspective rendering of a half-length portrait image;

[0025]

[0026] in, Denotes the edited image after the diffusion model and video stylization editing, G edit (w id ,c m ) represents the corresponding view angle c based on identity latent variable rendering m Video_style represents the video stylization algorithm.

[0027] Preferably, the personalized three-dimensional generative adversarial network after the second fine-tuning is trained based on the edited portrait image and the corresponding inverse mapping parameters, and the loss function includes:

[0028]

[0029] Among them, c m Represents the preset surround camera parameters, G edit (w id ,c m ) represents the image rendered based on the identity latent variable, Represents the edited image after diffusion model and video stylization editing.

[0030] According to the present invention, a 3D dynamic portrait editing system based on a fine-tuned 3D generative adversarial network is provided, comprising:

[0031] Module M1: Obtain each frame of the portrait image in the portrait video, preprocess the obtained portrait image, and use the pre-trained generative adversarial network to obtain the inverse mapping parameters corresponding to the portrait image based on the preprocessed portrait image and the corresponding loss function;

[0032] Module M2: obtaining a portrait facial image based on the preprocessed portrait image, constructing a personalized three-dimensional generative adversarial network, and fine-tuning the personalized three-dimensional generative adversarial network based on the portrait facial image and the corresponding inverse mapping parameters to obtain a fine-tuned personalized three-dimensional generative adversarial network;

[0033] Module M3: performing a second fine-tuning on the fine-tuned personalized three-dimensional generative adversarial network based on each frame of the portrait image in the portrait video and the corresponding inverse mapping parameters to obtain a personalized three-dimensional generative adversarial network after the second fine-tuning;

[0034] Module M4: Based on the inverse mapping identity information parameters, the personalized 3D generative adversarial network after the second fine-tuning is used to render the identity portrait multi-view image, and the obtained portrait image is edited using the diffusion model and the corresponding text prompt to obtain the edited portrait image, and the personalized 3D generative adversarial network after the second fine-tuning is trained based on the edited portrait image and the corresponding inverse mapping parameters to obtain an optimized stylized 3D generative adversarial network.

[0035] Preferably, the module M1 comprises: preprocessing the acquired portrait image, including: cropping and aligning the face of the acquired portrait image, and extracting camera parameters;

[0036] The loss function in the module M1 includes:

[0037]

[0038] Among them, w id Indicates the identity information of the portrait; o n Indicates the action information of the portrait; c′ n Represents the camera parameters corresponding to the preprocessed image, G raw (w id +o n ,c n ) represents the generated sample image, I′ n represents the preprocessed input image, L lpips and λ lpips Represents the learnable perceptual image block similarity loss and the corresponding weight, L L2 represents the Euclidean distance loss, λ L2 and reg are the weights corresponding to balancing the Euclidean distance loss and the action information penalty loss, respectively.

[0039] Preferably, the module M2 comprises: fine-tuning a personalized three-dimensional generative adversarial network based on the portrait facial image and the corresponding inverse mapping parameters, and the loss function comprises: (2) in, Representing personalized 3D generative adversarial networks; The module M3 includes: performing a second fine-tuning on the fine-tuned personalized three-dimensional generative adversarial network based on each portrait image frame in the portrait video and the corresponding inverse mapping parameters, and the loss function includes: (3) in, The camera parameters corresponding to the image without preprocessing, represents each portrait image frame in the portrait video, Representing personalized half-body 3D generative adversarial network; The module M4 renders a multi-view image of the identity portrait through a personalized three-dimensional generative adversarial network based on the inverse mapping identity information parameters, and edits the obtained portrait image using a diffusion model and corresponding text prompts to obtain an edited portrait image, including: Module M4.1: Based on the acquired portrait identity information parameters The personalized 3D generative adversarial network after the second fine-tuning is used to render the identity neural radiation field to obtain a multi-view portrait image; Module M4.2: Editing frontal portraits using diffusion models and editing multi-view portrait images using video stylization methods;

[0040] Where Diff represents the pre-trained text-guided image editing diffusion model, G edit represents the 3D generative adversarial network to be edited and fine-tuned, c m_front Indicates the camera parameters corresponding to the frontal view; Represents a frontal perspective The corresponding unedited half-length portrait image, represents an edited frontal perspective rendering of a half-length portrait image;

[0050]

[0051] in, Denotes the edited image after the diffusion model and video stylization editing, G edit (w id ,c m ) represents the corresponding perspective based on identity latent variable rendering, c m Video_style represents the video stylization algorithm;

[0052] The personalized three-dimensional generative adversarial network after the second fine-tuning is trained based on the edited portrait image and the corresponding inverse mapping parameters, and the loss function includes:

[0053]

[0054] Among them, c m Represents the preset surround camera parameters, G edit (w id ,c m ) represents the image rendered based on the identity latent variable, Represents the edited image after diffusion model and video stylization editing.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] 1. The present invention proposes a 3D dynamic portrait editing method based on a fine-tuned 3D generative adversarial network using a diffusion model. By applying the joint 3D generative adversarial network inverse mapping, fine-tuning the 3D generative adversarial network based on aligned images, fine-tuning the 3D generative adversarial network based on original images, rendering the identity neural radiation field, using the diffusion model and video stylization to edit the rendered image and fine-tuning the 3D generative adversarial network and other technologies, the problem of uncontrollable stylization and low consistency in generating 3D stylized Avatars from videos is effectively solved, thereby improving the quality of 3D dynamic portrait editing.

[0057] 2. The present invention effectively decouples identity, motion and camera information by combining the inverse mapping of the 3D generative adversarial network, and obtains inverse mapping parameters that can characterize 3D dynamic portraits;

[0058] 3. The present invention achieves a high matching degree between the 3D generative adversarial network and the given portrait by fine-tuning based on the aligned image and the original image respectively;

[0059] 4. The present invention utilizes the representativeness of the identity radiation field for dynamic portraits and realizes the editing of dynamic portraits by fine-tuning the identity neural radiation field; and utilizes the multi-frame consistency of video stylization to realize multi-perspective consistent image editing. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0061] Figure 1 It is a flow chart of the 3D dynamic portrait editing method based on fine-tuning the 3D generative adversarial network.

[0062] Figure 2 This is a schematic diagram of the effect generated by two fine-tuning and editing.

[0063] Figure 3 It is a 360-degree schematic diagram of the generated effect before and after editing.

[0064] Figure 4 This is a schematic diagram of the representative effect of the 3D dynamic portrait editing method based on fine-tuning the 3D generative adversarial network. DETAILED DESCRIPTION

[0065] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0066] Example 1

[0067] According to a 3D dynamic portrait editing method based on fine-tuning a 3D generative adversarial network provided by the present invention, Figure 1 As shown, including:

[0068] Step S1: obtaining each frame of a portrait image in a portrait video, and preprocessing the obtained portrait image, using a pre-trained generative adversarial network, based on the preprocessed portrait image and the corresponding loss function, to obtain an inverse mapping parameter corresponding to the portrait image;

[0069] In this embodiment, the training of the generative adversarial network requires that the data have the same structural properties, which requires that the input images be aligned. In order to complete the preprocessing of the portrait video, the facial key points are extracted and the image is cropped and aligned to meet the data requirements of the generative adversarial network. The posture parameters of the face are extracted as the posture input of the 3D generative adversarial network.

[0070] The inverse mapping of the generative adversarial network is used to inversely map the video portrait to the hidden variables of the generative adversarial network. The generative adversarial network can generate real images from hidden variables, and vice versa. Given a real image, the corresponding hidden variables can be solved. This process is called inverse mapping. Given a video, in order to achieve the inverse mapping of the hidden variables with consistent identity, the hidden variables w that decouple identity and action information are used. id +o n Represents the hidden variable corresponding to each frame, where w id Indicates the identity information of the portrait, o n Represents the action information of the portrait. The joint inverse mapping uses formula (1) to express the loss function:

[0071]

[0072] In the formula, c′ n The camera parameters corresponding to the preprocessed image, G raw (w id +o n ,c n ) represents the generated sample image, I′ n represents the preprocessed input image, L lpips and λ lpips represents the learned perceptual image patch similarity (LPIPS) loss and its corresponding weight, L L2 represents the Euclidean distance loss, λ L2 and regThey are the weights corresponding to balancing the Euclidean distance loss and the action information penalty loss, and take empirical values.

[0073] Step S2: obtaining a portrait facial image based on the preprocessed portrait image, constructing a personalized three-dimensional generative adversarial network, and fine-tuning the personalized three-dimensional generative adversarial network based on the portrait facial image and the corresponding inverse mapping parameters to obtain a fine-tuned personalized three-dimensional generative adversarial network; In this embodiment, after completing the joint latent variable inverse mapping, the generated image and the preprocessed image are constrained by the loss function, and a personalized three-dimensional generative adversarial network is optimized; wherein the loss function of the fine-tuned three-dimensional generative adversarial network is expressed by formula (2): (2) In the formula, The camera parameters corresponding to the preprocessed image, represents the preprocessed input image, Representation Personalization for 3D Generative Adversarial Networks.

[0078] Step S3: performing a second fine-tuning on the fine-tuned personalized three-dimensional generative adversarial network based on each frame of the portrait image in the portrait video and the corresponding inverse mapping parameters to obtain a personalized three-dimensional generative adversarial network after the second fine-tuning;

[0079] The personalized 3D generative adversarial network optimized on the cropped and aligned image is fine-tuned twice to make it consistent with the properties of the original half-body image; the loss function of the second fine-tuned 3D generative adversarial network is expressed by formula (3):

[0080]

[0081] In the formula, c n The camera parameters corresponding to the image without preprocessing, I n represents the input image without preprocessing, G″ raw Representing personalized half-body 3D generative adversarial networks.

[0082] Step S4: Rendering a multi-view image of the identity portrait through the personalized 3D generative adversarial network after the second fine-tuning based on the inverse mapping identity information parameters, and editing the obtained portrait image using a diffusion model and corresponding text prompts to obtain an edited portrait image, and training the personalized 3D generative adversarial network after the second fine-tuning based on the edited portrait image and the corresponding inverse mapping parameters to obtain an optimized stylized 3D generative adversarial network.

[0083] In this embodiment, the identity neural radiation field is rendered to obtain a multi-view portrait image, a diffusion model is used to edit the frontal portrait, and a video stylization method is used to edit the multi-view portrait. edit (w id ,c m ) represents the image rendered based on the identity latent variable, and the positive image G is selected edit (w id ,c m_front ) can be edited using the diffusion model and text controls to obtain, Based on the edited frontal image and the unedited side image, multi-view image editing is completed using video stylization:

[0084]

[0085] Where Diff represents the pre-trained text-guided image editing diffusion model, including but not limited to the stable diffusion model (SD); G edit represents the 3D generative adversarial network to be edited and fine-tuned, i.e., the personalized 3D generative adversarial network after the second fine-tuning, (G edit Initialized to G″ raw );c m_front Indicates the camera parameters corresponding to the frontal view; Represents a frontal perspective The corresponding unedited half-length portrait image, represents an edited frontal perspective rendering of a half-length portrait image;

[0086] The loss function is used to constrain the generated image and the edited image, and the edited personalized 3D generative adversarial network is optimized. The loss function of the 3D generative network is expressed by formula (4):

[0087]

[0088] Among them, c m Represents the preset surround camera parameters, G edit (w id ,c m ) represents the image rendered based on the identity latent variable, Represents the edited image after diffusion model and video stylization editing.

[0089] The present invention also provides a three-dimensional dynamic portrait editing system based on a fine-tuned three-dimensional generative adversarial network. The three-dimensional dynamic portrait editing system based on a fine-tuned three-dimensional generative adversarial network can be implemented by executing the process steps of the three-dimensional dynamic portrait editing method based on the fine-tuned three-dimensional generative adversarial network, that is, those skilled in the art can understand the three-dimensional dynamic portrait editing method based on the fine-tuned three-dimensional generative adversarial network as a preferred implementation of the three-dimensional dynamic portrait editing system based on the fine-tuned three-dimensional generative adversarial network.

[0090] The present invention achieves personalized generator fine-tuning through two fine-tunings. Specifically, two cases are selected to further illustrate the effectiveness of the two fine-tunings; Figure 2 As shown in the figure, for each case, the rendering after direct inverse mapping of the half-body and head, the results of fine-tuning the 3D generative adversarial network on the head image alone, the results of fine-tuning the 3D generative adversarial network on the half-body image alone, and the results of the two fine-tunings proposed by us are shown. The two fine-tunings proposed by the present invention can better capture the full body and expression attributes and generate rendering results consistent with the original video.

[0091] The present invention proposes to synthesize and edit dynamic 3D portraits based on 3D generative adversarial networks. Specifically, a case study is selected to discuss the 3D capabilities of the method; Figure 3 As shown in the figure. The first row shows the 360-degree display effect of the two postures after inverse mapping. The second row shows the 360-degree display effect of the correspondingly edited postures. It can be seen that the method of the present invention can ensure high-quality reconstruction and rendering of 360 degrees.

[0092] Furthermore, a portrait video sequence was selected for discussion; Figure 4 As shown in the figure, the first row shows 7 half-body video sequences with different postures and expressions, and the following three rows show the editing results synthesized under different text commands with high consistency and in line with the text commands. Among them, the results in the last row also show the rendering results of multiple perspectives. The present invention can achieve free high-quality editing results.

[0093] The computer hardware conditions used in the experiment of this invention are: an NVIDIA RTX 3090 graphics card (video memory size 24G), i9-10980XE processor. The input image resolution is 512×512.

[0094] The present invention fine-tunes the pre-trained 3D GAN, and performs 1000 iterations of joint latent variable inversion, fine-tuning based on aligned images, fine-tuning based on original images, and fine-tuning based on edited images. For a ten-second (300-frame) video, the joint latent variable inverse mapping takes about 80 minutes, and the three fine-tunings take 10 minutes each. The time to edit an image using the diffusion model and video stylization only takes about ten seconds.

[0095] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0096] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A 3D dynamic portrait editing method based on fine-tuning 3D generative adversarial network, characterized in that: include: Step S1: obtaining each frame of a portrait image in a portrait video, and preprocessing the obtained portrait image, using a pre-trained generative adversarial network, based on the preprocessed portrait image and the corresponding loss function, to obtain an inverse mapping parameter corresponding to the portrait image; Step S2: obtaining a portrait facial image based on the preprocessed portrait image, constructing a personalized three-dimensional generative adversarial network, and fine-tuning the personalized three-dimensional generative adversarial network based on the portrait facial image and the corresponding inverse mapping parameters to obtain a fine-tuned personalized three-dimensional generative adversarial network; Step S3: performing a second fine-tuning on the fine-tuned personalized three-dimensional generative adversarial network based on each frame of the portrait image in the portrait video and the corresponding inverse mapping parameters to obtain a personalized three-dimensional generative adversarial network after the second fine-tuning; Step S4: Rendering a multi-view image of the identity portrait through the personalized 3D generative adversarial network after the second fine-tuning based on the inverse mapping identity information parameters, and editing the obtained portrait image using a diffusion model and corresponding text prompts to obtain an edited portrait image, and training the personalized 3D generative adversarial network after the second fine-tuning based on the edited portrait image and the corresponding inverse mapping parameters to obtain an optimized stylized 3D generative adversarial network.

2. The 3D dynamic portrait editing method based on fine-tuning 3D generative adversarial network according to claim 1, characterized in that: The step S1 includes: preprocessing the acquired portrait image, including: cropping and aligning the face of the acquired portrait image, and extracting camera parameters.

3. The 3D dynamic portrait editing method based on fine-tuning 3D generative adversarial network according to claim 1, characterized in that: The loss function in step S1 includes: Among them, w id Indicates the identity information of the portrait; o n Indicates the action information of the portrait; c′ n Represents the camera parameters corresponding to the preprocessed image, G raw (w id +o n ,c n ) represents the generated sample image, I′ n represents the preprocessed input image, L lpips and λ lpips Represents the learnable perceptual image block similarity loss and the corresponding weight, L L2 represents the Euclidean distance loss, λ L2 and reg are the weights corresponding to balancing the Euclidean distance loss and the action information penalty loss, respectively.

4. The 3D dynamic portrait editing method based on fine-tuning 3D generative adversarial network according to claim 1, characterized in that: The step S2 includes: fine-tuning the personalized three-dimensional generative adversarial network based on the portrait facial image and the corresponding inverse mapping parameters, and the loss function includes: （2） in, Representation Personalization for 3D Generative Adversarial Networks.

5. The 3D dynamic portrait editing method based on fine-tuning 3D generative adversarial network according to claim 1, characterized in that: The step S3 includes: performing a second fine-tuning on the fine-tuned personalized three-dimensional generative adversarial network based on each portrait image frame in the portrait video and the corresponding inverse mapping parameters, and the loss function includes: Among them, c n The camera parameters corresponding to the image without preprocessing, I n Represents each portrait image frame in the portrait video, G″ raw Representing personalized half-body 3D generative adversarial networks.

6. The 3D dynamic portrait editing method based on fine-tuning 3D generative adversarial network according to claim 1, characterized in that: In step S4, based on the inverse mapped identity information parameters, a personalized three-dimensional generative adversarial network is used to render the identity portrait multi-view image, and the obtained portrait image is edited using a diffusion model and corresponding text prompts to obtain an edited portrait image, including: Step S4.1: Based on the acquired portrait identity information parameter w id The personalized 3D generative adversarial network after the second fine-tuning is used to render the identity neural radiation field to obtain a multi-view portrait image; Step S4.2: Edit the frontal portrait using the diffusion model, and edit the multi-view portrait images using the video stylization method; Where Diff represents the pre-trained text-guided image editing diffusion model, G edit represents the 3D generative adversarial network to be edited and fine-tuned, c m_front Indicates the camera parameters corresponding to the frontal view; Represents a frontal perspective The corresponding unedited half-length portrait image, represents an edited frontal perspective rendering of a half-length portrait image; in, Denotes the edited image after the diffusion model and video stylization editing, G edit (w id ,c m ) represents the corresponding view angle c based on identity latent variable rendering m Video_style represents the video stylization algorithm.

7. The 3D dynamic portrait editing method based on fine-tuning 3D generative adversarial network according to claim 1, characterized in that: The personalized three-dimensional generative adversarial network after the second fine-tuning is trained based on the edited portrait image and the corresponding inverse mapping parameters, and the loss function includes: Among them, c m Represents the preset surround camera parameters, G edit (w id ,c m ) represents the image rendered based on the identity latent variable, Represents the edited image after diffusion model and video stylization editing.

8. A 3D dynamic portrait editing system based on a fine-tuned 3D generative adversarial network, characterized in that: include: Module M1: Obtain each frame of the portrait image in the portrait video, preprocess the obtained portrait image, and use the pre-trained generative adversarial network to obtain the inverse mapping parameters corresponding to the portrait image based on the preprocessed portrait image and the corresponding loss function; Module M2: obtaining a portrait facial image based on the preprocessed portrait image, constructing a personalized three-dimensional generative adversarial network, and fine-tuning the personalized three-dimensional generative adversarial network based on the portrait facial image and the corresponding inverse mapping parameters to obtain a fine-tuned personalized three-dimensional generative adversarial network; Module M3: performing a second fine-tuning on the fine-tuned personalized three-dimensional generative adversarial network based on each frame of the portrait image in the portrait video and the corresponding inverse mapping parameters to obtain a personalized three-dimensional generative adversarial network after the second fine-tuning; Module M4: Based on the inverse mapping identity information parameters, the personalized 3D generative adversarial network after the second fine-tuning is used to render the identity portrait multi-view image, and the obtained portrait image is edited using the diffusion model and the corresponding text prompt to obtain the edited portrait image, and the personalized 3D generative adversarial network after the second fine-tuning is trained based on the edited portrait image and the corresponding inverse mapping parameters to obtain an optimized stylized 3D generative adversarial network.

9. The 3D dynamic portrait editing system based on fine-tuning 3D generative adversarial network according to claim 8, characterized in that: The module M1 includes: preprocessing the acquired portrait image, including: cropping and aligning the face of the acquired portrait image, and extracting camera parameters; The loss function in the module M1 includes: Among them, w id Indicates the identity information of the portrait; o n Indicates the action information of the portrait; c′ n Represents the camera parameters corresponding to the preprocessed image, G raw (w id +o n ,c n ) represents the generated sample image, I′ n represents the preprocessed input image, L lpips and λ lpips Represents the learnable perceptual image block similarity loss and the corresponding weight, L L2 represents the Euclidean distance loss, λ L2 and reg are the weights corresponding to balancing the Euclidean distance loss and the action information penalty loss, respectively.

10. The 3D dynamic portrait editing system based on fine-tuning 3D generative adversarial network according to claim 8, characterized in that: The module M2 includes: fine-tuning a personalized three-dimensional generative adversarial network based on a portrait facial image and corresponding inverse mapping parameters, and the loss function includes: （2） in, Representing personalized 3D generative adversarial networks; The module M3 includes: performing a second fine-tuning on the fine-tuned personalized three-dimensional generative adversarial network based on each portrait image frame in the portrait video and the corresponding inverse mapping parameters, and the loss function includes: （3） in, The camera parameters corresponding to the image without preprocessing, represents each portrait image frame in the portrait video, Representing personalized half-body 3D generative adversarial network; The module M4 renders a multi-view image of the identity portrait through a personalized three-dimensional generative adversarial network based on the inverse mapping identity information parameters, and edits the obtained portrait image using a diffusion model and corresponding text prompts to obtain an edited portrait image, including: Module M4.1: Based on the acquired portrait identity information parameters The personalized 3D generative adversarial network after the second fine-tuning is used to render the identity neural radiation field to obtain a multi-view portrait image; Module M4.2: Editing frontal portraits using diffusion models and editing multi-view portrait images using video stylization methods; in, represents the pre-trained text-guided image editing diffusion model, represents the 3D generative adversarial network to be edited and fine-tuned, Indicates the camera parameters corresponding to the frontal view; Represents a frontal perspective The corresponding unedited half-length portrait image, represents an edited frontal perspective rendering of a half-length portrait image; in, represents the edited image after the diffusion model and video stylization editing, Represents the corresponding perspective based on identity latent variable rendering images, represents the video stylization algorithm; The personalized three-dimensional generative adversarial network after the second fine-tuning is trained based on the edited portrait image and the corresponding inverse mapping parameters, and the loss function includes: （4） in, Represents the preset surround camera parameters, represents an image rendered based on the identity latent variable, Represents the edited image after diffusion model and video stylization editing.

Images