3D field speaker face generation defense method, system, device and medium

Abstract

The present application relates to the technical field of computer vision, artificial intelligence and multimedia security, in particular to a 3D field speaker face generation defense method, system, device and medium, which can effectively protect the reference video from the source, so that it cannot be used to generate high-quality speaker face video based on reinforcement learning of the adversarial interference method and system. By constructing a reinforcement learning environment, the reference video protection problem is modeled as a sequential decision process; the strategy network is used to learn the generation of adversarial interference actions, and a double-target reward function is designed to optimize the interference effect and visual concealment; finally, the deep reinforcement learning algorithm is iteratively trained to enable the model to autonomously master the optimal interference strategy.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, artificial intelligence and multimedia security technology, specifically to a 3D field speaking face generation defense method, system, device and medium. Background Technology

[0002] With the rapid development of generative AI technology, especially the maturity of deep generative models (such as GAN, Diffusion Model, NeRF, etc.), the technology of generating highly realistic and temporally coherent talking face videos based on a single reference image or a short video clip has become practical. This type of talking face generation technology (commonly known as Talking-Face Generation, TFG) has shown enormous application potential and social value in many fields such as film and television production, virtual digital humans, online education, and remote conferencing.

[0003] However, the double-edged sword effect of technology is particularly evident in this field. The widespread adoption and open-source nature of facial recognition technology has significantly lowered the barrier to malicious use, bringing with it serious security challenges and ethical risks. Attackers can easily generate video content containing statements that a target never made, using a publicly available video of that person. This allows them to impersonate others, commit online fraud, spread misinformation, and engage in defamation, posing a serious threat to personal privacy, social justice, and even national security.

[0004] Currently, defense technologies against the generation of illegally speaking facial videos can be mainly divided into two paths: The first category is post-event passive detection technology. This type of technology aims to develop high-precision classifiers or discrimination models to determine the authenticity of a given video. Although this type of method has achieved high accuracy on certain specific datasets, it has inherent limitations: (1) Lag: the defense action occurs after the fake video is generated and spread, and the damage may have already occurred; (2) Adversarial: with the continuous improvement of the quality of the generation model and the adversarial attacks (such as adversarial examples) launched by attackers against the detection model, the detection technology always faces an increasingly difficult challenge and it is difficult to form a reliable long-term defense; (3) Applicability: the detection model usually needs to be trained on a specific dataset, and its generalization ability may significantly decrease when applied to new generation methods or unknown identities.

[0005] The second category is proactive defense techniques. To overcome the drawbacks of passive detection, researchers began exploring interventions at the data source level (i.e., before the reference video is misused). The main idea is to embed tiny perturbations (i.e., adversarial examples) that are imperceptible to the human eye into the original reference video, preventing generative models trained or inferred from this "contaminated" video from outputting realistic results.

[0006] However, existing active defense methods all show significant inadequacy when facing different mainstream generation paradigms: For the generation method based on the diffusion model, its unique technical challenges are: (1) Target uncertainty: As a generative model, the diffusion model does not fit a definite and fixed target, but learns the mapping from noise to data distribution. The lack of a clear optimization target makes it difficult for traditional adversarial perturbation to find a stable and universal attack direction; (2) Lack of temporal consistency: The perturbation strategy based on single-frame images naturally lacks consideration and constraint on the temporal consistency of the video, making it difficult to produce a stable and coherent destructive effect in continuous video frames.

[0007] For the generation method based on 3D field, its unique technical challenges are: (1) Dispersed attack surface in multiple stages: The non-end-to-end, multi-stage generation process of 3D field model makes the attack surface dispersed in multiple relatively independent stages such as 3D reconstruction, rendering, and driving. Simple migration of disturbances is difficult to penetrate and effectively destroy the entire generation process; (2) Strong model specificity: The disturbance designed for a specific 3D representation or rendering architecture has weak generalization ability to other structures or parameters.

[0008] In addition, both paradigms face the following systemic defects: (1) It is difficult to balance concealment and effectiveness: the perturbation applied to achieve the interference effect is often too large, which leads to a decrease in the visual quality of the processed reference video and affects normal use; (2) Lack of a unified optimization framework: the inherently contradictory goals of "interference effectiveness" and "visual concealment" have not been systematically weighed and optimized under a unified framework; (3) Single defense target: existing methods are usually designed for specific types of generative models and lack universality for different generative paradigms.

[0009] It is evident that existing facial recognition technology can generate high-fidelity videos, posing risks of abuse and fraud. Existing defense technologies suffer from passive lag and insufficient generalization capabilities. The current technology lacks a proactive defense system specifically designed to target complex video generation models, automatically learn and generate efficient, covert, and universal interference signals, and systematically balance multiple defense targets. This severely restricts the development of proactive defense capabilities against deepfakes. Summary of the Invention

[0010] In view of this, the present invention provides a 3D field speaking face generation defense method, system, device and medium, which can effectively protect the reference video from the source and prevent it from being used to generate high-quality speaking face videos. This is a reinforcement learning-based adversarial interference method and system.

[0011] To achieve the objectives of this invention, the following technical solutions are provided.

[0012] A 3D field speaking face generation defense method is provided to protect video from being illegally used to generate high-quality speaking face videos. The method includes: Environment construction steps: Input the original video to be protected into the environment as a reference video. The state s of the environment is the feature extracted after preprocessing the current reference video. The feature includes one or more spatiotemporal visual features related to facial identity, expression, posture and motion. Interference application steps: The policy network generates an adversarial interference action a based on the current state s, adds the interference to the current reference video, and obtains a new reference video after interference; Video generation and evaluation steps: Input the new reference video into a pre-trained speaking face generation model to obtain the generated speaking face video; Reward calculation steps: Calculate the reward value r based on the generated speaking face video; the design goal of the reward value r is to make the quality of the generated video as poor as possible, while minimizing the visual difference between the new reference video after interference and the original reference video. Network training steps: Train the policy network using a deep reinforcement learning algorithm; through iterative optimization, enable the policy network to learn to generate effective perturbation patterns; The reward value r is calculated using one or a combination of the following two methods: (1) Calculated based on predefined evaluation indicators, including quality evaluation indicators and difference evaluation indicators; (2) Based on reward network calculation, the reward network is trained through supervised learning, and uses the reference video and the output of the corresponding speaking face generation model as input to predict the quality score of the generated video.

[0013] The calculation of the reward value r is configured as follows: The lower the quality assessment score of the generated speaking face video, the higher the reward value; Meanwhile, the lower the difference score between the new reference video after interference and the original reference video, the higher the reward value.

[0014] Specifically, when calculating the reward value using predefined evaluation metrics, the composition of the quality evaluation score is adjusted according to the target defense generation model type: When the defense target is a generative model based on a 3D field, the quality assessment score focuses on measuring the degree of failure of its reconstruction and driving capabilities, and consists of one or more combinations of the following indicators: perceptual quality indicators, pixel-level fidelity indicators, and temporal consistency indicators. When the defense target is a generative model based on a diffusion model, the quality assessment score focuses on measuring the degree of rigidity in its content generation, and consists of one or more of the following indicators: lack of lip-syncing, rigidity of facial expressions, and motion entropy.

[0015] Specifically, when calculating the reward value using predefined evaluation metrics, the difference score is composed of one or more combinations of the following metrics: pixel-level difference metrics and structural similarity metrics; when calculating the reward value using a reward network, the training method of the reward network includes: collecting a large number of reference videos and videos generated by a speaking face generation model; scoring the quality of the generated videos; training the reward network using the reference videos and generated videos as inputs and the corresponding quality scores as supervision signals; after training, the reward network can predict its quality score based on the input reference videos and generated videos.

[0016] The adversarial interference action a is a pixel-level perturbation applied to each frame of the reference video, or a feature-level perturbation applied to the video features.

[0017] This invention also proposes a 3D field speaking face generation defense system, capable of executing the 3D field speaking face generation defense method described in this invention, the system comprising: Video preprocessing module: used to extract spatiotemporal visual features related to facial identity, expression, posture and motion from reference videos to form environmental states; Policy network module: Receives state features and outputs adversarial interference actions; Interference application module: applies interference actions to the original reference video to generate a new protected reference video; Video generation module: Integrates a pre-trained speaking face generation model for evaluating the effect of interference; Reward calculation module: Calculates the reward based on the quality of the generated video and the distortion of the reference video. The reward calculation module is configured to perform one of the following two calculation methods: First calculation method: Calculate the reward value based on predefined quality assessment indicators and difference assessment indicators; The second calculation method is to calculate the reward value based on a reward network trained through supervised learning, wherein the reward network takes a reference video and a generated video of the speaking face as input and outputs a quality score. Training engine module: Used to train and optimize the policy network using reinforcement learning algorithms.

[0018] Wherein, when the reward calculation module adopts the first calculation method, it includes: The indicator calculation unit is used to calculate the predefined quality assessment indicators and difference assessment indicators; The reward fusion unit is used to fuse the calculation results of the indicators into a final reward value according to a preset weighting strategy.

[0019] When the reward calculation module uses the second calculation method, it includes: The reward network unit is the neural network trained through supervised learning; The reward fusion unit is used to convert the quality score predicted by the reward network into a final reward value.

[0020] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the 3D field speaking face generation defense method of the present invention.

[0021] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the present invention.

[0022] Beneficial effects First, this invention models the reference video protection problem as a sequential decision-making process by constructing a reinforcement learning environment; it utilizes a policy network to learn and generate adversarial interference actions, and designs a bi-objective reward function to simultaneously optimize interference effectiveness and visual concealment; finally, through iterative training using a deep reinforcement learning algorithm, the model autonomously masters the optimal interference strategy. This method, through environmental interaction mechanisms and multi-objective optimization strategies, effectively enhances the initiative and adaptability of defense, significantly improves the protection effect on reference videos, and can cope with various advanced speaking face generation models. It overcomes the problems of passive lag, insufficient generalization ability, and difficulty in dealing with high-quality video generation models found in existing speaking face generation defense technologies.

[0023] Secondly, this invention possesses strong robustness and practicality, capable of resisting common post-video processing attacks. During training, simulated attack steps such as image scaling, compression (e.g., JPEG encoding), and sanitization (e.g., latent space widening) are introduced, and corresponding loss functions are constructed and optimized to ensure that the embedded defense signal retains its interference effect even after the video has undergone compression, transcoding, or sanitization. This design makes the defense scheme more survivable in real network environments (such as social media video uploads and online streaming media transmissions), avoiding defense failures caused by video processing, and improving the reliability and long-term defense value of the technology.

[0024] Third, this invention significantly improves processing efficiency while ensuring defensive effectiveness, making it suitable for real-time or large-scale video processing scenarios. For noise generation in the Diffusion model, a recursive generation strategy based on the previous frame fine-tuning network is adopted, reducing redundant computation and resource consumption, thus achieving rapid noise generation. Furthermore, through temporal optimization and loss function design, the computation process is optimized in both the training and inference phases, enabling the entire defense framework to efficiently process long video sequences, reducing deployment costs, and providing a feasible proactive protection solution for real-time applications such as online education and remote conferencing. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a 3D field speaking face generation defense method according to an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of a 3D field speaking face generation defense system according to an embodiment of the present invention.

[0027] Figure 3 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0029] This invention proposes a 3D field-based defensive method for speaker face generation, implemented using reinforcement learning, to protect speaker videos from adversarial interference caused by the abuse of speaker face generation models. By constructing a closed-loop environment encompassing state feature extraction, perturbation action generation, video synthesis, and evaluation, a deep reinforcement learning algorithm is used to train an agent (policy network) to generate covert and destructive pixel-level perturbations. This results in a significant decrease in the output quality of speaker face generation models based on 3D fields (such as NeRF neural radiation fields), while ensuring that the perturbations are not visually noticeable in the original video. This method is applicable to various reinforcement learning frameworks, exhibiting high flexibility and adaptability.

[0030] The present invention provides a 3D field speaking face generation defense method as follows: Figure 1 As shown, it includes the following steps: Step S110, Environment Construction Step: The original video to be protected is used as the reference video input into the reinforcement learning environment. The state s of the environment is constructed, which is a high-dimensional feature vector composed of multiple visual features extracted from the reference video. These features should be able to fully represent the identity, expression, posture, and motion information of the face. As an exemplary and non-limiting implementation, the state vector can be composed of the following features concatenated: Coordinate data of facial key points detected from video frames (e.g., including but not limited to point sets of key parts such as eyes, nose, mouth, and facial contours); Features that characterize inter-frame motion (e.g., optical flow feature vectors calculated using optical flow algorithms); Parameters characterizing facial muscle movement (e.g., blendshape coefficients or action unit parameters obtained by fitting a 3D face model). Parameters characterizing the global motion of the head (e.g., the rotation and translation vectors of the head relative to the camera).

[0031] It is understood that the above feature list is exemplary. Those skilled in the art can select a subset of the above features, or add other visual features that can characterize facial attributes (such as depth map features, texture features, identity embedding features, etc.), as long as they serve the purpose of state construction. The specific dimensions and composition of the state vector should not be considered as a limitation of the present invention. These features together constitute a comprehensive digital description of the facial attributes of the current video frame.

[0032] Step S120, Interference Application Step: Policy Network ( The function receives the current state s as input and outputs an adversarial interference action a. The adversarial interference action a can be a pixel-level perturbation applied to each frame of the reference video (such as a small change in the RGB values ​​of an image), or a feature-level perturbation applied to the high-level features extracted in step S110. To ensure the concealment of the interference, the output range of action a is usually constrained within a finite interval (e.g., by using a tanh activation function to constrain the perturbation within a certain range). Action 'a' is applied to the current reference video, resulting in a new reference video after interference.

[0033] Step S130, Video Generation and Evaluation: The new reference video obtained in step S120 with added perturbation is input into a pre-trained 3D field-based speaking face generation model (e.g., a Neural Radiation Field (NeRF) model or a 3D Gaussian Sputtering model). This generation model will output the corresponding speaking face video based on the perturbated reference video.

[0034] Step S140, Reward Calculation Step: Based on the speaking face video generated in step S130, calculate the reward value r for this action. The design of the reward function is one of the core aspects of this invention, and its goal is to guide the policy network to learn effective interference patterns. Specifically, the calculation of the reward value r needs to satisfy two objectives: The quality of the generated speaking face video should be as poor as possible; This ensures that the visual difference between the new reference video after interference and the original reference video is as small as possible.

[0035] The reward value r is obtained by weighting and fusing the two scores above according to their contribution direction. By reasonably adjusting the weight coefficients, the relative importance of the two optimization objectives of "reducing the quality of generated videos" and "reducing visual differences" is better balanced.

[0036] In this basic embodiment, a predefined evaluation index is used to calculate the reward value r. Specifically, the reward r can be calculated using the following formula:

[0037] Among them, Quality_Score is the quality assessment score of the generated video, and the lower the score, the worse the quality; Difference_Score is the difference score of the reference video before and after interference, and the lower the score, the smaller the difference. and The weighting coefficients are used to balance the relative importance of the two objectives. The Quality_Score can be composed of a combination of perceptual quality metrics such as Learning Perceptual Image Patch Similarity (LPIPS) and Lip Keypoint Distance (LMD) and temporal consistency metrics. The Difference_Score can be composed of pixel-level or structural similarity metrics such as Mean Squared Error (MSE) and Structural Similarity Index (SSIM).

[0038] Step S150, Network Training Step: Train the policy network using a deep reinforcement learning algorithm. Collect a large number of data tuples on states, actions, rewards, and new states by repeatedly allowing the agent to interact in the constructed environment (execute steps S110-S140). And use this data to iteratively update the parameters of the policy network. The goal of training is to maximize the expected cumulative reward (i.e., to find the perturbation strategy that produces the worst generation quality and the minimum visual difference). During training, techniques such as experience replay, target networks, and gradient pruning are used to improve stability and convergence speed.

[0039] The present invention provides specific application examples of value function-based DQN, policy gradient-based PPO, maximum entropy-based SAC, and hierarchical reinforcement learning architectures to illustrate the method of the present invention.

[0040] Example 1: An Interference Method Based on Deep Q-Network (DQN) This embodiment details a specific implementation of the Deep Q-Network (DQN) algorithm. As a representative method for value function approximation, the DQN algorithm learns the optimal action value function through a neural network, thereby guiding the generation of interference strategies.

[0041] The implementation of the interference application step (S120) differs significantly from that of the policy search-based algorithm. The policy network, specifically the Q-network, functions to approximate the optimal action-value function. Action generation is not directly output by the policy network, but rather decided by querying the value estimation results of the Q-network. Specifically, in the current state s, the state vector s is input into the trained Q-network, which generates action vectors for each possible candidate action in the action space. A corresponding Q-value is calculated, representing the long-term value of performing the action. The agent then selects the final action from these candidate actions based on a certain explore-exploitation strategy (e.g., an ε-greedy strategy). During the testing or deployment phase, a greedy strategy is typically used, which directly selects the action with the highest Q-value as the output. The mathematical expression of this greedy strategy is as follows:

[0042] in, Let 'a' represent the state at the current time step 't', 'a' represent all candidate actions in the action space, and 'Q' be the action value function. These are the parameters of the Q-network. For this application, the action space needs to be designed in a discrete or parameterized form. For example, the intensity of pixel-level perturbations can be discretized into several fixed levels, or the perturbation direction can be discretized into a finite number of patterns, thus enabling DQN to process them.

[0043] In the network training step (S150), the deep Q-network algorithm is used to train the policy network. The policy network is the Q-network, and its function is to approximate the optimal action-value function. This function represents the expected cumulative reward obtained after performing action a in state s. The Q-network employs a deep neural network structure, specifically a multilayer perceptron. In an exemplary configuration, the network includes an input layer, at least one hidden layer, and an output layer. The dimension of the input layer matches the dimension of the state feature vector s; the hidden layers are typically composed of fully connected layers, and can be one or more, each containing several neurons, with ReLU (Rectified Linear Unit) activation functions to introduce nonlinear transformations; the dimension of the output layer is consistent with the dimension of the action space, and each output node corresponds to the estimated Q-value for performing a specific action a in a given state s.

[0044] The training process specifically includes an experience replay mechanism and a target network technique, and its specific operation is as follows: The specific implementation of the experience replay mechanism is as follows: the system maintains a fixed-capacity experience replay pool to store data tuples generated by the interaction between the agent and the environment. ,in Indicates the state at time t. Indicates the action to be performed. This indicates the immediate reward received. This represents the next state transitioned to after performing an action. Once the amount of data in the replay pool reaches a certain size, a small batch of data is randomly and uniformly sampled from the replay pool each time during training. This random sampling method helps break the temporal correlation between data points, making the training data closer to independent and identically distributed data, thereby significantly improving the training stability and efficiency of the neural network.

[0045] The specific implementation of the target network technology is as follows: In addition to the online Q-network (main network) used for real-time parameter updates, a target Q-network with the same structure but delayed parameter updates is maintained synchronously. The parameters of the target network are periodically (e.g., after every N updates of the main network parameters) hard-updated (completely replicated) from the online Q-network, or slowly synchronized using a soft-updating method (such as exponential moving average). When calculating the target value of Q-learning, the target network is used for estimation, and its calculation formula can be expressed as:

[0046] in, γ represents the immediate reward the agent receives after performing an action at time step t, and γ is a discount factor used to weigh the importance of current rewards against future rewards. This represents the next state that the agent transitions to after performing an action, where a' indicates the state in the next action. All possible actions, For the target Q-network, θ - These are the parameters of the target network. This method can fix the stability of the target value over a period of time, effectively mitigating training oscillations or divergence caused by frequent changes in the target value.

[0047] The specific process of network optimization is as follows: minimizing the predicted value of the online Q-network. With target value The temporal difference error between the two values ​​is used to update the network parameters θ. The loss function typically employs Mean Squared Error (MSE) or Huber loss to quantify the difference between the network's predicted and target values, and is optimized using gradient descent. Its definition is:

[0048] Where D represents the data distribution sampled from the experience replay pool, s represents the current state, a represents the action performed, r represents the immediate reward obtained, and s' represents the next state transitioned to. Target value The calculation formula is ,in θ' is the discount factor used to weigh the importance of current rewards against future rewards, and θ' is the parameter of the target network. The network parameters are updated using gradient descent. The optimizer can be Adam (Adaptive Moment Estimation) or RMSprop, and the learning rate is set to a small positive value (e.g., 0.001 or 0.0001) to ensure smooth convergence during training.

[0049] Through the synergistic effect of the aforementioned experience replay, target network, and gradient descent optimization, the Q-network can gradually learn parameters that approximate the real action value function, thereby ultimately obtaining a policy network that can generate effective anti-interference strategies based on the input video state.

[0050] Example 2: An interference method based on Proximity Policy Optimization (PPO) This embodiment details another implementation of the Proximal Policy Optimization (PPO) algorithm. As a representative advancement in policy gradient methods, the PPO algorithm demonstrates significant advantages in training stability and sample efficiency through innovations such as the introduction of pruning mechanisms.

[0051] The implementation of the interference application step (S120) differs fundamentally from the value function-based method. The policy network (often called the Actor network) directly parameterizes the policy function. In other words, it directly learns the mapping relationship from state to action. Specifically, the Actor network takes the state feature vector s as input, and its output layer is designed to parameterize the probability distribution of the action. For continuous action spaces (such as pixel-level perturbations), it is usually assumed that the action follows a Gaussian distribution. In this case, the network output contains two parts: one representing the distribution mean. A vector, and a vector representing the distribution variance. The vector (or logarithmic standard deviation) In actual action generation, the final action 'a' is obtained by sampling from this parameterized Gaussian distribution. The mathematical expression of this action sampling process is as follows:

[0052] Where s represents the current state input, The policy network represents the network based on the state s and parameters. The calculated mean of the action distribution, This represents the corresponding distribution variance. Let N be the parameters of the policy network, and let N represent a Gaussian distribution. This action selection mechanism based on distribution sampling naturally introduces randomness, which is beneficial for policy exploration during training. Furthermore, to ensure the concealment of interference, the sampled actions are usually truncated, constraining them to a finite range (e.g., ...). ),in This indicates the preset action boundary threshold.

[0053] In the reward calculation step (S140), the characteristics of the PPO algorithm require appropriate processing of the reward signal. Typically, a dominance function estimate is introduced to more accurately assess the relative merit of the action. Dominance function This represents the degree of advantage of performing action a relative to the average level in state s. It can be calculated using methods such as Generalized Advantage Estimation (GAE), and the bias and variance of the estimation are balanced by combining multi-step time-series difference error. Its basic form is:

[0054] in, Indicates the state at time step t. Indicates the state The action to be performed Here, γ is the index for future time steps, γ is the discount factor used to weigh the importance of future rewards, and λ is the GAE smoothing parameter used to balance bias and variance. For timing difference error, Indicates the state Execute action The instant reward obtained afterward Representing state The value estimate, Indicates the next state The value estimates are all calculated by another value network (Critic network). Through this advantage estimation, the PPO algorithm can more effectively evaluate the long-term value of actions, thereby improving the stability and efficiency of policy optimization.

[0055] In the network training step (S150), the Proximal Policy Optimization (PPO) algorithm is used to jointly train the Actor and Critic networks. Its core lies in designing an objective function that incorporates a pruning mechanism. This mechanism avoids drastic fluctuations during training by limiting the difference between the old and new policies, thus ensuring stability. Specifically, the PPO objective function consists of three parts: policy loss... Value function loss Entropy Rewards The total loss function is:

[0056] in, This represents the expectation of empirical data based on time-step sampling. This is the policy loss after pruning, used to constrain the magnitude of policy updates. The value function loss is used to optimize the accuracy of the Critic network's estimation of state values. The entropy reward is used to encourage exploration and prevent the policy from prematurely converging to a local optimum. These are the weighting coefficients for the loss in the value function. is the weighting coefficient of the entropy reward item.

[0057] Strategy Loss The specific implementation involves calculating the probability ratio between the old and new strategies, expressed as follows:

[0058] in, This represents a new policy with parameter θ in state θ. Select action The probability density, The parameter is The probability density of the old policy choosing the same action under the same state, where θ is the parameter of the current policy to be optimized. This is the policy parameter from the previous iteration. This probability ratio... It directly quantifies the degree of change in behavior distribution before and after the strategy update, and is the core input of the PPO pruning mechanism.

[0059] A pruned version of the objective function is constructed, which ensures training stability by limiting the step size of policy updates. This objective function is defined as follows:

[0060] Where θ represents the network parameters of the policy to be optimized. This represents the expectation based on the current sampled trajectory data. This represents the probability ratio between the old and new strategies. Indicates the state Next action The post-estimated advantage function value measures how well the action is performed relative to the average level. `clip` is the clipping function, which adjusts the probability ratio. Limited to [1- , 1+ Within the interval of ], The pruning range parameter (usually set to 0.1-0.2) controls the maximum magnitude of policy updates. This design forms a pessimistic estimate by taking the minimum value between the pruned target and the pruned target, thereby ensuring that the magnitude of policy updates is limited within a reasonable range and effectively preventing policy mutations during training.

[0061] Value function loss The specific implementation involves training the Critic network by minimizing the error between the predicted and target values ​​of the value network. This loss function is used to improve the accuracy of the value function estimation, and typically employs mean squared error loss.

[0062] in, Represents the state of a value network with parameter θ. Value prediction This represents the state obtained through target value calculation methods such as actual return or advantage function estimation. The value objective is defined by θ, which represents the parameters to be optimized in the value network. This loss function updates the parameters of the Critic network using gradient descent, aiming to optimize the network's predictions. It can more accurately approximate the true state value, thus providing a stable and reliable value benchmark for policy updates in Actor networks.

[0063] Entropy Rewards The specific implementation involves adding a policy entropy reward to the objective function to encourage exploration and prevent the policy from prematurely converging to a local optimum. This entropy reward term maintains exploration capability by quantifying the randomness of the policy, and its expression is:

[0064] Where S represents the entropy of the policy distribution, Indicates the state Below is the action probability distribution output by the policy network with parameter θ. This represents the expectation based on the sampled state distribution. Policy entropy. The larger the value of the discrete action space (or the corresponding differential entropy (continuous action space)), the stronger the randomness and exploratory nature of the strategy. By introducing this reward, we can effectively prevent the strategy from converging prematurely to a single deterministic behavior, thereby promoting the exploration of better strategies.

[0065] The network is trained using gradient descent for optimization, with Adam being a suitable optimizer. The learning rate is typically set to a small positive value (e.g., 0.0003). Through multiple iterative updates, the policy network learns adversarial strategies that effectively reduce the quality of generated videos while maintaining the stealth of the interference.

[0066] Example 3: An interference method based on soft actor-critic (SAC) This embodiment details the implementation of the Soft Actor-Critic (SAC) algorithm within the maximum entropy reinforcement learning framework. As a representative method of maximum entropy reinforcement learning, the SAC algorithm enhances the algorithm's exploration capability and robustness by introducing a policy entropy term into the objective function. This allows the algorithm to pursue high cumulative rewards while maintaining policy randomness.

[0067] In the disturbance application step (S120), the implementation combines the advantages of value function approximation and policy search. The policy network (called the actor network) directly learns a stochastic policy that aims to maximize not only the expected cumulative reward but also the policy entropy. Specifically, the actor network takes the state feature vector s as input and outputs the probability distribution parameters of the actions. For continuous action spaces, a Gaussian distribution is typically used for modeling, and the network output includes the mean. and standard deviation Unlike PPO, SAC uses reparameterization techniques to generate actions:

[0068] in, This represents the state input at the current time step t. The policy network represents the state. The mean of the action distribution calculated with parameter θ, Let θ represent the standard deviation of the corresponding distribution, and θ be the parameter to be optimized in the policy network. To obtain from the standard Gaussian distribution Random noise vector sampled in the middle, The identity matrix is ​​used. This mechanism allows actions to have both deterministic components (determined by the mean) and a certain degree of randomness (determined by the standard deviation). Furthermore, by shifting the source of randomness to the input noise ξ, the actions... The gradient with respect to the network parameter θ can be directly calculated, thus allowing the error to propagate effectively back through the network.

[0069] In the network training step (S150), the soft actor-critic algorithm is used for end-to-end training. The SAC algorithm maintains three key networks simultaneously: the actor network (policy network), the soft Q-value network (critic network), and the soft state value network. Its training process is achieved by alternately optimizing these networks.

[0070] The specific update method for the soft Q-value network is as follows: SAC contains two independent soft Q-value networks ( and The target Q-network is updated by minimizing the soft Bellman error. The objective value of the soft Q-function is calculated by comprehensively considering the contributions of immediate reward, discounted state value, and policy entropy, and its expression is:

[0071] in, The target value of the soft Q function, Indicates the state Execute action The instant reward obtained afterward This is a discount factor used to weigh the importance of future returns. This indicates the target value network's response to the next state. The value estimation, where α is a temperature parameter used to control the relative weight of the entropy regularization term, Indicates the target policy in state Select action The probability density of the Q-network. The loss function of the Q-network measures the difference between the current Q-value estimate and the target value, and improves the accuracy of the Q-value estimate by minimizing this error. It is defined as:

[0072] in, The parameter is The loss value of the Q network, Let represent the expected distribution of data sampled from the experience replay pool D, s represent the current state, a represent the action performed in the current state, r represent the immediate reward obtained after performing the action, and s' represent the next state to transition to. The parameter is Q-network for state-action pairs The value estimate, The target Q-value is calculated based on the soft Bellman equation. This loss function updates the parameters of the two Q-networks by minimizing the mean squared error between the current Q-value estimate and the target value. and This dual-Q network design helps reduce the problem of overvaluation.

[0073] The specific update method for the soft-state value network is as follows: the value network is trained by minimizing a loss function. This loss function aims to ensure that the predictions of the value network are consistent with the expected soft Q-value based on the current policy, and is defined as follows:

[0074] in, This represents the loss function of a value network with parameter ψ. This represents the state sampled from the experience replay pool D. Expectations Representing the value network in relation to state Value prediction Indicates according to the current strategy In state downsampling action Expectations This represents the state-action pairs in two soft Q networks. A smaller Q-value estimate, where α is the temperature parameter. For strategy Next action The logarithmic probability. This loss function is obtained by adjusting the predicted value of the value network. To improve the accuracy of state value estimation, we approximate the expected target value calculated based on the current policy and the double-Q network.

[0075] The specific update method for the actor network is as follows: the policy network is optimized by minimizing the KL divergence. This loss function aims to align the policy distribution with the distribution guided by the exponential dominance function, and its expression is:

[0076] in, Let θ represent the loss function of a policy network with parameter θ. This represents the state sampled from the experience replay pool D. Expectations (P‖Q) represents the KL divergence, used to measure the difference between probability distributions P and Q. Indicates the policy network in state The output action probability distribution, where α is the temperature parameter. Indicates the state of the two soft Q networks. The minimum Q value for each action is given below. The partition function is used to normalize the distribution of exponentially increased Q-values. This loss function minimizes the KL divergence between the current policy distribution and the energy basis distribution composed of exponentially increased Q-values, making the policy more inclined to choose actions with higher Q-values, while maintaining a certain degree of randomness to achieve efficient exploration.

[0077] In practice, reparameterized gradient estimation is typically employed, updating the policy by maximizing the following objective function. This objective function aims to increase action value while maintaining the stochasticity of the policy, and its expression is:

[0078] in, This represents the optimization objective function of the policy network. This indicates that the state is sampled from the experience replay pool D. and through strategy Sampling action The joint expectations This represents the state-action pairs in two soft Q networks. A smaller Q-value estimate is used to reduce the overestimation bias, and α is a temperature parameter used to control the weight of the entropy regularization term. For the policy network in state Select action The objective function improves policy performance by maximizing the expected Q-value, while also utilizing the entropy term. This encourages exploration and prevents strategies from converging prematurely to local optima.

[0079] The specific implementation of automatic temperature parameter adjustment is as follows: SAC introduces an optimizable temperature parameter α, which is automatically adjusted by minimizing the following loss function. This loss function aims to maintain the policy entropy near the target threshold, and its expression is:

[0080] Where L(α) represents the loss function of the temperature parameter α, Indicates according to the current strategy In state downsampling action The expected value, where α is an automatically adjustable temperature parameter. Representation strategy In state Select action The logarithmic probability, The target entropy value is typically set to a negative action dimension (e.g., ...). This loss function constrains the actual entropy value of the current policy. Approaching the target entropy This allows the algorithm to adaptively adjust the exploration intensity: when the policy entropy is lower than the target entropy, the loss function pushes α down to reduce the entropy weight; when the policy entropy is higher than the target entropy, it pushes α up to strengthen entropy regularization. This automatic adjustment mechanism enables the algorithm to adaptively balance the trade-off between reward maximization and exploration.

[0081] During training, an experience replay mechanism is used to store interactive data, and stochastic gradient descent is used to alternately optimize the network. The optimizer can be Adam, and the learning rate is typically set to a small positive value (e.g., 0.0003). By introducing a maximum entropy framework, SAC can learn more exploratory and robust perturbation policies, exhibiting superior performance in complex environments.

[0082] Example 4: An Interference Method Based on Hierarchical Reinforcement Learning (HRL) This embodiment details the implementation using a Hierarchical Reinforcement Learning (HRL) framework, specifically designed for the defense needs of long video sequences. This method effectively addresses the credit allocation problem and non-stationarity challenges in long sequence decision-making by introducing differentiation in time scale and decision hierarchy.

[0083] In the environment construction step (S110), a hierarchical state representation method is used to decompose the complete environment state into two levels: global state and local state. Global state It contains long-term spatiotemporal features spanning multiple frames, constructed by aggregating information such as facial keypoint trajectories, optical flow statistics, and head motion trends from N consecutive frames (e.g., N=10), and is used to characterize the overall properties of video segments. Local state It focuses on the visual features of a single frame or a short window (such as three consecutive frames), including facial key points, expression parameters, and instantaneous motion features of the current frame, providing fine-grained information for frame-level decision-making.

[0084] In the interference application step (S120), a two-level policy network architecture is adopted, corresponding to different decision time scales. The higher-level policy (Manager) makes a decision every K frames (e.g., K=10) and outputs an abstract global interference command. This instruction can be represented as a encoded vector of the target interference region (such as the mouth or eyes), interference intensity level, or interference type. The lower-level policy (Worker) receives instructions from higher levels. and the current local state Generate specific frame-level interference actions The interaction between the two-level strategies is achieved through an internal reward mechanism, with the goal of the lower-level strategy being to achieve the sub-goals specified by the higher-level instructions.

[0085] The specific implementation of the high-level strategy is as follows: High-level strategy network global state As input, output global interference command This instruction is represented through a target embedding space, whose update frequency is much lower than that of the underlying policy (e.g., once every 10 frames). The value function of the high-level policy is used to evaluate the long-term impact of the global state, and is defined as:

[0086] in, Indicates the global state The high-level state value function is defined as follows: E represents the expectation operator, γ is the discount factor used to measure the current value of future rewards, K is the high-level decision interval (i.e., the high-level policy executes a decision only once every K time steps), and i is the index of the high-level decision step. Indicates the first The value function considers the cumulative discounted returns of the high-level rewards obtained at K-step intervals, allowing high-level strategies to focus on long-term strategic planning rather than instantaneous tactical responses.

[0087] The specific implementation of the underlying policy is as follows: Underlying Policy Network By high-level instructions and local state Input, output frame-level interference action The goal of the underlying policy is to maximize the accumulated intrinsic reward under the guidance of higher-level instructions. The value function, used to evaluate the short-term performance of the underlying policy given a higher-level instruction, is defined as follows:

[0088] in, Indicates local state and high-level instructions The underlying state value function is defined by E, which represents the expectation operator, γ, which is a discount factor used to weigh the importance of future immediate rewards, K, which is the high-level decision interval and determines the execution cycle of the underlying policy, and k is the index of the underlying time step. This represents the underlying internal reward obtained at time step t+k. This internal reward... It typically consists of two parts: first, external rewards provided by the environment; and second, measures used to evaluate lower-level actions and higher-level instructions. Consistent internal rewards. Through this value function, the underlying strategy can focus on precise execution results in the short term, under the macro-guidance of higher-level instructions.

[0089] In the reward calculation step (S140), a hierarchical reward design is adopted, comprising two components: global rewards and local rewards. Global rewards... The overall interference effect over a long period is evaluated, such as the average quality degradation and temporal consistency of video segments. The reward function quantifies the global interference performance by weightedly combining multiple quality metrics; its expression is:

[0090] Local rewards The evaluation considers the impact of interference within a single frame or short time window, including immediate quality degradation and the degree of matching with higher-level instructions. The reward function guides the optimization of the underlying strategy by comprehensively considering the current interference effect and instruction follow-up; its expression is:

[0091] Among them, Alignment_Score measures the consistency between the underlying actions and the higher-level instructions, and can be calculated through cosine similarity or a specially designed matching network.

[0092] In the network training step (S150), a hierarchical reinforcement learning algorithm, such as Option-Critic or HIRO, is used to co-train the two-level policies. The training process consists of two stages: first, the high-level policy is fixed, and the low-level policy is trained to implement the given instructions; then, the low-level policy is fixed, and the high-level policy is trained to generate the optimal instruction sequence. Finally, the co-convergence of the two-level policies is achieved through alternating optimization.

[0093] The specific implementation of high-level policy training is as follows: the high-level policy parameters are updated by maximizing the expected cumulative global reward. The policy gradient formula guides the high-level policy to update in the direction of increasing the global reward through the advantage function; its expression is:

[0094] in, Describe the objective function of the high-level strategy. Regarding strategy parameters The gradient of E represents the expectation based on high-level empirical sampling. Indicates the high-level strategy in the global state Select global command below The probability, Let the logarithmic gradient of this probability be... This represents the high-level advantage function, used to evaluate the global state. Execute command The degree of advantage relative to the average level is typically calculated using generalized advantage estimation (GAE). This gradient update formula weights policy updates using an advantage function, thereby increasing the probability of selecting high-reward instructions while decreasing the probability of selecting low-reward instructions, thus effectively optimizing the decision quality of high-level policies.

[0095] The specific implementation of the underlying policy training is as follows: the underlying policy parameters are updated by maximizing the internal reward and instruction matching degree. The policy gradient formula guides the optimization direction of the underlying policy through the local dominance function, and its expression is:

[0096] in, Represents the underlying policy objective function Regarding strategy parameters The gradient of E represents the expectation based on the underlying empirical sampling. This indicates that the underlying strategy is in a local state. The probability of choosing action a under the condition of higher-level instruction g. This is the logarithmic gradient of the conditional probability. This represents the underlying advantage function, used to comprehensively evaluate the local state given a higher-level instruction g. The advantage of performing action 'a' relative to the average level. This advantage function considers both the immediate rewards provided by the environment and the alignment between the underlying action and the higher-level instruction 'g', ensuring that the underlying policy can effectively execute the macro-level instructions issued by the higher level while pursuing short-term returns.

[0097] During training, a hierarchical experience replay mechanism is employed to store global and local interaction data separately, and different discount factors are used to balance short-term and long-term objectives. Through this hierarchical architecture, the system can simultaneously optimize the overall defense effect of long video sequences and the accuracy of single-frame interference, significantly improving its defense capabilities against complex video content.

[0098] In summary, the present invention demonstrates the flexibility and wide applicability of its technical solutions through the above-described embodiments. It should be understood that the value function-based DQN, policy gradient-based PPO, maximum entropy-based SAC, and hierarchical reinforcement learning architecture listed are merely exemplary solutions for implementing the core concept of the present invention, and are not intended to limit the scope of protection of the present invention.

[0099] Based on the foregoing embodiments, the reward calculation of the present invention can be specifically configured according to the paradigm of the target generation model in order to achieve effective defense against different types of generation models.

[0100] When defending against generative models based on 3D fields, reward calculations need to guide the agent to disrupt its 3D reconstruction and rendering process. In this case, the quality assessment score should focus on measuring the degree of degradation in the generated video in terms of structural integrity, texture realism, and temporal consistency. Specifically, one or more combinations of perceptual quality metrics, pixel-level fidelity metrics, and temporal consistency metrics can be used to quantify the degree of failure in model reconstruction and driving capabilities, guiding the agent to generate interference patterns that effectively cause blurring, distortion, or flickering in the output video.

[0101] When defending against generative models based on diffusion models, reward calculations should guide the agent to make its generated content more rigid. In this case, quality assessment scores should focus on measuring the lack of lip-sync, facial expression richness, and motion diversity in the generated video. Specifically, indicators such as lip-sync scarcity, facial expression rigidity, and motion entropy can be used to quantify the rigidity of the model's content generation, guiding the agent to generate interfering patterns that effectively "freeze" the model's output, preventing it from producing vivid lip-sync and facial expressions that match the audio driver.

[0102] It is understandable that the reward calculation strategies for different paradigms mentioned above can be combined with any of the aforementioned reinforcement learning architectures to form a complete defense scheme.

[0103] The core of this invention lies in the fundamental concept of generating adversarial interference through reinforcement learning closed loops. Inspired by this concept, those skilled in the art can readily apply it to other reinforcement learning algorithm frameworks, such as, but not limited to, Asynchronous Advantage Actor-Commentator (A3C), Deep Deterministic Policy Gradient (DDPG), and Dual-Delay Deep Deterministic Policy Gradient (TD3), or make adaptive adjustments to the network structure and reward function design in the above embodiments. Any application of the basic principle of this invention—using reinforcement learning to generate interference to defend against 3D field speaking face generation models—should fall within the protection scope of this invention.

[0104] This invention also provides a 3D field speaking face generation defense system, capable of executing any of the defense methods described above, the system comprising: The video preprocessing module 1 is used to extract spatiotemporal visual features related to facial identity, expression, posture, and motion from the input reference video to be protected, and to construct a state vector s representing the current environmental state. Specifically, this module integrates a computer vision feature extraction unit, which can automatically extract and fuse various visual features representing facial attributes and dynamics from video frames, realizing the transformation from raw video frames to standardized state vectors.

[0105] The policy network module 2, connected to the video preprocessing module 1, receives the state feature vector s and outputs adversarial interference action a. This module is the core of the defense strategy and has a built-in trained policy network (such as a deep neural network) that can generate effective pixel-level or feature-level perturbations based on the current video state.

[0106] The interference application module 3, connected to the policy network module 2, applies the adversarial interference action a to the original reference video to generate a new protected reference video. This module includes a perturbation fusion unit, capable of applying perturbations to the video data in a superimposed or transformed manner, ensuring that the processed video format meets the input requirements of the downstream model.

[0107] The video generation module 4, connected to the interference application module 3, is used to input the protected new reference video into a pre-trained 3D field-based speaking face generation model (e.g., a model based on neural radiation fields or 3D Gaussian sputtering) and obtain the generated speaking face video. This module encapsulates the interaction interface with the complex generation model and is responsible for data transmission and triggering the synthesis process.

[0108] The reward calculation module 5, connected to the video generation module 4, is used to calculate the reward value r based on the quality of the generated speaking face video and the distortion of the new reference video relative to the original video. It includes an index calculation unit 51, a reward network unit 52, and a reward fusion unit 53. The reward calculation module 5 is a key feedback loop in the reinforcement learning training closed loop and is configured to support at least two selectable reward calculation strategies. These strategies include a strategy based on predefined indices and a strategy based on a reward network. Specifically, the strategy based on predefined indices involves calling the index calculation unit 51 to calculate predefined quality assessment indices (such as perceptual quality and lip-sync) and difference assessment indices (such as structural similarity and pixel error), and fusing them into the final reward value r according to a preset weighting formula. Specifically, the strategy based on a reward network involves calling the reward network unit 52, a neural network trained through supervised learning and interference application. This network can directly predict a comprehensive quality score based on the input reference video and the generated video, using this score as the primary basis for the reward value r, while also allowing for fine-tuning by combining simple difference indices. The reward fusion unit 53 is used to manage system configuration, and according to the selected reward calculation strategy, it routes and processes the output of the corresponding unit to generate the final reward value r.

[0109] The training engine module 6, connected to both the policy network module 2 and the reward calculation module 5, is used to train and optimize the network parameters in the policy network module 2 using reinforcement learning algorithms. This module is the "brain" of the system, responsible for driving the entire learning process. It integrates training mechanisms such as experience replay, gradient descent optimization, and target network update, and supports the implementation of various reinforcement learning algorithms such as DQN, PPO, and SAC. Figure 2 This is a schematic diagram of a 3D field speaking face generation defense system according to an embodiment of the present invention.

[0110] Furthermore, the policy network module 2 can be configured to support multiple network architectures, such as fully connected networks or convolutional networks, and constrain the perturbation range through specific output layer designs (such as using activation functions like tanh).

[0111] Furthermore, the reward fusion unit 53 in the reward calculation module 5 can be configured to support flexible reward function customization, allowing the weight coefficients of the quality assessment index and the difference assessment index to be adjusted according to different defense objectives, or conditions to select the index calculation result or the reward network prediction result as the final reward under specific circumstances.

[0112] Furthermore, the training engine module 6 can be configured to support modular algorithm plugins, facilitating the integration of different reinforcement learning algorithms, and includes functions such as training status monitoring, performance evaluation, and model saving to ensure the reliability and reproducibility of the training process.

[0113] Those skilled in the art will understand that the modules in the above embodiments can be distributed in the system as described, or they can be modified accordingly and distributed in one or more systems different from the above embodiments. The modules in the above embodiments can be combined into one, or they can be further divided into multiple.

[0114] This application also provides an electronic device. Figure 3 The structure of an electronic device provided in an embodiment of the present invention is illustrated. For example, the electronic device 30 may include a processor 31, a memory 32, and a transmission device 33. The processor is used to execute the 3D field speaking face generation defense method mentioned in the above embodiments. The processor and the memory can be connected via a bus or other means, taking a bus connection as an example. The transmission device can be connected to the processor and the memory via wired or wireless means. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and program instructions corresponding to the 3D field speaking face generation defense method in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs and instructions stored in the memory, thereby implementing the 3D field speaking face generation defense method in the above method embodiments. The memory may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created by the processor, etc. In addition, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, and this remote memory may be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. One or more of these memories are stored in the memory and, when executed by the processor, perform the 3D field speaking face generation defense method of the embodiments.

[0115] In another aspect, this application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus described above; or it may be a standalone computer-readable storage medium not assembled into the device. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the 3D field speaking face generation defense method described in this application.

[0116] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0117] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A 3D field speaking face generation defense method, characterized in that, The method for protecting video from being illegally used to generate high-quality speaking face videos includes: Environment construction steps: Input the original video to be protected into the environment as a reference video. The state s of the environment is the feature extracted after preprocessing the current reference video. The feature includes one or more spatiotemporal visual features related to facial identity, expression, posture and motion. Interference application steps: The policy network generates an adversarial interference action a based on the current state s, adds the interference to the current reference video, and obtains a new reference video after interference; Video generation and evaluation steps: Input the new reference video into a pre-trained speaking face generation model to obtain the generated speaking face video; Reward calculation steps: Calculate the reward value r based on the generated speaking face video; the calculation of the reward value r is configured as follows: The lower the quality assessment score of the generated speaking face video, the higher the reward value; Meanwhile, the lower the difference score between the new reference video after interference and the original reference video, the higher the reward value; Network training steps: Train the policy network using a deep reinforcement learning algorithm; through iterative optimization, enable the policy network to learn to generate effective perturbation patterns; The reward value r is calculated using one or a combination of the following two methods: (1) Calculated based on predefined evaluation indicators, including quality evaluation indicators and difference evaluation indicators; (2) Based on reward network calculation, the reward network is trained through supervised learning, and uses the reference video and the output of the corresponding speaking face generation model as input to predict the quality score of the generated video.

2. The method according to claim 1, characterized in that, When calculating the reward value using predefined evaluation metrics, the composition of the quality evaluation score is adjusted according to the generative model type of the target defense: When the defense target is a generative model based on a 3D field, the quality assessment score focuses on measuring the degree of failure of its reconstruction and driving capabilities, and consists of one or more combinations of the following indicators: perceptual quality indicators, pixel-level fidelity indicators, and temporal consistency indicators. When the defense target is a generative model based on a diffusion model, the quality assessment score focuses on measuring the degree of rigidity in its content generation, and consists of one or more of the following indicators: lack of lip-syncing, rigidity of facial expressions, and motion entropy.

3. The method according to claim 1, characterized in that, When calculating the reward value using predefined evaluation metrics, the difference score is composed of one or more combinations of the following metrics: pixel-level difference metrics and structural similarity metrics; When a reward network is used to calculate the reward value, the training method of the reward network includes: collecting a large number of reference videos and videos generated by a speaking face generation model; scoring the quality of the generated videos; training the reward network with the reference videos and generated videos as inputs and the corresponding quality scores as supervision signals; after training, the reward network can predict the quality score based on the input reference videos and generated videos.

4. The method according to any one of claims 1-3, characterized in that, The adversarial interference action a is a pixel-level perturbation applied to each frame of the reference video, or a feature-level perturbation applied to the video features.

5. A 3D field speaking face generation defense system, characterized in that, The system is capable of executing the 3D field speaking face generation defense method according to any one of claims 1 to 4, the system comprising: Video preprocessing module: used to extract spatiotemporal visual features related to facial identity, expression, posture and motion from reference videos to form environmental states; Policy network module: Receives state features and outputs adversarial interference actions; Interference application module: applies interference actions to the original reference video to generate a new protected reference video; Video generation module: Integrates a pre-trained speaking face generation model for evaluating the effect of interference; Reward calculation module: Calculates the reward based on the quality of the generated video and the distortion of the reference video. The reward calculation module is configured to perform one of the following two calculation methods: First calculation method: Calculate the reward value based on predefined quality assessment indicators and difference assessment indicators; The second calculation method is to calculate the reward value based on a reward network trained through supervised learning, wherein the reward network takes a reference video and a generated video of the speaking face as input and outputs a quality score. Training engine module: Used to train and optimize the policy network using reinforcement learning algorithms.

6. The system according to claim 5, characterized in that, When the reward calculation module uses the first calculation method, it includes: The indicator calculation unit is used to calculate the predefined quality assessment indicators and difference assessment indicators; The reward fusion unit is used to fuse the calculation results of the indicators into a final reward value according to a preset weighting strategy.

7. The system according to claim 5 or 6, characterized in that, When the reward calculation module uses the second calculation method, it includes: The reward network unit is the neural network trained through supervised learning; The reward fusion unit is used to convert the quality score predicted by the reward network into a final reward value.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the 3D field speaking face generation defense method as described in any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 4.

Images