A detection model generation method and system

By acquiring and processing rich visual depth forged data sets, performing multiple feature extraction and feature selection, and combining preset strategies to train detection models, the problems of insufficient sample data and poor adaptability of application scenarios in the existing technology are solved, and high-precision forged content detection is achieved.

CN119273929BActive Publication Date: 2025-04-29BEIJING YOUKUN TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411794849.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-06
Publication Date
2025-04-29
Estimated Expiration
2044-12-06

AI Technical Summary

Technical Problem

The existing forged detection technology lacks rich sample data, especially in-depth forged data for different application scenarios, which limits the generalization ability and detection effect of the model. The existing technology has different performances in different application scenarios, and lacks fine-tuning and optimization for specific scenarios, resulting in poor detection results.

Method used

Feature extraction, feature selection and model training are performed by obtaining the target data set, including visual depth forgery data and real data. Feature extraction uses a variety of features, such as image features, deep pseudo-video defects, deep pseudo-image defects, image digital features and video digital features. Feature selection obtains the most important feature set through methods such as variance screening, correlation analysis and recursive feature elimination. Then use the preset loss function, optimizer and model parameters to train the detection model in combination with the preset strategy.

Benefits of technology

Through the comprehensive feature extraction and analysis of deep forged data, combined with rich sample training and fine-tuning for different application scenarios, high-precision detection of forged content is achieved, and the generalization ability and detection effect of the model are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119273929B_ABST
    Figure CN119273929B_ABST
Patent Text Reader

Abstract

The present application discloses a detection model generation method and system, which obtains a target data set, the target data set includes visual deep fake data and real data, the visual deep fake data includes corresponding annotated data attribute information and fake operation information; performs feature extraction on the target data set to obtain a first feature set; performs feature selection on the first feature set to obtain a second feature set; based on the second feature set, uses a preset loss function, a preset optimizer and preset model parameters, combined with a preset strategy training to obtain a detection model for detecting visual deep fake content, the preset strategy is used to characterize that within a preset number of training cycles, the model performance improvement is less than a preset threshold, and training is stopped. Through the comprehensive feature extraction and analysis of deep fake data, combined with rich sample training and fine-tuning for different application scenarios, high-precision detection of fake content is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a detection model generation method and system. Background Art

[0002] Existing forgery detection technologies often lack rich sample data, especially deep forgery data for different application scenarios, which limits the generalization ability of the model and the detection effect. Existing forgery detection technologies perform differently in different application scenarios and lack fine-tuning and optimization for specific scenarios, resulting in poor results in practical applications. Existing forgery detection technologies usually rely on only a single feature analysis method, such as image features or deep learning features, and fail to fully utilize the combined advantages of multiple features, affecting the accuracy and robustness of detection. Summary of the invention

[0003] Based on the above problems, the present application provides a detection model generation method and system.

[0004] The embodiments of the present application disclose the following technical solutions:

[0005] In a first aspect, a detection model generation method is provided, the method comprising:

[0006] Acquire a target data set, wherein the target data set includes visual deep fake data and real data, wherein the visual deep fake data includes corresponding annotated data attribute information and fake operation information;

[0007] Performing feature extraction on the target data set to obtain a first feature set, wherein the first feature set includes a result obtained by performing feature extraction on the target data set based on at least one of an image feature, a preset neural network, a deep fake video defect, a deep fake image defect, an image digital feature, and a video digital feature;

[0008] Performing feature selection on the first feature set to obtain a second feature set;

[0009] Based on the second feature set, a preset loss function, a preset optimizer, and preset model parameters are used in combination with a preset strategy to obtain a detection model for detecting visual deep fake content. The preset strategy is used to stop training when the improvement in model performance is less than a preset threshold within a preset number of training cycles.

[0010] In a possible implementation manner, obtaining the target data set includes:

[0011] Obtain real data and fake data samples from the target data channel;

[0012] Generate visual deep fake data based on the acquired fake data samples;

[0013] Combine the acquired real data and the generated visual deep fake data to obtain a preliminary data set;

[0014] Data enhancement operations and data filtering operations are sequentially performed on the prepared data set to obtain a target data set, wherein the data enhancement operation includes performing at least one of rotation, scaling, flipping, cropping, color conversion, and noise addition on the data samples in the prepared data set; and the data filtering operation includes screening the data samples in the target data set based on at least one of statistical analysis, cluster analysis, or machine learning algorithms to identify and exclude data samples that do not meet preset quality standards.

[0015] In a possible implementation manner, performing feature selection on the first feature set to obtain a second feature set includes:

[0016] Based on variance screening and correlation analysis, the features in the first feature set whose importance does not meet the preset importance standard are eliminated; the features in the first feature set are subjected to dimensionality reduction processing; based on recursive feature elimination, the features in the first feature set are eliminated to obtain a second feature set.

[0017] In a possible implementation, the first feature set includes time domain features, spatial domain features, and frequency domain features obtained by extracting features based on image features, and extracting features from the target data set to obtain the first feature set includes:

[0018] The time domain characteristics of the target data set are determined by analyzing the pixel-level difference information between consecutive frames, the motion pattern and motion law of the object; the spatial domain characteristics of the target data set are determined by analyzing the texture information and edge information of the image; the images in the target data set are converted into the frequency domain, the spectral characteristics and multi-scale characteristics of the images in the target data set are analyzed, and the frequency domain characteristics of the target data set are determined.

[0019] In a possible implementation, the first feature set includes facial features and object movement features obtained by extracting features of deep fake video defects, and the feature extraction of the target data set to obtain the first feature set includes:

[0020] Based on the consistency of blinking frequency, head movement information, facial expression and context information, the facial features of the target data set are determined; based on the object movement trajectory and object occlusion transition information, the object movement features are determined.

[0021] In a possible implementation, the first feature set includes a splicing trace feature and a copy-paste trace feature obtained by extracting features based on deep fake image defects, and extracting features from the target data set to obtain the first feature set includes:

[0022] Based on whether the colors of different areas of the image meet the preset consistency conditions, whether the textures of different areas of the image meet the preset consistency conditions, whether the lighting of different areas of the image meets the preset consistency conditions, and the geometric transformation trace information in the image, the splicing trace characteristics of the target data set are determined; and whether there are repeated texture areas in the image is detected to determine the copy and paste trace characteristics of the target data set.

[0023] In a possible implementation, the first feature set includes camera features, compression trace features, and metadata features extracted based on image digital features, and the feature extraction of the target data set to obtain the first feature set includes:

[0024] Based on sensor noise information in the image and whether the image has lens distortion, the camera characteristics of the target data set are determined; based on quantization error and block effect in the image, the compression trace characteristics of the target data set are determined; based on metadata information of the image or video, the metadata characteristics of the target data set are determined.

[0025] In a possible implementation, the first feature set includes a frame rate feature, a coding format feature, and a timestamp feature extracted based on a video digital feature, and the feature extraction of the target data set to obtain the first feature set includes:

[0026] Determine whether the video frame rate meets the preset stability condition and determine the frame rate characteristics of the target data set; determine the encoding format characteristics of the target data set based on the encoding format of the video; determine whether the timestamp of the video meets the preset continuity condition and determine the timestamp characteristics of the target data set.

[0027] In a possible implementation, the method further includes:

[0028] Detection scenario requirement information is obtained, and model parameters of the detection model are adjusted based on the detection scenario requirement information, wherein the model parameters are used to balance the false alarm rate and the false negative rate of the detection model.

[0029] In a second aspect, a detection model generation system is provided, comprising:

[0030] An acquisition unit, configured to acquire a target data set, wherein the target data set includes visual deep fake data and real data, wherein the visual deep fake data includes correspondingly annotated data attribute information and fake operation information;

[0031] an extraction unit, configured to perform feature extraction on the target data set to obtain a first feature set, wherein the first feature set includes a result obtained by performing feature extraction on the target data set based on at least one of an image feature, a preset neural network, a deep fake video defect, a deep fake image defect, an image digital feature, and a video digital feature;

[0032] A selection unit, configured to perform feature selection on the first feature set to obtain a second feature set;

[0033] A training unit is used to obtain a detection model for detecting visual deep fake content based on a second feature set, using a preset loss function, a preset optimizer, and preset model parameters in combination with a preset strategy for training, wherein the preset strategy is used to stop training when the improvement in model performance is less than a preset threshold within a preset number of training cycles.

[0034] Compared with the prior art, this application has the following beneficial effects:

[0035] Obtain a target data set, the target data set includes visual deep fake data and real data, the visual deep fake data includes corresponding annotated data attribute information and fake operation information; perform feature extraction on the target data set to obtain a first feature set, the first feature set includes the result of feature extraction on the target data set based on at least one of image features, preset neural networks, deep fake video defects, deep fake image defects, image digital features, and video digital features; perform feature selection on the first feature set to obtain a second feature set; based on the second feature set, use a preset loss function, a preset optimizer, and preset model parameters, combined with a preset strategy training to obtain a detection model for detecting visual deep fake content, the preset strategy is used to characterize that within a preset number of training cycles, the model performance improvement is less than a preset threshold, and the training is stopped. Through the comprehensive feature extraction and analysis of deep fake data, combined with rich sample training and fine-tuning for different application scenarios, high-precision detection of fake content is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 A model determination flow chart provided for an embodiment of the present application;

[0038] Figure 2 A structural diagram of a detection model generation system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

[0041] As mentioned above, with the rapid development of deep learning technology, the technology of forged content generation based on visual deep learning has become increasingly mature, bringing many challenges to society. At present, most existing forgery detection algorithms only extract one or several features of forged data for discriminant analysis. This method is difficult to fully capture the subtle differences in forged content, resulting in low accuracy in forged content detection.

[0042] At present, most existing forgery detection algorithms only extract one or several features of forged data for discriminant analysis. This method is difficult to fully capture the subtle differences in forged content, resulting in low detection accuracy of forged content. For example, some algorithms rely only on image features, such as texture analysis or edge detection, while ignoring other important features, such as time domain features and frequency domain features. The limitations of this single feature analysis method make the detection results vulnerable to improvements in forgery technology and become invalid.

[0043] In addition, existing forgery detection technologies often lack rich sample data, especially deep forgery data for different application scenarios. This limits the generalization ability and detection effect of the model. As forgery technology is constantly updated, newly emerged forged content may be significantly different from the samples in the existing dataset, resulting in poor performance of the model when facing new forgery content.

[0044] Existing forgery detection technologies perform differently in different application scenarios and lack fine-tuning and optimization for specific scenarios, resulting in poor results in practical applications. For example, on social media platforms and news websites, the form and characteristics of forged content may vary significantly, but existing detection algorithms often cannot adapt well to these differences, thus affecting the accuracy and reliability of detection.

[0045] Finally, existing forgery detection technologies usually rely on a single feature analysis method, such as image features or deep learning features, and fail to fully utilize the combined advantages of multiple features. This affects the accuracy and robustness of detection. For example, a detection algorithm that relies solely on image features may not be able to effectively identify content that has been processed by advanced forgery technology, while a comprehensive analysis method that combines multiple features can significantly improve the detection effect.

[0046] In order to solve the above problems, the embodiments of the present application achieve high-precision detection of forged content through comprehensive feature extraction and analysis of deep fake data, combined with rich sample training and fine-tuning for different application scenarios.

[0047] It should be noted that the detection model generation method, system, device and medium provided in the present application can be applied to the field of computer technology. The above is only an example and does not limit the application field of the detection model generation method, system, device and medium provided in the present application. In addition, the embodiments of the present application may not limit the execution subject of the detection model generation. For example, the detection model generation method of the embodiment of the present application can be applied to data processing devices such as terminal devices or servers. Among them, the terminal device can be an electronic device such as a computer, a personal digital assistant (PDA). The server can be an independent server, a cloud server, or a cluster server composed of multiple servers.

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] The following describes a detection model generation method provided by the present application through an embodiment. Figure 1 ,Should Figure 1 A model determination flow chart provided in an embodiment of the present application, the method comprising:

[0050] S101: Obtain a target data set.

[0051] The target data set includes visual deep fake data and real data, and the visual deep fake data includes corresponding annotated data attribute information and fake operation information.

[0052] The target dataset is constructed based on sample data and application scenario data. In one possible implementation, sample data may include real data obtained from public channels, and real data containing faces, images, and videos may be obtained from social media platforms using interfaces or crawler technology. These data usually contain rich scenes and backgrounds, which helps the model learn more diverse features. News pictures and videos can be downloaded from authoritative news websites. These resources often contain images and video materials of real events and people.

[0053] In one possible implementation, the sample data may include forged data generated using a deep fake dataset. During the collection process, existing deep fake datasets such as FaceForensics++ and DeepFake DetectionDataset are selected. These datasets contain a large number of faces, images, and videos that have been processed with deep fakes. Deep fake generation tools (such as FaceSwap and DeepFaceLab) are used to forge real data to generate different types of forged data including faces, images, and videos. By adjusting the forgery parameters (such as the degree of forgery, the method of forgery, etc.), a variety of forged data can be generated.

[0054] After data collection is completed, the collected data can be annotated with high quality to clarify whether each piece of data is real or forged. The annotation information can include data type (face, image, video, etc.), data scenario (such as news, social media, etc.), forgery authenticity (such as low, medium, high, etc.), forgery method (such as face replacement, voice replacement, etc.) and forgery area (such as face area, background area, etc.).

[0055] In a possible implementation, the application scenario data may include real data obtained from public channels of the application scenario. Similar to obtaining rich sample data, real data is obtained from social media and news websites of relevant application scenarios. These data usually contain real faces, images, and video materials in specific scenarios.

[0056] Application scenario data can include real data collected on-site in real scenarios. On-site collection is carried out in specific application scenarios (such as airports, stations, banks, public places, etc.) to obtain real face, image and video data. Such data usually has higher authenticity and representativeness, which helps improve the performance of the model in specific scenarios.

[0057] Application scenario data may include forged data generated using deep fake datasets of application scenarios. During the acquisition process, deep fake datasets related to application scenarios are selected. These datasets usually contain forged faces, images, and videos in specific scenarios. Deep fake generation tools are used to forge real data to generate forged data related to application scenarios. By adjusting forgery parameters and scene settings, forged data that better meets actual application needs can be generated.

[0058] The collected application scenario data is annotated with high quality to clarify the authenticity and forgery information of each piece of data. The rich sample data and application scenario data are integrated to form a comprehensive data set containing real data and forged data. This data set will be used to train the deep forgery recognition model to improve the recognition performance and generalization ability of the model.

[0059] The collected data is preprocessed, including data enhancement and data filtering.

[0060] Data enhancement aims to increase the diversity and richness of the data set through a series of operations, thereby improving the generalization ability of the model. The following are several data enhancement methods that can be used in the embodiments of this application. In actual application scenarios, at least one of the following methods can be selected according to actual needs:

[0061] The first is rotation and scaling. Images or videos are rotated at different angles and scaled at different ratios to simulate shooting effects from different perspectives. These operations can increase the diversity of the data set and enable the model to recognize objects at different angles and sizes.

[0062] The second method is to add noise. Random noise is added to the image or video to simulate the noise interference in the real scene. This helps the model remain robust in noisy environments.

[0063] The third type is color jitter. This method changes the color distribution of the image, such as brightness, contrast, saturation, etc., to simulate the effects of shooting under different lighting conditions. This helps the model maintain stable performance under different lighting conditions.

[0064] The fourth type is median filtering and Gaussian filtering. Median filtering is used to remove salt and pepper noise in the image and protect edge information. Gaussian filtering is used to smooth the image, remove high-frequency noise, and make the image smoother. These operations help the model maintain accuracy when processing images of different qualities.

[0065] The fifth method is cropping. Cropping the area of ​​interest in the image or video to reduce redundant information and improve the pertinence of the dataset. This helps the model focus more on key features and improves recognition accuracy.

[0066] Data filtering aims to remove abnormal, duplicate and unclearly labeled data in the data set to ensure the quality and balance of the data. The following are several data filtering methods that can be used in the embodiments of this application. In actual application scenarios, at least one of the following methods can be selected according to actual needs:

[0067] The first is abnormal data filtering: strictly control the data collection, transportation and storage process to avoid abnormal situations. Perform random tests on each batch of data to promptly discover and handle problems such as files that cannot be opened, files that do not exist, or low collection quality. During feature extraction, retain the corresponding exception handling mechanism to deal with possible abnormal situations.

[0068] The second method is to clean up duplicate data: clarify the location, method and time of the batch data source in order to track and clean up duplicate data. Perform batch sampling and comparison on each batch of data to find and delete duplicate data. Use hash algorithms to perform uniqueness detection on images or videos to quickly identify and delete duplicate data.

[0069] The third type is filtering of unclearly labeled data: When labeling data, ensure that each piece of data has a clear true or false label. Filter out data that is unclear whether it is true or false by labeling it. Use manual review or machine learning algorithms to re-label or classify unclearly labeled data.

[0070] Through effective data enhancement and filtering, the performance and generalization ability of the deep fake identification system can be significantly improved. Data enhancement increases the diversity and quantity of data and improves the robustness of the model; data filtering ensures the quality and balance of the data and avoids the impact of abnormal and duplicate data on model training. These measures work together to enable the model to perform better in real scenarios.

[0071] S102: Extract features from the target data set to obtain a first feature set.

[0072] The embodiments of the present application can extract features from time domain features, spatial domain features, frequency domain features, object motion features, device features, metadata features, video frame rate features, and deep learning features. These technologies provide a rich perspective when detecting deep fake videos, and can capture the differences between fake videos and real videos from multiple dimensions.

[0073] The first type, temporal features, focuses on the changes in the time series of video frames. By capturing these changes, the anomalies of forged videos can be revealed.

[0074] The pixel difference between consecutive frames is obtained by calculating the difference between corresponding pixels between adjacent frames of the video. This method can detect inter-frame discontinuities in deep fake videos. Since fake videos may introduce subtle pixel differences during synthesis or editing, these differences will be amplified in the pixel difference between consecutive frames, thus revealing the traces of forgery. For example, if someone forges the face synthesis of two people in a video, the pixel difference between consecutive frames may increase significantly in the face area, because the pixel difference between the forged face and the real background will be more obvious in the inter-frame changes.

[0075] Optical flow is a motion vector field that describes the change of pixels in an image over time. By calculating the speed and direction of each pixel, an optical flow feature can be obtained. Optical flow features can detect unnatural motion in forged videos. Since forged videos may introduce unnatural motion trajectories during the synthesis process, these trajectories will be identified in the optical flow features. For example, in a forged video, changes in facial expressions or head rotation may appear unnatural, and optical flow features can capture these unnatural motion trajectories, thereby revealing traces of forgery.

[0076] The consistent temporal features are obtained by learning the long-term dependencies between video frames through a temporal model. This method can detect inconsistencies in forged videos. Since forged videos may not maintain consistent changes in facial features or backgrounds over time, these inconsistencies will be identified in the consistent temporal features. For example, if a forged video maintains a certain facial expression or background for a period of time, but then suddenly changes, this inconsistency will be captured in the consistent temporal features, revealing traces of forgery.

[0077] In deep fake video detection, by calculating at least one of the three time domain features: continuous frame pixel difference, optical flow features, and consistent time domain features, the anomalies and inconsistencies in the fake video can be effectively captured, thereby providing strong support for subsequent fake detection.

[0078] Second, in deep fake video detection, spatial feature extraction is also an indispensable step. Spatial features mainly focus on the spatial information of video frames. By capturing this spatial information, the anomalies of fake videos can be further revealed.

[0079] Color space domain features By analyzing the color features in the forged video, we can find the color inconsistency or lighting mismatch that occurs during the synthesis process. Some forged content is not natural enough in the processing of light and shadow, which is particularly obvious in the color space domain. For example, RGB (red, green, blue) is the most commonly used color space, which is based on the human eye's perception of color. By analyzing the color distribution of the forged video in the RGB color space, we can find the color inconsistency or lighting mismatch. The HSV (hue, saturation, value) color space decomposes color information into three components: hue, saturation, and value. In the HSV space, color anomalies in forged videos may be more intuitive, because changes in hue and saturation often reflect the unnaturalness of the synthesis process.

[0080] Fake videos may introduce color inconsistencies or lighting mismatches during the synthesis process, which will be identified in the color spatial domain features. For example, if the color of a certain area in the fake video is significantly different from the surrounding area, or the lighting conditions do not match the real scene, these anomalies will be captured in the color spatial domain features.

[0081] Texture spatial features detect forgery traces by analyzing the texture information in the video. By analyzing the texture features of the face or other objects in the video, it is possible to detect subtle texture anomalies in forgeries, such as texture changes in areas such as skin and hair that do not match the real content.

[0082] Exemplarily, LBP is an operator that describes the local texture features of an image by generating a binary pattern by comparing the grayscale values ​​of each pixel with those of its neighboring pixels. LBP features can capture subtle texture anomalies of faces or other objects in forged videos. For example, if the skin texture in a forged video does not match the real skin texture, or if the texture changes in the hair area are unnatural, these anomalies will be identified in the LBP features. Gabor filter is a linear filter used for image texture analysis that can capture frequency and direction information in an image. Gabor filter can detect texture anomalies in forged videos. For example, if the texture of a certain area in a forged video is inconsistent with the surrounding area, or if the texture direction is unnatural, these anomalies will be captured in the Gabor filter features.

[0083] Edge detection can reveal blurred or unnatural edges in specific areas of a fake video, which may be caused by poor edge processing when processing facial contours or when combining different video clips.

[0084] Exemplarily, Canny edge detection is a multi-stage algorithm for detecting edges in an image. It combines steps such as Gaussian filtering, gradient calculation, non-maximum suppression, and double threshold detection. Canny edge detection can reveal blurred or unnatural edges in specific areas of a forged video. For example, if a forged video has poor edge processing when processing facial contours or synthesizing different video clips, these anomalies will be identified in the Canny edge detection feature. The Sobel operator is a discrete differential operator for edge detection that combines Gaussian smoothing and differential derivation to calculate image gradients. The Sobel operator can detect edge anomalies in forged videos. Similar to Canny edge detection, if the edge processing in a forged video is unnatural or blurred, these anomalies will be captured in the Sobel operator feature.

[0085] The above three spatial feature extraction methods can help identify unnatural traces in deep fake videos. By combining these methods, anomalies in fake videos can be analyzed and detected more comprehensively. Each method has its own unique advantages, and combined use can improve the accuracy and robustness of detection.

[0086] The third type is frequency domain features. There are two main frequency domain feature extraction methods: Fourier transform and wavelet transform.

[0087] By converting the image into frequency domain analysis through Fourier transform, frequency anomalies in forged videos can be found. Forged content may show abnormal high-frequency or low-frequency signals in the frequency domain, especially in the details. By analyzing the spectral features of different scales through wavelet transform, anomalies in forged content at multiple scales can be revealed, such as the frequency distribution of texture, edge or structural features does not match the real content.

[0088] The fourth type is object motion features. For example, motion trajectory and occlusion features: By detecting the motion trajectory of objects in the video, it can be found that the motion of objects in the fake video does not conform to physical rules or the occlusion processing is unnatural. For example, in deep fakes, the synthetic face may not be synchronized with the motion trajectory of the real body.

[0089] Fifth, device features. For example, sensor noise and lens distortion features: Deep fake videos usually do not contain the noise and distortion introduced by the sensor or lens in real videos. By analyzing these features, fake videos can be identified. For example, fake content may not be able to truly simulate the optical characteristics of the camera device. Compression features: Deep fake videos may have quantization errors or block effects during the compression process. These features can help detect fake content in some scenarios.

[0090] The sixth type is metadata features. For example, time metadata can be used to analyze metadata such as the creation time and modification time of the video to find the editing traces of the forged video. The forger may forget to synchronize the time information in the metadata, resulting in inconsistent metadata.

[0091] The seventh type is video frame rate feature. For example, a fake video may have frame rate changes in different clips. By detecting the frame rate feature, traces of editing or synthesis in the video can be revealed.

[0092] The eighth type is deep learning features. By introducing the attention mechanism, key areas in the video can be identified, such as subtle traces of forgery in key parts such as the face and eyes. The attention mechanism can help the model focus on analyzing the key forgery areas in the video.

[0093] CNN networks can extract deep features of images, including high-level semantic features of objects. This helps identify unnatural content in fake videos, such as abnormal facial expressions or mismatched texture details. RNN networks can process video sequence data, helping the model learn the temporal dependencies between video frames and identify dynamic anomalies in fake videos. By deepening the number of layers in the CNN network, higher-level semantic features can be extracted to identify detailed differences in deep fake videos.

[0094] Ninth, facial features. By analyzing the micro-expressions, blinking frequency, and head movements of the face in the fake video, that is, blinking frequency, head movement, and micro-expression features, anomalies can be identified. For example, the fake video may not correctly simulate the natural facial dynamics of the person.

[0095] S103: Perform feature selection on the first feature set to obtain a second feature set.

[0096] Feature selection is an important step in the preprocessing stage of machine learning, which aims to select the most effective subset from the original feature set to improve the performance of the model, reduce training time and improve the interpretability of the model. The feature selection methods that can be applied in the embodiments of this application are divided into three categories: filtering method, embedding method and wrapping method.

[0097] The filtering method is independent of any machine learning algorithm, but relies on the statistical properties of the features themselves to select features. This method is usually simple to calculate and fast, but may not take into account the relationship between the features and the model. For example, features with variances less than the preset variance are removed. That is, features with variances that are too small, which means that the feature does not change much and may contribute less to the prediction of the target variable. You can set a threshold, such as 0.1, to remove all features with variances below this threshold. Remove features with low correlation with the target variable. The Pearson Correlation Coefficient or other correlation metrics can be used to evaluate the linear relationship between features and the target variable. Set a threshold, such as 0.3, to remove all features with correlations below this threshold.

[0098] Embedding method is a method to automatically complete feature selection during model training. This type of method embeds the process of feature selection into the process of model training and selects features by optimizing the objective function. Principal component analysis PCA is an unsupervised dimensionality reduction technique that reduces the dimension of features by converting the original features into a set of new, orthogonal features (principal components). The new features are linear combinations of the original features and are sorted by variance, usually retaining only the first few principal components. Linear discriminant analysis LDA is a supervised dimensionality reduction technique that aims to find a projection direction so that samples of different categories are as separated as possible in the projected space. Unlike PCA, LDA takes into account the information of category labels while reducing the dimensionality.

[0099] The wrapper method directly uses the performance of the machine learning model to evaluate the quality of the feature subset. This method is usually more time-consuming than the other two methods because it requires repeated training of the model, but it can obtain a better feature subset. Recursive feature elimination RFE is a model-based feature selection method that first trains a model, then removes the feature with the smallest weight, and repeats this process until a predetermined number of features is reached. RFE can be used in conjunction with a variety of machine learning models, such as logistic regression, decision trees, and support vector machines.

[0100] By performing feature selection through filtering, embedding, and wrapping, reducing unnecessary features can reduce model complexity and speed up training. By removing irrelevant or redundant features, the model can focus more on useful signals, thereby improving prediction performance. Fewer features means faster model training and inference speed. Feature selection helps build simpler models and improves model interpretability. It reduces the risk of overfitting and enables the model to perform better on unseen data.

[0101] S104. Based on the second feature set, a preset loss function, a preset optimizer, and preset model parameters are used in combination with a preset strategy training to obtain a detection model for detecting visual deep fake content.

[0102] The classifier is the core of the detection model, which is used to classify the extracted features and determine the authenticity of the data. Commonly used classifiers include logistic regression, support vector machine, random forest, gradient boosting decision tree, and deep learning models (such as convolutional neural network CNN, recurrent neural network RNN, etc.). When choosing a classifier, you can choose according to the specific nature of the problem and the characteristics of the data.

[0103] The loss function is used to measure the difference between the model's predicted results and the actual results. In the authenticity detection task, the cross-entropy loss function can be used. It is suitable for binary classification problems and can well reflect the difference between the probability distribution predicted by the model and the actual label.

[0104] The optimizer is used to update the parameters of the model to minimize the loss function. The Adam optimizer is an adaptive learning rate optimization algorithm based on first-order moment estimation and second-order moment estimation. It has the advantages of fast convergence speed and strong adaptability, so it has been widely used in the field of deep learning. When training the detection model, you can choose the Adam optimizer to update the parameters of the model.

[0105] The learning rate determines the step size of the model parameter update. Too large a learning rate may cause the model to fail to converge, while too small a learning rate will slow down the convergence. Therefore, it is necessary to set a suitable learning rate to balance the convergence speed and generalization ability of the model. In this example, the learning rate is set to 0.01, which is a relatively conservative but generally effective value.

[0106] Batch size determines the number of samples used to update model parameters during each training. A larger batch size can speed up model training, but may increase memory usage and computational costs; a smaller batch size can improve the generalization ability of the model, but may slow down training. Therefore, it is necessary to set an appropriate batch size based on hardware conditions and actual needs.

[0107] During the training process, a portion of neurons are randomly turned off to reduce the complex co-adaptation between neurons and prevent the model from overfitting. Usually used after the fully connected layer, the Dropout ratio (i.e. the proportion of neurons turned off) can be selected from 0.2 to 0.5. By adding the absolute value (L1) or square (L2) of the weight as a penalty term to the loss function, the size of the model parameters is suppressed, thereby reducing the complexity of the model and improving the generalization ability. L2 regularization (also known as weight decay) is more commonly used because it does not produce sparse solutions and is more suitable for most application scenarios.

[0108] By monitoring the performance of the model on the validation set, training is stopped when the performance on the validation set no longer improves to avoid overfitting the model to the training data.

[0109] Therefore, by adjusting the classifier, loss function, optimizer, learning rate, batch size, and taking measures such as overfitting prevention and early stopping of training, we can approach the learning upper limit of deep fake data and features as much as possible, thereby building and training an efficient and accurate authenticity detection model.

[0110] In one possible implementation, the judgment and evaluation of the model performance can be carried out in at least one of the following ways: FPS used to measure the real-time processing speed of the model, accuracy reflecting the overall correctness of the model in all predictions, recall evaluating the model's ability to detect positive examples (such as fake videos), precision measuring the proportion of samples predicted by the model to be positive that are actually positive: F1-score, the harmonic mean of the combined precision and recall.

[0111] Obtain real and forged video or image datasets, perform necessary preprocessing on the data, such as labeling, cropping, scaling, etc. Input the acquired datasets into the detection model and obtain prediction results, including TP, TN, FP, FN, etc. Calculate the evaluation indicators defined above based on the collected prediction results. Organize the calculation results and generate an analysis report containing all evaluation indicators.

[0112] If the model needs to deal with different types of forgery techniques (such as DeepFake, FaceSwap, GAN, etc.), a detailed comparative analysis of the precision, recall, F1-score, etc. under different forgery categories can be performed to evaluate the model's adaptability to different forgery methods.

[0113] In a possible implementation, the model structure and / or model parameters are adjusted according to the evaluation results.

[0114] For example, the model's hyperparameters such as learning rate, number of iterations, and regularization coefficient can be adjusted to accelerate training and obtain better model results.

[0115] For example, you can increase the training data: balance the data samples and collect more data for training. You can change the model structure in the detection model, such as increasing the number of network layers and adjusting the size of the convolution kernel. Fusion of the prediction results of multiple models can improve the robustness of the model. You can also develop an adaptive learning algorithm so that the model can adapt to the ever-changing counterfeiting technology. By adjusting the above data and model algorithms, the upper limit of the deep fake recognition model can be increased as much as possible.

[0116] In the process of deepfake detection, in order to ensure the accuracy and adaptability of the algorithm, the threshold needs to be adjusted dynamically. Specifically, different application scenarios, data distribution, and changes in detection targets may affect the performance of the detection algorithm. Therefore, the process of threshold setting and adjustment needs to be optimized based on these factors to improve the performance of the algorithm.

[0117] In the first case, the threshold is adjusted according to the forgery method.

[0118] Different forgery techniques (such as FaceSwap, DeepFake, GAN generation, etc.) will produce different characteristics. A single fixed threshold may not be applicable to all types of forged videos. Therefore, the threshold can be dynamically adjusted according to the distribution characteristics of the training data to optimize the detection effect.

[0119] By analyzing the features generated by different forgery methods, a threshold adjustment model is established, which can dynamically adjust the threshold according to the features of the input data. If the FaceSwap forgery video is detected, the threshold can be set higher because the features of this forgery method are more obvious; while for GAN-generated videos, the threshold may need to be set lower because the features of this forgery method are more obscure.

[0120] In the second case, adjust the threshold according to the application scenario.

[0121] In some application scenarios that require extremely high accuracy (such as forensic identification or news verification), the algorithm requires stricter detection standards. Raising the threshold will ensure that only videos with obvious forgery characteristics are marked as forged, thereby reducing the false alarm rate. In high-accuracy scenarios such as forensic identification, the threshold can be set to above 0.9 to ensure that only highly suspicious videos are marked as forged.

[0122] In some scenarios where it is necessary to capture as much fake content as possible (such as large-scale video surveillance or automatic review of online platforms), the algorithm may tend to prefer a lower threshold. Lowering the threshold allows more suspected fake videos to be detected, even if it means increasing the chance of false positives. In high-recall scenarios such as automatic review of online platforms, the threshold can be set to around 0.5 to ensure that as many potential fake videos as possible are captured.

[0123] In the third case, in actual applications, a balance can be struck between the false alarm rate and the missed alarm rate. A threshold that is too high may cause some slightly forged videos to be missed (high missed alarm rate), while a threshold that is too low may misdetect some real videos as forged (high false alarm rate). By continuously adjusting the threshold, the system can find the best balance point according to actual needs in different scenarios. In news verification, the threshold can be set to 0.7 first, and then fine-tuned according to the actual detection results to gradually optimize the threshold.

[0124] In the fourth case, as the algorithm is applied to new data or real-time scenarios, an adaptive threshold adjustment mechanism is introduced. For example, by monitoring the model's detection results, false positives, and false negatives, the algorithm can automatically optimize the threshold setting and gradually improve the detection ability of different counterfeiting methods, thereby enhancing its adaptability. Use online learning or reinforcement learning techniques to dynamically adjust the threshold based on new data. The system can regularly collect new detection results and automatically adjust the threshold through machine learning algorithms to make it more suitable for the current data distribution.

[0125] In the fifth case, the threshold is adjusted based on user feedback or manual review results.

[0126] Establish a feedback mechanism to allow users or auditors to submit cases of false positives or false negatives, and the system can dynamically modify the threshold settings based on these feedbacks. If a user reports that a video is falsely reported, the system can lower the threshold for such videos; if a user reports that a fake video is not detected, the system can raise the threshold for such videos.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0128] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0129] The above are some specific implementations of the detection model generation method provided in the embodiment of the present application. Based on this, the present application also provides a corresponding detection model generation system. The system provided in the embodiment of the present application will be introduced from the perspective of functional modularization. Figure 2 A structural diagram of a detection model generation system provided in an embodiment of the present application.

[0130] The system comprises:

[0131] An acquisition unit 110 is used to acquire a target data set, wherein the target data set includes visual deep fake data and real data, and the visual deep fake data includes corresponding annotated data attribute information and fake operation information;

[0132] An extraction unit 111 is used to perform feature extraction on the target data set to obtain a first feature set, wherein the first feature set includes a result obtained by performing feature extraction on the target data set based on at least one of an image feature, a preset neural network, a deep fake video defect, a deep fake image defect, an image digital feature, and a video digital feature;

[0133] A selection unit 112, configured to perform feature selection on the first feature set to obtain a second feature set;

[0134] The training unit 113 is used to obtain a detection model for detecting visual deep fake content based on the second feature set, using a preset loss function, a preset optimizer and preset model parameters, combined with a preset strategy training. The preset strategy is used to indicate that training is stopped when the improvement in model performance is less than a preset threshold within a preset number of training cycles.

[0135] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the test method solutions provided in the embodiments of the present application.

[0136] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the test method described in any embodiment of the present application.

[0137] The computer storage medium stores codes, and when the codes are executed, a device executing the codes implements the test method described in any embodiment of the present application.

[0138] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0139] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0140] It should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0141] It should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0142] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0143] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0144] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A detection model generation method, characterized in that: The method comprises: Acquire a target data set, the target data set includes visual deep fake data and real data, the visual deep fake data includes corresponding annotated data attribute information and forgery operation information; the target data set is constructed based on sample data and application scenario data, and the method for determining the visual deep fake data includes: using a deep fake generation tool to forge real data to generate different types of forged data; and by adjusting forgery parameters, generating diversified forged data, the forgery parameters include forgery degree and forgery method; the annotated information includes data type, data scenario, forgery authenticity, forgery method and forgery area; Performing multi-faceted feature extraction on the target data set to obtain a first feature set, wherein the first feature set includes a result obtained by performing feature extraction on the target data set based on at least one of image features, a preset neural network, deep fake video defects, deep fake image defects, image digital features, and video digital features; the multi-faceted feature extraction includes performing multi-faceted feature extraction from time domain features, spatial domain features, frequency domain features to object motion features, device features, metadata features, video frame rate features, and deep learning features; Performing feature selection on the first feature set to obtain a second feature set; Based on the second feature set, a detection model for detecting visual deep fake content is obtained by using a preset loss function, a preset optimizer, and preset model parameters in combination with a preset strategy training, wherein the preset strategy is used to indicate that training is stopped when the improvement in model performance is less than a preset threshold within a preset number of training cycles; The method further comprises: adjusting the model structure and / or model parameters according to the model performance evaluation result; The adjusting of the model structure and / or model parameters according to the model performance evaluation result includes: Adjust the model’s learning rate, number of iterations, and regularization coefficient to speed up training; Increase training data to balance data samples and collect more data for training; increase the number of network layers and adjust the size of convolution kernels; fuse the prediction results of multiple models to improve the robustness of the model; develop adaptive learning algorithms to enable the model to adapt to the ever-changing counterfeiting technology to improve the upper limit of the deep fake recognition model; The method further includes: dynamically adjusting the threshold based on different factors; The dynamically adjusting the threshold based on different factors includes: Adjustment based on counterfeiting methods, application scenarios, false positive and false negative balance, adaptive threshold or feedback; Adjustments based on forgery methods include: Analyze the features generated by different forgery techniques, establish a threshold adjustment model, dynamically adjust the threshold based on the features of the input data, and set a higher threshold for FaceSwap forged videos with obvious features than for GAN-generated videos with obscure features. The adjustment methods based on application scenarios include: In scenarios where high accuracy is required, the threshold is raised to above 0.9 to reduce the false alarm rate; in scenarios where more forged content needs to be captured, the threshold is lowered to 0.5 to increase the detection coverage; Methods based on balancing false positives and false negatives include: By continuously adjusting the threshold, we can find the best balance between the false positive rate and the false negative rate. The methods based on adaptive threshold adjustment include: By monitoring the model's detection results, false positive and false negative rates, using online learning or reinforcement learning techniques, regularly collecting new detection results, and automatically optimizing threshold settings with the help of machine learning algorithms, the model's adaptability to detecting different counterfeiting methods can be enhanced; Ways to adjust based on feedback include: Establish a feedback mechanism to adjust the threshold based on user feedback or manual review results; if users report that a video is a false alarm, lower the threshold for this type of video; if there is feedback that a fake video has not been detected, increase the threshold for this type of video.

2. The method according to claim 1, characterized in that The step of obtaining a target data set includes: Obtain real data and fake data samples from the target data channel; Generate visual deep fake data based on the acquired fake data samples; Combine the acquired real data and the generated visual deep fake data to obtain a preliminary data set; Data enhancement operations and data filtering operations are sequentially performed on the prepared data set to obtain a target data set, wherein the data enhancement operation includes performing at least one of rotation, scaling, flipping, cropping, color conversion, and noise addition on the data samples in the prepared data set; and the data filtering operation includes screening the data samples in the target data set based on at least one of statistical analysis, cluster analysis, or machine learning algorithms to identify and exclude data samples that do not meet preset quality standards.

3. The method according to claim 1, characterized in that The performing feature selection on the first feature set to obtain a second feature set includes: Based on variance screening and correlation analysis, the features in the first feature set whose importance does not meet the preset importance standard are eliminated; the features in the first feature set are subjected to dimensionality reduction processing; based on recursive feature elimination, the features in the first feature set are eliminated to obtain a second feature set.

4. The method according to claim 1, characterized in that: The first feature set includes time domain features, spatial domain features, and frequency domain features obtained by extracting features based on image features, and the feature extraction of the target data set to obtain the first feature set includes: The time domain characteristics of the target data set are determined by analyzing the pixel-level difference information between consecutive frames, the motion pattern and motion law of the object; the spatial domain characteristics of the target data set are determined by analyzing the texture information and edge information of the image; the images in the target data set are converted into the frequency domain, the spectral characteristics and multi-scale characteristics of the images in the target data set are analyzed, and the frequency domain characteristics of the target data set are determined.

5. The method according to claim 1, characterized in that The first feature set includes facial features and object movement features obtained by feature extraction of deep fake video defects, and the feature extraction of the target data set to obtain the first feature set includes: Based on the consistency of blinking frequency, head movement information, facial expression and context information, the facial features of the target data set are determined; based on the object movement trajectory and object occlusion transition information, the object movement features are determined.

6. The method according to claim 1, characterized in that The first feature set includes a splicing trace feature and a copy-paste trace feature obtained by extracting features based on deep fake image defects, and the first feature set is obtained by extracting features from the target data set, including: Based on whether the colors of different areas of the image meet the preset consistency conditions, whether the textures of different areas of the image meet the preset consistency conditions, whether the lighting of different areas of the image meets the preset consistency conditions, and the geometric transformation trace information in the image, the splicing trace characteristics of the target data set are determined; and whether there are repeated texture areas in the image is detected to determine the copy and paste trace characteristics of the target data set.

7. The method according to claim 1, characterized in that The first feature set includes camera features, compression trace features and metadata features obtained based on image digital feature extraction, and the feature extraction of the target data set to obtain the first feature set includes: Based on sensor noise information in the image and whether the image has lens distortion, the camera characteristics of the target data set are determined; based on quantization error and block effect in the image, the compression trace characteristics of the target data set are determined; based on metadata information of the image or video, the metadata characteristics of the target data set are determined.

8. The method according to claim 1, characterized in that The first feature set includes frame rate features, encoding format features and timestamp features extracted based on video digital features, and the feature extraction of the target data set to obtain the first feature set includes: Determine whether the video frame rate meets the preset stability condition and determine the frame rate characteristics of the target data set; determine the encoding format characteristics of the target data set based on the encoding format of the video; determine whether the timestamp of the video meets the preset continuity condition and determine the timestamp characteristics of the target data set.

9. The method according to claim 1, characterized in that: The method further comprises: Detection scenario requirement information is obtained, and model parameters of the detection model are adjusted based on the detection scenario requirement information, wherein the model parameters are used to balance the false alarm rate and the false negative rate of the detection model.

10. A detection model generation system, characterized in that: include: An acquisition unit, configured to acquire a target data set, wherein the target data set includes visual deep fake data and real data, wherein the visual deep fake data includes correspondingly annotated data attribute information and fake operation information; The target data set is constructed based on sample data and application scenario data. The method for determining the visual deep fake data includes: using a deep fake generation tool to forge real data to generate different types of fake data; and generating diversified fake data by adjusting fake parameters, wherein the fake parameters include fake degree and fake method; the annotation information includes data type, data scenario, fake authenticity, fake method and fake area; An extraction unit is used to perform multi-faceted feature extraction on the target data set to obtain a first feature set, wherein the first feature set includes a result obtained by performing feature extraction on the target data set based on at least one of image features, a preset neural network, deep fake video defects, deep fake image defects, image digital features, and video digital features; the multi-faceted feature extraction includes performing multi-faceted feature extraction from time domain features, spatial domain features, frequency domain features to object motion features, device features, metadata features, video frame rate features, and deep learning features; A selection unit, configured to perform feature selection on the first feature set to obtain a second feature set; A training unit, configured to obtain a detection model for detecting visual deep fake content based on a second feature set, using a preset loss function, a preset optimizer, and preset model parameters in combination with a preset strategy for training, wherein the preset strategy is used to indicate that training is stopped when the improvement in model performance is less than a preset threshold within a preset number of training cycles; and to adjust the model structure and / or model parameters according to a model performance evaluation result; The adjusting of the model structure and / or model parameters according to the model performance evaluation result includes: Adjust the model’s learning rate, number of iterations, and regularization coefficient to speed up training; Increase training data to balance data samples and collect more data for training; increase the number of network layers and adjust the size of convolution kernels; fuse the prediction results of multiple models to improve the robustness of the model; develop adaptive learning algorithms to enable the model to adapt to the ever-changing counterfeiting technology to improve the upper limit of the deep fake recognition model; Dynamically adjust thresholds based on different factors; The dynamically adjusting the threshold based on different factors includes: Adjustment based on counterfeiting methods, application scenarios, false positive and false negative balance, adaptive threshold or feedback; Adjustments based on forgery methods include: Analyze the features generated by different forgery techniques, establish a threshold adjustment model, dynamically adjust the threshold based on the features of the input data, and set a higher threshold for FaceSwap forged videos with obvious features than for GAN-generated videos with obscure features. The adjustment methods based on application scenarios include: In scenarios where high accuracy is required, the threshold is raised to above 0.9 to reduce the false alarm rate; in scenarios where more forged content needs to be captured, the threshold is lowered to 0.5 to increase the detection coverage; Methods based on balancing false positives and false negatives include: By continuously adjusting the threshold, we can find the best balance between the false positive rate and the false negative rate. The methods based on adaptive threshold adjustment include: By monitoring the model's detection results, false positive and false negative rates, using online learning or reinforcement learning techniques, regularly collecting new detection results, and automatically optimizing threshold settings with the help of machine learning algorithms, the model's adaptability to detecting different counterfeiting methods can be enhanced; Ways to adjust based on feedback include: Establish a feedback mechanism to adjust the threshold based on user feedback or manual review results; if users report that a video is a false alarm, lower the threshold for this type of video; if there is feedback that a fake video has not been detected, increase the threshold for this type of video.

Citation Information

Patent Citations

  • Forgery video detection method, device and equipment

    CN117649621A

  • Face forgery detection method based on image and video multi-domain fusion

    CN118351427A