Counterfeit face detection method, system and device and storage medium

Abstract

The invention discloses a forged face detection method, system and device and a storage medium, and relates to the technical field of deep learning, and the method comprises the steps: extracting face features from a to-be-detected image; matching a face shape feature template corresponding to the face features from a database; in combination with the face shape feature template, extracting face shape features from the to-be-detected image; recovering a recovered image by using a diffusion model in combination with the face shape features and the condition mask; and comparing the restored image with the to-be-detected image to judge whether the to-be-detected image is a faked face image or not. And extracting detailed face shape features from the to-be-detected image in combination with the face shape feature template, recovering a high-fidelity image by using a diffusion model in combination with the face shape features and the condition mask, and judging whether the to-be-detected image is a face-changing counterfeit image by comparing the recovered image with the to-be-detected image.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a forged face detection method, system, device and storage medium. Background Art

[0002] With the rapid development of deep learning technology, counterfeiting technology has gradually matured. People are facing the abuse of digital identities, the spread of false information, and threats to privacy and security. Traditional detection methods are difficult to effectively identify.

[0003] However, current deepfake detection methods often rely too much on global information, which causes the detection model to incorrectly learn the global information of the person, thus affecting the generalization ability of the model. Summary of the invention

[0004] The main purpose of the present invention is to provide a forged face detection method, system, device and storage medium, which extracts detailed face shape features from the image to be detected in combination with a face shape feature template, uses a diffusion model to combine face shape features and conditional masks to restore a high-fidelity image, and judges whether the image to be detected is a face-changing forged image by comparing the restored image with the image to be detected.

[0005] In order to achieve the above objectives, the embodiments of the present application provide the following technical solutions: According to a first aspect of an embodiment of the present application, a forged face detection method is provided, the method comprising: Extract facial features from the image to be detected; Matching a facial shape feature template corresponding to the facial feature from a database; Extracting face shape features from the image to be detected in combination with the face shape feature template; Restoring a restored image by combining the face shape features and the conditional mask using a diffusion model; By comparing the restored image with the image to be detected, it is determined whether the image to be detected is a forged face image.

[0006] Optionally, extracting the face shape feature from the image to be detected in combination with the face shape feature template includes: Locating set facial feature points from the image to be detected; Using the set facial feature points, mapping the three-dimensional shape information in the facial shape feature template to the corresponding position in the image to be detected to obtain a three-dimensional facial shape model; The face shape features are extracted from the three-dimensional face shape model.

[0007] Optionally, the conditional mask is used to indicate a set area for face detection; the conditional mask is generated based on facial landmarks to cover the set area of ​​the target face.

[0008] Optionally, restoring the restored image by using the diffusion model in combination with the face shape feature and the conditional mask includes: gradually adding noise to the image to be detected to obtain a noise image, so as to simulate the beginning of a diffusion process; Applying the conditional mask to the noisy image to obtain a masked image to preserve features during a simulated diffusion process; A restored image is restored from the masked image using the diffusion model.

[0009] Optionally, the comparing the restored image with the image to be detected to determine whether the image to be detected is a forged face image includes: If the difference between the restored image and the image to be detected in the set area meets the set condition, it is determined that the image to be detected is a forged face image.

[0010] Optionally, matching a face shape feature template corresponding to the face feature from a database includes: Matching the facial features with facial features stored in the database to confirm the identity of the person in the image to be detected; If the person's identity is confirmed successfully, a face shape feature template uniquely corresponding to the person's identity is matched from the pre-registered face shape template set in the database.

[0011] Optionally, the face shape feature template construction process includes: Determine a base model according to an average face shape of a plurality of face data; According to the principal component analysis technology, the principal component components of several faces are extracted respectively, and each principal component component represents the shape change pattern of the face part; Assigning a coefficient of variation to each principal component, the coefficient of variation characterizing the degree to which the shape varies along the principal component; The base model and all principal components are weighted and summed according to the variation coefficient to obtain a reconstructed three-dimensional face shape; The reconstructed three-dimensional facial features are bound to the person identity information corresponding to the facial data to obtain a facial shape feature template and store it in the database.

[0012] According to a second aspect of an embodiment of the present application, a forged face detection system is provided, the system comprising: A face feature extraction module is used to extract face features from the image to be detected; A feature template matching module, used for matching a face shape feature template corresponding to the face feature from a database; A face shape feature module, used to extract face shape features from the image to be detected in combination with the face shape feature template; An image restoration module, used to restore a restored image by using a diffusion model combined with the face shape features and a conditional mask; The forgery detection module is used to determine whether the image to be detected is a forged image of a human face by comparing the restored image with the image to be detected.

[0013] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0014] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement the method described in the first aspect above.

[0015] In summary, the embodiments of the present application provide a forged face detection method, system, device and storage medium, which extracts facial features from the image to be detected; matches the facial shape feature template corresponding to the facial features from the database; extracts facial shape features from the image to be detected in combination with the facial shape feature template; restores a restored image by combining the facial shape features with a conditional mask using a diffusion model; and determines whether the image to be detected is a forged face image by comparing the restored image with the image to be detected. Detailed facial shape features are extracted from the image to be detected in combination with the facial shape feature template, and a high-fidelity image is restored by combining the facial shape features with a conditional mask using a diffusion model. By comparing the restored image with the image to be detected, it is determined whether the image to be detected is a forged face image. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0017] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0018] Figure 1 A schematic diagram of a fake face detection method provided in an embodiment of the present application; Figure 2 A flowchart of face-changing forgery detection provided in an embodiment of the present application; Figure 3 A schematic diagram of a facial 3D shape extraction process provided in an embodiment of the present application; Figure 4 A schematic diagram of the 3D-Aware Masked Diffusion detection process provided in an embodiment of the present application; Figure 5 A user interface feedback diagram provided for an embodiment of the present application; Figure 6 A schematic diagram of a fake face detection system provided in an embodiment of the present application; Figure 7 A structural diagram of an electronic device provided in an embodiment of the present application is shown; Figure 8 A diagram showing a computer-readable storage medium provided by an embodiment of the present application.

[0019] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0022] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0023] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0024] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0025] Figure 1 A forged face detection method provided in an embodiment of the present application is shown, the method comprising: Step 101: extracting facial features from the image to be detected; Step 102: matching a face shape feature template corresponding to the face feature from a database; Step 103: extracting face shape features from the image to be detected in combination with the face shape feature template; Step 104: restoring a restored image by combining the face shape features and the conditional mask using a diffusion model; Step 105: By comparing the restored image with the image to be detected, it is determined whether the image to be detected is a forged face image.

[0026] Extract key facial features from the image to be detected to ensure the accuracy and reliability of the features. Match the facial shape feature template corresponding to the extracted facial features from the database, and use the pre-registered template to improve the accuracy of detection. Combined with the facial shape feature template, extract detailed facial shape features from the image to be detected, providing rich information for subsequent forgery detection. Use the diffusion model to combine facial shape features and conditional masks to restore a high-fidelity image, and compare the restored image with the image to be detected to determine whether there is a face-changing forgery.

[0027] In a possible implementation, in step 102, matching a face shape feature template corresponding to the face feature from a database includes: The facial features are matched with the facial features stored in the database to confirm the identity of the person in the image to be detected; if the identity of the person is successfully confirmed, a facial shape feature template uniquely corresponding to the identity of the person is matched from the facial shape template set pre-registered in the database.

[0028] Key facial features are extracted from the image to be detected through deep learning algorithms (such as convolutional neural networks, CNN) to ensure the accuracy and reliability of the features. These features can uniquely identify a person's identity, providing a solid foundation for subsequent template matching and forgery detection. The facial shape feature template corresponding to the extracted facial features is matched from the database, and the pre-registered template is used to improve the accuracy of detection. The template matching process ensures that the identity of the person in the image to be detected is consistent with the identity in the database, thereby improving the reliability of detection.

[0029] In a possible implementation, the facial shape feature template construction process includes: determining a base model based on the average facial shape of a number of facial data; extracting the principal component components of a number of faces respectively according to the principal component analysis technology, each principal component component representing the shape change pattern of a facial part; assigning a variation coefficient to each principal component component, the variation coefficient representing the degree of shape change along the principal component component; weightedly summing the base model and all principal component components according to the variation coefficient to obtain a reconstructed three-dimensional facial shape; binding the reconstructed three-dimensional facial features to the person identity information corresponding to the facial data to obtain a facial shape feature template and store it in the database.

[0030] The base model is determined based on the average face shape of several face data. This base model represents the shape of the average face and provides a basis for subsequent feature extraction and model construction. The principal component components of several faces are extracted using the principal component analysis technology. Each principal component component represents the shape change pattern of the face part. These principal component components are arranged in order according to the size of the variance, and the front principal component components contain more shape change information. For example, the first 10 principal component components are extracted through PCA, which represent the shape change patterns of face width, nose bridge height, eye distance, etc.

[0031] Assign a variation coefficient to each principal component, which characterizes the degree of shape change along the principal component. By adjusting these variation coefficients, face models of different shapes can be generated. For example, assigning a variation coefficient of 0.5 to the first principal component (face width) indicates that the face width changes by 0.5 times the average value. The basis model and all principal components are weighted and summed according to the variation coefficient to obtain the reconstructed three-dimensional face shape. This reconstructed three-dimensional face shape can better reflect the shape characteristics of a specific face. The reconstructed three-dimensional face features are bound to the person identity information corresponding to the face data to obtain the face shape feature template and store it in the database. This step ensures the unique correspondence between each template and a specific person identity, which facilitates subsequent matching and detection.

[0032] In a possible implementation, in step 103, extracting the face shape feature from the image to be detected in combination with the face shape feature template includes: The method comprises the steps of locating set facial feature points from the image to be detected; using the set facial feature points, mapping the three-dimensional shape information in the facial shape feature template to the corresponding position in the image to be detected to obtain a three-dimensional facial shape model; and extracting the facial shape features from the three-dimensional facial shape model.

[0033] In face-swapping forgery detection, accurately extracting facial shape features from the image to be detected is a key step. The embodiment of the present application aims to extract high-precision three-dimensional facial shape features from the image to be detected by combining a facial shape feature template, thereby providing a reliable basis for subsequent forgery detection. The set facial feature points are located from the image to be detected to ensure the accuracy and reliability of the feature points. Using these feature points, the three-dimensional shape information in the facial shape feature template is mapped to the corresponding position in the image to be detected, and a detailed three-dimensional facial shape model is constructed. Detailed facial shape features are extracted from the constructed three-dimensional facial shape model to provide rich information for subsequent forgery detection.

[0034] Use facial key point detection algorithms (such as algorithms based on deep learning) to locate the set facial feature points, such as eyes, nose, mouth, etc., in the image to be detected. These feature points are two-dimensional coordinate points and are used for subsequent three-dimensional model fitting. For example, locate the feature points such as the center of Zhang San's eyes, the tip of the nose, and the corners of his mouth in the image to be detected. Further map the three-dimensional shape information (such as facial contour, facial features, etc.) in the matched facial shape feature template to the corresponding position in the image to be detected. This step uses an optimization algorithm (such as the ICP algorithm) to align the three-dimensional points in the template with the key points in the two-dimensional image to construct a detailed three-dimensional facial shape model. Detailed facial shape features, such as facial contour, eye distance, nose bridge height, etc., are extracted from the constructed three-dimensional facial shape model. These features describe the three-dimensional shape characteristics of the face and provide rich information for subsequent forgery detection.

[0035] In a possible implementation, the conditional mask is used to indicate a set area for face detection; the conditional mask is generated based on facial landmarks to cover the set area of ​​the target face.

[0036] Conditional masks are used to indicate the set areas for face detection, ensuring that these areas remain unchanged during the diffusion process. This step improves the accuracy and reliability of detection by generating conditional masks based on facial landmarks to cover the set areas of the target face. For example, a conditional mask is generated to cover key areas such as Zhang San's eyes, nose, and mouth to ensure that these areas remain unchanged during the diffusion process.

[0037] In a possible implementation, in step 104, restoring the restored image by using the diffusion model in combination with the face shape feature and the conditional mask includes: Noise is gradually added to the image to be detected to obtain a noise image to simulate the beginning of a diffusion process; the conditional mask is applied to the noise image to obtain a masked image to maintain features during the simulated diffusion process; and a restored image is restored from the masked image using the diffusion model.

[0038] The beginning of the diffusion process is simulated by gradually adding noise to the image to be detected to obtain a noisy image. A conditional mask is applied to the noisy image to obtain a masked image to maintain key features during the simulated diffusion process. A high-fidelity image is recovered from the masked image using the diffusion model. For example, a noisy image is obtained by gradually adding noise to the image to be detected. A conditional mask is applied to the noisy image to ensure that key areas such as eyes, nose, and mouth remain unchanged during the diffusion process. A high-fidelity image is recovered using the diffusion model, and the restored image is highly consistent with the original image in key areas.

[0039] In a possible implementation manner, in step 105, comparing the restored image with the image to be detected to determine whether the image to be detected is a forged face image includes: If the difference between the restored image and the image to be detected in the set area meets the set condition, it is determined that the image to be detected is a forged face image.

[0040] By comparing the difference between the restored image and the image to be detected in the set area, it is determined whether the image to be detected is a face-swapping forged image. If the difference between the restored image and the image to be detected in the set area meets the set conditions, it is determined to be a face-swapping forged image. For example, the difference between the restored image and the image to be detected in key areas such as eyes, nose, and mouth is compared. If the differences in these areas are significant (such as eye shape, nose height, etc.), it is determined to be a face-swapping forged image; otherwise, it is determined to be a real face image.

[0041] The details of a fake face image in certain areas (such as eyes, nose, mouth, etc.) may differ significantly from the real image. The embodiment of the present application aims to accurately detect the differences in these key areas by combining conditional masks and diffusion models, so as to accurately determine whether the image to be detected is a fake face-changing image. By generating a conditional mask, the set areas of the target face are covered to ensure that these areas remain unchanged during the diffusion process. The diffusion model is used to combine the facial shape features and the conditional mask to restore a high-fidelity image. By comparing the differences in the set areas between the restored image and the image to be detected, it is determined whether there is a face-changing fake.

[0042] Figure 2 The following is a flowchart of face-changing forgery detection operation, which is applied to a face-changing forgery detection integrated machine and specifically includes the following steps: Step 1: Input the image to be detected: The user uploads the face image to be detected through the touch screen of the all-in-one machine or through a connected computer. The software interface of the all-in-one machine will prompt the user to select an image file or directly take a photo and upload it. This step is the basis of the entire detection process and ensures the accuracy and effectiveness of subsequent processing.

[0043] Step 2: Image feature extraction: The system uses deep learning algorithms, such as convolutional neural networks (CNNs), to extract key facial features from images.

[0044] Step 3: Person identity confirmation module: The extracted features are matched with the facial features stored in the database to confirm the identity of the person.

[0045] Identity confirmation result: If the identity confirmation is successful, the process continues to the template selection module; if not confirmed, the system will prompt the user to re-enter the image.

[0046] The deep learning model built into the all-in-one machine automatically processes the uploaded images, extracts key facial feature points, and matches them with facial features in the database to determine the identity of the person in the image to be detected. This process involves complex image processing and pattern recognition technology, among which deep learning models, especially convolutional neural networks (CNNs), play a core role in feature extraction.

[0047] Step 4: Template selection module: Once the identity is confirmed, the system selects a facial 3D shape feature template corresponding to the person from the pre-registered facial shape template set.

[0048] These templates are built using 3D Morphable Model (3DMM) technology, which contains the three-dimensional shape features of the faces of different people. 3DMM technology builds face models through principal component analysis (PCA), which can be expressed as: in, is the reconstructed 3D face shape, is the base model, is the i-th principal component, is the corresponding coefficient.

[0049] Step 5: Facial 3D shape extraction module: Use 3D Morphable Model (3DMM) technology to extract the 3D shape features of the face from the image to be detected.

[0050] Using 3DMM technology, the all-in-one machine extracts the three-dimensional shape features of the face from the image to be detected. This step involves mapping the two-dimensional image information into three-dimensional space to obtain a three-dimensional model of the face. The application of 3DMM technology makes it possible to extract three-dimensional shape features from two-dimensional images, providing strong support for subsequent face-changing forgery detection.

[0051] Step 6: 3D-Aware Masked Diffusion Detection: Combining facial 3D shape features and conditional masks, the diffusion model technology is used to perform high-fidelity face-changing forgery detection.

[0052] The all-in-one machine inputs the extracted 3D shape features and conditional masks into the 3D-Aware Masked Diffusion module. This module uses the generation capability of the diffusion model, combined with facial 3D shape features and conditional masks, to perform high-fidelity face-changing forgery detection.

[0053] Step 7: Detection results: The system analyzes the output of the 3D-Aware Masked Diffusion module to determine whether the image is a face-swap forgery.

[0054] Based on the detection results of the 3D-Aware Masked Diffusion module, the all-in-one machine will output whether the image to be detected is a face-swap forged image. If the detected face-swap forged features are significantly different from the real face features, it is judged to be a forged image; otherwise, it is judged to be a real face image. The design of this output module takes into account the needs of users and provides intuitive detection results, allowing users to quickly take corresponding actions based on the detection results.

[0055] User interface feedback: The all-in-one machine provides test results to users through the user interface, including graphic display and text description. The user interface is designed to be intuitive and easy to understand, providing clear instructions such as "Test result: real" or "Test result: fake". This intuitive feedback mechanism ensures that users can quickly understand the test results and make decisions accordingly.

[0056] The method provided by the embodiment of the present application is described in detail below in conjunction with the accompanying drawings. The technical solution of the present invention includes the following key modules: Part 1: Person identity confirmation module: extracts the person's facial features from the image to be detected and matches them with the facial features in the database to confirm the identity of the person in the image to be detected.

[0057] This module is the first line of defense of the entire system and is responsible for extracting key facial features from the image to be detected. This process involves the application of deep learning algorithms, especially convolutional neural networks (CNNs), which can learn useful feature representations from complex image data. Through its multi-layer structure, CNN can capture features from low-level to high-level, thus providing strong feature support for subsequent identity confirmation.

[0058] In the feature extraction process, the output of CNN is used as the basis for comparison with the pre-stored facial features in the database. This comparison process can be achieved by a variety of distance metrics, such as Euclidean distance, cosine similarity or Mahalanobis distance. These metrics can quantify the similarity or difference between two feature vectors. Specifically, the feature matching process can be expressed as: Similarity = .

[0059] in, is the feature of the image to be detected, is the feature of the corresponding identity in the database, is a feature of a non-corresponding identity in the database, is the distance metric function.

[0060] Part II: Template selection module: Once the identity of a person is confirmed, a facial 3D shape feature template corresponding to the person is selected from a pre-registered facial shape template set.

[0061] Once the identity of the person in the image to be detected is confirmed, the template selection module comes into play. The task of this module is to select the facial 3D shape feature template corresponding to the confirmed identity from the pre-registered facial shape template set. These templates are built based on 3D Morphable Model (3DMM) technology, which is a statistical model that can describe the natural variations in face shape and texture.

[0062] 3DMM technology constructs a face model through principal component analysis (PCA), and the process can be expressed as: ;in, is the reconstructed 3D face shape, is the base model, representing the average face shape, is the i-th principal component, representing the change in face shape, are the corresponding coefficients, which determine how the face shape varies along these principal components.

[0063] Part III: Facial 3D shape extraction module: Use 3D Morphable Model (3DMM) technology to extract facial 3D shape features from the image to be detected.

[0064] Figure 3 The schematic diagram of the facial 3D shape extraction process is shown. The facial 3D shape extraction module is the core of the present invention, which uses 3DMM technology to extract the 3D shape features of the face from the image to be detected. This process not only involves restoring 3D information from a 2D image, but also requires ensuring the accuracy and completeness of this information.

[0065] The application of 3DMM technology makes it possible to extract three-dimensional shape features from two-dimensional images. By comparing the image to be detected with the base model and the principal component, the coefficients can be determined. , thereby reconstructing the 3D shape of the face. This process is crucial for subsequent face-swapping forgery detection because it provides a highly detailed 3D model of the face that can be used to compare with the real 3D model of the face.

[0066] Step 1: 2D image: input original face image.

[0067] Step 2: Feature point location: locate key facial feature points in the image.

[0068] Step 3: Key feature points: feature points used for 3D model fitting.

[0069] Step 4: 3D model fitting: Use key feature points to build a preliminary 3D model.

[0070] Step 5: 3D face model: The final 3D face shape model.

[0071] Step 6: Parameter extraction: Extract shape parameters from the 3D model, which are used for subsequent face-changing forgery detection.

[0072] Part 4: 3D-Aware Masked Diffusion Module: Through 3D-Aware Masked Diffusion technology, combined with facial three-dimensional shape features and conditional masks, high-fidelity face-changing forgery detection is performed.

[0073] The 3D-Aware Masked Diffusion module is another key innovation of the present invention. This module uses the latest diffusion model technology, combined with facial 3D shape features and conditional masks, to perform high-fidelity face-changing forgery detection. The diffusion model is a generative model that transfers data from its original distribution to a known simple distribution by gradually adding noise, and then learns how to reverse this process.

[0074] In the present invention, this process can be expressed as: ;in, is the original image, is the image after adding noise, is the noise added at each step. The inverse of this process, i.e., recovering the original image from the noise, is learned by the diffusion model.

[0075] In this paper, the diffusion model not only considers the two-dimensional information of the image, but also maintains the shape of the source face through conditional masks and 3D-aware landmarks as conditional inputs. This 3D-aware processing method enables the model to better understand and process the three-dimensional structure of the face, thereby improving the accuracy and robustness of detection.

[0076] Figure 4 The 3D-Aware Masked Diffusion detection process is shown, which specifically includes the following steps: Step 1: Input image: the image to be detected.

[0077] Step 2: Adding Noise: Adding noise to the image simulates the beginning of the diffusion process.

[0078] Step 3: Noise image: The image after adding noise is used for subsequent diffusion model processing.

[0079] Step 4: Conditional mask application: A conditional mask is applied to maintain the shape characteristics of the source face.

[0080] Step 5: Masked Image: The image after applying the mask for feature preservation.

[0081] Step 6: Feature Preservation: Maintain key features during the diffusion process to ensure the consistency of the face shape.

[0082] Step 7: High-fidelity image: High-fidelity image restored by the diffusion model for forgery detection.

[0083] Step 8: Forgery Detection: Analyze high-fidelity images to detect whether there is face-swapping forgery.

[0084] Step 9: Whether it is forged: Determine whether the image is forged based on the detection results.

[0085] Step 10: Output: Output the detection results and mark them as real or fake.

[0086] Part 5: Output module: Based on the detection results, output whether the image to be detected is a face-swapped forged image.

[0087] Finally, the output module will output the final judgment of whether the image to be detected is a face-swapping forged image based on the detection results of the 3D-Aware Masked Diffusion module. This judgment is based on a comprehensive analysis of the three-dimensional shape features and conditional masks. If the detected face-swapping forged features are significantly different from the real face features, it is judged to be a forged image; otherwise, it is judged to be a real face image.

[0088] Figure 5 The user interface feedback diagram is shown. Specifically including: User interface: the overall user interface, providing intuitive operation and feedback. Image display area: the area that displays the image to be detected, where users can view the uploaded image. Image to be detected: the image currently being processed, displayed in the image display area. Result indication: an indicator that displays the detection result, such as "detecting" or "completed". Detection result: a specific detection result, such as "real" or "fake", and possible confidence scores. Detailed information: provides detailed information on the detection result, including the technology used, a brief description of the detection process, etc. Mahalanobis distance value: displays the Mahalanobis distance value used for judgment, which is a method of measuring feature similarity. Operation buttons: including buttons for "re-detection" and "save results", and users can re-detect or save the current detection result as needed.

[0089] In summary, the embodiment of the present application provides a forged face detection method, which extracts face features from the image to be detected; matches the face shape feature template corresponding to the face features from the database; extracts face shape features from the image to be detected in combination with the face shape feature template; restores a restored image by combining the face shape features and conditional masks using a diffusion model; and determines whether the image to be detected is a forged face image by comparing the restored image with the image to be detected. Detailed face shape features are extracted from the image to be detected in combination with the face shape feature template, and a high-fidelity image is restored by combining the face shape features and conditional masks using a diffusion model. By comparing the restored image with the image to be detected, it is determined whether the image to be detected is a forged face image.

[0090] Based on the same technical concept, the embodiment of the present application also provides a fake face detection system, such as Figure 6 As shown, the system comprises: The face feature extraction module 601 is used to extract face features from the image to be detected; A feature template matching module 602 is used to match a face shape feature template corresponding to the face feature from a database; A face shape feature module 603 is used to extract face shape features from the image to be detected in combination with the face shape feature template; An image restoration module 604 is used to restore a restored image by using a diffusion model combined with the face shape features and a conditional mask; The forgery detection module 605 is used to determine whether the image to be detected is a forged image of a human face by comparing the restored image with the image to be detected.

[0091] The present application also provides an electronic device corresponding to the method provided in the above embodiment. Figure 7 , which shows an electronic device diagram provided by some embodiments of the present application. The electronic device 20 may include: a processor 200, a memory 201, a bus 202 and a communication interface 203, wherein the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and the processor 200 executes the method provided by any of the aforementioned embodiments of the present application when running the computer program.

[0092] The memory 201 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one physical port (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0093] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs, and the processor 200 executes the programs after receiving execution instructions. The method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 200, or implemented by the processor 200.

[0094] The processor 200 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 200. The above processor 200 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

[0095] The electronic device provided in the embodiment of the present application and the method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, operated or implemented by them.

[0096] The present application also provides a computer-readable storage medium corresponding to the method provided in the above embodiment. Figure 8The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the method provided in any of the aforementioned embodiments is executed.

[0097] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for detecting fake faces, characterized in that: The method comprises: Extract facial features from the image to be detected; Matching a facial shape feature template corresponding to the facial feature from a database; Extracting face shape features from the image to be detected in combination with the face shape feature template; Restoring a restored image by combining the face shape features and the conditional mask using a diffusion model; By comparing the restored image with the image to be detected, determining whether the image to be detected is a forged face image; In combination with the face shape feature template, extracting face shape features from the image to be detected includes: Locating set facial feature points from the image to be detected; Using the set facial feature points, mapping the three-dimensional shape information in the facial shape feature template to the corresponding position in the image to be detected to obtain a three-dimensional facial shape model; Extracting the face shape features from the three-dimensional face shape model; The conditional mask is used to indicate a set area for face detection; the conditional mask is generated based on facial landmarks to cover the set area of ​​the target face.

2. The method according to claim 1, characterized in that The method of restoring the restored image by combining the face shape feature with the conditional mask using the diffusion model includes: gradually adding noise to the image to be detected to obtain a noise image, so as to simulate the beginning of a diffusion process; Applying the conditional mask to the noisy image to obtain a masked image to preserve features during a simulated diffusion process; A restored image is restored from the masked image using the diffusion model.

3. The method according to claim 2, characterized in that The step of comparing the restored image with the image to be detected to determine whether the image to be detected is a forged face image includes: If the difference between the restored image and the image to be detected in the set area meets the set condition, it is determined that the image to be detected is a forged face image.

4. The method according to claim 1, characterized in that Matching a face shape feature template corresponding to the face feature from a database includes: Matching the facial features with facial features stored in the database to confirm the identity of the person in the image to be detected; If the person's identity is confirmed successfully, a facial shape feature template uniquely corresponding to the person's identity is matched from the facial shape template set pre-registered in the database.

5. The method according to claim 4, characterized in that The face shape feature template construction process includes: Determine a base model according to an average face shape of a plurality of face data; According to the principal component analysis technology, the principal component components of several faces are extracted respectively, and each principal component component represents the shape change pattern of the face part; Assigning a coefficient of variation to each principal component, the coefficient of variation characterizing the degree to which the shape varies along the principal component; The base model and all principal components are weighted and summed according to the variation coefficient to obtain a reconstructed three-dimensional face shape; The reconstructed three-dimensional facial features are bound to the person identity information corresponding to the facial data to obtain a facial shape feature template and store it in the database.

6. A fake face detection system, characterized in that: The system comprises: A face feature extraction module is used to extract face features from the image to be detected; A feature template matching module, used for matching a face shape feature template corresponding to the face feature from a database; A face shape feature module, used to extract face shape features from the image to be detected in combination with the face shape feature template; An image restoration module, used to restore a restored image by using a diffusion model combined with the face shape features and a conditional mask; The forgery detection module is used to determine whether the image to be detected is a forged image of a human face by comparing the restored image with the image to be detected.

7. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1-5.

Images