Method and system for identifying authenticity of forged face image based on likelihood ratio

Abstract

The invention provides a counterfeit face image authenticity identification method and system based on a likelihood ratio, and the method comprises the steps: building an extractor through a feature extraction module based on a plurality of advanced face key point detection models, and obtaining the contour and local features of a face; the model training module is used for training a contour and local similarity calculation model by using a convolutional neural network and a reference ArcFace loss function; the similarity distribution statistics module is used for counting contours and local similarity distribution probabilities of homologous and non-homologous faces by applying advanced methods such as stratified sampling and the like; and the likelihood ratio calculation module obtains an identification likelihood ratio result according to the calculated similarity, and judges the authenticity of the image through a set threshold value. According to the method, multiple biological feature extraction is carried out on the deep counterfeit face, the similarity difference between features of the counterfeit face and the source face is utilized, the identification likelihood ratio of the deep counterfeit face is obtained through a likelihood ratio method of Bayesian estimation, and the deep counterfeit face can be effectively identified.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and recognition, and particularly relates to a method and system for authenticating the authenticity of forged face images based on likelihood ratio. Background Art

[0002] With the rapid development of artificial intelligence technology, the Deepfake technology has become increasingly mature. The synthetic face images or videos created by means of deep learning bring novel experiences while also causing serious security risks, such as identity theft, false information dissemination, etc.

[0003] Currently, the detection of Deepfake images mainly relies on computer vision and deep learning technologies. Common methods include artifact recognition method and unoriented method. The artifact recognition method is based on the inconsistencies in the fusion process, such as features like blinking and head pose; the unoriented method constructs a detection model by means of convolutional neural network (CNN) and generative adversarial network (GAN), and trains the neural network with a large amount of real and forged data to identify forged features.

[0004] However, these methods generally have defects such as poor generalization ability, dependence on specific identity information, and weak adaptability to new forgery means. Moreover, in application scenarios such as forensic science identification, the detection results often lack sufficient theoretical basis and interpretability, and it is difficult to meet the actual needs.

[0005] In view of this, it is very meaningful to propose a method and system for authenticating the authenticity of forged face images based on likelihood ratio. Summary of the Invention

[0006] Aiming at the problems that the existing detection methods generally have defects such as poor generalization ability, dependence on specific identity information, and weak adaptability to new forgery means, and in application scenarios such as forensic science identification, the detection results often lack sufficient theoretical basis and interpretability, the present invention provides a method and system for authenticating the authenticity of forged face images based on likelihood ratio.

[0007] In a first aspect, the present invention proposes a method for authenticating the authenticity of forged face images based on likelihood ratio, and the method includes the following steps:

[0008] S1. Construct a contour extractor and a face local extractor, detect the face on the image based on a preset face key point detection model, and extract the face contour and specified face local features;

[0009] S2. Build and train a contour similarity calculation model, the backbone network of the contour similarity calculation model is a convolutional neural network, accepts an input with width, height and channels being (n, n, 3) respectively, and is trained using a training loss function;

[0010] S3. Build and train no less than three face local similarity calculation models, including human eye, human mouth, human nose, human ear and human eyebrow similarity comparison models. The model structure, training method and training loss function of the face local similarity calculation model are the same as those of the contour similarity calculation model;

[0011] S4. Use the same calculation method to respectively count the similarities between homologous face contours and non - homologous face contours, and calculate the contour similarity distribution probability; and count the multiple local similarities between homologous faces and non - homologous faces, and calculate the multi - local similarity distribution probability;

[0012] S5. Through the contour similarity calculation model and the face local similarity calculation model, obtain the contour similarity between the forged face and the suspected source face and the multi - local similarity including human eyes, human mouth, human nose, human ears and human eyebrows, and further calculate the likelihood ratio result for the identification of the forged face.

[0013] Preferably, S5 specifically includes:

[0014] Obtain the contour similarity δ between the forged face and the suspected source face through the contour similarity calculation model and the face local similarity calculation model contour , multi - local similarities δ eye , δ mouth , δ nose , δ brow , δ ear , and calculate the likelihood ratio result for the identification of the forged face based on the following formula:

[0015] The likelihood ratio of the face contour is:

[0016]

[0017] The likelihood ratio of the facial features is:

[0018]

[0019] where I = {eye, mouth, nose, brow, ear,...}, and N is the number of face local similarity models;

[0020] Finally, the likelihood ratio result for the identification of the forged face is:

[0021]

[0022] The larger the r value, the greater the forgery probability. Judge the forgery probability according to the set threshold.

[0023] More preferably, it further includes: The specific judgment threshold r is determined by the statistical results of the contour similarity calculation model and the face local similarity calculation model. When r ≥ 10000, it indicates a high forgery probability.

[0024] Further preferably, in S4, the calculation formula of the calculation method is as follows:

[0025]

[0026] Among them, where H p represents that the samples are homologous, and H d represents that the samples are not homologous; M g (δ) represents the number of comparison times when the similarity is δ in the ratio of homologous face contour images; M i (δ) represents the number of comparison times when the similarity is δ in the ratio of non - homologous face contour images; the value range of δ is an integer from 0 to 100; N g represents the total number of comparison times or the number of relevant samples of homologous samples; N i represents the total number of comparison times or the number of relevant samples of non - homologous samples.

[0027] Preferably, in S2, the training loss function is set as the ArcFace loss function:

[0028]

[0029] W j T x i = ||W i ||||x i ||cosθ j

[0030] Among them, m is the number of pictures in a batch; K is the number of categories in the dataset, that is, the number of person IDs in the dataset; s is the scaling factor; η is the angular margin; W j T x i is the prediction result, where W j and x i are the weight matrix and the input data vector in the model respectively.

[0031] Further preferably, on the basis of the ArcFace loss function, the training loss functions of the contour similarity calculation model and the face local similarity calculation model also include the Softmax loss function, the Cosine loss function and the Triplet loss function, and parameter adjustment and optimization are carried out according to different datasets and application scenarios.

[0032] Preferably, in S4, when statistically analyzing the distribution probabilities of the contours and local similarities of homologous and non - homologous faces, data sampling and statistical analysis methods such as stratified sampling, adaptive sampling, kernel density estimation and Bayesian network analysis are also included to further calculate the distribution probabilities.

[0033] Preferably, in S1, the preset face key point detection model includes the face key point detection model in the dlib library, the MTCNN model based on the convolutional neural network, and the HRNet model.

[0034] In a second aspect, an authenticity identification system for forged face images based on the likelihood ratio provided by an embodiment of the present invention includes:

[0035] A feature extraction module, configured to construct a contour extractor and a face local extractor, and the contour extractor and the face local extractor detect the face in the input image based on a conventional face key point detection model, and extract the face contour and specified face local features;

[0036] A model training module, which includes a contour similarity calculation model training unit and no less than three face local similarity calculation model training units. The backbone networks of the contour similarity calculation model and the face local similarity calculation model are convolutional neural networks, which accept inputs with widths, heights, and channels of (n, n, 3) respectively, and the contour similarity calculation model training unit uses a training loss function to train the contour similarity calculation model;

[0037] A similarity distribution statistics module, configured to statistically analyze the similarity between homologous face contours and non-homologous face contours, and calculate the contour similarity distribution probability. At the same time, this module is also used to statistically analyze the multiple local similarities between homologous faces and non-homologous faces, and calculate the multi-local similarity distribution probability. The calculation method is the same as the contour likelihood ratio statistics;

[0038] A likelihood ratio calculation module, configured to respectively obtain the contour similarity between the forged face and the suspected source face and the multi-local similarity including human eyes, human mouths, human noses, human ears, and human eyebrows through the contour similarity calculation model and the face local similarity calculation model, and further calculate the identification likelihood ratio result of the forged face.

[0039] In a third aspect, an embodiment of the present invention provides an electronic device, including: one or more processors; a storage device, configured to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method described in any implementation manner in the first aspect.

[0040] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] The method disclosed in the present invention extracts multiple biometric features from deepfake faces, and utilizes the similarity difference between the features of the fake face and the source face. Through the likelihood ratio method of Bayesian estimation, the identification likelihood ratio value of the deepfake face is obtained, and the deepfake face can be effectively identified. This identification method has good generalization ability and can adapt to new forgery techniques. At the same time, the computational complexity is relatively low, which can meet the real-time detection requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The drawings illustrate the embodiments and, together with the description, are used to explain the principles of the present invention. Other embodiments and many of the intended advantages of the embodiments will be readily apparent as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily drawn to scale. Like reference numerals refer to corresponding like parts.

[0044] Figure 1 is an exemplary device architecture diagram to which an embodiment of the present invention can be applied;

[0045] Figure 2 is a schematic flowchart of the method for authenticating the authenticity of a forged face image based on the likelihood ratio according to an embodiment of the present invention;

[0046] Figure 3 is an overall calculation flowchart of the method for authenticating the authenticity of a forged face image based on the likelihood ratio according to a specific embodiment of the present invention;

[0047] Figure 4 is a statistical flowchart of the contour similarity distribution according to a specific embodiment of the present invention;

[0048] Figure 5 is a schematic diagram of the system for authenticating the authenticity of a forged face image based on the likelihood ratio according to an embodiment of the present invention;

[0049] Figure 6 is a schematic structural diagram of a computer device of an electronic device suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only for explaining the related invention and not for limiting the invention. Additionally, it should be noted that for the sake of description, only parts related to the relevant invention are shown in the drawings.

[0051] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the drawings and embodiments.

[0052] Figure 1 Fig. 1 shows an exemplary system architecture 100 to which the likelihood ratio-based forged face image authenticity identification method or the likelihood ratio-based forged face image authenticity identification system according to the embodiments of the present invention can be applied.

[0053] As Figure 1 shown in Fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0054] Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0055] The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they may be installed in the above-listed electronic devices. They may be implemented as multiple software or software modules (such as software or software modules for providing distributed services), or may be implemented as a single software or software module. No specific limitation is made here.

[0056] The server 105 may be a server that provides various services, such as a background information processing server that processes the verification request information sent by the terminal devices 101, 102, 103. The background information processing server may analyze and process the received verification request information and obtain a processing result.

[0057] It should be noted that the likelihood ratio-based forged face image authenticity identification method provided by the embodiments of the present invention is generally executed by the server 105. Correspondingly, the likelihood ratio-based forged face image authenticity identification system is generally set in the server 105. In addition, the likelihood ratio-based forged face image authenticity identification method provided by the embodiments of the present invention is generally executed by the terminal devices 101, 102, 103. Correspondingly, the likelihood ratio-based forged face image authenticity identification system is generally set in the terminal devices 101, 102, 103.

[0058] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers or as a single server. When the server is software, it can be implemented as multiple software or software modules (such as those used to provide distributed services), or as a single software or multiple software modules, and no specific limitations are made here.

[0059] It should be understood that Figure 1 the numbers of the terminal devices, the network, and the server in

[0060] Deepfake technology refers to the technology of using artificial intelligence, especially deep learning technology, to create synthetic images or videos. This technology is often used to replace the facial features of a person in a video to make it look like another person, or to create completely virtual characters. To counter the potential hazards brought by deepfakes, the detection technology of deepfake images has emerged.

[0061] The detection of deepfake images mainly relies on computer vision and deep learning technology to identify and prevent forged content in images. Currently, in addition to some traditional detection methods, it mainly focuses on the artifact-specific method and the undirected approach. Among them, the artifact-specific method is based on the inconsistencies generated during the fusion process, such as blinking, head posture, etc. The undirected approach is to build a detection model based on convolutional neural network (CNN) and generative adversarial network (GAN), collect a variety of different deepfake networks, and generate a dedicated dataset. By learning a large number of real and forged data, the neural network is allowed to analyze its features and solve the deepfake detection and identification tasks in different scenarios.

[0062] Although certain progress has been made in the current identification technology of deepfake images, there are still shortcomings such as insufficient generalization ability, dependence on identity information, and limitations to specific forgery methods. Moreover, although there has been extensive research on the detection of deepfake images, there is still little work related to the identification of deepfake images for forensic science purposes, and the detection and identification results often lack sufficient theoretical basis. Therefore, the original deep synthesis and generation methods can only perform detection, give the probability of detection and discrimination, have poor interpretability and visualization effects, cannot explicitly display the traces of forgery, and cannot explain the reasons for forgery.

[0063] The method proposed in the embodiments of the present invention mainly aims at the problem of poor interpretability of the original detection method, and proposes a method for face identity authentication based on the likelihood ratio. This method uses a multiple-depth feature extractor to extract the biometric features of the forged face and the source face. The similarity between the features is calculated using a neural network, and the likelihood ratio of different features is calculated by statistically analyzing the distribution of the similarity of different network features. The authentication result of the deepfake face is obtained through the ratio between the likelihood ratios. Through this method, the deepfake face can be effectively authenticated. It does not depend on the forgery method and has good generalization ability. At the same time, its authentication result is more in line with the statistical concept, making the authenticity authentication of the face generated by deep synthesis more interpretable and more applicable to the application scenarios related to forensic science authentication.

[0064] In a first aspect, an embodiment of the present invention discloses a method for authenticating the authenticity of a forged face image based on the likelihood ratio, as Figure 2 shown, the method includes the following steps:

[0065] S1. Construct a contour extractor and a face local extractor, detect the face on the image based on a preset face key point detection model, and extract the face contour and specified face local features;

[0066] In this step, the preset face key point detection model includes the face key point detection model in the dlib library, the MTCNN model based on a convolutional neural network, the HRNet model, etc.

[0067] S2. Build and train a contour similarity calculation model. The backbone network of the contour similarity calculation model is a convolutional neural network, which accepts an input with width, height, and channels of (n, n, 3) respectively, and is trained using a training loss function;

[0068] Specifically, the training loss functions of the contour similarity calculation model and the face local similarity calculation model, on the basis of the ArcFace loss function, also include the Softmax loss function, the Cosine loss function, the Triplet loss function, etc., and the parameters are adjusted and optimized according to different data sets and application scenarios.

[0069] In this embodiment, the training loss function is set to the ArcFace loss function:

[0070]

[0071] W j T x i =||W i ||||x i ||cosθ j

[0072] Among them, m is the number of images in a batch; K is the number of categories in the dataset, that is, the number of person IDs in the dataset; s is the scaling factor; η is the angular spacing; W j T x i is the prediction result, where W j and x i are the weight matrix and the input data vector in the model respectively.

[0073] S3. Build and train at least three face local similarity calculation models, including human eye, human mouth, human nose, human ear and human eyebrow similarity comparison models. The model structure, training method and training loss function of the face local similarity calculation model are the same as those of the contour similarity calculation model;

[0074] S4. Use the same calculation method to respectively count the similarities between homologous face contours and non-homologous face contours, and calculate the contour similarity distribution probability; and count the multiple local similarities between homologous faces and non-homologous faces, and calculate the multi-local similarity distribution probability;

[0075] Preferably, when counting the contour and local similarity distribution probabilities of homologous and non-homologous faces, it also includes data sampling and statistical analysis methods such as stratified sampling, adaptive sampling, kernel density estimation and Bayesian network analysis to further calculate the distribution probability.

[0076] In this embodiment, the calculation formulas for the contour similarity distribution probability and the multi-local similarity distribution probability are as follows:

[0077]

[0078] Among them, where H p represents that the samples are homologous, and H d represents that the samples are not homologous; M g (δ) represents the number of times the similarity of δ appears in the comparison of homologous face contour images; M i (δ) represents the number of times the similarity of δ appears in the comparison of non-homologous face contour images; the value range of δ is an integer from 0 to 100; N g represents the total number of comparisons or the number of relevant samples of homologous samples; N i represents the total number of comparisons or the number of relevant samples of non-homologous samples.

[0079] S5. Through the contour similarity calculation model and the face local similarity calculation model, respectively obtain the contour similarity between the forged face and the suspected source face and the multi-local similarity including human eyes, human mouth, human nose, human ears and human eyebrows, and further calculate the identification likelihood ratio result of the forged face.

[0080] Specifically, in this embodiment, this step specifically includes:

[0081] Obtain the contour similarity δ between the forged face and the suspected source face through the contour similarity calculation model and the local face similarity calculation model contour , multiple local similarities δ eye , δ mouth , δ nose , δ brow , δ ear , and calculate the likelihood ratio result of the forged face identification based on the following formula:

[0082] The likelihood ratio of the face contour is:

[0083]

[0084] The likelihood ratio of the facial features is:

[0085]

[0086] where I = {eye, mouth, nose, brow, ear,...}, and N is the number of local face similarity models;

[0087] Finally, obtain the likelihood ratio result of the forged face identification as:

[0088]

[0089] The larger the r value, the greater the forgery probability, and the forgery probability is judged according to the set threshold.

[0090] Furthermore, the specific judgment threshold r is determined by the statistical results of the contour similarity calculation model and the local face similarity calculation model. In this embodiment, when r ≥ 10000, it indicates a high forgery probability.

[0091] In a specific embodiment, as Figure 3 shown, the steps of the method for authenticating the authenticity of a forged face image based on likelihood ratio of the present invention are specifically as follows:

[0092] Step 1: Construct a contour extractor and a local face extractor. Detect the face on the image, and extract the face contour and the specified local face. There are various construction methods for the extractor. Here, the extractor is constructed based on a conventional face key point detection model. According to the face key point detection results, extract the local face features.

[0093] Step 2: Build and train a contour similarity calculation model. The backbone network of this model is based on a face similarity calculation network, which is a convolutional neural network and accepts an input with width, height, and channels of (n, n, 3) respectively. The training loss function is as follows:

[0094]

[0095] W j T x i = ||W i ||||x i ||cosθ j

[0096] where m is the number of images in a batch; K is the number of categories in the dataset, i.e., the number of person IDs in the dataset; s is the scaling factor; η is the angular spacing; W j T x i is the prediction result. Here, L refers to the ArcFace loss function.

[0097] Step 3: Build and train at least three face local similarity calculation models. Including but not limited to the human eye similarity comparison model, human mouth similarity comparison model, human nose similarity comparison model, human ear similarity comparison model, human eyebrow similarity comparison model. The model structure, training method, and training loss function are all the same as those of the contour similarity calculation model.

[0098] Step 4: Statistically analyze the similarities between homologous and non-homologous face contours and calculate the contour similarity distribution. The method for calculating the similarity distribution probability of homologous and non-homologous face contour images is as follows:

[0099]

[0100] where, where H p indicates that the samples are homologous, and H d indicates that the samples are not homologous; M g (δ) represents the number of comparison times with a similarity of δ in the homologous face contour image ratio; M i (δ) represents the number of comparison times with a similarity of δ in the non-homologous face contour image ratio; δ takes integer values ranging from 0 to 100.

[0101] Figure 4 shows the contour similarity distribution statistical flowchart of this embodiment.

[0102] Step 5: Statistically analyze the multiple local similarities between homologous and non-homologous faces and calculate the multi-local similarity distribution. The calculation method is the same as the contour likelihood ratio statistics in Step 4.

[0103] Step 6: Calculate and obtain the identification likelihood ratio result of the forged face. Through the similarity calculation model, obtain the contour similarity δ contour , multi-local similarity δ eye , δ mouth , δ nose , δ brow , δear .

[0104] Since the likelihood ratio based on similarity can be approximately calculated as:

[0105]

[0106] where g represents the probability density distribution of the similarity score S(x,y), and I represents the background information. At N g and N i being large enough, the left and right sides of the approximate equation are approximately equal.

[0107] Then the likelihood ratio of the face contour is:

[0108]

[0109] The likelihood ratio of the facial features is:

[0110]

[0111] where I = {eye, mouth, nose, brow, ear,...}, and N is the number of local face similarity models.

[0112] Finally, the likelihood ratio result of forged face identification is:

[0113]

[0114] The larger the r value, the greater the forgery probability. The specific judgment threshold is determined by the statistical results of the similarity calculation model. Generally speaking, r ≥ 10000.

[0115] With the wide application of deepfake technology, deepfake face identification technology has become one of the important topics in network security. Its application prospect is very broad, involving multiple industries and social fields, including social media, financial banking, political election security, entertainment industry, etc. However, since most technologies directly discriminate based on neural network models, their results are often not interpretable. The method in this paper extracts multiple biometric features from deepfake faces, and uses the similarity difference between the features of forged faces and source faces. Through the likelihood ratio method of Bayesian estimation, the likelihood ratio value of deepfake face identification is obtained, which can effectively identify deepfake faces. This identification method has good generalization ability and can adapt to new forgery technologies. At the same time, the computational complexity is relatively low, which can meet the real-time detection requirements.

[0116] For further reference Figure 5 , as an implementation of the methods shown in the above figures, this application provides an embodiment of a forged face image authenticity identification system based on likelihood ratio. This system embodiment is related to Figure 2The method embodiments shown correspond to a system that can be specifically applied to various electronic devices.

[0117] In a second aspect, embodiments of the present invention also disclose a system for authenticating the authenticity of forged face images based on the likelihood ratio, as Figure 5 shown. The system includes: a feature extraction module 51, a model training module 52, a similarity distribution statistics module 53, and a likelihood ratio calculation module 54.

[0118] In this embodiment, the feature extraction module 51 is used to construct a contour extractor and a face local extractor. The contour extractor and the face local extractor detect faces in the input image based on a conventional face key point detection model, and extract the face contour and specified face local features. The model training module 52 includes a contour similarity calculation model training unit and no less than three face local similarity calculation model training units. The backbone networks of the contour similarity calculation model and the face local similarity calculation model are convolutional neural networks, which accept inputs with widths, heights, and channels of (n, n, 3) respectively. And the contour similarity calculation model training unit uses a training loss function to train the contour similarity calculation model.

[0119] The similarity distribution statistics module 53 is used to statistically analyze the similarities between homologous face contours and non-homologous face contours, and calculate the contour similarity distribution probability. At the same time, this module is also used to statistically analyze the similarities of multiple parts between homologous faces and non-homologous faces, and calculate the multi-local similarity distribution probability. The calculation method is the same as that of the contour likelihood ratio statistics. The likelihood ratio calculation module 54 is used to obtain the contour similarity between the forged face and the suspected source face, and the multi-local similarity including human eyes, human mouth, human nose, human ears, and human eyebrows, through the contour similarity calculation model and the face local similarity calculation model, and further calculate the identification likelihood ratio result of the forged face.

[0120] The functions of the above modules correspond to the methods and will not be elaborated here.

[0121] In a specific embodiment, the specific implementation manner of the system is as follows:

[0122] System initialization and model training: When the system starts, first load the selected face key point detection model, and train the contour similarity calculation model and the face local similarity calculation model according to the preset parameters and training data. The training data should cover a large number of homologous and non-homologous face images, and should include samples under different ages, genders, races, postures, and lighting conditions to ensure the generalization ability of the model. During the training process, continuously adjust the weights and parameters of the model according to the feedback of the loss function until the model converges and reaches the predetermined performance indicators.

[0123] Image Input and Feature Extraction: Input the face image to be identified into the system. The feature extraction module 51 processes the image using the constructed extractor to obtain the face contour and local feature information, and transmits them to the subsequent modules. For the quality and format of the input image, the system should have certain preprocessing capabilities, such as enhancing blurred images and converting images of different formats, to ensure the accuracy of feature extraction.

[0124] Similarity Calculation and Distribution Statistics: The model trained by the model training module 52 calculates the similarity of the extracted features to obtain the similarity scores of the contour and local parts. The similarity distribution statistics module 53 calculates the similarity distribution probabilities of homologous and non-homologous faces using the selected statistical method based on these scores. During the statistical process, it is necessary to ensure the accuracy and integrity of the data, and reasonably process or mark abnormal data.

[0125] Likelihood Ratio Calculation and Result Judgment: The likelihood ratio calculation module 54 calculates the likelihood ratio according to the data obtained in the previous steps according to the formula and compares it with the preset threshold. If the calculated value is greater than or equal to 10000, it is determined that the possibility of the face image being forged is relatively high; otherwise, it is determined to be highly authentic. The system should output the identification result in an intuitive way, such as marking the authenticity information on the image or generating an identification report, and record the relevant intermediate data and processing process for subsequent analysis and verification.

[0126] During the entire implementation process, the system should be regularly evaluated for performance and the model updated, introducing new datasets and improved algorithms to continuously enhance the system's identification ability and adaptability, ensuring its effectiveness in dealing with constantly evolving deepfake technologies.

[0127] The following refers to Figure 6 , which shows a schematic structural diagram of a computer device 600 of an electronic device (such as Figure 1 the server or terminal device shown) suitable for implementing the embodiments of the present invention. Figure 6 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0128] As Figure 6As shown, the computer device 600 includes a central processing unit (CPU) 601 and a graphics processing unit (GPU) 602, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 603 or a program loaded from a storage section 609 into a random access memory (RAM) 604. In the RAM 604, various programs and data required for the operation of the device 600 are also stored. The CPU 601, GPU 602, ROM 603, and RAM 604 are connected to each other via a bus 605. An input / output (I / O) interface 606 is also connected to the bus 605.

[0129] The following components are connected to the I / O interface 606: an input section 607 including a keyboard, a mouse, etc.; an output section 608 including, for example, a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 609 including a hard disk, etc.; and a communication section 610 including a network interface card such as a LAN card, a modem, etc. The communication section 610 performs communication processing via a network such as the Internet. A drive 611 may also be connected to the I / O interface 606 as needed. A removable medium 612, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 611 as needed so that a computer program read from it can be installed into the storage section 609 as needed.

[0130] Specifically, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 610, and / or installed from the removable medium 612. When the computer program is executed by the central processing unit (CPU) 601 and the graphics processing unit (GPU) 602, the above functions defined in the method of the present invention are executed.

[0131] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium, a computer-readable medium, or any combination of the two. The computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus, or component, or any combination of the above. More specific examples of the computer-readable medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution device, apparatus, or component. In the present invention, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution device, apparatus, or component. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0132] The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of devices, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based device that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0134] The modules described in the embodiments of the present invention can be implemented in software or in hardware. The described modules can also be provided in a processor.

[0135] As another aspect, the present invention also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device is caused to: execute the methods and steps described in the first aspect.

[0136] The above description is only a preferred embodiment of the present invention and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features having similar functions disclosed in the present invention.

Claims

1. A method for identifying the authenticity of a forged face image based on likelihood ratio, characterized in that: The method comprises the following steps: S1, constructing a contour extractor and a face local extractor, detecting the face on the image based on a preset face key point detection model, and extracting the face contour and designated face local features; S2. Build and train a contour similarity calculation model, where the backbone network of the contour similarity calculation model is a convolutional neural network, which accepts inputs with width and height channels of (n, n, 3) respectively, and is trained using a training loss function; S3, build and train no less than three local face similarity calculation models, including human eye, human mouth, human nose, human ear and human eyebrow similarity comparison models, and the model structure, training method and training loss function of the local face similarity calculation model are consistent with the contour similarity calculation model; S4, using the same calculation method to respectively count the similarities between the contours of homologous faces and non-homologous faces, and calculate the contour similarity distribution probability; and count the multiple local similarities between the homologous faces and non-homologous faces, and calculate the multiple local similarity distribution probability; S5. The contour similarity calculation model and the face local similarity calculation model are used to respectively obtain the contour similarity and multiple local similarities including eyes, mouth, nose, ears and eyebrows between the forged face and the suspected source face, and further calculate the identification likelihood ratio result of the forged face.

2. The method for authenticity identification of forged facial images based on likelihood ratio according to claim 1, characterized in that: S5 specifically includes: The contour similarity δ between the forged face and the suspected source face is obtained by using the contour similarity calculation model and the face local similarity calculation model. contour , multiple local similarity δ eye ,δ mouth ,δ nose ,δ brow ,δ ear , and calculate the identification likelihood ratio of the forged face based on the following formula: The face contour likelihood ratio is: The likelihood ratio of facial features is: Where I = {eye, mouth, nose, brow, ear, ...}, N is the number of local face similarity models; The final result of the likelihood ratio of forged face identification is: The larger the r value is, the greater the probability of forgery is, and the probability of forgery is determined based on the set threshold.

3. The method for identifying the authenticity of a forged face image based on likelihood ratio according to claim 2, characterized in that: Also includes: The specific judgment threshold r is determined by the statistical results of the contour similarity calculation model and the face local similarity calculation model. When r≥10000, it indicates that the probability of forgery is high.

4. The method for authenticity identification of forged facial images based on likelihood ratio according to claim 1, characterized in that: In S4, the calculation formula of the calculation method is as follows: Among them, H p Indicates that the samples are homologous, H d Indicates that the samples are not homologous; M g (δ) represents the number of times the similarity δ appears in the face contour images of the same source; M i (δ) represents the number of comparisons with a similarity of δ in the non-homologous face profile image ratio; the value range of δ is an integer from 0 to 100; N g Indicates the total number of comparisons of homologous samples or the number of related samples; N i Represents the total number of alignments of non-homologous samples or the number of related samples.

5. The method for authenticity identification of forged facial images based on likelihood ratio according to claim 1, characterized in that: In S2, the training loss function is set to the ArcFace loss function: W j T x i =||W i ||||x i ||cosθ j Where m is the number of images in a batch; K is the number of categories in the dataset, that is, the number of person IDs in the dataset; s is the scaling factor; η is the angular spacing; W j T x i is the prediction result, where W j and x i are the weight matrix and input data vector in the model respectively.

6. The method for authenticity identification of forged facial images based on likelihood ratio according to claim 5, characterized in that: The training loss functions of the contour similarity calculation model and the face local similarity calculation model include, based on the ArcFace loss function, a Softmax loss function, a Cosine loss function and a Triplet loss function, and parameters are adjusted and optimized according to different data sets and application scenarios.

7. The method for authenticity identification of forged facial images based on likelihood ratio according to claim 1, characterized in that: In S4, when calculating the distribution probability of contours and local similarities of homologous and non-homologous faces, data sampling and statistical analysis methods such as stratified sampling, adaptive sampling, kernel density estimation and Bayesian network analysis are also included to further calculate the distribution probability.

8. The method for authenticity identification of forged facial images based on likelihood ratio according to claim 1, characterized in that: In S1, the preset facial key point detection models include the facial key point detection model in the dlib library, the MTCNN model based on convolutional neural network, and the HRNet model.

9. A system for identifying the authenticity of forged facial images based on likelihood ratio, characterized in that: include: A feature extraction module, used to construct a contour extractor and a face local extractor, wherein the contour extractor and the face local extractor detect the face in the input image based on a conventional face key point detection model, and extract the face contour and the specified face local features; A model training module, comprising a contour similarity calculation model training unit and no less than three face local similarity calculation model training units, the backbone networks of the contour similarity calculation model and the face local similarity calculation model are convolutional neural networks, accepting inputs of width and height channels of (n, n, 3) respectively, and the contour similarity calculation model training unit uses a training loss function to train the contour similarity calculation model; Similarity distribution statistics module, used to count the similarity between the contours of homologous faces and non-homologous faces, and calculate the contour similarity distribution probability. At the same time, this module is also used to count the multiple local similarities between homologous faces and non-homologous faces, and calculate the multiple local similarity distribution probability. The calculation method is consistent with the contour likelihood ratio statistics; The likelihood ratio calculation module is used to obtain the contour similarity and multiple local similarities including eyes, mouth, nose, ears and eyebrows between the forged face and the suspected source face through the contour similarity calculation model and the face local similarity calculation model, and further calculate the identification likelihood ratio result of the forged face.

10. An electronic device comprising: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images