Face authenticity detection method, device and equipment and computer readable storage medium

Abstract

The application provides a face authenticity detection method, device and equipment and a computer readable storage medium. The method comprises the following steps: acquiring multimedia data to be detected, pre-processing the multimedia data to be detected to obtain a face image to be detected; acquiring a trained detection model, the trained detection model being obtained through knowledge distillation training of feature knowledge provided by a plurality of trained expert models, and different trained expert models being trained by using training sample data corresponding to different forgery means; performing prediction processing on the face image to be detected by using the trained detection model to obtain a detection result of the face image to be detected; and determining a detection result of the multimedia data to be detected based on the detection result of the face image to be detected. Through the application, the accuracy and comprehensiveness of face authenticity detection can be improved.

Description

Technical Field

[0001] This application relates to machine learning technology, and more particularly to a method, apparatus, device, and computer-readable storage medium for detecting the authenticity of faces. Background Technology

[0002] Currently, facial recognition has been applied to all aspects of daily life, such as electronic payments and online identity authentication, making the detection of fake faces particularly important. Deepfake technology has driven the emerging development of the entertainment and cultural exchange industries, but it also poses a significant potential threat to facial security. Related technologies for facial recognition can be divided into two main categories: one primarily focuses on biometric information, using blink frequency or heart rate and other biological characteristics to identify fake videos; the other primarily utilizes neural networks to extract discriminative features related to forgery methods from statistical features of given image data, thereby classifying forged images or videos. However, current facial recognition methods can only identify fake facial images created using certain forgery techniques, resulting in poor accuracy. Summary of the Invention

[0003] This application provides a method, apparatus, and computer-readable storage medium for detecting the authenticity of faces, which can improve the accuracy of face authenticity detection.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a method for detecting the authenticity of a face, including: Acquire the multimedia data to be detected, preprocess the multimedia data to be detected, and obtain the face image to be detected; A trained detection model is obtained by knowledge distillation training using feature knowledge provided by multiple trained expert models. Different trained expert models are trained using training sample data corresponding to different forgery methods. The trained detection model is used to predict the face image to be detected, and the detection result of the face image to be detected is obtained. The detection result of the multimedia data to be detected is determined based on the detection result of the face image to be detected.

[0005] This application provides a face authenticity detection device, including: The first acquisition module is used to acquire multimedia data to be detected, and to preprocess the multimedia data to be detected to obtain a face image to be detected. The second acquisition module is used to acquire the trained detection model. The trained detection model is obtained by knowledge distillation training through feature knowledge provided by multiple trained expert models. Different trained expert models are trained using training sample data corresponding to different forgery methods. The prediction processing module is used to perform prediction processing on the face image to be detected using the trained detection model to obtain the detection result of the face image to be detected. The first determining module is used to determine the detection result of the multimedia data to be detected based on the detection result of the face image to be detected.

[0006] In some embodiments, the device further includes: The third acquisition module is used to acquire training sample data, a preset detection model, and a first loss function of the detection model. The training sample data includes multiple training sample images and label information of each training sample image. The fourth acquisition module is used to acquire multiple trained expert models; The second determining module is used to determine an adaptive knowledge distillation loss function using the plurality of trained expert models, the detection model, and the plurality of training sample images; The first training module is used to jointly train the detection model using the adaptive knowledge distillation loss function, the first loss function, and the training sample data to obtain a trained detection model.

[0007] In some embodiments, the device further includes: The fifth acquisition module is used to acquire M preset expert models and the second loss function corresponding to each expert model, where M is an integer greater than 1; The sixth acquisition module is used to acquire information on the forgery methods of each training sample image, divide the training sample data based on the forgery method information, and obtain real training sample images and forged training sample images corresponding to each forgery method. The third determining module is used to determine the real training sample image and the forged training sample image corresponding to the i-th forgery method as the training data subset corresponding to the i-th expert model, where i=1, 2, ..., M; The second training module is used to train the i-th expert model using the training data subset corresponding to the i-th expert model and the second loss function corresponding to the i-th expert model, so as to obtain the trained i-th expert model.

[0008] In some embodiments, the second determining module is further configured to: The j-th training sample image is predicted using the multiple trained expert models to obtain the prediction vector of the j-th training sample image, where j = 1, 2, ..., N, and N is the total number of training sample images. Based on the prediction vector of the j-th training sample image, determine the distillation weights of each expert model for the j-th training sample image; The distillation loss function of the j-th training sample image in each expert model is determined using the multiple trained expert models and the detection model. The adaptive knowledge distillation loss function is obtained by weighting and summing the distillation weights of each expert model for the N training sample images and the distillation loss function of each expert model for the N training sample images.

[0009] In some embodiments, the second determining module is further configured to: Based on the prediction vector of the j-th training sample image, obtain the prediction value of the j-th training sample image in each expert model; Obtain the discrimination threshold for each expert model; Based on the predicted value of the j-th training sample image in each expert model and the discrimination threshold of each expert model, the distillation weights of each expert model for the j-th training sample image are determined.

[0010] In some embodiments, the second determining module is further configured to: When the predicted value of the j-th training sample image corresponding to the i-th expert model is greater than the discrimination threshold of the i-th expert model, the distillation weight of the i-th expert model for the j-th training sample image is determined based on the predicted value and the discrimination threshold. When the predicted value of the j-th training sample image corresponding to the i-th expert model is less than or equal to the discrimination threshold of the i-th expert model, the first preset value is determined as the distillation weight of the i-th expert model for the j-th training sample image.

[0011] In some embodiments, the second determining module is further configured to: The j-th training sample image is subjected to feature extraction using each trained expert model to obtain the expert features of the j-th training sample image. The detection model is used to extract features from the j-th training sample image to obtain the detection features of the j-th training sample image; Based on the detection features of the j-th training sample image and the expert features of the j-th training sample image, the distillation loss function of the j-th training sample image in each expert model is determined.

[0012] In some embodiments, when the multimedia data to be detected is video data, the first acquisition module is further configured to: The multimedia data to be detected is decoded to obtain multiple video frame images; Extract a predetermined number of target video frame images from the plurality of video frame images; The face image region and background image region in each target video frame image are determined, and the background image region in each target video frame image is deleted to obtain each face image to be detected.

[0013] In some embodiments, the first determining module is further configured to: Determine the number of target face images in the face image to be detected that are identified as fake face images; When the number of target face images exceeds a preset threshold, the detection result of the multimedia data to be detected is determined to be a fake face video.

[0014] In some embodiments, the device further includes: The first marking module is used to add a fake face image mark to the target video frame image corresponding to the face image to be detected when the detection result of the face image to be detected is a fake face image; The second labeling module is used to add fake face image labels to each video frame image when the detection result of the multimedia data to be detected is a fake face video.

[0015] This application provides a computer device, including: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method provided in the embodiments of this application.

[0016] This application provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by a processor, they implement the face authenticity detection method provided in this application.

[0017] This application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the face authenticity detection method provided in this application.

[0018] The embodiments of this application have the following beneficial effects: After acquiring the multimedia data to be detected, the data is first preprocessed to obtain the face image to be detected. Then, a trained detection model is acquired. This trained detection model is trained by knowledge distillation using feature knowledge provided by multiple trained expert models. Different trained expert models are trained using training sample data corresponding to different forgery methods. In other words, the trained detection model is trained through multi-expert learning and integrates the feature knowledge learned by multiple trained expert models. Therefore, when using the trained detection model to predict the face image to be detected, it can accurately detect fake face images using various forgery methods. Finally, the detection result of the multimedia data to be detected is determined based on the detection result of the face image to be detected, thereby improving the accuracy of the detection result of the multimedia data to be detected. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the architecture of the face authenticity detection system provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the server 400 provided in the embodiment of this application; Figure 3 This is a schematic diagram illustrating an implementation process of the face authenticity detection method provided in this application embodiment; Figure 4 A schematic diagram illustrating the training process of the detection model provided in this application embodiment; Figure 5 A schematic diagram illustrating another implementation process of the face authenticity detection method provided in the embodiments of this application; Figure 6 A schematic diagram illustrating the training process of the detection model provided in this application embodiment; Figure 7 This is a schematic diagram illustrating the implementation process of knowledge distillation in an embodiment of this application. Figure 8 This diagram illustrates the distillation weight calculation of face images using the same forgery method by different expert models provided in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0022] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0024] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0025] 1) Multi-expert learning refers to the learning process in which a model is guided by feature knowledge provided by multiple expert models; 2) Knowledge distillation, the core idea is to first train a complex network model, and then use the output of this complex network and the real labels of the data to train a smaller network. Therefore, the knowledge distillation framework usually includes a complex model (called the expert model) and a small model (called the student model).

[0026] This application provides a method, apparatus, device, and computer-readable storage medium for face authentication, which addresses the diversity of forgery methods. First, it models the statistical characteristics of various forgery methods using pre-trained models. Then, it performs knowledge distillation on multiple pre-trained models, integrating diverse model knowledge into a single model. This mitigates the impact of diverse forgery methods on the performance of the discrimination model. The following describes exemplary applications of the computer device provided in this application. This computer device can be implemented as various types of user terminals such as laptops, tablets, desktop computers, set-top boxes, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), or as a server. The following describes exemplary applications when the device is implemented as a server.

[0027] See Figure 1 , Figure 1This is a schematic diagram of the network architecture of the face authenticity detection system 100 provided in this application embodiment, as shown below. Figure 1 As shown, the network architecture includes: terminal 200, network 300 and server 400. Terminal 200 is connected to server 400 through network 300. Network 300 can be a wide area network or a local area network, or a combination of both.

[0028] Terminal 200 can be equipped with various applications, such as camera applications, image editing applications, video viewing applications, shopping applications, and instant messaging applications. Currently, users can use terminal 200 to capture and record video files, and upload these video files to server 400, which then publishes the video files.

[0029] In this embodiment, after receiving a video file uploaded by the terminal 200, the server 400 decodes and extracts frames from the video file to obtain multiple target video frame images. Then, the face image region in the target video frame images is identified as the face image to be detected. The server 400 uses a trained detection model to perform prediction processing on the face image to be detected, and obtains the detection result of the face image to be detected. The trained detection model is obtained by knowledge distillation training using feature knowledge provided by multiple trained expert models. Different trained expert models are trained using training sample data corresponding to different forgery methods. Therefore, the trained detection model can have a good recognition effect on different forgery methods. Finally, the server 400 determines the detection result of the video file based on the detection result of the face image to be detected, thereby ensuring the accuracy of the detection result.

[0030] In this embodiment, if the server 400 detects a real face in a video file, it publishes the video file and sends a second notification message of successful publication to the terminal 200. If the server 400 detects a fake face in a video file, it sends a first notification message of publication failure to the terminal 200 and does not publish the video file.

[0031] It should be noted that, in this embodiment, the face authenticity detection method provided by this application is only illustrated using video publishing as an example. This face authenticity detection method is applicable to all application scenarios that require face authenticity detection, such as judicial verification and identity authentication.

[0032] In another embodiment, the face authenticity detection method can be used for various services in instant messaging, such as verifying user identity during payment.

[0033] In some embodiments, server 400 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminal 200 may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, smart vehicle terminal, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0034] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the server 400 provided in an embodiment of this application. Figure 2 The server 400 shown includes at least one processor 410, at least one network interface 420, a bus system 430, and memory 440. The various components in the server 400 are coupled together via the bus system 430. It is understood that the bus system 430 is used to implement communication between these components. In addition to a data bus, the bus system 430 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 430.

[0035] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0036] The memory 440 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 440 may optionally include one or more storage devices physically located away from the processor 410.

[0037] The memory 440 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 440 described in this application embodiment is intended to include any suitable type of memory.

[0038] In some embodiments, memory 440 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0039] Operating system 441 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 442 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc. In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A face authenticity detection device 443 stored in memory 440 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: a first acquisition module 4431, a second acquisition module 4432, a prediction processing module 4433, and a first determination module 4434. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0040] In other embodiments, the apparatus provided in this application can be implemented in hardware. As an example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the face authenticity detection method provided in this application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0041] To better understand the methods provided in the embodiments of this application, we will first explain artificial intelligence, its various branches, and the application fields involved in the methods provided in the embodiments of this application.

[0042] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0043] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning. The solutions provided in this application's embodiments relate to machine learning and other AI technologies, which will be described below.

[0044] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0045] The method for detecting the authenticity of faces provided in this application will be described in conjunction with exemplary applications and implementations of the server provided in the embodiments of this application.

[0046] This application provides a method for detecting the authenticity of faces. This method is applied to a computer device, which can be a server or a terminal. In this application embodiment, the computer device is described as a server. Figure 3 This is a flowchart illustrating the face authenticity detection method provided in the embodiments of this application. The following will be combined with... Figure 3 The steps of the face authenticity detection method provided in the embodiments of this application are described.

[0047] Step S101: Obtain the multimedia data to be detected, and preprocess the multimedia data to be detected to obtain the face image to be detected.

[0048] Here, the multimedia data to be detected can be either video data or image data. When the multimedia data to be detected is video data, this step can be implemented by decoding the video and extracting frames to obtain multiple target video frame images. Then, the face image regions of each target video frame image are identified, and background removal, size adjustment, etc., are performed to obtain the face images to be detected. In this case, multiple face images to be detected are obtained. When the multimedia data to be detected is image data, this step can be implemented by performing face recognition on the image data, determining the face image regions, and performing background removal, size adjustment, etc., to obtain the face images to be detected.

[0049] Step S102: Obtain the trained detection model.

[0050] The trained detection model is obtained by knowledge distillation training using feature knowledge provided by multiple trained expert models. Different trained expert models are trained using training sample data corresponding to different forgery methods. In other words, the trained detection model integrates the knowledge learned by each trained expert model and has high detection accuracy for face images forged by different forgery methods.

[0051] Step S103: Use the trained detection model to perform prediction processing on the face image to be detected, and obtain the detection result of the face image to be detected.

[0052] The detection result of a face image can characterize whether the face image is a real face or a fake face. In practice, the detection result can be 1 when the face image is detected as a real face, and 0 when the face image is detected as a fake face. When there are multiple face images to be detected, each face image corresponds to one detection result.

[0053] Step S104: Determine the detection result of the multimedia data to be detected based on the detection result of the face image to be detected.

[0054] In this step, when the multimedia data to be detected is a single image, the detection result of the face image to be detected is directly determined as the detection result of the multimedia data to be detected. When the multimedia data to be detected is video data, the number of fake faces detected in multiple face images corresponding to the video data can be counted, and the detection result of the multimedia data to be detected can be determined based on the number of fake faces.

[0055] In the face authenticity detection method provided in this application embodiment, after acquiring the multimedia data to be detected, the multimedia data to be detected is first preprocessed to obtain the face image to be detected, and then a trained detection model is acquired. The trained detection model is obtained by knowledge distillation training using feature knowledge provided by multiple trained expert models. Different trained expert models are trained using training sample data corresponding to different forgery methods. That is, the trained detection model is obtained through multi-expert learning and integrates the feature knowledge learned by multiple trained expert models. Therefore, by using the trained detection model to predict the face image to be detected, it can accurately detect fake face images of various forgery methods. Finally, the detection result of the multimedia data to be detected is determined based on the detection result of the face image to be detected, thereby improving the accuracy of the detection result of the multimedia data to be detected.

[0056] In some embodiments, prior to step S102, it may also be possible to use, as follows: Figure 4 The trained detection model is obtained through steps S001 to S004 as shown below. Figure 4 Each step is explained.

[0057] Step S001: Obtain training sample data, a preset detection model, and the first loss function of the detection model.

[0058] The training sample data includes multiple training sample images and their respective label information. The label information of the training sample images can characterize whether they are real or fake face images. The preset detection model can be a small-scale and simple neural network model. The first loss function of this detection model can be the cross-entropy function or other types of loss functions.

[0059] Step S002: Obtain multiple trained expert models.

[0060] The network structure of a well-trained expert model is larger and more complex than that of a prediction model. A well-trained expert model can be a single complex network or a collection of several networks, possessing good performance and generalization ability. In the embodiments of this application, different well-trained expert models are trained using training sample images obtained through different forgery methods. Compared to the discriminative model obtained through mixed training, these expert models exhibit superior performance in discriminating their respective forgery methods.

[0061] Step S003: Determine the adaptive knowledge distillation loss function using the multiple trained expert models, the detection model, and the multiple training sample images.

[0062] In this step, multiple training sample images can be predicted using various trained expert models to obtain predicted values. Then, based on the predicted values, the distillation weights of each training sample image in each trained expert model are determined. The multiple training sample images are then input into the expert model and the detection model respectively to obtain expert features and detection features. Based on the expert features and detection features, the distillation loss function of each training sample is determined. Finally, the adaptive knowledge distillation loss function is determined based on the distillation loss function and distillation weights of each training sample.

[0063] Step S004: Using the adaptive knowledge distillation loss function, the first loss function, and the training sample data, the detection model is jointly trained to obtain a trained detection model.

[0064] In this step, each training sample image from the training sample data is input into the detection model to obtain the prediction result of each training sample image. Then, the prediction result and the label information of each training sample image are used to perform backpropagation training on the detection model, thereby adjusting the parameters of the detection model until the training termination condition is determined based on the adaptive knowledge distillation loss function and the first loss function. At this point, the trained detection model is obtained.

[0065] Due to the differences between various forgery methods, the flaw features and discrimination patterns carried in the generated forged face images also differ. Expert models experience a significant performance drop when judging forgery methods they haven't encountered before. Therefore, in steps S001 to S004, knowledge distillation is used to extract the discrimination knowledge for different forgery methods possessed by different expert models and integrate it into the detection model. This allows the trained detection model to not only have faster recognition and detection efficiency but also to simultaneously capture flaws generated by multiple forgery methods, thereby enabling the identification of fake face images using various forgery methods and improving detection accuracy.

[0066] In some embodiments, the above step S003, "determining the adaptive knowledge distillation loss function using the plurality of trained expert models, the detection model, and the plurality of training sample images," can be implemented by the following steps S0031 to S0034, and each step is described below.

[0067] Step S0031: Use the multiple trained expert models to perform prediction processing on the j-th training sample image to obtain the prediction vector of the j-th training sample image.

[0068] j = 1, 2, ..., N, where N is the total number of training sample images. Assuming there are M trained expert models, the dimension of the prediction vector for the j-th training sample image is 1*M. Each predicted value in the prediction vector is a real number between 0 and 1, representing the probability that the training sample image is a real face image.

[0069] Step S0032: Based on the prediction vector of the j-th training sample image, determine the distillation weights of each expert model for the j-th training sample image.

[0070] In implementation, a discrimination threshold is first set for each expert model. If the predicted probability value of the expert model for the j-th training sample image exceeds the discrimination threshold, the distillation weight of the expert model for the j-th training sample image is set to be linearly related to the difference between the probability prediction and the threshold. Conversely, if the predicted probability value of the expert model for the j-th training sample image does not exceed the discrimination threshold, the distillation weight is reset to a first preset value. The first preset value can be 0 or a positive number close to 0, indicating that the expert model has no discrimination ability for the training sample image.

[0071] Step S0033: Determine the distillation loss function of the j-th training sample image in each expert model using the multiple trained expert models and the detection model.

[0072] In this step, multiple training sample images are input into the expert model and the detection model respectively to obtain expert features and detection features. Then, the distillation loss function of each training sample is determined by calculating the mean square error between the expert features and the detection features.

[0073] Step S0034: The distillation weights of each expert model for the N training sample images and the distillation loss functions of the N training sample images in each expert model are weighted and summed to obtain the adaptive knowledge distillation loss function.

[0074] In the embodiments described above, steps S0031 to S0034, when determining the adaptive knowledge distillation loss function, firstly, based on the prediction vectors of each training sample image, the distillation weights of the expert model for each training sample image are determined, and then the sample-level distillation loss function is determined. Then, the distillation weights and the sample-level distillation loss function are weighted and summed to obtain the adaptive knowledge distillation loss function of the detection model. This ensures that the adaptive knowledge distillation loss function can effectively measure the differences between forged face images produced by different forgery methods, and stabilizes the distillation process of diversity distillation.

[0075] In some embodiments, step S0032, "determining the distillation weights of each expert model for the j-th training sample image based on the prediction vector of the j-th training sample image," can be implemented through the following steps: Step S321: Based on the prediction vector of the j-th training sample image, obtain the prediction value of the j-th training sample image in each expert model.

[0076] The predicted value is a real number between 0 and 1, representing the probability that the j-th training sample image predicted by the expert model is a real face image.

[0077] Step S322: Obtain the discrimination threshold for each expert model.

[0078] In the embodiments of this application, the discrimination threshold of different expert models can be the same, for example, the discrimination threshold of each expert model is 0.5; the discrimination threshold of different expert models can be different, for example, the discrimination threshold of expert model A is 0.5, the discrimination threshold of expert model B is 0.6, the discrimination threshold of expert model C is 0.65, and so on.

[0079] Step S323: Based on the predicted value of the j-th training sample image in each expert model and the discrimination threshold of each expert model, determine the distillation weights of each expert model for the j-th training sample image.

[0080] In implementing step S323, when the predicted value of the j-th training sample image corresponding to the i-th expert model is greater than the discrimination threshold of the i-th expert model, the distillation weights of the i-th expert model for the j-th training sample image are determined based on the predicted value and the discrimination threshold. Further, the distillation weights can be set to have a linear relationship with the difference between the probability prediction and the threshold. In actual implementation, this can be determined using formula (1-1): (1-1); in, The distillation weights of the i-th expert model for the j-th training sample image are: Let be the predicted probability value of the i-th expert model for the j-th training sample image. is the discrimination threshold for the i-th expert model.

[0081] The expert model's discrimination threshold is pre-set, therefore As a fixed value, it can be seen from formula (1-1) that the distillation weight of the i-th expert model for the j-th training sample image is proportional to the difference between the prediction probability value and the discrimination threshold of the i-th expert model for the j-th training sample image.

[0082] When the predicted value of the j-th training sample image corresponding to the i-th expert model is less than or equal to the discrimination threshold of the i-th expert model, the first preset value is determined as the distillation weight of the i-th expert model for the j-th training sample image. In actual implementation, the first preset value can be 0 or a real number close to 0, such as 0.01, 0.008, etc.

[0083] In the embodiments described in steps S321 to S323 above, when determining the distillation weights of each expert model for the j-th training sample image, if the predicted probability value of the expert model for the j-th training sample image exceeds the threshold, it is determined that the expert model has a certain discriminative ability for the training sample image. The distillation weights are set to have a linear relationship with the difference between the probability prediction and the threshold. Conversely, if the predicted probability value of the expert model for the j-th training sample image does not exceed the threshold, it is considered that the expert model has no discriminative ability for the training sample image and cannot provide effective model knowledge to assist the learning of the final detection model. Its weights are reset to a first preset value, which can be 0 or a positive number close to 0. This makes the process of transferring diverse domains more stable, and the domain sample knowledge after distillation weighting is more robust.

[0084] In some embodiments, step S0033, "determining the distillation loss function of the j-th training sample image in each expert model using the plurality of trained expert models and the detection model," can be achieved through the following steps: Step S331: Use each trained expert model to extract features from the j-th training sample image to obtain the expert features of the j-th training sample image.

[0085] In implementation, the j-th training sample image can be input into each of the trained expert models. The feature extraction modules in each trained expert model can then extract features from the j-th training sample image, thereby obtaining the expert features of the j-th training sample image. These expert features can be feature vectors, for example, a 1*256 feature vector.

[0086] Step S332: Use the detection model to extract features from the j-th training sample image to obtain the detection features of the j-th training sample image.

[0087] Similar to step S331, this step can be implemented by inputting the j-th training sample image into the detection model, so that the feature extraction module in the detection model can extract features from the j-th training sample image to obtain the detection features of the j-th training sample image. This detection feature is also a vector, and has the same dimension as the expert features, for example, it can be a 1*256 feature vector.

[0088] Step S333: Determine the distillation loss function of the j-th training sample image in each expert model based on the detection features of the j-th training sample image and each expert feature of the j-th training sample image.

[0089] Here, the mean squared error between the detection features of the j-th training sample image and each expert feature of the j-th training sample image can be calculated separately to determine the distillation loss function of the j-th training sample image in each expert model. In practical implementation, the distillation loss function of the j-th training sample image in the i-th expert model can be determined by formula (1-2): (1-2); Wherein, the detection features of the j-th training sample image The expert features obtained from the j-th training sample image through the i-th expert model are: The dimensions of both the detection features and the expert features are 1*P, where P is an integer greater than 1.

[0090] In some embodiments, each trained expert model can be obtained through the following steps: Step S201: Obtain M preset expert models and the second loss function corresponding to each expert model.

[0091] M is an integer greater than 1. In this embodiment, the training sample images include forged face images created using several forgery methods, thus requiring several expert models. For example, if there are four forged face images created using four different methods, then four expert models are set up.

[0092] In this embodiment, cross-entropy can be used as the second loss function of the expert model. In actual implementation, the second loss function of the i-th expert model can be determined by formula (2-1) provided in other embodiments.

[0093] Step S202: Obtain the forgery method information of each training sample image, divide the training sample data based on the forgery method information, and obtain the real training sample image and the forged training sample image corresponding to each forgery method.

[0094] In the embodiments of this application, each training sample image has a forgery method information. In implementation, a corresponding forgery method information value can be set for various forgery methods. For example, when the training sample image is a real face image, the forgery method information value is 0; when the training sample image is a training sample image forged by Deepfake, the forgery method information is 1; when the training sample image is a training sample image forged by Faceswap, the forgery method information is 2, and so on.

[0095] When dividing training sample data based on forgery method information, training sample images with the same forgery method information value can be classified as forged training sample images belonging to the same forgery method. Training sample images with a forgery method information value of 0 are real training sample images, and training sample images with a forgery method information value of non-zero are forged training sample images.

[0096] Step S203: The real training sample image and the forged training sample image corresponding to the i-th forgery method are determined as the training data subset corresponding to the i-th expert model.

[0097] Where i = 1, 2, ..., M.

[0098] Step S204: Using the training data subset corresponding to the i-th expert model and the second loss function corresponding to the i-th expert model, train the i-th expert model to obtain the trained i-th expert model.

[0099] In this step, the i-th expert model can be input into the i-th expert model by taking each training sample image from the training data subset corresponding to the i-th expert model, obtaining the prediction value of the i-th expert model for each training sample image, and then performing backpropagation training on the i-th expert model based on the prediction value and label information of each training sample image. When the training termination condition is determined based on the second loss function, the i-th expert model is obtained.

[0100] In the embodiments described in steps S201 to S204 above, when training the expert model, the training sample data is classified according to the forgery method information, and the expert model is trained using training sample images of the same forgery method, so that the trained expert model has a better performance in judging fake face images of their respective forgery methods.

[0101] Based on the foregoing embodiments, this application further provides a method for detecting the authenticity of faces, applied to... Figure 1 The network architecture shown is Figure 5 This is a schematic diagram illustrating another implementation process of the face authenticity detection method provided in the embodiments of this application, as follows: Figure 5 As shown, the process includes: In step S301, the terminal acquires video data through its own image acquisition device and determines the video data to be uploaded based on the acquired video data.

[0102] In this step, the collected video data can be directly identified as the video data to be uploaded, or the collected video data can be edited, such as adding special effects, voice changing, adding background music, face swapping, etc., to obtain the processed video data, and then the processed video data can be identified as the video data to be uploaded.

[0103] In step S302, the terminal responds to the video data upload operation by sending a video publishing request to the server.

[0104] The video publishing request includes the video data to be uploaded.

[0105] In step S303, the server obtains the video data to be uploaded and identifies the video data to be uploaded as the multimedia data to be detected.

[0106] After receiving a video publishing request, the server parses the request to obtain the video data to be uploaded and identifies it as the multimedia data to be tested. In this embodiment, before performing face verification on the multimedia data to be tested, the server may also perform other legality and compliance checks. If the video data to be uploaded is determined to meet the legality and compliance conditions, face verification is then performed through the following steps; if the video data to be uploaded is determined to not meet the legality and compliance conditions, a first notification message of publishing failure is directly returned to the terminal. The reason for the publishing failure may be carried in this first notification message.

[0107] In step S304, the server decodes the multimedia data to be detected to obtain multiple video frame images.

[0108] Decoding the multimedia data to be detected is also known as decompressing the multimedia data to be detected. After decompression, multiple video frame images from the multimedia data to be detected can be obtained.

[0109] In step S305, the server extracts a preset number of target video frame images from multiple video frame images.

[0110] In implementation, this step can be carried out by extracting a preset number of target video frame images from multiple video frame images according to a preset interval duration, or by randomly extracting a preset number of target video frame images from multiple video frame images, or by extracting a preset number of target video frame images from multiple video frame images according to frame type, for example, by extracting a preset number of keyframe images.

[0111] Step S306: The server determines the face image region and background image region in each target video frame image, deletes the background image region in each target video frame image, and obtains each face image to be detected.

[0112] To better capture face-related features, this step requires removing background information other than faces from each target video frame image. Therefore, face detection is first performed on each target video frame image to identify the face image region and the background image region. Then, the face image region is magnified to retain the complete face region. Finally, the target video frame image is cropped according to the edge of the magnified face image region to obtain the face image to be detected.

[0113] Step S307: The server obtains the trained detection model.

[0114] The trained detection model is obtained by knowledge distillation using feature knowledge provided by multiple trained expert models. Different trained expert models are trained using training sample data corresponding to different forgery methods.

[0115] Step S308: The server uses the trained detection model to perform prediction processing on the face image to be detected, and obtains the detection result of the face image to be detected.

[0116] The detection result of the face image to be detected is a binary classification result, that is, the detection result can be a real face image or a fake face image.

[0117] Step S309: When the detection result of the face image to be detected is a fake face image, the server adds a fake face image marker to the target video frame image corresponding to the face image to be detected.

[0118] In implementation, this step can be achieved by adding a watermark of the fake face image to the target video frame image corresponding to the detected fake face image, or by adding an overlay to indicate in the watermark or overlay that the target video frame image is a fake face image.

[0119] Step S310: The server determines the number of target face images in the face image to be detected that are detected as fake face images.

[0120] The number of target face images can be 0 or an integer greater than 0. When the number of target face images is 0, it means that all the face images to be detected are real face images. When the number of target face images is an integer greater than 0, it means that there are fake face images among the face images to be detected.

[0121] Step S311: The server determines whether the number of target face images is greater than a preset threshold.

[0122] The number threshold is an integer, meaning it can be 0 or a positive integer. When the number of target face images is greater than the preset number threshold, proceed to step S312; when the number of target face images is less than or equal to the number threshold, proceed to step S315.

[0123] In step S312, the server determines that the detection result of the multimedia data to be detected is a fake face video.

[0124] Step S313: The server adds fake face image tags to each video frame image.

[0125] In this embodiment of the application, when it is determined that the detection result of the multimedia data to be detected is a fake face video, fake face image markers can be added to each video frame image in the multimedia data to be detected, such as watermarks or overlays.

[0126] In step S314, the server sends a first notification message to the terminal indicating that the publishing failed.

[0127] The first notification message included the reason for the failure to upload, namely that the video to be uploaded was a fake face video.

[0128] Step S315: The server determines that the detection result of the multimedia data to be detected is a real face video, and publishes the multimedia data to be detected.

[0129] Step S316: The server sends a second notification message to the terminal indicating successful publication.

[0130] In the face authenticity detection method provided in this application embodiment, when a terminal needs to upload its own collected or produced video data to the server to request publication, the server first obtains multiple video frame images from the video data through decoding processing, then extracts a certain number of target video frames, and performs face recognition to determine the face image to be detected. Then, using a trained detection model, the server determines the detection result of the face image to be detected. Since the trained detection model is a lightweight detection model with high detection efficiency, and because it integrates the forgery feature knowledge learned by multiple expert models through knowledge transfer, it has a high detection accuracy for fake images using various forgery methods. Therefore, after obtaining the detection result of the face image to be detected through the detection model, the server can determine the detection result of the multimedia data to be detected based on the detection result of the face image to be detected. If the detection result of the multimedia data to be detected is a real face video, the multimedia data to be detected is published. If the detection result of the multimedia data to be detected is a fake face video, a first notification message of publication failure is returned to the terminal, thereby ensuring and improving the credibility of the multimedia publishing platform and maintaining a healthy ecosystem for the multimedia publishing platform.

[0131] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0132] Deepfake technology has driven the emerging development of the entertainment and cultural exchange industries, but it also poses a significant potential threat to facial security. The facial authenticity detection method provided in this application can be applied to fake face detection products, such as upgrading facial verification technology, improving the security of facial payment, identity authentication, and other services, aiding in evidence verification, and preventing the use of Deepfakes-related technologies to forge evidence. On multimedia platforms, the widespread dissemination of face-swapping videos has led to a decline in media credibility and can easily mislead users. The facial authenticity detection method provided in this application can help multimedia platforms screen videos, adding prominent markers such as "made by Deepfake" to detected fake videos, ensuring the credibility of the video content and guaranteeing social credibility.

[0133] The face authenticity detection method provided in this application can be implemented through the following steps: S601. Obtain the video dataset for training the face forensics detection model and define the algorithm objective.

[0134] S602. Preprocess the dataset and divide it into multiple sub-datasets according to different fraud methods.

[0135] S603. Train expert detection models for each forgery method on the subset dataset (corresponding to expert models in other embodiments).

[0136] S604. Integrate the student detection model (corresponding to the detection model in other embodiments) on the total dataset using a joint expert model.

[0137] S605. Use the final student detection model to detect the authenticity of face images.

[0138] Face authentication is a binary classification problem. Typically, given an image or video as input to a detection model, the model outputs a label indicating whether the face is real or fake. However, in real-world scenarios, detection models need to be able to detect multiple different forgery methods simultaneously. Therefore, common detection models are trained using a mixture of fake face images generated by various forgery methods. Considering the differences between different forgery methods, networks trained with simply mixed data are limited by their learning capabilities and struggle to simultaneously capture flaws produced by multiple forgery methods.

[0139] To address this situation, the detection model used in the face authentication provided in this application embodiment is as follows: Figure 6 The two-stage learning and training process shown below is combined with the following. Figure 6 Please provide an explanation.

[0140] The first phase, 601, involves training multiple expert models.

[0141] The face detection method provided in this application primarily targets image input. Therefore, when the object to be detected is a video, it is first necessary to preprocess the video to obtain suitable face images. For a given video, it is considered as a collection of frames. Considering the varying lengths of the videos, to ensure fairness in training, a fixed number of frames are randomly sampled from each video as candidate frames. After obtaining the candidate frames, background information other than the face needs to be removed from each image to better capture face-related features. The dlib library is used to detect the face portion of the image. After obtaining the face bounding box, it is enlarged by 1.3 times to retain the complete face region. The image is then cropped according to the enlarged bounding box, retaining only the face portion. The resulting cropped image is used as input to the face detection model.

[0142] Generally, facial forensics datasets typically contain natural face videos and fake face videos generated using various spoofing techniques. After the preprocessing described above, the resulting natural face images are referred to as... The various forgery methods used to create fake faces are collectively referred to as Due to the differences between various forgery methods, the forged face is further subdivided. If there are M different forgery methods, in this embodiment, the forged image produced by the i-th forgery method is referred to as Fi. Therefore, .

[0143] To fully capture the flaws and characteristics of different forgery methods, in this stage, a separate expert model is trained for each forgery method to detect it. For ease of representation, in this embodiment, the first... The training method for one expert model will be explained here; the same applies to the training methods for other expert models.

[0144] Defined for the first The subset of datasets used to train the expert models is When forged faces are generated using the same forgery method, the consistency of flaw features becomes stronger compared to all fake face data, making it easier to capture discrimination patterns related to the forgery method. The training mode of the expert model is consistent with the training mode of a general binary classifier model. In this model, cross-entropy is chosen as the loss function.

[0145] Definition of the first The encoder of each expert model is Its classifier is Then input an image. The features output by the model are The classification result is For ease of description, the classification results will be represented using... express.

[0146] Regarding the first For each model, the training objective function can be expressed using formula (2-1): (2-1); in, This represents the label of the k-th sample image. For example... Figure 6 As shown, expert models are trained sequentially on the subsets of datasets corresponding to different forgery methods.

[0147] Compared to the discriminative models obtained through hybrid training, these expert models exhibit superior performance in identifying their respective forgery methods. However, due to the differences between various forgery methods, the flawed features and discrimination patterns carried in the generated fake face images also differ. This leads to a significant performance drop in expert models when judging forgery methods they have not encountered before. Therefore, the second stage of training integrates the discriminative knowledge of different experts regarding various forgery methods.

[0148] In the second training phase 602, knowledge from multiple expert models is integrated into a single discriminative model through knowledge transfer. Since knowledge transfer is required, a knowledge distillation method is employed. Because the expert models are trained on multiple fraud domains, each domain expert can discriminate different sample domains, resulting in varying discriminative abilities for each sample. For example, an expert model trained on Deepfakes is suitable for detecting fraudulent samples created using Deepfakes techniques, while its discriminative ability against other fraud methods depends on its similarity to Deepfakes techniques. Based on this, this application proposes an adaptive knowledge distillation method and provides an effective weight adaptation strategy.

[0149] In this embodiment of the application, the loss of adaptive knowledge distillation is modeled as Equation (2-2): (2-2); in Indicates the first l The expert model for the first k Distillation weights for each data sample Indicates a single sample The l The expert model uses a distillation loss function. The overall modeling approach is primarily a weighted, sample-level adaptive loss function. Considering the diversity of domain knowledge inherited by the expert model, this embodiment designs an effective strategy to model the distillation weights in the formula, making the transfer process across diverse domains stable and the weighted domain sample knowledge more robust.

[0150] Figure 7 This is a schematic diagram illustrating the implementation process of knowledge distillation provided in this application embodiment. For a sample image, the student feature 701 corresponding to the sample image is obtained through the feature extractor of the student model, and the expert features of the sample image are obtained through the feature extractors of each expert model. In this application embodiment, it is assumed that there are four expert models, so four expert features are obtained, for example... Figure 7 In the expert features 702, 703, 704, and 705, during knowledge distillation, the distillation loss function for each expert model of the sample image is first determined based on the student features and each expert feature. Then, based on the predicted vectors of the sample image in each expert model, the distillation weights of each expert model are determined according to the following formula (2-3). Then, based on each distillation loss function and each distillation weight, the loss function of adaptive knowledge distillation is determined. .

[0151] In implementation, samples are input into various expert models to obtain probability prediction vectors. For each sample, each expert model outputs a probability prediction to determine whether the sample is true or false. The vector length is the number of expert models. To transform this probability prediction vector into a distilled weight vector, a prediction threshold is first determined. If the probability output of the expert model exceeds its threshold, it is considered that the expert model has a certain discriminative ability for a specific sample, and its discriminative weight is linearly related to the difference between the probability prediction and the threshold. Conversely, if the output exceeds the threshold, it is considered that the model has no discriminative ability for a specific sample and cannot provide effective model knowledge to assist the learning of the final student model, and its weight is reset to 0. In implementation, the distillation weight can be determined by formula (2-3): (2-3); in, Indicates the first l The decision threshold of an expert model.

[0152] Figure 8 This is a schematic diagram illustrating the distillation weight calculation of face images using the same forgery method by different expert models provided in the embodiments of this application, such as... Figure 8 The original face images 801 and 802 shown are both generated using Deepfakes techniques. On an expert model trained in the Deepfakes domain, the prediction probability of both original face images 801 and 802 is 1.00, and their distillation weights are both 1.00. On an expert model trained in the NeuralTexture domain, the prediction probability of original face image 801 is 0.3496, with a distillation weight of 0; the prediction probability of original face image 802 is 0.8838, with a distillation weight of 0.68. It can be seen that the output weights generated by the expert model trained in the NeuralTexture domain differ significantly between the two original face images 801 and 802. The adaptive weight design proposed in this embodiment can effectively measure this difference and use it as an auxiliary weight for diversity distillation, stabilizing the distillation process.

[0153] In some embodiments, other methods can be used to determine adaptive distillation weights, such as using a pre-trained classifier to uniformly measure the prediction difficulty and uncertainty of samples to determine adaptive distillation weights, or introducing a series of methods in the course learning method to measure the difficulty of samples to extract adaptive distillation weights.

[0154] The loss function obtained from adaptive knowledge distillation Then, this loss function and the student model's loss function can be used. The student model is trained to obtain a well-trained student model, which is then used as a detection model to detect the authenticity of face images. Since the detection model learns the knowledge of various expert models through knowledge transfer during training, it has a good ability to identify fake images using various forgery methods.

[0155] The face authentication method provided in this application achieves excellent detection results on multiple metrics of the FaceForensics++ dataset, and shows a significant performance improvement compared to related face authentication methods. Furthermore, t-SNE visualization can be performed on the fake and real image features extracted by the model, allowing for a direct observation of the increased distance between the distributions of real and fake features, and a smaller variance within each feature cluster, thus fully demonstrating the effectiveness of the face authentication method provided in this application.

[0156] The following description continues to illustrate the exemplary structure of the face authenticity detection device 443 provided in this application embodiment as a software module. In some embodiments, such as... Figure 2 As shown, the software module stored in the face authenticity detection device 443 in the memory 440 may include: The first acquisition module 4431 is used to acquire multimedia data to be detected, preprocess the multimedia data to be detected, and obtain a face image to be detected. The second acquisition module 4432 is used to acquire the trained detection model. The trained detection model is obtained by knowledge distillation training through feature knowledge provided by multiple trained expert models. Different trained expert models are trained using training sample data corresponding to different forgery methods. The prediction processing module 4433 is used to perform prediction processing on the face image to be detected using the trained detection model to obtain the detection result of the face image to be detected. The first determining module 4434 is used to determine the detection result of the multimedia data to be detected based on the detection result of the face image to be detected.

[0157] In some embodiments, the device further includes: The third acquisition module is used to acquire training sample data, a preset detection model, and a first loss function of the detection model. The training sample data includes multiple training sample images and label information of each training sample image. The fourth acquisition module is used to acquire multiple trained expert models; The second determining module is used to determine an adaptive knowledge distillation loss function using the plurality of trained expert models, the detection model, and the plurality of training sample images; The first training module is used to jointly train the detection model using the adaptive knowledge distillation loss function, the first loss function, and the training sample data to obtain a trained detection model.

[0158] In some embodiments, the device further includes: The fifth acquisition module is used to acquire M preset expert models and the second loss function corresponding to each expert model, where M is an integer greater than 1; The sixth acquisition module is used to acquire information on the forgery methods of each training sample image, divide the training sample data based on the forgery method information, and obtain real training sample images and forged training sample images corresponding to each forgery method. The third determining module is used to determine the real training sample image and the forged training sample image corresponding to the i-th forgery method as the training data subset corresponding to the i-th expert model, where i=1, 2, ..., M; The second training module is used to train the i-th expert model using the training data subset corresponding to the i-th expert model and the second loss function corresponding to the i-th expert model, so as to obtain the trained i-th expert model.

[0159] In some embodiments, the second determining module is further configured to: The j-th training sample image is predicted using the multiple trained expert models to obtain the prediction vector of the j-th training sample image, where j = 1, 2, ..., N, and N is the total number of training sample images. Based on the prediction vector of the j-th training sample image, determine the distillation weights of each expert model for the j-th training sample image; The distillation loss function of the j-th training sample image in each expert model is determined using the multiple trained expert models and the detection model. The adaptive knowledge distillation loss function is obtained by weighting and summing the distillation weights of each expert model for the N training sample images and the distillation loss function of each expert model for the N training sample images.

[0160] In some embodiments, the second determining module is further configured to: Based on the prediction vector of the j-th training sample image, obtain the prediction value of the j-th training sample image in each expert model; Obtain the discrimination threshold for each expert model; Based on the predicted value of the j-th training sample image in each expert model and the discrimination threshold of each expert model, the distillation weights of each expert model for the j-th training sample image are determined.

[0161] In some embodiments, the second determining module is further configured to: When the predicted value of the j-th training sample image corresponding to the i-th expert model is greater than the discrimination threshold of the i-th expert model, the distillation weight of the i-th expert model for the j-th training sample image is determined based on the predicted value and the discrimination threshold. When the predicted value of the j-th training sample image corresponding to the i-th expert model is less than or equal to the discrimination threshold of the i-th expert model, the first preset value is determined as the distillation weight of the i-th expert model for the j-th training sample image.

[0162] In some embodiments, the second determining module is further configured to: The j-th training sample image is subjected to feature extraction using each trained expert model to obtain the expert features of the j-th training sample image. The detection model is used to extract features from the j-th training sample image to obtain the detection features of the j-th training sample image; Based on the detection features of the j-th training sample image and the expert features of the j-th training sample image, the distillation loss function of the j-th training sample image in each expert model is determined.

[0163] In some embodiments, when the multimedia data to be detected is video data, the first acquisition module is further configured to: The multimedia data to be detected is decoded to obtain multiple video frame images; Extract a predetermined number of target video frame images from the plurality of video frame images; The face image region and background image region in each target video frame image are determined, and the background image region in each target video frame image is deleted to obtain each face image to be detected.

[0164] In some embodiments, the first determining module is further configured to: Determine the number of target face images in the face image to be detected that are identified as fake face images; When the number of target face images exceeds a preset threshold, the detection result of the multimedia data to be detected is determined to be a fake face video.

[0165] In some embodiments, the device further includes: The first marking module is used to add a fake face image mark to the target video frame image corresponding to the face image to be detected when the detection result of the face image to be detected is a fake face image; The second labeling module is used to add fake face image labels to each video frame image when the detection result of the multimedia data to be detected is a fake face video.

[0166] It should be noted that the description of the face authenticity detection device in this application is similar to the description of the method embodiments described above, and has similar beneficial effects. For technical details not disclosed in this device embodiment, please refer to the description of the method embodiments of this application for understanding.

[0167] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the face authenticity detection method described in this application.

[0168] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the face authenticity detection method provided in this application. For example, ... Figure 3 , Figure 4 , Figure 5 The method for detecting the authenticity of a face is shown.

[0169] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0170] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0171] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0172] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0173] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for detecting the authenticity of a face, characterized in that, The method includes: Acquire the multimedia data to be detected, preprocess the multimedia data to be detected, and obtain the face image to be detected; A trained detection model is obtained, which is trained by knowledge distillation using feature knowledge provided by multiple trained expert models. Different trained expert models are trained using training sample data corresponding to different forgery methods in the following manner: Acquire training sample data, which includes multiple training sample images and label information for each training sample image; Obtain M preset expert models and the second loss function corresponding to each expert model, where M is an integer greater than 1; Obtain information on the forgery methods of each training sample image, divide the training sample data based on the forgery method information, and obtain real training sample images and forged training sample images corresponding to each forgery method; The real training sample images and the forged training sample images corresponding to the i-th forgery method are determined as the training data subset corresponding to the i-th expert model, where i = 1, 2, ..., M; Using the training data subset corresponding to the i-th expert model and the second loss function corresponding to the i-th expert model, the i-th expert model is trained to obtain the trained i-th expert model; The trained detection model is used to predict the face image to be detected, and the detection result of the face image to be detected is obtained. The detection result of the multimedia data to be detected is determined based on the detection result of the face image to be detected.

2. The method according to claim 1, characterized in that, The method further includes: Obtain a preset detection model and a first loss function of the detection model; Obtain multiple pre-trained expert models; An adaptive knowledge distillation loss function is determined using the multiple trained expert models, the detection model, and the multiple training sample images. The detection model is jointly trained using the adaptive knowledge distillation loss function, the first loss function, and the training sample data to obtain a trained detection model.

3. The method according to claim 2, characterized in that, The step of determining the adaptive knowledge distillation loss function using the multiple trained expert models, the detection model, and the multiple training sample images includes: The j-th training sample image is predicted using the multiple trained expert models to obtain the prediction vector of the j-th training sample image, where j = 1, 2, ..., N, and N is the total number of training sample images. Based on the prediction vector of the j-th training sample image, determine the distillation weights of each expert model for the j-th training sample image; The distillation loss function of the j-th training sample image in each expert model is determined using the multiple trained expert models and the detection model. The adaptive knowledge distillation loss function is obtained by weighting and summing the distillation weights of each expert model for the N training sample images and the distillation loss function of each expert model for the N training sample images.

4. The method according to claim 3, characterized in that, The step of determining the distillation weights of each expert model for the j-th training sample image based on the prediction vector of the j-th training sample image includes: Based on the prediction vector of the j-th training sample image, obtain the prediction value of the j-th training sample image in each expert model; Obtain the discrimination threshold for each expert model; Based on the predicted value of the j-th training sample image in each expert model and the discrimination threshold of each expert model, the distillation weights of each expert model for the j-th training sample image are determined.

5. The method according to claim 4, characterized in that, The step of determining the distillation weights of each expert model for the j-th training sample image based on the predicted value of the j-th training sample image in each expert model and the discrimination threshold of each expert model includes: When the predicted value of the j-th training sample image corresponding to the i-th expert model is greater than the discrimination threshold of the i-th expert model, the distillation weight of the i-th expert model for the j-th training sample image is determined based on the predicted value and the discrimination threshold. When the predicted value of the j-th training sample image corresponding to the i-th expert model is less than or equal to the discrimination threshold of the i-th expert model, the first preset value is determined as the distillation weight of the i-th expert model for the j-th training sample image.

6. The method according to claim 3, characterized in that, The step of determining the distillation loss function of the j-th training sample image in each expert model using the multiple trained expert models and the detection model includes: The j-th training sample image is extracted using each trained expert model to obtain the expert features of the j-th training sample image. The detection model is used to extract features from the j-th training sample image to obtain the detection features of the j-th training sample image; Based on the detection features of the j-th training sample image and the expert features of the j-th training sample image, the distillation loss function of the j-th training sample image in each expert model is determined.

7. The method according to any one of claims 1 to 6, characterized in that, When the multimedia data to be detected is video data, the preprocessing of the multimedia data to be detected to obtain the face image to be detected includes: The multimedia data to be detected is decoded to obtain multiple video frame images; Extract a predetermined number of target video frame images from the plurality of video frame images; The face image region and background image region in each target video frame image are determined, and the background image region in each target video frame image is deleted to obtain each face image to be detected.

8. The method according to claim 7, characterized in that, The step of determining the detection result of the multimedia data to be detected based on the detection result of the face image to be detected includes: Determine the number of target face images in the face image to be detected that are identified as fake face images; When the number of target face images exceeds a preset threshold, the detection result of the multimedia data to be detected is determined to be a fake face video.

9. The method according to claim 8, characterized in that, The method further includes: When the detection result of the face image to be detected is a fake face image, a fake face image marker is added to the target video frame image corresponding to the face image to be detected; When the detection result of the multimedia data to be detected is a fake face video, a fake face image marker is added to each video frame image.

10. A face authenticity detection device, characterized in that, The device includes: The first acquisition module is used to acquire multimedia data to be detected, and to preprocess the multimedia data to be detected to obtain a face image to be detected. The second acquisition module is used to acquire a trained detection model. The trained detection model is obtained through knowledge distillation training using feature knowledge provided by multiple trained expert models. Different trained expert models are trained using training sample data corresponding to different forgery methods in the following way: acquiring training sample data, which includes multiple training sample images and label information for each training sample image; acquiring M preset expert models and a second loss function corresponding to each expert model, where M is an integer greater than 1; acquiring forgery method information for each training sample image, dividing the training sample data based on the forgery method information to obtain real training sample images and forged training sample images corresponding to each forgery method; determining the real training sample images and the forged training sample images corresponding to the i-th forgery method as the training data subset corresponding to the i-th expert model, where i = 1, 2, ..., M; and training the i-th expert model using the training data subset corresponding to the i-th expert model and the second loss function corresponding to the i-th expert model to obtain the trained i-th expert model. The prediction processing module is used to perform prediction processing on the face image to be detected using the trained detection model to obtain the detection result of the face image to be detected. The first determining module is used to determine the detection result of the multimedia data to be detected based on the detection result of the face image to be detected.

11. The apparatus according to claim 10, characterized in that, Also includes: The third acquisition module is used to acquire a preset detection model and a first loss function of the detection model; The fourth acquisition module is used to acquire multiple trained expert models; The second determining module is used to determine an adaptive knowledge distillation loss function using the plurality of trained expert models, the detection model, and the plurality of training sample images; The first training module is used to jointly train the detection model using the adaptive knowledge distillation loss function, the first loss function, and the training sample data to obtain a trained detection model.

12. The apparatus according to claim 11, characterized in that, The second determining module is further configured to: The j-th training sample image is predicted using the multiple trained expert models to obtain the prediction vector of the j-th training sample image, where j = 1, 2, ..., N, and N is the total number of training sample images. Based on the prediction vector of the j-th training sample image, the distillation weights of each expert model for the j-th training sample image are determined. The distillation loss function of the j-th training sample image in each expert model is determined using the multiple trained expert models and the detection model. The distillation weights of each expert model for N training sample images and the distillation loss function of the N training sample images in each expert model are weighted and summed to obtain the adaptive knowledge distillation loss function.

13. A computer device, characterized in that, The computer device includes: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method according to any one of claims 1 to 9.

14. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the method according to any one of claims 1 to 9.

15. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the method described in any one of claims 1 to 9.

Images