Method and device for detecting fake person videos by integrating title information

Abstract

The present invention belongs to the field of video processing and network information security technology, and specifically relates to a method and device for detecting forged person videos that integrate title information. The method comprises first creating a dictionary of person names, extracting person name information from the title of a video to be detected that contains a face and a title based on text rule matching; then inputting the person name extracted from the video title as a keyword into the Internet for search, entering the entry of the corresponding person, extracting the person image, and saving the face area as a reference face image; secondly extracting the video frames in the video to be detected frame by frame, and saving the face area as a face image to be detected; inputting the face image to be detected and the reference face image into a pre-trained detection model M, and if the output of M is 1, marking the current video frame as forged. The present invention obtains the real face image of the target object from the Internet as prior knowledge based on the title information, and accurately detects the authenticity of the forged video.

Description

Technical Field

[0001] The invention belongs to the technical field of video processing and network information security, and in particular relates to a method and a device for detecting forged character videos integrating title information. Background Art

[0002] In recent years, researchers in academia and industry have conducted research on the detection of fake videos of important people with faces, and proposed a series of detection models based on deep learning. Based on the principle that such fake videos are usually generated by partial replacement methods such as face or lip replacement, these detection models usually convert the detection of fake videos into the detection of fake face areas in fake videos. However, due to the rapid development of deep fake technology and the increasingly hidden clues of fakes, these fake detection models generally have poor generalization ability and are unable to detect fake videos of unknown categories.

[0003] Compared with the general public, the audio and video information of key target figures such as national dignitaries, public figures, and cultural and sports stars is easier to obtain through public means, and once published, it usually brings better dissemination effects, so they are the key targets of forgery. At the same time, compared with general forged videos, the forged videos of these key targets will cause more severe effects once they are spread.

[0004] To this end, the present invention conducts research on forged character video detection technology for key figures such as national dignitaries, public figures, and cultural and sports stars, and provides a solution for authenticity discrimination by introducing prior knowledge for such videos with the names of key figures in the published video titles. Summary of the invention

[0005] In order to publish fake character videos with fake target object names in the titles, the present invention proposes a fake character video detection method and device that integrates title information, obtains the real face image of the target object from the Internet as prior knowledge based on the title information, and accurately detects the authenticity of the fake video.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] The present invention provides a method for detecting fake person videos by integrating title information, comprising the following steps:

[0008] Step 1, create a dictionary of person names, extract the name information of the person from the title of the video to be detected with a title containing a face based on text rule matching. If the match is successful, return the person's name and go to step 2. Otherwise, set the reference face image to empty and go to step 3.

[0009] Step 2: input the name of the person extracted from the video title as a keyword into the Internet for search. If no entry for the corresponding person is found, set the reference face image to be empty and proceed to step 3; otherwise, enter the entry for the corresponding person, extract the image of the person, and then use the face detection model to locate the face, and save the face area as the reference face image;

[0010] Step 3, extract the video frames in the video to be detected frame by frame, use the face detection model to locate the face, and save the face area as the face image to be detected; input the face image to be detected and the reference face image in step 2 or the reference face image set to be empty into the pre-trained detection model M. If the output of M is 1, the current video frame is marked as forged, otherwise it is judged to be true;

[0011] Step 4, counting the detection results of all video frames in the video to be detected, if the number of video frames judged as forged is greater than the number of video frames judged as real, then output the video to be detected as a forged video, otherwise, output the video to be detected as a real video.

[0012] Furthermore, the face detection model is a RetinaFace model.

[0013] Furthermore, the detection model M includes a face recognition model M1 and a forgery detection model M2, the face recognition model M1 is used to detect whether the face image to be detected in the video frame and the reference face image of the object to be forged are the same person, and the forgery detection model M2 is used to detect whether the face image to be detected in the video frame has traces of forgery.

[0014] Furthermore, the detection of whether the face recognition model M1 is the same person includes:

[0015] After the face images captured from the video to be detected and the face images captured from the reference image are input into the face recognition model M1, the features of the input images are extracted based on two AlexNet networks respectively. For each face image, 1000-dimensional features are first extracted, and then the features of the two face images are concatenated into 2000-dimensional features. The 2000-dimensional features are then converted into a 2-dimensional vector output through a fully connected layer, corresponding to the probabilities that the two face images are the same person and not the same person respectively. Finally, the value of the first dimension is output to reflect the similarity of the two faces.

[0016] Furthermore, the training process of the face recognition model M1 includes:

[0017] Collect the face images of i people, use the RetinaFace model to crop the face areas in these face images, and get 2 face area images for each person {f i (j)}, where i represents the ID number of the person, j represents the number of the face image, and the value range of j is 1-2;

[0018] Two face images of the same person form a positive sample pair: {f i (1) , f i (2)}, a total of i pairs of positive sample pairs are obtained; the first face image of the person numbered i and the first face image of the person numbered i+1 form a negative sample pair: A total of i negative sample pairs are obtained;

[0019] The above i positive sample pairs and negative sample pairs are sent to the face recognition model M1 for training, where the output of the positive sample pair is 1 and the output of the negative sample pair is 0;

[0020] The range of the output value r1 of the face recognition model M1 is [0,1]. The larger the value is, the more similar the face image in the video to be detected is to the reference face image of the object to be forged.

[0021] Furthermore, the forgery detection model M2 is used to perform authenticity detection on the face image to be detected, including:

[0022] After the face image captured from the video to be detected is input into the forgery detection model M2, the feature extraction of the input image is performed based on the AlexNet network. For each face image, 1000-dimensional features are extracted, and then these 1000-dimensional features are converted into a 2-dimensional vector output through a fully connected layer, corresponding to the probabilities of the face image being real and forged, and finally the value of the first dimension is output as the probability that the image is a real image.

[0023] Furthermore, the training process of the forgery detection model M2 includes:

[0024] Collect i real face images and i face images generated based on deep fakes, use the RetinaFace model to crop the face areas in these face images, and get the positive sample set p = {p1,p2,…,p i} and negative sample set n={n1,n2,…,n i}, input p and n into the forgery detection model M2 for training, where the output of each sample in the positive sample set is 1, and the output of each sample in the negative sample set is 0; the range of the output value r2 of the forgery detection model M2 is [0,1]. The larger the value, the greater the probability that the face image in the video to be detected is a real face image.

[0025] Furthermore, the authenticity score of the face image in the video to be detected is r=(r1+r2) / 2. If r is greater than a preset threshold, the face image to be detected is determined to be a real face, otherwise it is a fake face.

[0026] The present invention also provides a forged person video detection device integrating title information, comprising:

[0027] The character name extraction module in the video title is used to create a character name dictionary and extract the character name information from the title of the video to be detected with a title containing a face based on text rule matching. If the match is successful, the character name is returned, otherwise, the reference face image is set to empty;

[0028] The reference face image extraction module is used to input the name of the person extracted from the video title as a keyword into the Internet for search. If no entry of the corresponding person is found, the reference face image is set to be empty; otherwise, the entry of the corresponding person is entered, the image of the person is extracted, and the face detection model is used to locate the face, and the face area is saved as the reference face image;

[0029] The single-frame video frame authenticity judgment module is used to extract the video frames in the video to be detected frame by frame, locate the face using the face detection model, and save the face area as the face image to be detected; the face image to be detected and the reference face image or the reference face image set to be empty are input into the pre-trained detection model M. If the output of M is 1, the current video frame is marked as forged, otherwise it is judged to be authentic;

[0030] The authenticity determination module of the video to be detected is used to count the detection results of all video frames in the video to be detected. If the number of video frames judged as forged is greater than the number of video frames judged as real, the video to be detected is output as a forged video, otherwise, the video to be detected is output as a real video.

[0031] Furthermore, the face detection model is a RetinaFace model; the detection model M includes a face recognition model M1 and a forgery detection model M2, the face recognition model M1 is used to detect whether the face image to be detected in the video frame and the reference face image of the object to be forged are the same person, and the forgery detection model M2 is used to detect whether the face image to be detected in the video frame has traces of forgery.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] The forged person video detection method integrating title information of the present invention extracts the person's name based on the video title information, and obtains the real face image of the target object from the Internet as prior knowledge, which is used as the basis for judging whether the forged face is similar or not, and then combined with the authenticity detection of the face image to be detected, the authenticity of the forged video is detected from two perspectives of "likeness" and "authenticity", thereby improving the accuracy of forged video face detection with the forged object name as prior information. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 It is a flowchart of a method for detecting fake person videos by integrating title information according to an embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of a flow chart of authenticity determination of a single video frame according to an embodiment of the present invention;

[0037] Figure 3 It is a schematic diagram of the process of image processing by the face recognition model M1 according to an embodiment of the present invention;

[0038] Figure 4 It is a schematic diagram of the structure of the AlexNet network of an embodiment of the present invention;

[0039] Figure 5 2 is a schematic diagram of the process of image processing by the forgery detection model M2 according to an embodiment of the present invention;

[0040] Figure 6 is a structural block diagram of a forged person video detection device integrating title information according to an embodiment of the present invention;

[0041] Figure 7 The present invention is a schematic diagram of a process of extracting a face image and a person's name from a video to be detected according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0043] For a video V with a title and containing a face to be detected, this embodiment provides a method for detecting a fake person video by integrating the title information, such as Figure 1 As shown, the method comprises the following steps:

[0044] Step S11, create a celebrity name dictionary in advance, extract the name information of the person from the title of the video V to be detected based on text rule matching. If the match is successful, return the celebrity's name N and go to step S12. Otherwise, set the reference face image I to empty and go to step S13.

[0045] In step S12, the name N extracted from the video title is input as a keyword into Baidu Encyclopedia for search. If no entry for the corresponding person is found, the reference face image I is set to empty and the process proceeds to step S13. Otherwise, the entry for the corresponding person is entered, the image of the person is extracted, and the face detection model is used to locate the face, and the face area is saved as the reference face image I.

[0046] Step S13, extract the video frames in the video to be detected V frame by frame, locate the face using the face detection model, and save the face area as the face image to be detected F i ; The face image to be detected F i The reference face image I of step S12 or the reference face image set to be empty (the empty reference face image may come from step S11 or step S12) is input into the pre-trained detection model M. If the output of M is 1, the current video frame is marked as forged, otherwise it is judged to be real.

[0047] Step S14, counting the detection results of all video frames in the video to be detected V, if the number of video frames judged as forged is greater than the number of video frames judged as real, then output the video to be detected as a forged video, otherwise, output the video to be detected as a real video.

[0048] Preferably, the face detection model adopts the classic RetinaFace model; Figure 2 As shown, the detection model M includes a face recognition model M1 and a forgery detection model M2. The face recognition model M1 is used to detect the face image F to be detected in the video frame. i Whether the face image I to be forged is the same person as the reference face image I, that is, whether it is similar or not. The forgery detection model M2 is used to detect the face image F to be detected in the video frame. i Whether there are traces of forgery, that is, whether it is true or not; finally, a judgment is made based on the results of M1 and M2. For the video to be detected that has not retrieved the reference face image I from Baidu Encyclopedia, the detection result of M2 is directly used.

[0049] Using the face recognition model M1 to detect whether it is the same person includes:

[0050] like Figure 3 As shown in the figure, after the face image captured from the video to be detected and the face image captured from the reference image are input into the face recognition model M1, two AlexNet networks (network structure as shown in Figure 4 As shown in the figure, the feature extraction of the input image is performed. For each face image, 1000-dimensional features are extracted respectively. Then the features of the two face images are concatenated into 2000-dimensional features. The 2000-dimensional features are converted into a 2-dimensional vector output through a fully connected layer (fc), which respectively corresponds to the probability that the two face images are the same person and not the same person. Finally, the value of the first dimension is output to reflect the similarity of the two faces.

[0051] Figure 4 In the above, fc stands for fully connected layer, pool stands for pooling layer, and conv stands for convolution layer. The first parameter after the convolution layer indicates the number of convolution kernels, and the second parameter indicates the step size of the convolution. For example, the number of convolution kernels in the first convolution layer is 96, the step size is 4, and 11×11 indicates the matrix size of the convolution kernel; the number of convolution kernels in the second convolution layer is 256, and the default step size is 1. The parameter after the fully connected layer indicates the dimension of the output after the operation.

[0052] Specifically, the training process of the face recognition model M1 includes:

[0053] Collect 100 face images of people, use the RetinaFace model to crop the face areas in these face images, and get 2 face area images for each person. i (j)}, where i represents the ID number of the person, ranging from 1 to 100, and j represents the ID of the face image, ranging from 1 to 2.

[0054] Two face images of the same person form a positive sample pair: {f i (1) , f i (2)}, a total of 100 pairs of positive sample pairs are obtained; the first face image of the person numbered i and the first face image of the person numbered i+1 form a negative sample pair: 1<<i<<99, when i=100, the negative sample pair constructed is A total of 100 negative sample pairs are obtained.

[0055] The above 100 positive sample pairs and negative sample pairs are sent to the face recognition model M1 for training, where the output of the positive sample pairs is 1 and the output of the negative sample pairs is 0.

[0056] The range of the output value r1 of the face recognition model M1 is [0,1]. The larger the value, the more likely it is that there is a face image F in the video to be detected. i The more similar it is to the reference face image I of the object to be forged.

[0057] Using the forgery detection model M2 to detect the authenticity of the face image to be detected includes:

[0058] like Figure 5 As shown in the figure, after the face image captured from the video to be detected is input into the forgery detection model M2, based on the AlexNet network (network structure as shown in Figure 4 For each face image, a 1000-dimensional feature is extracted, and then this 1000-dimensional feature is converted into a 2-dimensional vector output through a fully connected layer (fc), which corresponds to the probability that the face image is real and forged, and finally the value of the first dimension is output as the probability that the image is a real image.

[0059] Specifically, the training process of the forgery detection model M2 includes:

[0060] We collect 100 real face images and 100 face images generated based on deepfakes, and use the RetinaFace model to crop the face areas in these face images to obtain a positive sample set p = {p1, p2, …, p i} and negative sample set n={n1,n2,…,n i}, input p and n into the forgery detection model M2 for training, where the output of each sample in the positive sample set is 1, and the output of each sample in the negative sample set is 0; the range of the output value r2 of the forgery detection model M2 is [0,1]. The larger the value, the smaller the face image F in the video to be detected. i The greater the probability that it is a real face image.

[0061] Finally, the face image F in the video to be detected i The true-false score r = (r1 + r2) / 2, if r is greater than a preset threshold, the face image to be detected is determined to be a real face, otherwise it is a fake face. In practical applications, the threshold T can be set to 0.5.

[0062] Corresponding to the forged person video detection method integrating title information, as Figure 6 As shown, this embodiment also proposes a forged person video detection device integrating title information, including:

[0063] The character name extraction module 61 in the video title is used to create a character name dictionary and extract the character name information from the title of the video to be detected with a title containing a face based on text rule matching. If the match is successful, the character name is returned, otherwise, the reference face image is set to empty.

[0064] The reference face image extraction module 62 is used to input the name of the person extracted from the video title as a keyword into the Internet for searching. If no entry for the corresponding person is found, the reference face image is set to empty; otherwise, the entry for the corresponding person is entered, the image of the person is extracted, and the face detection model is used to locate the face, and the face area is saved as a reference face image.

[0065] The single-frame video frame authenticity judgment module 63 is used to extract the video frames in the video to be detected frame by frame, use the face detection model to locate the face, and save the face area as the face image to be detected; the face image to be detected and the reference face image or the reference face image set to be empty are input into the pre-trained detection model M. If the output of M is 1, the current video frame is marked as forged, otherwise it is judged to be authentic.

[0066] The authenticity determination module 64 of the video to be detected is used to count the detection results of all video frames in the video to be detected. If the number of video frames judged as forged is greater than the number of video frames judged as authentic, the video to be detected is output as a forged video, otherwise, the video to be detected is output as an authentic video.

[0067] Preferably, the face detection model is a RetinaFace model; the detection model M includes a face recognition model M1 and a forgery detection model M2, the face recognition model M1 is used to detect whether the face image to be detected in the video frame and the reference face image of the object to be forged are the same person, and the forgery detection model M2 is used to detect whether the face image to be detected in the video frame has traces of forgery.

[0068] A specific example is given below to facilitate a better understanding of the present invention.

[0069] Now we need to detect the authenticity of a video titled "Someone's latest speech". We extract the video frames in this video one by one for authenticity detection. The detection process for each video frame is as follows:

[0070] First, for regular expression matching of the title text, use the created dictionary of celebrity names.

[0071] Then, the name "someone" of the protagonist in the video extracted from the title is input into Baidu Encyclopedia for search, and the relevant entry is entered, and the face image of the relevant person "someone" is extracted from the above entry as a reference face image.

[0072] for Figure 7 The face image and the reference face image extracted from the video to be detected are used to extract the face features based on the AlexNet network and input into the trained face recognition model M1. The process of M1 detecting the similarity of the two faces is as shown in steps S21-S29:

[0073] Step S21, for the input image, first reshape it into a size of 227×227×3, and then send it to the first convolution layer, which has 96 convolution kernels, each of which has a size of 11×11×3 and a step size of 4. Because (227-11) / 4+1=55, the output after convolution is 55×55×96, and then filter these output values ​​with Relu. The data dimension after Relu is also 55×55×96, and then send it to Max pool. Max pool uses a 3×3 pooling kernel with a step size of 2. The data dimension after Max pool is 27×27×96. Finally, LRN (Local Response Normalization) is used for normalization to obtain an output with a dimension of 27×27×96.

[0074] Step S22, the above 27×27×96 output is used as the input of the second convolutional layer. The second convolutional layer has 256 convolution kernels, each of which has a size of 5×5 and a step size of 1. The dimension after convolution is 27×27×256. Then, these output values ​​are filtered by Relu. The data dimension after Relu is also 27×27×256. Max pool uses a 3×3 pooling kernel with a step size of 2. The data dimension after Max pool is 13×13×256. Finally, LRN (Local Response Normalization) is used for normalization to obtain an output with a dimension of 13×13×256.

[0075] In step S23, the above 13×13×256 output is used as the input of the third convolutional layer. The third convolutional layer has 384 convolution kernels, each of which has a size of 3×3 and a step size of 1. The dimension after convolution is 13×13×384. Then, these output values ​​are filtered with Relu, and the data dimension after Relu is also 13×13×384 as the output of this layer.

[0076] In step S24, the above 13×13×384 output is used as the input of the fourth convolutional layer. The fourth convolutional layer has 384 convolution kernels, each of which has a size of 3×3 and a step size of 1. The dimension after convolution is 13×13×384. Then, these output values ​​are filtered with Relu, and the data dimension after Relu is also 13×13×384 as the output of this layer.

[0077] Step S25, the above 13×13×384 output is used as the input of the 5th convolutional layer. The 5th convolutional layer has 256 convolution kernels, each of which has a size of 3×3 and a step size of 1. The dimension after convolution is 13×13×256. Then, these output values ​​are filtered by Relu. The data dimension after Relu is also 13×13×256. Max pool uses a 3×3 pooling kernel with a step size of 2. The data dimension after Max pool is 6×6×256. Finally, LRN (Local Response Normalization) is used for normalization to obtain an output with a dimension of 6×6×256.

[0078] In step S26, the above 6×6×256 output is sent to the sixth fully connected layer, linearly mapped into a 4096×1 dimensional output, and then passes through ReLU and Drop out to output a 4096×1 dimensional result.

[0079] In step S27, the above 4096×1 output is sent to the fully connected layer of the 7th layer, linearly mapped into a 4096×1 dimensional output, and then through ReLU and Drop out, a 4096×1 dimensional result is output.

[0080] In step S28, the above 4096×1 output is sent to the 8th fully connected layer and linearly mapped into a 1000×1 dimensional output.

[0081] Step S29, concatenate the 1000-dimensional features of the extracted face image to be detected and the reference face image into 2000-dimensional features, send them to the final fully connected layer, take the output data of the first dimension, and obtain the output r1=0.8 of M1.

[0082] Will Figure 7 The face image extracted from the video to be detected is input into the trained forgery detection model M2 to extract features. The processing flow is the same as steps S21-28 above, but the parameters of the AlexNet network in M2 are different from those in M1, and the output r2 of M2 is obtained to be 0.6.

[0083] Finally, the score of the above video frame is (r1+r2) / 2=(0.8+0.6) / 2=0.7. Since 0.7>0.5, the video frame is judged as a real video frame.

[0084] The above steps are repeated for each video frame, and the detection results of all video frames in the video to be detected are counted. If the number of video frames judged as forged is greater than the number of video frames judged as real, the video to be detected is output as a forged video, otherwise, the video to be detected is output as a real video.

[0085] It should be noted that, in this article, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus.

[0086] Finally, it should be noted that the above is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for detecting fake person videos by integrating title information, characterized in that: The following steps are involved: Step 1, create a dictionary of person names, extract the name information of the person from the title of the video to be detected with a title containing a face based on text rule matching. If the match is successful, return the person's name and go to step 2. Otherwise, set the reference face image to empty and go to step 3. Step 2: input the name of the person extracted from the video title as a keyword into the Internet for searching. If no entry of the corresponding person is found, set the reference face image to be empty and proceed to step 3; otherwise, enter the entry of the corresponding person, extract the image of the person, and then use the face detection model to locate the face, and save the face area as the reference face image; the face detection model is the RetinaFace model; Step 3, extract the video frames in the video to be detected frame by frame, use the face detection model to locate the face, and save the face area as the face image to be detected; input the face image to be detected and the reference face image in step 2 or the reference face image set to be empty into the pre-trained detection model M. If the output of M is 1, the current video frame is marked as forged, otherwise it is judged to be true; The detection model M includes a face recognition model M1 and a forgery detection model M2, wherein the face recognition model M1 is used to detect whether the face image to be detected in the video frame and the reference face image of the object to be forged are the same person, and the forgery detection model M2 is used to detect whether the face image to be detected in the video frame has traces of forgery; Using the face recognition model M1 to detect whether the two faces are the same person includes: after inputting the face image captured from the video to be detected and the face image captured from the reference image into the face recognition model M1, respectively extracting the features of the input images based on two AlexNet networks, first extracting 1000-dimensional features from each face image, then concatenating the features of the two face images into 2000-dimensional features, and then converting the 2000-dimensional features into a 2-dimensional vector output through a fully connected layer, respectively corresponding to the probability that the two face images are the same person and are not the same person, and finally outputting the value of the first dimension to reflect the similarity of the two faces; Using the forgery detection model M2 to detect the authenticity of the face image to be detected includes: after the face image captured from the video to be detected is input into the forgery detection model M2, the feature extraction of the input image is performed based on the AlexNet network, 1000-dimensional features are extracted for each face image, and then the 1000-dimensional features are converted into a 2-dimensional vector output through a fully connected layer, which respectively corresponds to the probability that the face image is real and forged, and finally the value of the first dimension is output as the probability that the image is a real image; Step 4, counting the detection results of all video frames in the video to be detected, if the number of video frames judged as forged is greater than the number of video frames judged as real, then output the video to be detected as a forged video, otherwise, output the video to be detected as a real video.

2. The forged person video detection method integrating title information according to claim 1 is characterized in that: The training process of the face recognition model M1 includes: Collect the face images of i people, use the RetinaFace model to crop the face areas in these face images, and get 2 face area images for each person {f i (j) }, where i represents the ID number of the person, j represents the number of the face image, and the value range of j is 1-2; Two face images of the same person form a positive sample pair: {f i (1) ,f i (2) }, a total of i pairs of positive sample pairs are obtained; the first face image of the person numbered i and the first face image of the person numbered i+1 form a negative sample pair: A total of i negative sample pairs are obtained; The above i positive sample pairs and negative sample pairs are sent to the face recognition model M1 for training, where the output of the positive sample pair is 1 and the output of the negative sample pair is 0; The range of the output value r1 of the face recognition model M1 is [0,1]. The larger the value is, the more similar the face image in the video to be detected is to the reference face image of the object to be forged.

3. The method for detecting fake person videos by integrating title information according to claim 2, characterized in that: The training process of the forgery detection model M2 includes: Collect i real face images and i face images generated based on deep fakes, use the RetinaFace model to crop the face areas in these face images, and get the positive sample set p = {p1,p2,…,p i } and negative sample set n={n1,n2,…,n i }, input p and n into the forgery detection model M2 for training, where the output of each sample in the positive sample set is 1, and the output of each sample in the negative sample set is 0; the range of the output value r2 of the forgery detection model M2 is [0,1]. The larger the value, the greater the probability that the face image in the video to be detected is a real face image.

4. The method for detecting fake person videos by integrating title information according to claim 3, characterized in that: The authenticity score of the face image in the video to be detected is r=(r1+r2) / 2. If r is greater than a preset threshold, the face image to be detected is determined to be a real face, otherwise it is a fake face.

5. A forged person video detection device integrating title information, characterized in that: The device is used to implement the forged person video detection method integrated with title information as described in any one of claims 1 to 4, comprising: The character name extraction module in the video title is used to create a character name dictionary and extract the character name information from the title of the video to be detected with a title containing a face based on text rule matching. If the match is successful, the character name is returned, otherwise, the reference face image is set to empty; The reference face image extraction module is used to input the name of the person extracted from the video title as a keyword into the Internet for search. If no entry of the corresponding person is found, the reference face image is set to be empty; otherwise, the entry of the corresponding person is entered, the image of the person is extracted, and the face detection model is used to locate the face, and the face area is saved as the reference face image; The single-frame video frame authenticity judgment module is used to extract the video frames in the video to be detected frame by frame, locate the face using the face detection model, and save the face area as the face image to be detected; the face image to be detected and the reference face image or the reference face image set to be empty are input into the pre-trained detection model M. If the output of M is 1, the current video frame is marked as forged, otherwise it is judged to be authentic; The authenticity determination module of the video to be detected is used to count the detection results of all video frames in the video to be detected. If the number of video frames judged as forged is greater than the number of video frames judged as real, the video to be detected is output as a forged video, otherwise, the video to be detected is output as a real video.

Images