Method for carrying out real person verification through video

Abstract

The invention relates to the technical field of image recognition, in particular to a method for carrying out real person verification through a video. The technical problem is that according to a video real person verification method in the prior art, for patterns, expression pictures and the like similar to human face features, a system may be mistakenly regarded as a human face, the judgment capability is not high enough, and the verification system may not effectively resist attack of a fake human face. According to the technical scheme, the method for carrying out real person verification through the video comprises the steps that SCL is introduced, the distance from each category sample to the category center of the category sample is calculated, and the distance is minimized, so that the distinguishing capacity of a model for fake faces is enhanced; according to the method, the algorithm specially used for detecting face tampering is trained by using the deep learning technology, the difference between a real face and a forged face can be learned, and a suspicious forged face area can be automatically identified and marked, so that the accuracy and the safety of video real person verification can be greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method for real person verification through video. Background Art

[0002] The video real person verification method uses computer vision, image processing and biostatistics principles to extract features, match and identify faces in video streams. Specifically, the system will collect, locate and pre-process the facial images in the video, and then use advanced algorithms such as deep learning models to analyze facial features and compare them with known facial databases to verify whether the faces appearing in the video are real individuals. However, the video real person verification method under the existing technology usually detects and identifies based on some facial features such as grayscale features, relative position structural features, etc., and the system may mistakenly identify patterns and expression pictures with similar facial features as human faces. The judgment ability is not high enough, which may cause the verification system to be unable to effectively resist attacks by forged faces. Summary of the invention

[0003] In order to overcome the problem that the existing video real-person verification method usually detects and recognizes based on some facial features such as grayscale features, relative position structural features, etc., the system may mistakenly identify patterns and expression pictures with similar facial features as faces. The judgment ability is not high enough, which may cause the verification system to be unable to effectively resist attacks by forged faces.

[0004] The technical solution of the present invention is: a method for real person verification through video, comprising the following steps:

[0005] S1: Collect a large number of real face videos and fake face videos from public datasets and web crawler channels, and annotate the collected videos frame by frame;

[0006] S2: convert the video into continuous video frame images and perform image data preprocessing;

[0007] S3: Select a convolutional neural network model and perform model training. In the training, a metric learning mechanism is introduced to help the model learn an embedding space.

[0008] S4: Introducing the single center loss SCL by calculating the distance from each category sample to its category center and minimizing this distance to enhance the model's ability to distinguish fake faces. Combining metric learning and single center loss, the model parameters are continuously adjusted through the back propagation algorithm to minimize the loss function value.

[0009] S5: Input the video to be verified into the system, extract the video frames, use the pre-trained face detection model to detect the face area in each frame image, and perform precise positioning;

[0010] S6: Extract features from the detected face area and input the extracted features into the trained deep learning model for face tampering detection. The model uses the embedding space learned by metric learning and single center loss to output the tampering probability of each face area or the specific location and size of the tampered area;

[0011] S7: According to the output result of the model, mark the suspected forged face area in the video frame and output the final verification result;

[0012] S8: Set a tampering threshold according to actual needs to determine the verification result. When the tampering probability of a certain face area exceeds the threshold, it is determined to be a forged face; otherwise, it is determined to be a real face;

[0013] S9: For the entire video sequence, the tampering detection results of the face area in each frame are integrated, and whether the verification is successful is determined according to the set rules.

[0014] As a preferred method, a large number of real face videos and fake face videos are collected from public data sets and web crawler channels, and the collected videos are annotated frame by frame; the specific steps are as follows:

[0015] S201: Search and filter public datasets containing real faces and fake face videos, including LFW and CelebA;

[0016] S202: Using a crawler program to crawl real faces and fake face videos from websites and platforms;

[0017] S203: Preliminarily screening the collected videos, removing videos with low quality, repeated content or not meeting the requirements, classifying and arranging the screened videos according to real faces and forged faces, and storing them in a designated file directory;

[0018] S204: performing data enhancement processing on the screened videos, including rotation, scaling, cropping, and flipping, to increase data diversity and generalization capability, and performing color adjustment and noise addition processing on the videos to simulate video quality in different scenarios;

[0019] S205: Select the LabelImg annotation tool, and perform necessary configuration and adjustment on the tool, including setting the annotation category and annotation format;

[0020] S206: using a labeling tool to label the screened video frame by frame, marking the face area in each frame with a rectangular frame, and labeling it as "real face" or "forged face"; for forged faces, additionally using polygons and other shapes to accurately label the specific location and size of the tampered area;

[0021] S207: Storing the annotated data in a designated database, where the annotated data includes the video file name, frame number, location and size of the face region, and annotated category information.

[0022] Preferably, the video is converted into continuous video frame images, and image data preprocessing is performed; the specific steps are as follows:

[0023] S301: Obtaining video data to be processed;

[0024] S302: Decode the video using a video decoder to obtain a video frame sequence;

[0025] S303: extracting each frame of image from the decoded video frame sequence in time order, and saving it as an image file;

[0026] S304: For an image with noise, use a filtering algorithm to remove the noise and restore the clarity and details of the image;

[0027] S305: For the blurred image, a Sobel sharpening algorithm is used to perform blur processing to improve the clarity of the image;

[0028] S306: For a distorted image, select a geometric distortion algorithm and a color distortion algorithm to perform distortion correction;

[0029] S307: Performing face detection on the cleaned image using a pre-trained face detection model to obtain the position and size of the face area;

[0030] S308: cropping the image according to the result of face detection, and only retaining necessary information of the face area and its surroundings;

[0031] S309: performing normalization processing on the cropped image to adjust the image size to a uniform size.

[0032] As a preference, a convolutional neural network model is selected and model training is performed. A metric learning mechanism is introduced during the training to help the model learn an embedding space. The single center loss SCL is introduced to enhance the model's ability to distinguish fake faces by calculating the distance from each category sample to its category center and minimizing this distance. The metric learning and single center loss are combined, and the model parameters are continuously adjusted through the back propagation algorithm to minimize the loss function value. The specific steps are as follows:

[0033] S401: Select the convolutional neural network model architecture;

[0034] S402: Determine the number of layers, convolution kernel size, activation function parameters of the model, and initialize using a pre-trained model;

[0035] S403: Embedding a metric learning mechanism during the training of the deep learning model;

[0036] S404: defining a goal of an embedding space, in which samples of the same category, i.e., the distance between real faces, should be optimized to be as close as possible, while samples of different categories, i.e., the distance between forged faces and real faces, should be optimized to be as far as possible;

[0037] S405: Contrast loss and triplet loss are used to construct a specific form of loss function to reflect the distance between samples;

[0038] S406: Perform training iterations and calculate the value of the loss function, and update the weight of the model through a back propagation algorithm to minimize the value of the loss function;

[0039] S407: setting one or more feature extraction layers in the deep learning model to extract useful feature vectors from the input face image;

[0040] S408: extracting feature vectors, and using the extracted feature vectors as inputs of metric learning to calculate distances between samples;

[0041] S409: Define a single center loss function SCL. The single center loss calculates the distance between each category sample and the center of its corresponding category and minimizes this distance to enhance the model's ability to distinguish fake faces.

[0042] S410: before training starts, a category center is initialized for each category, where the category center is a random vector or a representative vector obtained through preprocessing;

[0043] S411: During the training process, the category center is continuously updated as the model weights are updated;

[0044] S412: In each training iteration, calculate the distance between each sample and the center of its corresponding category;

[0045] S413: using this distance as the input of the single center loss and calculating the value of the loss function;

[0046] S414: Update the weights and category centers of the model through the back-propagation algorithm to minimize the value of the loss function.

[0047] Preferably, the video to be verified is input into the system, and the video frames are extracted. The face area is detected in each frame using a pre-trained face detection model, and the face is accurately located. The specific steps are as follows:

[0048] S601: The system receives the video file to be verified and loads the video through the video processing library OpenCV;

[0049] S602: Use cv2.VideoCapture of the video processing library to read the video frame by frame;

[0050] S603: number each frame of image;

[0051] S604: storing the extracted video frames in a data list;

[0052] S605: Loading the trained face detection convolutional neural network model;

[0053] S606: For each frame of the extracted image, face detection is performed using the loaded face detection model, and the model outputs the coordinates and size of the face area, as well as the confidence score;

[0054] S607: Screening out possible face regions according to the confidence score and a preset threshold;

[0055] S608: Accurately locate the screened face area by further adjusting the coordinates and size of the face area;

[0056] S609: storing information such as coordinates, size, and confidence score of the face region detected in each frame of image in a data structure.

[0057] Preferably, feature extraction is performed on the detected face area, and the extracted features are input into the trained deep learning model for face tampering detection. The model uses the embedding space learned by metric learning and single center loss to output the tampering probability of each face area or the specific position and size of the tampered area. The specific steps are as follows:

[0058] S701: In each frame of the image, based on the output of the face detection and positioning step, confirm the coordinates and range of the face area;

[0059] S702: extracting texture information of the face region using image processing techniques, including Gabor filter and local binary pattern;

[0060] S703: extracting the contour and shape features of the facial features through facial key point detection, which reflect the overall structure and local details of the face;

[0061] S704: In the face area, statistically analyze the histograms of different color channels to extract color distribution information. The color features reflect the skin color and lighting conditions of the face.

[0062] S705: fusing the extracted texture features, shape features and color features through weighted fusion to form a comprehensive face feature vector;

[0063] S706: Inputting the fused facial feature vector into the trained deep learning model;

[0064] S707: The deep learning model processes the input facial feature vector and outputs the tampering probability of each facial region or the specific location and size of the tampering region.

[0065] Preferably, based on the output result of the model, a suspicious forged face area is marked in the video frame, and the final verification result is output; a tampering threshold is set according to actual needs to determine the verification result. When the tampering probability of a certain face area exceeds the threshold, it is determined to be a forged face; otherwise, it is determined to be a real face; the specific steps are as follows:

[0066] S801: Marking a suspected forged face area in a video frame according to an output result of the deep learning model;

[0067] S802: If the model outputs the specific location and size of the tampered area, a marking box is accurately drawn based on the information;

[0068] S803: If the model only outputs the tampering probability, it can be determined whether to mark it according to the set threshold;

[0069] S804: Setting a tampering threshold according to actual needs to determine the verification result. When the tampering probability of a certain face area exceeds the threshold, it is determined to be a forged face; otherwise, it is determined to be a real face;

[0070] S805: Perform tampering detection on each frame of the entire video sequence, and integrate the detection results of each frame, including counting the number of forged faces detected, and recording the position and size information of the forged faces in each frame;

[0071] S806: Output the final verification result according to the integrated detection results, including whether the video passes the verification, the number and location of the detected forged faces.

[0072] Preferably, when determining the verification result, the following steps are included:

[0073] S1001: Based on the needs of actual application scenarios, including security monitoring and content review, clarify the criteria for determining forged faces;

[0074] S1002: According to the performance of the deep learning model and the statistical results of historical data, a tampering threshold is set to distinguish between real faces and forged faces;

[0075] S1003: For each frame of image, use a pre-trained face detection model to detect and locate the face area;

[0076] S1004: For each face region, extract its feature vector and input it into the trained deep learning model to calculate the tampering probability;

[0077] S1005: Compare the calculated tampering probability with a set threshold value, and if it exceeds the threshold value, mark the face area as a forged face; otherwise, mark it as a real face;

[0078] S1006: setting a judgment rule for comprehensively considering the tampering detection result of the entire video sequence, the rule comprising: if more than a certain number of forged face areas appear in the video, or signs of forged faces appear in multiple consecutive frames of images, then judging that the verification has failed;

[0079] S1007: Outputting the final verification conclusion, which includes "verification passed" or "verification failed", and accompanying related forged face information and statistical data;

[0080] S1009: The output result is presented in the form of a text report, which lists in detail the location, size, tampering probability information of the area determined to be a forged face in each frame of the image, as well as the overall statistical data and determination results.

[0081] Beneficial effects of the present invention:

[0082] 1. Compared with the existing video real person verification method, which usually detects and recognizes based on some features of the face, such as grayscale features, relative position structure features, etc., the system may mistake patterns and expression pictures with similar facial features for faces, and the judgment ability is not high enough, which may cause the verification system to be unable to effectively resist the attack of forged faces; this method uses deep learning technology to train an algorithm specifically for detecting face tampering, which can learn the difference between real faces and forged faces, and automatically identify and mark suspicious forged face areas, thereby greatly improving the accuracy and security of video real person verification;

[0083] 2. By converting the video into continuous video frame images and preprocessing the image data, a convolutional neural network model is selected and the model is trained. A metric learning mechanism is introduced in the training to help the model learn an embedding space. The single center loss SCL is introduced to enhance the model's ability to distinguish forged faces by calculating the distance from each category sample to its category center and minimizing this distance. The model parameters are continuously adjusted through the back propagation algorithm in combination with metric learning and single center loss to minimize the loss function value. The video to be verified is input into the system, and the video frame is extracted. The face area is detected in each frame image using a pre-trained face detection model and accurately located. The features of the detected face area are extracted, and the extracted features are input into the trained deep learning model for face tampering detection. The model uses the embedding space learned by metric learning and single center loss to output the tampering probability of each face area or the specific location and size of the tampering area. According to the output results of the model, the suspicious forged face area is marked in the video frame, and the final verification result is output, thereby improving the accuracy and security of video real person verification.

[0084] 3. By locating the face area and accurately marking and outputting the specific position and size of the forged face, refined detection of forged faces is achieved, which helps users understand the face tampering in the video more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 What is shown is a schematic diagram of the result determination process of the method for real person verification through video of the present invention;

[0086] Figure 2 Shown is a schematic diagram of the face detection and positioning process of the method for real-person verification through video of the present invention. DETAILED DESCRIPTION

[0087] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0088] See also Figure 1-2 The present invention provides an embodiment: a method for real person verification through video, comprising the following steps:

[0089] S1: Collect a large number of real face videos and fake face videos from public datasets and web crawler channels, and annotate the collected videos frame by frame;

[0090] S2: convert the video into continuous video frame images and perform image data preprocessing;

[0091] S3: Select a convolutional neural network model and perform model training. In the training, a metric learning mechanism is introduced to help the model learn an embedding space.

[0092] S4: Introducing the single center loss SCL by calculating the distance from each category sample to its category center and minimizing this distance to enhance the model's ability to distinguish fake faces. Combining metric learning and single center loss, the model parameters are continuously adjusted through the back propagation algorithm to minimize the loss function value.

[0093] S5: Input the video to be verified into the system, extract the video frames, use the pre-trained face detection model to detect the face area in each frame image, and perform precise positioning;

[0094] S6: Extract features from the detected face area and input the extracted features into the trained deep learning model for face tampering detection. The model uses the embedding space learned by metric learning and single center loss to output the tampering probability of each face area or the specific location and size of the tampered area;

[0095] S7: According to the output result of the model, mark the suspected forged face area in the video frame and output the final verification result;

[0096] S8: Set a tampering threshold according to actual needs to determine the verification result. When the tampering probability of a certain face area exceeds the threshold, it is determined to be a forged face; otherwise, it is determined to be a real face;

[0097] S9: For the entire video sequence, the tampering detection results of the face area in each frame are integrated, and whether the verification is successful is determined according to the set rules.

[0098] As a preferred method, a large number of real face videos and fake face videos are collected from public data sets and web crawler channels, and the collected videos are annotated frame by frame; the specific steps are as follows:

[0099] S201: Search and filter public datasets containing real faces and fake face videos, including LFW and CelebA;

[0100] S202: Using a crawler program to crawl real faces and fake face videos from websites and platforms;

[0101] S203: Preliminarily screening the collected videos, removing videos with low quality, repeated content or not meeting the requirements, classifying and arranging the screened videos according to real faces and forged faces, and storing them in a designated file directory;

[0102] S204: performing data enhancement processing on the screened videos, including rotation, scaling, cropping, and flipping, to increase data diversity and generalization capability, and performing color adjustment and noise addition processing on the videos to simulate video quality in different scenarios;

[0103] S205: Select the LabelImg annotation tool, and perform necessary configuration and adjustment on the tool, including setting the annotation category and annotation format;

[0104] S206: using a labeling tool to label the screened video frame by frame, marking the face area in each frame with a rectangular frame, and labeling it as "real face" or "forged face"; for forged faces, additionally using polygons and other shapes to accurately label the specific location and size of the tampered area;

[0105] S207: Storing the annotated data in a designated database, where the annotated data includes the video file name, frame number, location and size of the face region, and annotated category information.

[0106] Preferably, the video is converted into continuous video frame images, and image data preprocessing is performed; the specific steps are as follows:

[0107] S301: Obtaining video data to be processed;

[0108] S302: Decode the video using a video decoder to obtain a video frame sequence;

[0109] S303: extracting each frame of image from the decoded video frame sequence in time order, and saving it as an image file;

[0110] S304: For an image with noise, use a filtering algorithm to remove the noise and restore the clarity and details of the image;

[0111] S305: For the blurred image, a Sobel sharpening algorithm is used to perform blur processing to improve the clarity of the image;

[0112] S306: For a distorted image, select a geometric distortion algorithm and a color distortion algorithm to perform distortion correction;

[0113] S307: Performing face detection on the cleaned image using a pre-trained face detection model to obtain the position and size of the face area;

[0114] S308: cropping the image according to the result of face detection, and only retaining necessary information of the face area and its surroundings;

[0115] S309: performing normalization processing on the cropped image to adjust the image size to a uniform size.

[0116] As a preference, a convolutional neural network model is selected and model training is performed. A metric learning mechanism is introduced during the training to help the model learn an embedding space. The single center loss SCL is introduced to enhance the model's ability to distinguish fake faces by calculating the distance from each category sample to its category center and minimizing this distance. The metric learning and single center loss are combined, and the model parameters are continuously adjusted through the back propagation algorithm to minimize the loss function value. The specific steps are as follows:

[0117] S401: Select the convolutional neural network model architecture;

[0118] S402: Determine the number of layers, convolution kernel size, activation function parameters of the model, and initialize using a pre-trained model;

[0119] S403: Embedding a metric learning mechanism during the training of the deep learning model;

[0120] S404: defining a goal of an embedding space, in which samples of the same category, i.e., the distance between real faces, should be optimized to be as close as possible, while samples of different categories, i.e., the distance between forged faces and real faces, should be optimized to be as far as possible;

[0121] S405: Contrast loss and triplet loss are used to construct a specific form of loss function to reflect the distance between samples;

[0122] S406: Perform training iterations and calculate the value of the loss function, and update the weight of the model through a back propagation algorithm to minimize the value of the loss function;

[0123] S407: setting one or more feature extraction layers in the deep learning model to extract useful feature vectors from the input face image;

[0124] S408: extracting feature vectors, and using the extracted feature vectors as inputs of metric learning to calculate distances between samples;

[0125] S409: Define a single center loss function SCL. The single center loss calculates the distance between each category sample and the center of its corresponding category and minimizes this distance to enhance the model's ability to distinguish fake faces.

[0126] S410: before training starts, a category center is initialized for each category, where the category center is a random vector or a representative vector obtained through preprocessing;

[0127] S411: During the training process, the category center is continuously updated as the model weights are updated;

[0128] S412: In each training iteration, calculate the distance between each sample and the center of its corresponding category;

[0129] S413: using this distance as the input of the single center loss and calculating the value of the loss function;

[0130] S414: Update the weights and category centers of the model through the back-propagation algorithm to minimize the value of the loss function.

[0131] Preferably, the video to be verified is input into the system, and the video frames are extracted. The face area is detected in each frame using a pre-trained face detection model, and the face is accurately located. The specific steps are as follows:

[0132] S601: The system receives the video file to be verified and loads the video through the video processing library OpenCV;

[0133] S602: Use cv2.VideoCapture of the video processing library to read the video frame by frame;

[0134] S603: number each frame of image;

[0135] S604: storing the extracted video frames in a data list;

[0136] S605: Loading the trained face detection convolutional neural network model;

[0137] S606: For each frame of the extracted image, face detection is performed using the loaded face detection model, and the model outputs the coordinates and size of the face area, as well as the confidence score;

[0138] S607: Screening out possible face regions according to the confidence score and a preset threshold;

[0139] S608: Accurately locate the screened face area by further adjusting the coordinates and size of the face area;

[0140] S609: storing information such as coordinates, size, and confidence score of the face region detected in each frame of image in a data structure.

[0141] Preferably, feature extraction is performed on the detected face area, and the extracted features are input into the trained deep learning model for face tampering detection. The model uses the embedding space learned by metric learning and single center loss to output the tampering probability of each face area or the specific position and size of the tampered area. The specific steps are as follows:

[0142] S701: In each frame of the image, based on the output of the face detection and positioning step, confirm the coordinates and range of the face area;

[0143] S702: extracting texture information of the face region using image processing techniques, including Gabor filter and local binary pattern;

[0144] S703: extracting the contour and shape features of the facial features through facial key point detection, which reflect the overall structure and local details of the face;

[0145] S704: In the face area, statistically analyze the histograms of different color channels to extract color distribution information. The color features reflect the skin color and lighting conditions of the face.

[0146] S705: fusing the extracted texture features, shape features and color features through weighted fusion to form a comprehensive face feature vector;

[0147] S706: Inputting the fused facial feature vector into the trained deep learning model;

[0148] S707: The deep learning model processes the input facial feature vector and outputs the tampering probability of each facial region or the specific location and size of the tampering region.

[0149] Preferably, based on the output result of the model, a suspicious forged face area is marked in the video frame, and the final verification result is output; a tampering threshold is set according to actual needs to determine the verification result. When the tampering probability of a certain face area exceeds the threshold, it is determined to be a forged face; otherwise, it is determined to be a real face; the specific steps are as follows:

[0150] S801: Marking a suspected forged face area in a video frame according to an output result of the deep learning model;

[0151] S802: If the model outputs the specific location and size of the tampered area, a marking box is accurately drawn based on the information;

[0152] S803: If the model only outputs the tampering probability, it can be determined whether to mark it according to the set threshold;

[0153] S804: Setting a tampering threshold according to actual needs to determine the verification result. When the tampering probability of a certain face area exceeds the threshold, it is determined to be a forged face; otherwise, it is determined to be a real face;

[0154] S805: Perform tampering detection on each frame of the entire video sequence, and integrate the detection results of each frame, including counting the number of forged faces detected, and recording the position and size information of the forged faces in each frame;

[0155] S806: Output the final verification result according to the integrated detection results, including whether the video passes the verification, the number and location of the detected forged faces.

[0156] Preferably, when determining the verification result, the following steps are included:

[0157] S1001: Based on the needs of actual application scenarios, including security monitoring and content review, clarify the criteria for determining forged faces;

[0158] S1002: According to the performance of the deep learning model and the statistical results of historical data, a tampering threshold is set to distinguish between real faces and forged faces;

[0159] S1003: For each frame of image, use a pre-trained face detection model to detect and locate the face area;

[0160] S1004: For each face region, extract its feature vector and input it into the trained deep learning model to calculate the tampering probability;

[0161] S1005: Compare the calculated tampering probability with a set threshold value, and if it exceeds the threshold value, mark the face area as a forged face; otherwise, mark it as a real face;

[0162] S1006: setting a judgment rule for comprehensively considering the tampering detection result of the entire video sequence, the rule comprising: if more than a certain number of forged face areas appear in the video, or signs of forged faces appear in multiple consecutive frames of images, then judging that the verification has failed;

[0163] S1007: Outputting the final verification conclusion, which includes "verification passed" or "verification failed", and accompanying related forged face information and statistical data;

[0164] S1009: The output result is presented in the form of a text report, which lists in detail the location, size, tampering probability information of the area determined to be a forged face in each frame of the image, as well as the overall statistical data and determination results.

[0165] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of those skilled in the art without departing from the spirit of the present invention.

Claims

1. A method for real person verification through video; characterized in that: The following steps are included: S1: Collect a large number of real face videos and fake face videos from public datasets and web crawler channels, and annotate the collected videos frame by frame; S2: convert the video into continuous video frame images and perform image data preprocessing; S3: Select a convolutional neural network model and perform model training. In the training, a metric learning mechanism is introduced to help the model learn an embedding space. S4: Introducing the single center loss SCL by calculating the distance from each category sample to its category center and minimizing this distance to enhance the model's ability to distinguish fake faces. Combining metric learning and single center loss, the model parameters are continuously adjusted through the back propagation algorithm to minimize the loss function value. S5: Input the video to be verified into the system, extract the video frames, use the pre-trained face detection model to detect the face area in each frame image, and perform precise positioning; S6: Extract features from the detected face area and input the extracted features into the trained deep learning model for face tampering detection. The model uses the embedding space learned by metric learning and single center loss to output the tampering probability of each face area or the specific location and size of the tampered area; S7: According to the output result of the model, mark the suspected forged face area in the video frame and output the final verification result; S8: Set a tampering threshold according to actual needs to determine the verification result. When the tampering probability of a certain face area exceeds the threshold, it is determined to be a forged face; otherwise, it is determined to be a real face; S9: For the entire video sequence, the tampering detection results of the face area in each frame are integrated, and whether the verification is successful is determined according to the set rules.

2. The method for real person verification through video according to claim 1, characterized in that: A large number of real face videos and fake face videos are collected from public data sets and web crawler channels, and the collected videos are annotated frame by frame; the specific steps are as follows: S201: Search and filter public datasets containing real faces and fake face videos, including LFW and CelebA; S202: Using a crawler program to crawl real faces and fake face videos from websites and platforms; S203: Preliminarily screening the collected videos, removing videos with low quality, repeated content or not meeting the requirements, classifying and arranging the screened videos according to real faces and forged faces, and storing them in a designated file directory; S204: performing data enhancement processing on the screened videos, including rotation, scaling, cropping, and flipping, to increase data diversity and generalization capability, and performing color adjustment and noise addition processing on the videos to simulate video quality in different scenarios; S205: Select the LabelImg annotation tool, and perform necessary configuration and adjustment on the tool, including setting the annotation category and annotation format; S206: using a labeling tool to label the screened video frame by frame, marking the face area in each frame with a rectangular frame, and labeling it as "real face" or "forged face"; for forged faces, additionally using polygons and other shapes to accurately label the specific location and size of the tampered area; S207: Storing the annotated data in a designated database, where the annotated data includes the video file name, frame number, location and size of the face region, and annotated category information.

3. A method for real person verification through video according to claim 2, characterized in that: Convert the video into continuous video frame images and perform image data preprocessing; the specific steps are as follows: S301: Obtaining video data to be processed; S302: Decode the video using a video decoder to obtain a video frame sequence; S303: extracting each frame of image from the decoded video frame sequence in time order, and saving it as an image file; S304: For an image with noise, use a filtering algorithm to remove the noise and restore the clarity and details of the image; S305: For the blurred image, a Sobel sharpening algorithm is used to perform blur processing to improve the clarity of the image; S306: For a distorted image, select a geometric distortion algorithm and a color distortion algorithm to perform distortion correction; S307: Performing face detection on the cleaned image using a pre-trained face detection model to obtain the position and size of the face area; S308: cropping the image according to the result of face detection, and only retaining necessary information of the face area and its surroundings; S309: performing normalization processing on the cropped image to adjust the image size to a uniform size.

4. The method for real person verification through video according to claim 3, characterized in that: Select a convolutional neural network model and perform model training. Introduce a metric learning mechanism during training to help the model learn an embedding space. Introduce a single center loss SCL to enhance the model's ability to distinguish fake faces by calculating the distance from each category sample to its category center and minimizing this distance. Combine metric learning and single center loss, and continuously adjust model parameters through the back propagation algorithm to minimize the loss function value. The specific steps are as follows: S401: Select the convolutional neural network model architecture; S402: Determine the number of layers, convolution kernel size, activation function parameters of the model, and initialize using a pre-trained model; S403: Embedding a metric learning mechanism during the training of the deep learning model; S404: defining a goal of an embedding space, in which samples of the same category, i.e., the distance between real faces, should be optimized to be as close as possible, while samples of different categories, i.e., the distance between forged faces and real faces, should be optimized to be as far as possible; S405: Contrast loss and triplet loss are used to construct a specific form of loss function to reflect the distance between samples; S406: Perform training iterations and calculate the value of the loss function, and update the weight of the model through a back propagation algorithm to minimize the value of the loss function; S407: setting one or more feature extraction layers in the deep learning model to extract useful feature vectors from the input face image; S408: Extract feature vectors, and use the extracted feature vectors as input for metric learning to calculate the distance between samples.

5. The method for real person verification through video according to claim 4, characterized in that: Select a convolutional neural network model and perform model training. In the training, introduce a metric learning mechanism to help the model learn an embedding space. Introduce the single center loss SCL to enhance the model's ability to distinguish fake faces by calculating the distance from each category sample to its category center and minimizing this distance. Combine metric learning and single center loss, and continuously adjust the model parameters through the back propagation algorithm to minimize the loss function value. The following steps are also included: S409: Define a single center loss function SCL. The single center loss calculates the distance between each category sample and the center of its corresponding category and minimizes this distance to enhance the model's ability to distinguish fake faces. S410: before training starts, a category center is initialized for each category, where the category center is a random vector or a representative vector obtained through preprocessing; S411: During the training process, the category center is continuously updated as the model weights are updated; S412: In each training iteration, calculate the distance between each sample and the center of its corresponding category; S413: using this distance as the input of the single center loss and calculating the value of the loss function; S414: Update the weights and category centers of the model through the back-propagation algorithm to minimize the value of the loss function.

6. A method for real person verification through video according to claim 5, characterized in that: Input the video to be verified into the system, extract the video frames, use the pre-trained face detection model to detect the face area in each frame, and accurately locate it; the specific steps are as follows: S601: The system receives the video file to be verified and loads the video through the video processing library OpenCV; S602: Use cv2.VideoCapture of the video processing library to read the video frame by frame; S603: number each frame of image; S604: storing the extracted video frames in a data list; S605: Loading the trained face detection convolutional neural network model; S606: For each frame of the extracted image, face detection is performed using the loaded face detection model, and the model outputs the coordinates and size of the face area, as well as the confidence score; S607: Screening out possible face regions according to the confidence score and a preset threshold; S608: Accurately locate the screened face area by further adjusting the coordinates and size of the face area; S609: storing information such as coordinates, size, and confidence score of the face region detected in each frame of image in a data structure.

7. A method for real person verification through video according to claim 6, characterized in that: Extract features from the detected face area and input the extracted features into the trained deep learning model for face tampering detection. The model uses the embedding space learned by metric learning and single center loss to output the tampering probability of each face area or the specific location and size of the tampered area. The specific steps are as follows: S701: In each frame of the image, based on the output of the face detection and positioning step, confirm the coordinates and range of the face area; S702: extracting texture information of the face region using image processing techniques, including Gabor filter and local binary pattern; S703: extracting the contour and shape features of the facial features through facial key point detection, which reflect the overall structure and local details of the face; S704: In the face area, statistically analyze the histograms of different color channels to extract color distribution information. The color features reflect the skin color and lighting conditions of the face. S705: fusing the extracted texture features, shape features and color features through weighted fusion to form a comprehensive face feature vector; S706: Inputting the fused facial feature vector into the trained deep learning model; S707: The deep learning model processes the input facial feature vector and outputs the tampering probability of each facial region or the specific location and size of the tampering region.

8. The method for real person verification through video according to claim 7, characterized in that: According to the output results of the model, suspicious forged face areas are marked in the video frame, and the final verification results are output; the tampering threshold is set according to actual needs to determine the verification results. When the tampering probability of a certain face area exceeds the threshold, it is determined to be a forged face; otherwise, it is determined to be a real face; the specific steps are as follows: S801: Marking a suspected forged face area in a video frame according to an output result of the deep learning model; S802: If the model outputs the specific location and size of the tampered area, a marking box is accurately drawn based on the information; S803: If the model only outputs the tampering probability, it can be determined whether to mark it according to the set threshold; S804: Set a tampering threshold according to actual needs to determine the verification result. When the tampering probability of a certain facial area exceeds the threshold, it is determined to be a forged face; otherwise, it is determined to be a real face.

9. A method for real person verification through video according to claim 8, characterized in that: According to the output results of the model, suspicious forged face areas are marked in the video frame, and the final verification results are output; the tampering threshold is set according to actual needs to determine the verification results. When the tampering probability of a certain face area exceeds the threshold, it is determined to be a forged face; otherwise, it is determined to be a real face; the following steps are also included: S805: Perform tampering detection on each frame of the entire video sequence, and integrate the detection results of each frame, including counting the number of forged faces detected, and recording the position and size information of the forged faces in each frame; S806: Output the final verification result according to the integrated detection results, including whether the video passes the verification, the number and location of the detected forged faces.

10. A method for real person verification through video according to claim 9, characterized in that: When determining the verification results, the following steps are included: S1001: Based on the needs of actual application scenarios, including security monitoring and content review, clarify the criteria for determining forged faces; S1002: According to the performance of the deep learning model and the statistical results of historical data, a tampering threshold is set to distinguish between real faces and forged faces; S1003: For each frame of the image, use a pre-trained face detection model to detect and locate the face area; S1004: For each face region, extract its feature vector. And input it into the trained deep learning model to calculate the probability of tampering; S1005: Compare the calculated tampering probability with a set threshold, and if it exceeds the threshold, mark the face area as a forged face; Otherwise, it is marked as a real face; S1006: setting a judgment rule for comprehensively considering the tampering detection result of the entire video sequence, the rule comprising: if more than a certain number of forged face areas appear in the video, or signs of forged faces appear in multiple consecutive frames of images, then judging that the verification has failed; S1007: Outputting the final verification conclusion, which includes "verification passed" or "verification failed", and accompanying related forged face information and statistical data; S1009: The output result is presented in the form of a text report, which lists in detail the location, size, tampering probability information of the area determined to be a forged face in each frame of the image, as well as the overall statistical data and determination results.

Images