Multi-face image local fake feature intelligent detection method based on generative adversarial network

Abstract

The application discloses a multi-face image local fake feature intelligent detection method based on a generative adversarial network. The method comprises the following steps: S1: inputting a face image true and false data set into a generative adversarial network for face key point extraction training; S2: determining position information positioning of the face key points to obtain a position set of face key local points; S3: performing feature segmentation on the multi-face image to obtain corresponding features of the face key local points; S4: inputting the face key local point position set and the corresponding features into the generative adversarial network to generate realistic false data to achieve the purpose of deceiving a discriminator; and S5: using a radial basis function as a kernel function to help capture the relationship between fake features and real features, and using a support vector machine classification model to perform fake detection. The method can realize face fake feature point positioning while consuming less computing amount, accurately detect local fake features of a face image, and is suitable for multi-angle face image fake detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image detection, and particularly relates to a multi-face image local fake feature intelligent detection method based on a generative adversarial network. BACKGROUND

[0002] Face image fake detection technology takes the statistical distribution difference between real face images and fake face images, tampering traces caused by image splicing, and the authenticity of the generated images as the basis for identification. Face fake can be used in film, television and other creations to enhance the aesthetic value of the creation. However, malicious application of face fake technology can cause harm. Therefore, it has great practical application value to study face image fake detection methods.

[0003] Some existing face image fake detection methods introduce the idea of unsupervised time series anomaly detection into face fake video detection and convert it into an unsupervised anomaly detection problem. Then, multi-granularity information such as offset, local and time is used to reconstruct a series of face feature points from the detected video. Finally, the error between the original sequence and the reconstructed sequence is obtained, and the product of the peak frequency and the integral of the reconstruction error is used to realize automatic detection of face fake videos.

[0004] Some designs a fake face detection method based on capsule detection. Local Xception network is used for feature extraction to reduce model parameters. In the main body, a capsule structure with attention mechanism is introduced to realize accurate positioning of the face. A synthetic multi-dimensional focal point is used as a cost function to improve the detection ability of face fake.

[0005] Some proposes a method of using local correlation representation for multi-modal contrast to realize effective detection of face fake. Through the method of shallow style representation, the correlation between levels is measured to realize the coding of local style information of the image, thereby improving the discriminability of the image in the spatial domain. By using the complementary characteristics of amplitude and phase, a double attention model based on block amplitude and phase is studied to capture the inconsistency of local correlation in the frequency domain, thereby realizing the detection of face fake.

[0006] Some proposes a new reverse residual mixed method for face recognition. Through the reverse residual mixed method, the parameters in the network are reduced, and the calculation amount is reduced. After learning the loss function, face detection and recognition in complex background and lighting conditions are realized.

[0007] However, with the continuous development of fake technology, existing detection algorithms are difficult to realize real-time processing of fake data. How to adapt to the constantly changing fake technology and constantly update and optimize the detection model with less fake data is a key problem that needs to be solved urgently. SUMMARY

[0008] To solve the above problems, the application provides a multi-face image local fake feature intelligent detection method based on a generative adversarial network, which can realize face fake feature point positioning while reducing the amount of calculation, accurately detect local fake features of a face image, and improve the universality of detection, and is suitable for multi-angle face image fake detection.

[0009] According to one aspect of the application, a multi-face image local fake feature intelligent detection method based on a generative adversarial network is provided, comprising the following steps

[0010] S1: inputting a face image true and false data set into a generative adversarial network for face key point extraction training, comprising the following steps

[0011] 1) inputting a batch of sample data in the face image true and false data set into a full connection layer;

[0012] 2) inputting the extracted data in the full connection layer into the generative adversarial network for network training to obtain key point positions;

[0013] 3) determining whether the error of the key point positions is less than a set threshold value;

[0014] 4) in step C), if the determination is no, the data continues to be iteratively operated and inputted into the full connection layer, and the step A) is jumped to for network training on the sample data;

[0015] 5) in step C), if the determination is yes, the next batch of sample data is inputted into the full connection layer, and the step A) is jumped to for network training on the next batch of sample data until the sample quantity is completely trained;

[0016] S2: determining the position information of the face key points to obtain a position set of the face key local points, comprising the following steps

[0017] A) training a multi-face feature image I through a first layer network to achieve a bias estimation S;

[0018] B) adding an initial image key point positioning estimation to obtain a first layer key point position S1;

[0019] C) calculating an affine matrix T for normalizing the input image to a standard shape to obtain a corrected face image T(I) and an image key point position T(S1), and generating a critical point heat map H t ;

[0020] D) obtaining the image key point position of each stage through multiple network iterative training;

[0021] E) extracting local features of the human face, and transforming two-dimensional coordinate points of the image into multi-dimensional feature vectors to obtain a position set of the key local points of the human face;

[0022] S3: performing feature segmentation on the multi-human face image to obtain corresponding features of the key local points of the human face, including the following steps

[0023] a) performing saliency detection on the fuzzy correlation feature quantity of the local features of the human face image to obtain a membership function of the fuzzy set of the features of the human face image;

[0024] b) using a hybrid space clustering algorithm based on multi-dimensional information to optimize image detection and cluster image edge weights to obtain fuzzy distributed features of the local features of the human face image;

[0025] c) performing image segmentation optimization through a fuzzy rough set to improve the accuracy of detection;

[0026] d) establishing a super-resolution recognition model of the human face image to obtain feature scores of a face shadow area and complete feature segmentation of the multi-human face image;

[0027] S4: inputting the position set of the key local points of the human face and the corresponding features into a generative adversarial network to generate realistic false data to achieve the purpose of deceiving a discriminator;

[0028] S5: using a radial basis function as a kernel function to help capture the relationship between the false features and the real features, and using a support vector machine classification model to perform forgery detection.

[0029] In some embodiments, in step 2), the extracted data in the full connection layer is the corrected human face image, the key points, and the feature map. This is beneficial in that it describes the specific content of the extracted data in the full connection layer.

[0030] In some embodiments, in step B), the critical point heat map H t is:

[0031]

[0032] In the formula, s i is the positioning of the i-th key local point in T(S1), and (g, f) is an image pixel point. This is beneficial in that it describes the expression formula of the critical point heat map.

[0033] In some embodiments, in step E), the position set of the key local points of the human face is:

[0034]

[0035] Wherein, n is the number of two-dimensional coordinates of sample image. Its advantage lies in that the expression formula of the position set to the key local points of human face is described.

[0036] In some embodiments, in step a), the iterative formula for significant detection is

[0037]

[0038] Wherein, U(v) is the membership function of the fuzzy set of human face features, B is the significant index, j is the pixel feature distribution function of face image, and K is the distribution coefficient. Its advantage lies in that the iterative formula for significant detection is described.

[0039] In some embodiments, in step b), the fuzzy distribution feature Y of local features of face image satisfies the formula:

[0040] Y = r (1) F (v + 1) q

[0041] Wherein:

[0042] r (1) = 0, v = 1, a (1) = 0, q ∈ Q

[0043] Wherein, r is the fuzzy related feature quantity of face image, v is the fuzzy related coefficient, a is the initial feature vector of face image, q is the pixel value of current face image, and Q is the maximum pixel value of current face image. Its advantage lies in that the calculation formula of the fuzzy distribution feature of local features of face image is described.

[0044] In some embodiments, in step c), the pheromone matrix W of image is:

[0045]

[0046] Wherein, Ω is the fuzzy information clustering result, theta is the Euler angle of information matrix, and

[0047]

[0048] Wherein, s is the pixel point of image, g(x) is the vector fusion function of single frame of image, x is the image frame number, and h(s) is the fuzzy set function representing image information. Its advantage lies in that the calculation formula of the pheromone matrix of image and the fuzzy information clustering result is described.

[0049] In some embodiments, in step d), the calculation formula of feature score M of face shadow area is:

[0050] M = Wmed(X j-1 ,…,X j ,…,X j+1)

[0051] In the formula, med() is a super function of an image, and X is the maximum frame number of the current image. The advantage is that the calculation formula of the feature value of the face shadow area is described.

[0052] In some embodiments, in step S4, the generator estimates the original image feature distribution, improves the generalization ability of the model by introducing random noise, generates realistic fake data, and achieves the purpose of cheating the discriminator. The advantage is that the generation of realistic fake data is described.

[0053] In some embodiments, in step S5, the radial basis function is used to optimize the kernel function parameters, and the cross-validation is used to find the parameter value with the highest accuracy in the actual image classification, so as to realize the forgery detection of the support vector machine classification model. The advantage is that the principle of using the radial basis function as the kernel function support vector machine classification model for forgery detection is described. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 The flow chart of the multi-face image local forgery feature intelligent detection method based on the generative adversarial network is shown in the embodiment of the application.

[0055] Figure 2 For Figure 1 The multi-face image recognition accuracy diagram for detecting the performance of the multi-face image local forgery feature intelligent detection method based on the generative adversarial network is shown in the embodiment of the application.

[0056] Figure 3 For Figure 1 The absolute error result diagram for testing the feature point positioning accuracy of the multi-face image local forgery feature intelligent detection method based on the generative adversarial network is shown in the embodiment of the application.

[0057] Figure 4 For Figure 1 The ROC curve diagram for face forgery detection of the multi-face image local forgery feature intelligent detection method based on the generative adversarial network is shown in the embodiment of the application. DETAILED DESCRIPTION

[0058] The application will be further described in detail below with reference to the accompanying drawings.

[0059] As Figure 1 shown, the flow of the multi-face image local feature forgery intelligent detection method is as follows:

[0060] Step S1: training the generative adversarial network to determine the position information of the face key points.

[0061] In this step, the face image authenticity data set is input into the generative adversarial network to train the generative adversarial network.

[0062] The training process is to input a batch of sample data in the face image authenticity data set into the full connection layer, extract the face image, key point and feature map and the like (the specific extraction method is described in the following steps), and input into the generative adversarial network for network training to obtain the key point position, and then judge whether the error of the key point position is less than a reasonable threshold set in advance.

[0063] When the judgment is no, the batch of sample data is continuously iterated and input into the full connection layer, and the above steps are repeated to train the adversarial network again until the judgment is yes.

[0064] When the judgment is yes, the batch of sample data is trained and qualified, and the next batch of sample data can be input into the full connection layer to repeat the above steps to train the adversarial network.

[0065] Step S2: Determine the position information of the face key point to obtain the position set of the face key local point.

[0066] In this step, the extracted multi-face feature image I is trained through the first layer network to achieve the bias estimation S', and the initial image key point positioning estimation is added to obtain the key point position S1 of this layer.

[0067] Then, the affine matrix T for normalizing the input image to the standard shape is calculated to obtain the corrected face image T(I) and the image key point position T(S1), and the critical point heat map H is generated from the critical point position estimation value generated in the previous stage t , to obtain the formula:

[0068]

[0069] In the formula, s i is the i-th image key local point positioning in T(S1), and (g, f) is the image pixel point.

[0070] After multiple network iteration training, the image key point position of each stage is obtained until the sample quantity is completely trained.

[0071] The method of key local point extraction is adopted to accurately extract the local features of the face, and the two-dimensional coordinate points of the image are converted into multi-dimensional feature vectors to obtain the position set of the face key local point.

[0072]

[0073] In formula (8), n is the number of two-dimensional coordinates of the sample image.

[0074] Step S3: feature segmentation is performed on the multi-face image to obtain corresponding features of the key local points of the face.

[0075] The local feature blur correlation feature quantity of the face image can be expressed as:

[0076] r(1) = 0, v = 1, a(1) = 0, q e Q

[0077] In the formula, r is the blur correlation feature quantity of the face image, v is the blur correlation coefficient, a is the initial feature vector of the face image, q is the pixel value of the current face image, and Q is the maximum pixel value of the current face image.

[0078] The membership function of the blur set of the face image feature obtained through the saliency detection can determine the degree to which each pixel or feature point belongs to a certain blur set. The iteration formula is:

[0079]

[0080] In the formula, U(v) is the membership function of the face feature blur set, B is the saliency index, j is the pixel feature distribution function of the face image, and K is the distribution coefficient.

[0081] The hybrid space clustering algorithm based on multi-dimensional information is used to optimize the image detection and cluster the image edge weight, and the blur distribution feature Y of the local feature of the face image is obtained, which satisfies the formula:

[0082] Y = r(1)F(v+1)q

[0083] The fuzzy information clustering result Ω can be expressed as:

[0084]

[0085] In the formula, s is the pixel point of the image, g(x) is the vector fusion function of the single frame of the image, x is the image frame number, and h(s) is the blur set function representing the image information.

[0086] The fuzzy rough set is further used to optimize the image segmentation, thereby improving the detection accuracy. At this time, the pheromone matrix W of the image is:

[0087]

[0088] In the formula, theta is the Euler angle of the information matrix.

[0089] On this basis, the super-resolution recognition model of the face image is established, the feature score M of the face shadow area is obtained, the multi-face image feature segmentation is completed, and more valuable information is provided for the subsequent forgery detection. The calculation formula of M is:

[0090] M = Wmed(X j-1 ,…,X j ,…,X j+1 )

[0091] where med() is the super function of the image, and X is the maximum frame number of the current image.

[0092] Step S4: input the face key local point position set and the corresponding features obtained in the above steps into the generative adversarial network, estimate the original image feature distribution by the generator of the adversarial network, introduce random noise to improve the generalization ability of the model, and generate very real fake data to achieve the purpose of cheating the discriminator.

[0093] where the discriminator needs to do is to judge whether a data is real or not, such as b is the input parameter of the actual data, if b is an image, the output represents the true probability D of the image. If it is 1, it is a 100% true image, and if it is 0, it is a false image.

[0094] In this way, a dynamic adversarial training game is formed in the adversarial network, so as to achieve a dynamic Nash equilibrium. The mathematical description of the adversarial training program is as follows:

[0095]

[0096] where V(D, G) is the loss function in the training process of the generative adversarial network, G is the generator of the generative adversarial network, D is the discriminator of the generative adversarial network, z is the added random noise, E is the mathematical expectation function, p b is the distribution probability of the real data, p z is the noise distribution probability.

[0097] Step S5: use radial basis function as kernel function to help capture the relationship between fake features and real features, and use support vector machine classification model for fake detection.

[0098] Because there is a complex nonlinear relationship between fake features and real features, using radial basis function as kernel function can help support vector machine to better capture these relationships. Among them, the radial basis function has the characteristics of local response, and has a larger response to the points with a closer distance in the input space, and a smaller response to the points with a farther distance. This characteristic makes the radial basis function pay attention to local details, which is very beneficial for detecting local fake features.

[0099] Therefore, the support vector machine classification model for fake detection can be as follows: n n

[0100] ​​

[0101] In the formula, y is a kernel function parameter. The radial basis function can optimize the parameter, and the cross-validation is used to find the parameter value with the highest accuracy in actual image classification, so as to realize the forgery detection.

[0102] The above is the flow of the multi-face image local feature forgery intelligent detection method. The detection performance experiment of the method is as follows.

[0103] In the experiment, 100 real and fake images of different faces were randomly selected as a training set, and the images were not considered under the conditions of light, expression, occlusion and angle. The number of layers of the generator network is four, the number of layers of the discriminator network is five, and the convolution kernel size is 3*3; the learning rate is set to 0.001, the iteration number is set to 500 times, and the above method is used to identify the multi-face image of different face numbers.

[0104] As shown in Figure 2 , the horizontal axis of the figure is the number of faces in the image, and the vertical axis is the accuracy of identification. As the number of faces in the image increases, the accuracy of the detection method in the application for identifying local forgery features in the image gradually decreases, but the overall accuracy remains around 0.88. This is because, as the number of faces in the image increases, the resolution of different faces in the image decreases, which reduces the clarity of the faces in the image. However, the detection method in the application can describe and process fuzzy phenomena by introducing the concept of membership degree through fuzzy rough set, which can ensure a certain recognition accuracy.

[0105] The following is a feature point positioning accuracy test.

[0106] In order to verify whether the detection method in the application can accurately locate the local forgery feature points of the face image, the absolute value of the error before and after the method is calculated. The smaller the absolute value of the error, the higher the accuracy of the method in locating the forgery feature points, and vice versa, which means that the method may have some deviation or deficiency.

[0107] As shown in Figure 3 , the horizontal axis of the figure represents the key point sequence number, and the vertical axis represents the accuracy of identification. The dotted line represents the face local feature point extraction result without using the detection method in the application, and the solid line represents the face local feature point extraction result using the detection method in the application.

[0108] As can be seen from the figure, the absolute value of the error of the face local feature point extraction without using the detection method in the application is high, while the absolute value of the error of the face local feature point extraction using the detection method in the application is low. Therefore, the detection method in the application can accurately locate the position information points of the face image, and further improve the accurate detection of the local features of the face image.

[0109] The following conducts a face forgery detection effect test.

[0110] In order to avoid the influence of image selection on the experimental results, the area of the ROC curve is selected as the experimental index of method performance, and the test results of the front and side of the face forgery image after extracting the feature points.

[0111] The ROC curve can comprehensively show the performance of the classification model at different thresholds. By calculating the area under the ROC curve (AUC), a quantitative index can be obtained to evaluate the overall performance of the model in distinguishing real faces from fake faces. Among them, the larger the AUC value, the better the performance of the model, which can more accurately identify fake face images.

[0112] As shown in Figure 4 , wherein (a) represents the ROC curve diagram of face front detection, (b) represents the ROC curve diagram of face front detection, the horizontal axis in the figure represents the false positive rate, the vertical axis represents the true positive rate, the dotted line represents the face forgery detection effect without using the detection method in the present application, and the solid line represents the face forgery detection effect using the detection method in the present application.

[0113] As can be seen from the figure, whether in the front or side state, the effect of local face image forgery detection using the detection method in the present application is better than that without using the detection method in the present application.

[0114] The detection results of the detection method in the present application and two other comparative methods are compared as follows.

[0115] Among them, the correct rate R n , the false detection rate R P , and the accuracy A C are used as indexes, the correct rate measures the ability of the method to correctly identify real and fake face images; the false detection rate reflects the proportion of real face images incorrectly identified as fake by the method; the accuracy considers all classification results, and the three indexes can give a comprehensive performance evaluation together. Through the comprehensive analysis of the three indexes, the performance of various methods can be more comprehensively understood.

[0116] The formulas of the above three indexes are as follows:

[0117]

[0118] Among them, TP is the number of fake images correctly detected, FN is the number of fake images incorrectly detected, FP is the number of real images incorrectly detected, and TN is the number of real images correctly detected.

[0119] Two comparison methods are: unsupervised face spoofing video detection based on reconstruction error (referred to as comparison method one), and spoofing face detection method based on self-attention capsule network (referred to as comparison method two).

[0120] The detection results are shown in Table 1.

[0121] Table 1 Method detection results

[0122]

[0123] As can be seen from the results in Table 1, the accuracy and detection precision of the method in the application are high, the value exceeds 99%; the false detection rate is less than 0.5%, which can more accurately and clearly identify the face spoofing image.

[0124] The above only describes some embodiments of the application. For those skilled in the art, without departing from the inventive concept, several modifications and improvements can be made, which are within the protection scope of the application.

Claims

1. A multi-face image local forgery feature intelligent detection method based on a generative adversarial network, characterized in that: The method comprises the following steps S1: inputting a face image true and false data set into a generative adversarial network for face key point extraction training, wherein The method comprises the following steps 1) inputting a batch of sample data in the face image true and false data set into a full connection layer; 2) inputting the extracted data in the full connection layer into the generative adversarial network for network training to obtain key point positions; 3) judging whether the error of the key point positions is less than a set threshold value; 4) in step C), if the judgment is no, the data continues to be iteratively operated and input into the full connection layer, and the step A) is jumped to for network training on the sample data; 5) in step C), if the judgment is yes, the next batch of sample data is input into the full connection layer, and the step A) is jumped to for network training on the next batch of sample data until all the sample data are trained; S2: determining the position information of the face key points to obtain a position set of face key local points, wherein the following steps are included A) training a plurality of face feature images I through a first layer network to obtain a bias estimation S; B) adding an initial image key point positioning estimation to obtain a first layer key point position S1; C) Compute an affine matrix T for normalizing the input image to a canonical shape, obtain a corrected face image T(I) and image keypoint positions T(S1), produce critical point heatmaps H t ; D) obtaining image key point positions at each stage through multiple network iterative training; E) extracting local features of the face, and converting two-dimensional coordinate points of the image into multi-dimensional feature vectors to obtain the position set of the face key local points; S3: performing feature segmentation on the plurality of face images to obtain corresponding features of the face key local points, wherein The method comprises the following steps a) performing saliency detection on the local feature blur correlation feature quantity of the face image to obtain a membership function of a blur set of the face image features; b) using a hybrid space clustering algorithm based on multi-dimensional information to optimize image detection and cluster image edge weight values to obtain a blur distribution feature of the local feature of the face image; c) using a fuzzy rough set to optimize image segmentation to improve the accuracy of detection; d) establishing a super-resolution recognition model of the face image to obtain a feature score of a face shadow area, and completing feature segmentation of the plurality of face images; S4: inputting the position set of the face key local points and the corresponding features into the generative adversarial network to generate realistic false data to achieve the purpose of deceiving a discriminator; S5: using a radial basis function as a kernel function to help capture the relationship between the fake features and the real features, and using a support vector machine classification model to perform fake detection.

2. The multi-face image local fake feature intelligent detection method based on a generative adversarial network according to claim 1, characterized in that: In step 2), the extracted data in the full connection layer are the corrected face image, key points and feature maps.

3. The multi-face image local fake feature intelligent detection method based on a generative adversarial network according to claim 1, characterized in that: In step B), the critical point thermogram H t is where s i is the position of the i-th keypoint in T(S1), and (g,f) is the pixel of the image.

4. The multi-face image local fake feature intelligent detection method based on a generative adversarial network according to claim 3, characterized in that: In step E), the position set of the face key local points is In step a), the iterative formula for saliency detection is 5. The method of claim 1, wherein the method is based on a generative adversarial network. In step b), the blur distribution feature Y of the local feature of the face image satisfies the formula Y=r(1)F(v+1)q 6. The multi-face image local fake feature intelligent detection method based on a generative adversarial network according to claim 5, characterized in that: wherein r(1)=0,v=1,a(1)=0,q∈Q ​ ​ In the formula, r is the blur-related feature quantity of the face image, v is the blur-related coefficient; a is the initial face image feature vector, q is the pixel value of the current face image, and Q is the maximum pixel value of the current face image.

7. The method of claim 1, wherein the method is based on a generative adversarial network. In step c), the pheromone matrix W of the image is Wherein, Omega is the fuzzy information clustering result, theta is the Euler angle of the information matrix, and In the formula, s is the pixel point of the image, g(x) is the vector fusion function of the image single frame, x is the image frame number, and h(s) is the fuzzy set function representing the image information.

8. The method of claim 1, wherein the method is based on a generative adversarial network. In step d), the calculation formula of the feature score M of the face shadow area is M = Wmed(X j-1 ,…,X j ,…,X j+1 ) In the formula, med() is the super-resolution function of the image, and X is the maximum frame number of the current image.

9. The method of claim 1, wherein the method is based on a generative adversarial network. In step S4, the generator estimates the original image feature distribution, improves the generalization ability of the model by introducing random noise, generates realistic fake data to achieve the purpose of cheating the discriminator.

10. The method of claim 1, wherein the method is based on a generative adversarial network. In step S5, the kernel function parameters are optimized using the radial basis function, and the cross-validation is used to find the parameter value with the highest accuracy in the actual image classification, so as to realize the forgery detection of the support vector machine classification model.

Images