Method and device for identifying same-egg twins based on facial fusion features and storage medium

Abstract

The invention relates to the field of biological identification, and discloses an identical twin identification method and device based on face fusion features and a storage medium, and the method comprises the steps: obtaining a face image; respectively extracting a facial aging feature FAF and a facial asymmetry feature ASF from the face image, and extracting a facial local texture feature by using a rotation-invariant and noise-insensitive binary pattern BRINT; carrying out normalization processing on the extracted features, and constructing difference fusion features through feature serial splicing; carrying out feature embedding on the difference fusion features through a multi-layer perceptron; and a Triplet loss function is used to measure the similarity degree between individuals of the same-egg twins, and face identity identification of the same-egg twins is realized. The method has the beneficial effect that high-precision identification of the same-egg twins can be realized without excessive dependence on hardware equipment.

Description

Technical Field

[0001] The present invention relates to the field of biological identification, and in particular to a method, device and storage medium for identifying identical twins based on facial fusion features. Background Art

[0002] Face recognition occupies an important position in the biometrics market due to its large user base and the advantages of the Internet. Compared with other biometric features, face recognition has the characteristics of uniqueness, easy collection, unchangeability and non-copyability. However, in recent years, the application of face recognition systems in real environments is showing some new security issues. For example, since identical twins have almost the same facial features, attackers may impersonate their twins to perform identity authentication. This kind of behavior is called a twin impersonation attack.

[0003] Regarding the problem of twin impersonation attacks, it is known that the existing mainstream commercial face recognition systems generally have low recognition accuracy for identical twins. Summary of the invention

[0004] The purpose of the present invention is to propose a method, device and storage medium for identifying identical twins based on facial fusion features, so as to solve the technical problems that traditional identical twin identification is not accurate enough and has a high dependence on hardware equipment.

[0005] The present invention provides a method for identifying identical twins based on facial fusion features, comprising the following steps:

[0006] S1: Get face image;

[0007] S2: Extract facial aging features FAF, facial asymmetry features ASF from face images, and extract facial local texture features using rotation-invariant and noise-insensitive binary patterns BRINT;

[0008] S3: Normalize the extracted features and construct differential fusion features through feature serial concatenation;

[0009] S4: embed the difference fusion features through multi-layer perceptron;

[0010] S5: Use the Triplet loss function to measure the similarity between identical twins and realize the facial identity recognition of identical twins.

[0011] A storage medium stores instructions and data for implementing a method for identifying identical twins based on facial fusion features.

[0012] A device for identifying identical twins based on facial fusion features comprises: a processor and the storage medium; the processor loads and executes instructions and data in the storage medium to implement a method for identifying identical twins based on facial fusion features.

[0013] The beneficial effects provided by the present invention are: using the differential fused facial features obtained by fusing facial aging features (FAF), facial asymmetry features (ASF), and rotationally invariant and noise-insensitive local binary patterns (BRINT), by analyzing the facial differences caused by non-genetic factors such as facial aging, behavioral expressions, and muscle movements, without the need for additional hardware equipment, more accurate and effective individual identity identification of identical twins can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic flow chart of the method of the present invention;

[0015] Figure 2 It is a schematic diagram of the self-quotient image of the present invention;

[0016] Figure 3 This is a schematic diagram of the key areas of facial aging characteristics of the present invention;

[0017] Figure 4 This is a schematic diagram of a symmetrical face of the present invention;

[0018] Figure 5 This is a network structure diagram of a multi-layer perceptron of the present invention;

[0019] Figure 6 This is a comparative analysis diagram of the identical twin identification performance test of different systems of the present invention;

[0020] Figure 7 It is a working schematic diagram of the hardware device of the present invention. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0022] Before formally describing the present invention, the scheme of the present invention is first generally described for easy understanding.

[0023] Please refer to Figure 1 , Figure 1 It is a schematic flow diagram of the method of the present invention;

[0024] The present invention provides a method for identifying identical twins based on facial fusion features, comprising the following steps:

[0025] S1: Get face image;

[0026] It should be noted that in step S1, the facial image obtained is a face image without a mask.

[0027] S2: Extract facial aging features FAF, facial asymmetry features ASF from face images, and extract facial local texture features using rotation-invariant and noise-insensitive binary patterns BRINT;

[0028] It should be noted that the facial aging feature FAF extracted from the face image in step S2 is specifically:

[0029] S211: Self-quotient image generation: Assume that the input image is I, the 3×3 Gaussian filter convolution kernel is G, and the self-quotient image SI is calculated as follows:

[0030] For all pixels I(x,y) on the input image I, select a 3×3 neighborhood window W and calculate the Gaussian spatial gradient of the pixel in all directions

[0031]

[0032] Regularize the spatial gradient value,

[0033]

[0034] Calculate the eigenvalue Z(x,y):

[0035]

[0036] Calculate window scale weights And based on this, we get the self-quotient image

[0037] Please refer to Figure 2 , Figure 2 It is an example of a self-sufficient image; it should be noted that the face images used in the present invention are all from public images in public data sets.

[0038] S212: Key area selection: Based on the calculated facial key points, the specific areas on both sides of the eyes, nose, and mouth of the face are cut out to effectively extract facial aging features; please refer to Figure 3 , Figure 3 is an example image of the selected facial region;

[0039] S213: Gabor filter calculation: Gabor features are used to specifically describe facial aging features. The kernel calculation formula of the two-dimensional Gabor wavelet function acting on the image is:

[0040]

[0041] Among them, u and v represent direction and scale respectively; z represents the coordinate point of a fixed position; Used to compensate for the attenuation of the energy spectrum; represents the Gaussian envelope function; exp(ik u,v z) represents an oscillating function, its real part is the cosine function, and its imaginary part is the sine function; represents the DC component; σ represents the size of the two-dimensional Gabor wavelet, that is, the radius of the Gaussian function; k u,v Represents the center frequency of the filter, describing the Gabor filter response in different directions and scales;

[0042] In the process of Gabor function feature extraction, the input image is convolved with the two-dimensional Gabor wavelet kernel. Suppose the input image is I(x, y), then I and the Gabor kernel G u,v The convolution between is calculated as:

[0043] O u,v (x,y)=I(x,y)*G u,v (x,y)

[0044] Among them, * represents the convolution factor; O u,v (x, y) represents the convolution image corresponding to scale u and scale v;

[0045] S214: Gradient Histogram HOG calculation: The Gabor features extracted from each part are used to calculate the HOG features, and the HOG features after vector embedding and splicing are used as a direct representation of facial aging features.

[0046] It should be noted that the extraction of the asymmetry feature ASF from the face image in step S2 specifically includes: asymmetry feature extraction based on singular value decomposition, asymmetry feature extraction based on Prucker analysis, and asymmetry feature construction.

[0047] As an embodiment, the asymmetric feature extraction based on singular value decomposition is specifically as follows:

[0048] S221: Construct feature subspace: embed each image of a data set with n images into a d-dimensional vector, construct a d×n matrix X, the rank of the matrix X is r, and decompose the matrix X into the following form through singular value decomposition:

[0049]

[0050] Among them, u i and v i are orthogonal singular vectors, Σ is a diagonal matrix containing the descending sequence of the square roots of the eigenvalues, and the characteristic matrix U is used to identify the characteristic subspace of the data set;

[0051] S222: Subspace projection: Perform left-right symmetry processing on the face and embed the processed left-right symmetric image to obtain a d-dimensional vector x i and x j , embed the above two vectors into the feature subspace U respectively:

[0052] y i =U T x i ,y j =U T x j

[0053] Preferably, as an embodiment of step S222, please refer to Figure 4 , Figure 4 is a schematic diagram of a symmetrical face, where Figure 4 (a) in the figure is the original face. Figure 4 (b) is a symmetrical face generated based on the left half of the face. Figure 4 (c) is a symmetrical face generated based on the right half of the face;

[0054] S223: Difference value calculation: by calculation The facial asymmetry feature y based on singular value decomposition is obtained.

[0055] As an embodiment, the asymmetry feature extraction based on Prucker analysis is specifically as follows:

[0056] S231: Data normalization, the calculation formula is:

[0057]

[0058]

[0059] Among them, X L represents the symmetrical face image obtained by the left half of the face, X R Represents a symmetrical face image obtained by the right half of the face;

[0060] S232: Calculate the optimal rotational affine transformation matrix:

[0061] First, singular value decomposition is used to calculate the left-right symmetrical face X L and X R The facial feature value U L , U R :

[0062]

[0063]

[0064] From U L To U R The affine transformation matrix [R t] of , where R is the rotation matrix and t is the translation vector. Since the input data is regularized, t is a zero vector; the formula for calculating the optimal value of R is:

[0065]

[0066] The optimal rotation matrix R is calculated by singular value decomposition:

[0067]

[0068] R=UV T

[0069] S233: Difference value calculation: Apply the obtained affine transformation matrix R to the left-symmetric facial feature U L , so that the left and right facial features meet the Pulitzer alignment condition; by calculating the right symmetrical facial feature U R and the rotated left-aligned face feature RU L The Euclidean distance between them gives the asymmetry characteristic d based on Prucker analysis:

[0070]

[0071] As an embodiment, the asymmetric feature construction is specifically as follows: the asymmetric features y and d obtained by the singular value-based method and the Prucker analysis-based method are spliced ​​along a unique dimension, and the feature vector ASF = [y, d] is used as a description of the left-right asymmetric features of a face.

[0072] S3: Normalize the extracted features and construct differential fusion features through feature serial concatenation;

[0073] As an example, step S3 serially merges the FAF, ASF, and BRINT facial features one by one to obtain differential fused facial features. Regularization processing is required when each facial feature is fused. The processing method is as follows:

[0074]

[0075] Where x is a one-dimensional feature vector, is the mean of the one-dimensional feature, s 2 is the variance of the one-dimensional feature.

[0076] S4: embed the difference fusion features through multi-layer perceptron;

[0077] It should be noted that step S4 uses a multi-layer perceptron to analyze the difference fusion features. The multi-layer perceptron consists of five fully connected layers and uses the ReLU activation function. It will eventually map the original input feature x to a 128-dimensional feature vector. Please refer to Figure 5 , Figure 5 The structure diagram of a multi-layer perceptron consisting of four fully connected layers.

[0078] S5: Use the Triplet loss function to measure the similarity between identical twins and realize the facial identity recognition of identical twins.

[0079] It should be noted that in step S5, the Triplet loss function is used to embed the fused feature x into the feature space R 128 The embedded feature y can be used to measure the difference between the identification features of identical twins. The Triplet loss function L t The calculation formula is:

[0080]

[0081] Among them, f(I a )、f(I p )、f(I n ) is a triple from the set of triples T (I α ,I p .I n ) is the embedding representation of the training dataset, α t is a given spacing used to further increase the distance between classes.

[0082] The above method can effectively distinguish the faces of identical twins. Figure 6 This is a comparative analysis chart of the identical twin identification performance test of different systems.

[0083] See also Figure 7 , Figure 7 4 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, and the hardware device specifically includes: a device 401 for identifying identical twins based on facial fusion features, a processor 402 and a storage medium 403.

[0084] A device 401 for identifying identical twins based on facial fusion features: The device 401 for identifying identical twins based on facial fusion features implements the method for identifying identical twins based on facial fusion features.

[0085] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the method for identifying identical twins based on facial fusion features.

[0086] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the method for identifying identical twins based on facial fusion features.

[0087] The beneficial effects of the present invention are: facial aging features (FAF), facial asymmetry features (ASF), and rotationally invariant and noise-insensitive local binary patterns (BRINT) are fused to obtain differential fused facial features, fully extracting facial differences caused by non-genetic factors such as facial aging, expressions, and muscle movements. Among them, FAF can effectively capture the regional features of facial wrinkles and smile lines caused by facial aging of twin individuals, ASF can effectively analyze the facial asymmetry of twin individuals caused by muscle movement, and BRINT can effectively extract facial texture details such as facial hair, birthmarks, scars, etc. of twin individuals. At the same time, the Triplet loss function is used for metric learning to increase the class spacing, and an embedded feature that can measure the similarity between identical twin individuals is obtained, so as to achieve effective distinction of identical twin faces.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying identical twins based on facial fusion features, characterized by: The method comprises the following steps: S1: Get face image; S2: Extract facial aging features FAF, facial asymmetry features ASF from face images, and extract facial local texture features using rotation-invariant and noise-insensitive binary patterns BRINT; S3: Normalize the extracted features and construct differential fusion features through feature serial concatenation; S4: embed the difference fusion features through multi-layer perceptron; S5: Use the Triplet loss function to measure the similarity between identical twins and realize the facial identity recognition of identical twins.

2. The method for identifying identical twins based on facial fusion features as claimed in claim 1, characterized in that: The facial aging feature FAF is extracted from the face image in step S2 as follows: S211: Self-quotient image generation: Assume that the input image is I, the 3×3 Gaussian filter convolution kernel is G, and the self-quotient image SI is calculated as follows: For all pixels I(x,y) on the input image I, select a 3×3 neighborhood window W and calculate the Gaussian spatial gradient of the pixel in all directions Regularize the spatial gradient value, Calculate the eigenvalue Z(x,y): Calculate window scale weights And based on this, we get the self-quotient image S212: Key area selection: based on the calculated key points of the face, specific areas on both sides of the eyes, nose and mouth of the face are cut out to effectively extract facial aging features; S213: Gabor filter calculation: Gabor features are used to specifically describe facial aging features. The kernel calculation formula of the two-dimensional Gabor wavelet function acting on the image is: Among them, u and v represent direction and scale respectively; z represents the coordinate point of a fixed position; Used to compensate for the attenuation of the energy spectrum; represents the Gaussian envelope function; exp(ik u,v z) represents an oscillating function, its real part is the cosine function, and its imaginary part is the sine function; represents the DC component; σ represents the size of the two-dimensional Gabor wavelet, that is, the radius of the Gaussian function; k u,v Represents the center frequency of the filter, describing the Gabor filter response in different directions and scales; In the process of Gabor function feature extraction, the input image is convolved with the two-dimensional Gabor wavelet kernel. Suppose the input image is I(x, y), then I and the Gabor kernel G u,v The convolution between is calculated as: O u,v (x,y)=I(x,y)*G u,v (x,y) Among them, * represents the convolution factor; O u,v (x, y) represents the convolution image corresponding to scale u and scale v; S214: Gradient Histogram HOG calculation: The Gabor features extracted from each part are used to calculate the HOG features, and the HOG features after vector embedding and splicing are used as a direct representation of facial aging features.

3. The method for identifying identical twins based on facial fusion features as claimed in claim 1, characterized in that: Extracting the asymmetry feature ASF from the face image in step S2 specifically includes: asymmetry feature extraction based on singular value decomposition, asymmetry feature extraction based on Prucker analysis, and asymmetry feature construction.

4. The method for identifying identical twins based on facial fusion features as claimed in claim 3, characterized in that: The specific asymmetric feature extraction based on singular value decomposition is: S221: Construct feature subspace: embed each image of a data set with n images into a d-dimensional vector, construct a d×n matrix X, the rank of the matrix X is r, and decompose the matrix X into the following form through singular value decomposition: Among them, u i and v i is an orthogonal singular vector, ∑ is a diagonal matrix containing a descending sequence of square roots of eigenvalues, and the characteristic matrix U is used to identify the characteristic subspace of the data set; S222: Subspace projection: Perform left-right symmetry processing on the face and embed the processed left-right symmetric image to obtain a d-dimensional vector x i and x j , embed the above two vectors into the feature subspace U respectively: y i =U T x i ,y j =U T x j S223: Difference value calculation: by calculation The facial asymmetry feature y based on singular value decomposition is obtained.

5. The method for identifying identical twins based on facial fusion features as claimed in claim 4, characterized in that: The asymmetry feature extraction based on Prucker analysis is as follows: S231: Data normalization, the calculation formula is: Among them, X L represents the symmetrical face image obtained by the left half of the face, X R Represents a symmetrical face image obtained by the right half of the face; S232: Calculate the optimal rotational affine transformation matrix: First, singular value decomposition is used to calculate the left-right symmetrical face X L and X R The facial feature value U L , U R : From U L To U R The affine transformation matrix [Rt] of , where R is the rotation matrix and t is the translation vector. Since the input data is regularized, t is a zero vector; the formula for calculating the optimal value of R is: The optimal rotation matrix R is calculated by singular value decomposition: R=UV T S233: Difference value calculation: Apply the obtained affine transformation matrix R to the left-symmetric facial feature U L , so that the left and right facial features meet the Pulitzer alignment condition; by calculating the right symmetrical facial feature U R and the rotated left-aligned face feature RU L The Euclidean distance between them gives the asymmetry characteristic d based on Prucker analysis:

6. The method for identifying identical twins based on facial fusion features as claimed in claim 5, characterized in that: The specific construction of asymmetric features is as follows: the asymmetric features y and d obtained by the singular value-based method and the Pluck analysis-based method are concatenated along a unique dimension, and the feature vector ASF = [y, d] is used as a description of the left-right asymmetric features of a face.

7. The method for identifying identical twins based on facial fusion features as claimed in claim 1, characterized in that: The process of extracting local facial texture features using the rotation-invariant and noise-insensitive binary pattern BRINT in step S2 is as follows: S241: image preprocessing, converting the image into a grayscale image; S242: sequentially calculating a BRINT_S descriptor with a dimension of 36, a BRINT_M descriptor with a dimension of 36, and a BRINT_C descriptor with a dimension of 2; S243: Calculate the joint histogram of the BRINT_S descriptor histogram and the BRINT_S descriptor histogram with the BRINT_C descriptor histogram respectively to obtain a BRINT_CS histogram and a BRINT_CM histogram with a dimension of 36×2; S244: The BRINT_CS histogram and the BRINT_CM histogram are concatenated to obtain a final BRINT histogram with a dimension of 36×2+36×2, which is used as a BRINT facial local texture feature for identifying facial images of identical twins.

8. The method according to claim 1, wherein: Step S4 uses a multi-layer perceptron to analyze the difference fusion features. The multi-layer perceptron consists of five fully connected layers and uses the ReLU activation function. It will eventually map the original input feature x to a 128-dimensional feature vector. In step S5, the Triplet loss function L t The calculation formula is: Among them, f(I a )、f(I p )、f(I n ) is a triple from the set of triples T (I α ,I p .I n ) is the embedding representation of the training dataset, α t is a given spacing used to further increase the distance between classes.

9. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a method for identifying identical twins based on facial fusion features as described in any one of claims 1 to 8.

10. A device for identifying identical twins based on facial fusion features, characterized in that: include: Processor and storage medium; the processor loads and executes instructions and data in the storage medium to implement a method for identifying identical twins based on facial fusion features as described in any one of claims 1 to 8.

Images