Face recognition method based on twin neural network

Abstract

Embodiments of the present application provide a face recognition method based on a twin neural network. The twin neural network comprises a first neural network and a second neural network which are completely the same. The face recognition method comprises: inputting a first image into the first neural network, and inputting a second image into the second neural network, wherein the first image and the second image are separately obtained by photographing a same object; acquiring a first output result of the first neural network with respect to the first image and a second output result of the second neural network with respect to the second image; and performing computational comparison on the first output result and the second output result, and determining a target object of the first image and the second image on the basis of the comparison result.

Description

Face recognition method based on twin neural network Technical Field

[0001] The present application relates to the field of power grid management technology, and in particular to a face recognition method based on a twin neural network. Background Art

[0002] With the rapid development of the power industry and the popularization of intelligent applications, power grid security and management have become important issues. Facial recognition technology, as a non-contact biometric recognition technology, is widely used in security monitoring and management of power grid systems.

[0003] However, due to the problems of illumination changes and occlusion in the power grid environment, the accuracy of face recognition methods is low and cannot meet the needs.

[0004] Summary of the Invention

[0005] The embodiment of the present application provides a face recognition method based on a twin neural network, which is used to solve the problem in the existing technology that the face recognition method has low accuracy and cannot meet the needs due to problems such as lighting changes and occlusion in the power grid environment.

[0006] The embodiment of the present application provides a face recognition method based on a twin neural network, wherein the twin neural network includes a first neural network and a second neural network that are completely identical. The face recognition method includes:

[0007] Inputting a first image into a first neural network and inputting a second image into a second neural network, wherein the first image and the second image are respectively obtained by photographing the same object;

[0008] Obtaining a first output result of the first neural network with respect to the first image and a second output result of the second neural network with respect to the second image;

[0009] The first output result and the second output result are compared and calculated, and the target objects of the first image and the second image are determined according to the comparison result.

[0010] In a feasible implementation, the comparison result is a similarity score, and the method for obtaining the similarity score includes:

[0011] The first neural network extracts a first feature of the first image and processes the first feature;

[0012] The second neural network extracts a second feature of the second image and processes the feature;

[0013] Calculate the difference between the first feature and the second feature, obtain a similarity score, and output the result.

[0014] In a feasible implementation, the first neural network and the second neural network both include a convolutional layer and a pooling layer;

[0015] The convolution layer is used to extract neighborhood spatial features from the first image or the second image, and the pooling layer is used to perform dimensionality reduction on the neighborhood spatial features.

[0016] In one feasible implementation, the contrast loss function is used to optimize the first neural network and the second neural network:

[0017] The contrastive loss function estimates the similarity of images based on the distance loss, where two similar points in the image have a smaller Euclidean distance and two different points have a larger Euclidean distance.

[0018] In one possible implementation, the contrastive loss function includes:

[0019] Here, x is either 1 or 0. If the model predicts that the input is similar, then x is 0, otherwise Y is 1;

[0020] max() is a function that represents the larger value between 0 and m-Dw, where Dw is defined as the Euclidean distance between the outputs of the twin neural networks;

[0021] D w According to the Euclidean distance calculation, it can be specifically calculated according to formula (2).

[0022] G w The output value represents the similarity, and y1 and y2 represent the input data pair.

[0023] In one feasible implementation, a triple loss function is used to optimize the first neural network and the second neural network, including:

[0024] The triple loss is trained by inputting training images into the first neural network and the second neural network respectively;

[0025] The training images include positive images, target images and negative images, wherein the positive images have a high similarity with the target images, and the negative images have a low similarity with the target images.

[0026] In one possible implementation, the triple loss function includes: λ(B,P,M)=max(||f((MA))-f(P)|| 2 -||f(B)-f(P)|| 2 +Γ,0)

[0027] Among them, λ(B,P,M) represents the calculation result of the loss function, f(B), f(P), and f(M) are the positive and negative feature embeddings of the target point, respectively, and Γ is the stretching factor.

[0028] The embodiment of the present application provides a face recognition method based on a twin neural network, wherein the twin neural network includes a completely identical first neural network and a second neural network. The face recognition method includes: inputting a first image into the first neural network and inputting a second image into the second neural network, wherein the first image and the second image are respectively obtained by photographing the same object; obtaining a first output result of the first neural network regarding the first image and a second output result of the second neural network regarding the second image; performing calculations and comparisons on the first output result and the second output result, and determining the target objects of the first image and the second image based on the comparison results. The two images of the same object are respectively calculated and compared by the first neural network and the second neural network to calculate the similarity and difference between the two images, and the target is identified based on the judgment results, thereby improving the accuracy of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings described herein are used to provide further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present application and do not constitute improper limitations on the present invention.

[0030] In the attached figure:

[0031] FIG1 is a schematic diagram of a face recognition method based on a twin neural network provided in one embodiment of the present application;

[0032] FIG2 is a schematic diagram of the structure of a twin neural network provided in one embodiment of the present application;

[0033] Figure 3 is a schematic diagram of pruning within the convolution kernel of a twin neural network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the technical solutions in this application, the following will provide a clear and complete description of the technical solutions in the embodiments of this application in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0035] With the rapid development of the power industry and the popularization of intelligent applications, power grid security and management have become important issues. Facial recognition technology, as a non-contact biometric recognition technology, is widely used in security monitoring and management of power grid systems.

[0036] However, due to the problems of illumination changes and occlusion in the power grid environment, the accuracy of face recognition methods is low and cannot meet the needs.

[0037] In order to solve the above problems, an embodiment of the present application provides a face recognition method based on a twin neural network. The solution provided by the embodiment of the present application will be described in detail below in conjunction with the drawings in the specification.

[0038] Figure 1 is a schematic diagram of a face recognition method based on a twin neural network provided in an embodiment of the present application.

[0039] 1 , an embodiment of the present application provides a face recognition method based on a twin neural network. The twin neural network includes a first neural network and a second neural network. The face recognition method includes:

[0040] S100: A first image is input into a first neural network.

[0041] In some examples, a first image may be captured by a recognition camera or a camera, and then the first image may be transmitted to the first neural network. The first image may be a photo taken by the recognition camera of the object to be captured.

[0042] S200: The second image is input into the second neural network.

[0043] Corresponding to S100, the second neural network receives the second image. For example, the second image can be captured by a recognition camera or a camera, and then the second image is transmitted to the second neural network. The second image can be a photo taken by the recognition camera of the object to be captured.

[0044] In addition, it should be noted that the first image and the second image are obtained by shooting the same object with different cameras at different angles.

[0045] S300: The first neural network and the second neural network perform calculations and comparisons on the first image and the second image respectively.

[0046] The first neural network performs calculations and comparisons on the first image based on the data in the database and the calculation model, and the second neural network performs calculations and comparisons on the second image based on the data in the database and the calculation model, thereby determining the target object of the first image and the second image and identifying the identity of the target image.

[0047] Exemplarily, the comparison result is a similarity score, and the method for obtaining the similarity score includes: extracting a first feature of the first image by a first neural network and processing it; extracting a second feature of the second image by a second neural network and processing it; calculating the difference between the first feature and the second feature, obtaining a similarity score, and outputting the result.

[0048] For example, a method for face recognition in power grids based on a twin neural network compares two images captured by two cameras—that is, two photos taken from different angles—and measures their similarity. This twin neural network consists of two identical neural networks, each receiving two copies of the input image and trained using shared weights. This network can learn the similarities and differences between images, resulting in higher accuracy in face recognition tasks. This is done to improve the accuracy and reliability of face recognition in power grids.

[0049] In other words, an embodiment of the present application provides a face recognition method based on a twin neural network, wherein the twin neural network includes a completely identical first neural network and a second neural network. The face recognition method includes: inputting a first image into the first neural network and inputting a second image into the second neural network, wherein the first image and the second image are respectively obtained by photographing the same object; obtaining a first output result of the first neural network regarding the first image and a second output result of the second neural network regarding the second image; performing calculations and comparisons on the first output result and the second output result, and determining the target objects of the first image and the second image based on the comparison results. The two images of the same object are respectively calculated and compared by the first neural network and the second neural network to calculate the similarity and difference between the two images, and the target is identified based on the judgment results, thereby improving the accuracy of face recognition.

[0050] In some examples, the first neural network and the second neural network both include convolutional layers and pooling layers; the convolutional layers are used to extract neighborhood spatial features from the first image and the second image, respectively, and the pooling layers are used to perform dimensionality reduction processing on the neighborhood spatial features.

[0051] The image inputs of the first neural network and the second neural network can be two photos from the same category or other categories. The twin neural networks will calculate the similarity scores between images of different categories. In the face recognition task, each face is considered to be a different category.

[0052] The output is 1 if the two photos are from the same class (i.e., identical); otherwise, it is zero (i.e., not identical).

[0053] In addition, each neural network max pooling defines a feature map block maximum to produce a downsampled (pooled) feature map, introducing a small degree of translation invariance, that is, most of the pooled output values ​​will not be significantly affected by small changes in the image.

[0054] Closely connected to the previous layer is the fully connected layer, where all neurons are interconnected with those in the previous layer. This is the most commonly used layer in neural networks, performing both feature propagation and feature mapping. This twin neural network only needs to distinguish between three categories: positive samples, negative samples, and target samples. Unlike classic datasets that require labeling for each image, creating training data for the twin neural network architecture is much easier.

[0055] To create a highly accurate model, we first need to collect and create a dataset. This dataset consists of images of all employees collected from different angles within the company. We then use the array() method to convert the data into a NumPy array. We convert the image values ​​from integers to floating-point arrays and divide them by 255, so that each value ranges between 0 and 1. We then import the data into the dataset and use the transform method to reduce the image size to 100 pixels in height and width.

[0056] Since we only use 0 and 1, we effectively force gradient descent, resulting in a better performing model. TensorFlow’s dataset generator enables us to use our anchor images, positive samples, and negative samples.

[0057] In some examples, in order to improve the accuracy of the twin neural network system for face recognition, a contrast loss function is used to optimize the first neural network and the second neural network.

[0058] Among them, the contrastive loss function estimates the similarity of images based on the loss of distance, where two similar points in the image have a smaller Euclidean distance and two different points have a larger Euclidean distance. Specifically, the contrastive loss function is used to learn embedding, which extracts the output vector information of the network and estimates the similarity, where two similar points have a smaller Euclidean distance and two different points have a larger Euclidean distance. If the positive samples are encoded in the same form and the negative samples are encoded in a different form, the loss is reduced. The contrastive loss function is shown in Formula (1).

[0059] Where x is either 1 or 0. If the model predicts that the inputs are similar, then x is 0, otherwise Y is 1. max() is the function that represents the larger value between 0 and m-Dw, where Dw is defined as the Euclidean distance between the outputs of the twin neural networks;

[0060] D is calculated based on the Euclidean distance, which can be calculated specifically according to formula (2).

[0061] G w The output value represents the similarity, and y1 and y2 represent the input data pair.

[0062] In other examples,

[0063] In one feasible implementation, a triple loss function is used to optimize the first neural network and the second neural network, including:

[0064] The triple loss is trained by inputting training images into the first neural network and the second neural network respectively;

[0065] The training images include positive images, target images and negative images, wherein the positive images have a high similarity with the target images, and the negative images have a low similarity with the target images.

[0066] In one possible implementation, the triple loss function includes: λ(B,P,M)=max(||f((MA))-f(P)|| 2 -||f(B)-f(P)|| 2 +Γ,0)

[0067] Among them, λ(B,P,M) represents the calculation result of the loss function, f(B), f(P), and f(M) are the positive and negative feature embeddings of the target point, respectively, and Γ is the stretching factor.

[0068] Exemplarily, the steps for training the first neural network and the second neural network are as follows:

[0069] First, the twin neural network is initialized using a loss function and an Adam function optimizer. Then, the first and second images are fed into the first and second neural networks, respectively. The loss is calculated using the first and second images. Finally, the model's gradient is determined, and an optimizer is used to update the weights.

[0070] In addition, the binary cross entropy loss is used to learn the embedding so that two similar points have a small Euclidean distance, while two different points have a large Euclidean distance. Finally, the optimizer adjusts the weight parameters to minimize the loss function;

[0071] For example, Adam is a well-known gradient descent optimization method. It is computationally efficient and uses very little memory.

[0072] Finally, the recognition and classification performance parameters can be evaluated using precision and recall values. Since the accuracy of each instruction varies, this approach also helps detect overfitting, which occurs when, for example, a function is overfitted.

[0073] Figure 2 is a schematic diagram of the structure of the twin neural network provided in one embodiment of the present application.

[0074] As shown in Figure 2, for example, two face images are used as input to the face recognition network, and the two related subnetworks of the twin neural network (the first neural network and the second neural network) perform forward propagation and backpropagation operations. Since the two neural networks have the same design, parameters and weights, these subnetworks will generate codes and calculate the distance between the two input images. It can be understood that the twin neural network is similar to the input of other convolutional neural networks, taking images as input and converting hidden features extracted by the neural network into numerical values. However, the output of the twin neural network is different from that of other convolutional neural networks. The classic CNN needs to fine-tune the parameters during training to correctly classify each picture.

[0075] In addition, when the Siamese neural network calculates the distance and similarity between the features of two input images, it does not directly assign the images to any output class, but instead uses a similarity function to compare the inputs.

[0076] In some examples, the twin neural network also employs convolution kernel pruning. This kernel pruning typically involves regularly pruning some weights within the kernel using a specific method. Specifically, each kernel that requires pruning has its weights pruned according to a specific operation. Structured pruning offers high operability at both the hardware and software levels.

[0077] It is understandable that convolution kernel pruning not only reduces computational complexity and memory usage, but also improves the network's generalization capabilities. Furthermore, it can accelerate the operation of neural networks on embedded systems and mobile devices, further improving the network's practicality and performance.

[0078] Figure 3 is a schematic diagram of pruning within the convolution kernel of a twin neural network provided in an embodiment of the present application.

[0079] As shown in Figure 3, the bottom brackets are the matrix of a layer of input feature map and a convolution kernel matrix. According to the traditional convolution operation method, the input matrix and the convolution kernel matrix will be split for operation, and finally the convolution operation results of each part will be merged. As shown in the brackets above, after the convolution kernel is split, a certain step size and bias can be set to perform structured pruning on the convolution kernel. In the figure, the step size is set to 2 and the bias is set to 0 or 1. The red numbers in the convolution kernel are the retained weights after pruning, and the black numbers are the pruned weights. After pruning, the entire convolution kernel matrix is ​​reduced from the original 12×2 to 6×2.

[0080] The retained convolution kernel weights are the red weights in the figure, and the weights actually used in the input feature map will also be reduced from the original 4×12 to 4×6, that is, only the weights corresponding to the retained weights after the convolution kernel pruning are used. The blue digital matrix in the figure is the feature map matrix output after pruning, and the corresponding black digital matrix is ​​the feature map matrix output before pruning.

[0081] Table 1. Recognition results of face recognition method based on twin neural network and advanced methods.

[0082] To evaluate the accuracy of the face recognition method based on the twin neural network, the present embodiment compared the recognition results with state-of-the-art methods, as shown in Table 1. After training with dense and sigmoid as activation functions, the results were excellent compared to other classifiers and those trained with other activation functions, achieving an accuracy of 96%. While various methods and results in current face recognition technology generally perform well, with accuracy rates between 90% and 96%, the twin neural network produced the best recognition results on the "Tagged Faces in the Wild" dataset.

[0083] Table 1. Recognition results of face recognition method based on twin neural network and advanced methods

[0084] Table 2. Statistics of experimental results of applying existing methods to the data sets of the embodiments of this application.

[0085] This application presents statistical results from tests using existing methods on the datasets of the present application's embodiments. The specific results are shown in Table 2. The results show that the ResNet method achieved a recognition rate of 81.21%. The TripleNet method achieved a recognition rate of 84.32%, and the FaceNet method achieved a recognition rate of 85.52%. The SN-LF method achieved a recognition rate of 90.31%. The present application's face recognition method based on the twin neural network achieved a recognition rate of 96%, far exceeding the recognition rates of other methods.

[0086] Table 2. Statistics of experimental results of existing methods applied to the data set of the present embodiment

[0087] It is easy to understand that those skilled in the art can combine, split, reorganize, etc. the embodiments of the present application based on the several embodiments provided in the present application to obtain other embodiments, and these embodiments do not exceed the scope of protection of the present application.

[0088] The above specific implementation methods further explain in detail the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above are only specific implementation methods of the embodiments of the present application and are not intended to limit the scope of protection of the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the scope of protection of the embodiments of the present application.

Claims

1. A face recognition method based on a twin neural network, characterized in that: The twin neural network includes a first neural network and a second neural network that are completely identical, and the face recognition method includes: Inputting a first image into the first neural network, and inputting a second image into the second neural network, wherein the first image and the second image are respectively obtained by photographing the same object; Obtaining a first output result of the first neural network with respect to the first image and a second output result of the second neural network with respect to the second image; The first output result and the second output result are calculated and compared, and the target objects of the first image and the second image are determined according to the comparison results.

2. The face recognition method based on twin neural network according to claim 1 is characterized in that: The comparison result is a similarity score, and a method for obtaining the similarity score includes: The first neural network extracts and processes a first feature of the first image; The second neural network extracts and processes a second feature of the second image; The difference between the first feature and the second feature is calculated to obtain a similarity score, and the result is output.

3. The face recognition method based on twin neural network according to claim 1 is characterized in that: The first neural network and the second neural network both include a convolutional layer and a pooling layer; The convolution layer is used to extract neighborhood space features from the first image or the second image, and the pooling layer is used to perform dimensionality reduction processing on the neighborhood space features.

4. The face recognition method based on twin neural network according to claim 1 is characterized in that: Before the first image or the second image is input, the first neural network and the second neural network are optimized using a contrast loss function: The contrast loss function estimates the similarity of images based on the loss of distance, wherein two similar points in the image have a smaller Euclidean distance and two different points have a larger Euclidean distance.

5. The face recognition method based on twin neural network according to claim 1, characterized in that: The contrast loss function includes: Among them, the value of x is 1 or 0. If the model predicts that the input is similar, then the value of x is 0, otherwise Y is 1; max() is a function that represents the larger value between 0 and m-Dw, where Dw is defined as the Euclidean distance between the outputs of the twin neural network; D w According to the Euclidean distance calculation, it can be specifically calculated according to formula (2). G w The output value represents the similarity, and y1 and y2 represent the input data pair.

6. The face recognition method based on twin neural network according to claim 1, characterized in that: Before the first image or the second image is input, the first neural network and the second neural network are optimized using a triple loss function, including: Training the triple loss by inputting training images to the first neural network and the second neural network respectively; The training images include a positive image, a target image and a negative image, wherein the positive image has a high similarity to the target image, and the negative image has a low similarity to the target image.

7. The face recognition method based on twin neural network according to claim 5 is characterized in that: The triple loss function includes: λ(B,P,M)=max(||f((MA))-f(P)|| 2 -||f(B)-f(P)|| 2 +Γ,0) Among them, λ(B,P,M) represents the calculation result of the loss function, f(B), f(P), and f(M) are the positive and negative feature embeddings of the target point, respectively, and Γ is the stretching factor.

Images