Face recognition method and device

Through the neural network model, multiple features are extracted and fused, and the deviation angle cosine value is calculated, and the face recognition model is trained, which solves the problem that image feature extraction and quality estimation cannot be correlated in the prior art, and improves the accuracy of face recognition.

CN114708625BActive Publication Date: 2024-11-15SHENZHEN XUMI YUNTU SPACE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111360868.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-17
Publication Date
2024-11-15
Estimated Expiration
2041-11-17

AI Technical Summary

Technical Problem

In the prior art, during the training process of the face recognition model, image feature extraction and feature quality estimation cannot promote each other, resulting in low recognition accuracy.

Method used

Multiple features of the input image are extracted through the neural network model, and feature fusion processing is performed, the deviation angle cosine value between the sample and the class center is calculated, and the model is trained using the loss function to improve the correlation between feature extraction and quality estimation.

Benefits of technology

The recognition accuracy of the face recognition model is improved, and the problem that image feature extraction and feature quality estimation cannot promote each other is solved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114708625B_ABST
    Figure CN114708625B_ABST
Patent Text Reader

Abstract

The present disclosure relates to the field of artificial intelligence technology, and provides a face recognition method and device. The method includes: extracting the first feature, the second feature, and the third feature of each sample in the input image through a neural network model, and performing feature fusion processing on the above three features to obtain the fusion feature of each sample, and then calculating the first mass score of each sample and the second mass score of the class center corresponding to each class; calculating the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs through the neural network model; training the neural network model through a loss function according to the first mass score of each sample, the second mass score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs; and performing face recognition using the trained neural network model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a face recognition method and device. Background Art

[0002] The common practice of existing face recognition algorithms is to treat all training samples equally. In the training of face recognition models, the training problems caused by different sample qualities and class centers of different qualities are not considered. Different sample qualities and class centers of different qualities actually involve image feature extraction and image feature quality estimation. In existing face recognition technology, the feature extraction task of face images and the quality estimation task of features are independent of each other. The feature extraction task and the quality estimation task are not related and cannot be related, which leads to the problem that image feature extraction and image feature quality estimation cannot promote each other in the training of face recognition models.

[0003] In the process of realizing the concept of the present disclosure, the inventors discovered that there are at least the following technical problems in the related art: in the training of the face recognition model, the image feature extraction and the image feature quality estimation cannot promote each other. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide a face recognition method and device to solve the problem in the prior art that, in the training of the face recognition model, image feature extraction and image feature quality estimation cannot promote each other.

[0005] According to a first aspect of an embodiment of the present disclosure, a face recognition method is provided, comprising: obtaining an input image, and extracting a first feature, a second feature, and a third feature of each sample in the input image through a neural network model, and calculating a cosine value of a deviation angle between each sample and a class center corresponding to a class to which each sample belongs through the neural network model, wherein the input image has multiple classes, each class corresponds to a class center, and each class has multiple samples; performing feature fusion processing on the first feature, the second feature, and the third feature of each sample to obtain a fused feature of each sample; interactively calculating at least one of the following features of each sample: the first feature, the third feature, and the fused feature to obtain a first quality score of each sample; calculating a second quality score of a class center corresponding to each class according to the quality scores of multiple samples in each class in the input image; training the neural network model through a loss function according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs; and performing face recognition using the trained neural network model.

[0006] According to a second aspect of an embodiment of the present disclosure, a face recognition device is provided, comprising: a first calculation module, configured to obtain an input image, and extract a first feature, a second feature, and a third feature of each sample in the input image through a neural network model, and calculate a cosine value of a deviation angle between each sample and a class center corresponding to a class to which each sample belongs through the neural network model, wherein the input image has multiple classes, each class corresponds to a class center, and each class has multiple samples; a feature fusion module, configured to perform feature fusion processing on the first feature, the second feature, and the third feature of each sample to obtain a fusion feature of each sample; a second calculation module, configured to interactively calculate at least one of the following features of each sample: the first feature, the third feature, and the fusion feature to obtain a first quality score of each sample; a third calculation module, configured to calculate a second quality score of a class center corresponding to each class according to the quality scores of multiple samples in each class in the input image; a model training module, configured to train the neural network model through a loss function according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs; and a face recognition module, configured to perform face recognition using the trained neural network model.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0009] Compared with the prior art, the embodiments of the present disclosure have the following beneficial effects: because the embodiments of the present disclosure extract the first feature, the second feature and the third feature of each sample in the input image through a neural network model, and perform feature fusion processing on the above three features to obtain the fused feature of each sample, and then obtain the first mass score of each sample and the second mass score of the class center corresponding to each class by calculation; calculate the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs through the neural network model; train the neural network model through a loss function according to the first mass score of each sample, the second mass score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs; and then use the trained neural network model for face recognition. Therefore, the above technical means can be used to solve the problem in the prior art that image feature extraction and image feature quality estimation cannot promote each other in the training of face recognition models, thereby improving the accuracy of face recognition models in recognizing faces. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of a face recognition method provided by an embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of a method for performing face recognition using a trained neural network model provided by an embodiment of the present disclosure;

[0014] Figure 4 is a structural schematic diagram of a face recognition device provided by an embodiment of the present disclosure;

[0015] Figure 5 is a structural diagram of a face recognition module provided by an embodiment of the present disclosure;

[0016] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0018] A face recognition method and device according to an embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0019] Figure 1 Schematic diagram of an application scenario of an embodiment of the present disclosure. The application scenario may include terminal devices 1, 2 and 3, a server 4 and a network 5.

[0020] Terminal devices 1, 2 and 3 can be hardware or software. When terminal devices 1, 2 and 3 are hardware, they can be various electronic devices with display screens and supporting communication with server 4, including but not limited to smart phones, tablet computers, laptop portable computers and desktop computers, etc.; when terminal devices 1, 2 and 3 are software, they can be installed in the above electronic devices. Terminal devices 1, 2 and 3 can be implemented as multiple software or software modules, or as a single software or software module, and the embodiments of the present disclosure are not limited to this. Furthermore, various applications can be installed on terminal devices 1, 2 and 3, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0021] The server 4 may be a server that provides various services, for example, a background server that receives a request sent by a terminal device that establishes a communication connection with the server, and the background server may receive and analyze the request sent by the terminal device, and generate a processing result. The server 4 may be a single server, or a server cluster composed of several servers, or a cloud computing service center, which is not limited in the embodiments of the present disclosure.

[0022] It should be noted that the server 4 can be hardware or software. When the server 4 is hardware, it can be various electronic devices that provide various services for the terminal devices 1, 2, and 3. When the server 4 is software, it can be multiple software or software modules that provide various services for the terminal devices 1, 2, and 3, or it can be a single software or software module that provides various services for the terminal devices 1, 2, and 3, and the embodiments of the present disclosure are not limited to this.

[0023] The network 5 can be a wired network connected by coaxial cable, twisted pair and optical fiber, or it can be a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), infrared, etc., which is not limited in the embodiments of the present disclosure.

[0024] The user can establish a communication connection with the server 4 via the network 5 through the terminal devices 1, 2 and 3 to receive or send information, etc. It should be noted that the specific types, quantities and combinations of the terminal devices 1, 2 and 3, the server 4 and the network 5 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present disclosure are not limited to this.

[0025] Figure 2 It is a flowchart of a face recognition method provided by an embodiment of the present disclosure. Figure 2 The face recognition method can be Figure 1 The terminal device or server executes. Figure 2 As shown, the face recognition method includes:

[0026] S201, obtaining an input image, and extracting a first feature, a second feature, and a third feature of each sample in the input image through a neural network model, and calculating a cosine value of a deviation angle between each sample and a class center corresponding to a class to which each sample belongs through the neural network model, wherein the input image has multiple classes, each class has a class center corresponding to it, and each class has multiple samples;

[0027] S202, performing feature fusion processing on the first feature, the second feature and the third feature of each sample to obtain a fusion feature of each sample;

[0028] S203, interactively calculating at least one of the following features of each sample: the first feature, the third feature, and the fusion feature, to obtain a first quality score of each sample;

[0029] S204, calculating a second mass score of a class center corresponding to each class according to the mass scores of a plurality of samples in each class in the input image;

[0030] S205, training a neural network model through a loss function according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs;

[0031] S206, using the trained neural network model to perform face recognition.

[0032] It should be noted that the input image obtained by the embodiment of the present disclosure is for training a neural network model. The input image includes multiple classes, each class corresponds to a positive class center and / or a negative class center, and each class has multiple samples. In the field of face recognition, a class can be a person, and a sample can be a picture of a person. The class center corresponding to a class can be understood as the average value of the features of all pictures of a person, the positive class center corresponding to a class can be understood as the average value of the features of the positive sample pictures of a person, and the negative class center corresponding to a class can be understood as the average value of the features of the negative sample pictures of a person. Samples whose features are greater than the preset threshold of face recognition are positive samples, and samples whose features are less than the preset threshold of face recognition are negative samples.

[0033] According to the technical solution provided by the embodiment of the present disclosure, because the embodiment of the present disclosure extracts the first feature, the second feature and the third feature of each sample in the input image through a neural network model, and performs feature fusion processing on the above three features to obtain the fusion feature of each sample, and then obtains the first mass score of each sample and the second mass score of the class center corresponding to each class by calculation; calculates the cosine value of the deviation angle of each sample and the class center corresponding to the class to which each sample belongs through the neural network model; trains the neural network model through a loss function based on the first mass score of each sample, the second mass score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle of each sample and the class center corresponding to the class to which each sample belongs; and then the trained neural network model can be used for face recognition. Therefore, the above technical means can solve the problem in the prior art that, in the training of the face recognition model, image feature extraction and image feature quality estimation cannot promote each other, thereby improving the accuracy of the face recognition model in recognizing faces.

[0034] In step S202, feature fusion processing is performed on the first feature, the second feature and the third feature of each sample to obtain the fused feature of each sample, including: inputting the input image into the neural network model, and outputting the first feature, the second feature and the third feature of each sample in the input image through the second stage, the third stage and the fourth stage of the neural network model respectively, wherein the neural network model has four stages; inputting the first feature of each sample into the first preset convolution layer, and outputting the fourth feature of each sample; inputting the second feature of each sample into the second preset convolution layer, and outputting the fifth feature of each sample; inputting the third feature of each sample into the third preset convolution layer, and outputting the sixth feature of each sample; performing feature splicing processing on the fourth feature, the fifth feature and the sixth feature of each sample to obtain the spliced ​​feature of each sample, wherein the feature fusion processing includes feature splicing processing; inputting the spliced ​​feature of each sample into the fourth preset convolution layer, and outputting the fused feature of each sample.

[0035] The four stages of the neural network model are well known to those skilled in the art and will not be described in detail here.

[0036] Specifically, the first preset convolution layer performs convolution on the first feature of the input with a convolution kernel of the first preset size, a downsampling step of the first preset step, and a number of channels of the first preset number, to obtain a fourth feature with a matrix dimension of the first preset dimension; by way of example, the first feature is convolved with a convolution kernel of 3x3, a downsampling step of 2, and a number of channels of 128 to obtain a fourth feature, and the fourth feature or the dimension of the matrix corresponding to the fourth feature is (14, 14, 128). The second preset convolution layer performs convolution on the second feature of the input with a convolution kernel of the second preset size and a number of channels of the second preset number to obtain a fifth feature with a matrix dimension of the second preset dimension; by way of example, the second feature is convolved with a convolution kernel of 3x3 and 128 channels to obtain a fifth feature, and the dimension of the fifth feature or the dimension of the matrix corresponding to the fifth feature is (14, 14, 128). The third preset convolution layer performs convolution with a third preset size and a third preset number of channels on the third feature of the input, and then performs a preset multiple of linear interpolation upsampling on the convolution result to obtain a sixth feature with a matrix dimension of the third preset dimension; for example, the third feature is convolved with a 3x3 convolution kernel and 128 channels, and then the convolution result is upsampled by 2 times linear interpolation to obtain the sixth feature, and the dimension of the sixth feature or the dimension of the matrix corresponding to the sixth feature is (14,14,128). The fourth preset convolution layer performs convolution with a fourth preset size and a fourth preset number of channels on the fourth feature of the input, and obtains a fusion feature with a matrix dimension of the fourth preset dimension; for example, the fourth feature is convolved with a 1x1 convolution kernel and 128 channels to obtain a convolution calculation result, and the convolution calculation result is used as a fusion feature. It should be noted that the fourth preset convolution layer can also perform normalization on the convolution calculation result, and the normalized result is used as a fusion feature.

[0037] According to the technical solution provided by the embodiment of the present disclosure, the first feature, the second feature and the third feature are processed by different preset convolutional layers respectively to obtain the fourth feature, the fifth feature and the sixth feature, and then the fused feature is obtained based on the fourth feature, the fifth feature and the sixth feature, thereby improving the similarity between the fused feature and the first feature, the second feature and the third feature.

[0038] In step S203, interactive calculation is performed on at least one of the following features of each sample: the first feature, the third feature and the fused feature to obtain a first quality score of each sample, including: interactive calculation is performed on the first feature and the fused feature of each sample to obtain a shallow quality score of each sample, wherein the first quality score includes a shallow quality score, including: inputting the first feature into the fifth preset convolutional layer and outputting the seventh feature; flattening the dimensions of the first matrix corresponding to the seventh feature and the second matrix corresponding to the fused feature, respectively, and multiplying the first matrix after the dimension flattening process by the transpose of the second matrix after the dimension flattening process to obtain a first product; calculating the first product using a normalized exponential function to obtain a first calculation result, and multiplying the first calculation result by the second matrix to obtain a second product; multiplying the second product by the first parameter matrix and then by the second parameter matrix to obtain a third product; calculating the third product using a sigmoid function to obtain a shallow quality score.

[0039] The feature of the sample in the embodiment of the present disclosure may be a feature map. Since the feature map may correspond to a matrix, for ease of understanding, each feature in the embodiment of the present disclosure may be directly understood as a matrix.

[0040] Specifically, the fifth preset convolution layer performs convolution on the first input feature with a convolution kernel of the fifth preset size and a fifth preset number of channels to obtain the seventh feature. For example, the fifth feature is convolved with a convolution kernel of 1x1 and 128 channels to obtain the seventh feature. Then the dimension of the seventh feature is flattened into a matrix A of (28x28, 128), and the dimension of the first matrix is ​​flattened into a matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the number of channels; A is multiplied by the transpose of B; the product result, that is, the first product, is subjected to a normalized exponential function softmax operation; the operation result is then multiplied by B; and the product result, that is, the second product, is externally connected to the parameter matrix W 11 (128, 64) and matrix W 12 (64, 1), and the third product is obtained; the third product is calculated using the sigmoid function to obtain the shallow quality score. It should be noted that after the third product is calculated using the sigmoid function, the calculation results can be averaged and the average value is used as the shallow quality score. The external parameter matrix W 11 It is the first parameter matrix, the external parameter matrix W 12 That is, the second parameter matrix, the first parameter matrix and the second parameter matrix can be preset.

[0041] The above steps can be understood as the following formula:

[0042]

[0043] is the shallow quality score, and T is the symbol of the transposition operation. 11 is the first parameter matrix, W 12 is the second parameter matrix, and mean is the mean function.

[0044] Because the shallow quality score is the output of the second stage of the neural network model, the shallow quality score can be understood as the influence weight of the sample on the class center corresponding to the sample class.

[0045] In step S203, interactive calculation is performed on at least one of the following features of each sample: the first feature, the third feature and the fused feature to obtain a first quality score of each sample, including: interactive calculation is performed on the third feature and the fused feature of each sample to obtain a middle-level quality score of each sample, wherein the first quality score includes the middle-level quality score, including: inputting the third feature into the fifth preset convolutional layer and outputting the eighth feature; respectively performing dimensional flattening processing on the third matrix corresponding to the eighth feature and the second matrix corresponding to the fused feature, and multiplying the third matrix after the dimensional flattening processing by the transpose of the second matrix after the dimensional flattening processing to obtain a fourth product; calculating the fourth product using a normalized exponential function to obtain a second calculation result, and multiplying the second calculation result by the second matrix to obtain a fifth product; multiplying the fifth product by the third parameter matrix and then by the fourth parameter matrix to obtain a sixth product; calculating the sixth product using a sigmoid function to obtain a middle-level quality score.

[0046] For example, the third feature is convolved with a kernel of 1x1 and 128 channels to obtain the eighth feature. Then the dimensions of the eighth feature are flattened into a matrix C of (7x7, 128), and the dimensions of the first matrix are flattened into a matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the number of channels; C is multiplied by the transpose of B; the product result, that is, the fourth product, is subjected to a normalized exponential function softmax operation; the result of the operation is then multiplied by B; and the product result, that is, the fifth product, is externalized to the parameter matrix W 21 (128, 64) and matrix W 22 (64, 1), and the sixth product is obtained; the sigmoid function is used to calculate the sixth product to obtain the middle layer quality score. It should be noted that after the sigmoid function is used to calculate the sixth product, the average value of the calculation results can be taken and the average value is used as the middle layer quality score. The external parameter matrix W 21 It is the third parameter matrix, the external parameter matrix W 22That is, the fourth parameter matrix, the third parameter matrix and the fourth parameter matrix can be preset.

[0047] The above steps can be understood as the following formula:

[0048]

[0049] is the middle layer mass fraction, W 21 is the third parameter matrix, W 22 is the fourth parameter matrix.

[0050] In step S203, interactive calculation is performed on at least one of the following features of each sample: the first feature, the third feature and the fused feature to obtain a first quality score of each sample, including: interactive calculation is performed on the fused feature of each sample to obtain a high-level quality score of each sample, wherein the first quality score includes a high-level quality score, including: inputting the fused feature into the fifth preset convolutional layer and outputting the ninth feature; respectively performing dimensionality flattening processing on the fourth matrix corresponding to the ninth feature and the second matrix corresponding to the fused feature, and multiplying the fourth matrix after the dimensionality flattening processing by the transpose of the second matrix after the dimensionality flattening processing to obtain a seventh product; using a normalized exponential function to calculate the seventh product to obtain a third calculation result, and using the third calculation result to multiply the second matrix to obtain an eighth product; multiplying the eighth product by the fifth parameter matrix and then by the sixth parameter matrix to obtain a ninth product; using a sigmoid function to calculate the ninth product to obtain a high-level quality score.

[0051] For example, the fusion feature is convolved with a convolution kernel of 1x1 and 128 channels to obtain the ninth feature. Then the dimension of the ninth feature is flattened into a matrix D of (14x14, 128), and the dimension of the first matrix is ​​flattened into a matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the number of channels; D is multiplied by the transpose of B; the product result, that is, the seventh product, is subjected to a normalized exponential function softmax operation; the result of the operation is then multiplied by B; and the product result, that is, the eighth product, is externalized to the parameter matrix W 31 (128, 64) and matrix W 32 (64, 1), and the ninth product is obtained; the ninth product is calculated using the sigmoid function to obtain the high-level quality score. It should be noted that after the ninth product is calculated using the sigmoid function, the average of the calculation results can be taken as the high-level quality score. The external parameter matrix W 31 It is the fifth parameter matrix, the external parameter matrix W 32 That is, the sixth parameter matrix, the fifth parameter matrix and the sixth parameter matrix may be preset.

[0052] The above steps can be understood as the following formula:

[0053]

[0054] is the high-level quality score, W 31 is the third parameter matrix, W 32 is the fourth parameter matrix.

[0055] In step S204, the second quality score of the class center corresponding to each class is calculated according to the quality scores of multiple samples in each class in the input image, including calculating the first queue score corresponding to each sample according to the middle-level quality score and the high-level quality score of each sample, wherein the first quality score includes: shallow quality score, middle-level quality score and high-level quality score; calculating the second queue score corresponding to each sample according to the shallow quality score of each sample; calculating the second quality score of the class center corresponding to each class according to the first queue score and the second queue score corresponding to the multiple samples in each class.

[0056] Specifically, the first queue score q is calculated according to the following formula: i,1 :

[0057]

[0058] The second queue score q is calculated according to the following formula i,2 :

[0059]

[0060] The second quality score γ of the class center corresponding to each class is calculated according to the following formula:

[0061] γAccording to

[0062] i is the serial number of samples in a class, R is the number of samples in a class, α is the adjustment parameter of the neural network model, which can be set to 0.2, and max(R) represents the number of samples of the class with the most samples among all classes.

[0063] In step S205, the neural network model is trained by a loss function according to the first mass score of each sample, the second mass score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs, including: training the neural network model by a cross entropy loss function according to the first mass score of each sample and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs, wherein the loss function includes a cross entropy loss function; wherein the first mass score includes: a shallow quality score, a middle quality score, and a high quality score; wherein the class center includes: a positive class center and a negative class center.

[0064] Specifically, the cross entropy loss function L1 is:

[0065]

[0066] s is the magnification factor, which can be set to 64, θ yi Represents the angle between the sample and the positive class center, θ j It represents the angle between the sample and the center of the negative class. i is the serial number of the positive sample in a class, j is the serial number of the negative sample in a class, m0 can be 0.35, m1 can be 0.25, N is the total number of all samples, and n is the total number of all negative samples.

[0067] In step S205, according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs, the neural network model is trained by a loss function, including: training the neural network model by a nearest neighbor optimization loss function, wherein the loss function includes a nearest neighbor optimization loss function, including: calculating the first sum of the inverse of the shallow quality score, the inverse of the middle quality score, and the inverse of the high quality score of each sample, Among them, the first quality score includes: shallow quality score, middle quality score and high quality score; according to the second quality score of the class center corresponding to the class to which each sample belongs and the cosine value of the deviation angle between each sample and the negative class center corresponding to the class to which each sample belongs, the neighbor optimization function is used to calculate the neighbor optimization result, wherein the class center includes: positive class center and negative class center; the second sum of the shallow quality score, middle quality score and high quality score of each sample is calculated, and the second sum corresponding to each sample is multiplied by the neighbor optimization result corresponding to each sample to obtain the tenth product; the first sum corresponding to each sample is added to the tenth product corresponding to each sample to obtain the third sum; the neural network model is trained by the third sum corresponding to each sample.

[0068] Specifically, the nearest neighbor optimization loss function L2 is:

[0069]

[0070] t can be set to 0.2, ∑ top10 cos(θ j -γt) is the nearest neighbor optimization result calculated by the nearest neighbor optimization function, z is the preset optimization number, assuming z is 10, ∑ top10 cos(θ j -γt) means that in a class, the top 10 negative samples with the highest similarity to the negative class center corresponding to this class are used to optimize the neural network model.

[0071] In an optional embodiment, when training the neural network model, it is also necessary to update or modify the cosine value cosθ of the deviation angle between each sample and the positive class center corresponding to the class to which each sample belongs. p , this step can be updated by the following formula cosine value cosθ p :

[0072] or

[0073] or

[0074] or

[0075]

[0076] m2 and m1, like m0, are parameters that can be set in advance.

[0077] Cosine value cosθ p The value of is related to whether the neural network model determines whether the sample is a positive sample or a negative sample. When a sample is input into the neural network model, the neural network model can determine whether it is a positive sample or a negative sample.

[0078] Figure 3 It is a flowchart of a method for performing face recognition using a trained neural network model provided in an embodiment of the present disclosure. Figure 3 The face recognition method can be Figure 1 The terminal device or server executes it.

[0079] like Figure 3 As shown, the face recognition method includes:

[0080] S301, upon detecting that a target user enters a preset area, obtaining a face image of the target user through an image acquisition device, and obtaining a face prototype image corresponding to the face image from a face detection database;

[0081] S302, respectively extracting the tenth feature and the eleventh feature corresponding to the face prototype image and the face image through a neural network model, and respectively calculating a first score and a second score corresponding to the tenth feature and the eleventh feature;

[0082] S303, calculating the Euclidean distance between the tenth feature and the eleventh feature;

[0083] S304, calculating the Euclidean transformation distance according to the first score, the second score and the Euclidean distance;

[0084] S305: When the Euclidean transformation distance is greater than a preset threshold, confirm that the face recognition is successful.

[0085] According to the technical solution provided by the embodiment of the present disclosure, because the embodiment of the present disclosure can respectively extract the tenth feature and the eleventh feature corresponding to the face prototype image and the face image through the neural network model, and respectively calculate the first score and the second score corresponding to the tenth feature and the eleventh feature, and then calculate the Euclidean transformation distance, when the Euclidean transformation distance is greater than the preset threshold, the face recognition is confirmed to be successful. Therefore, the above-mentioned technical means can solve the problem in the prior art that the threshold of face recognition is fixed in face recognition, and thus provide a face recognition solution based on a dynamic threshold.

[0086] Specifically, the Euclidean transformation distance is:

[0087] D'=D+Euclidean transformation (s1, s2)

[0088] f1 and f2 represent the tenth and eleventh features respectively, s1 and s2 represent the first score and the second score respectively, D is the Euclidean distance between f1 and f2, D' is the Euclidean transformation distance, β can be adjusted, generally 1.2, and f() can take the min() function.

[0089] It can be seen from the above formula that the Euclidean transformation distance changes according to the facial image of the target user acquired or detected by the image acquisition device, so the threshold of face recognition in the embodiment of the present disclosure changes dynamically.

[0090] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0091] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0092] Figure 4 FIG. 1 is a schematic diagram of a face recognition device provided by an embodiment of the present disclosure. Figure 4 As shown, the face recognition device includes:

[0093] The first calculation module 401 is configured to obtain an input image, and extract the first feature, the second feature and the third feature of each sample in the input image through a neural network model, and calculate the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs through the neural network model, wherein the input image has multiple classes, each class corresponds to a class center, and each class has multiple samples;

[0094] The feature fusion module 402 is configured to perform feature fusion processing on the first feature, the second feature and the third feature of each sample to obtain a fusion feature of each sample;

[0095] The second calculation module 403 is configured to interactively calculate at least one of the following features of each sample: the first feature, the third feature, and the fusion feature, to obtain a first quality score of each sample;

[0096] A third calculation module 404 is configured to calculate a second quality score of a class center corresponding to each class according to the quality scores of a plurality of samples in each class in the input image;

[0097] The model training module 405 is configured to train the neural network model through a loss function according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs;

[0098] The face recognition module 406 is configured to perform face recognition using the trained neural network model.

[0099] It should be noted that the input image obtained by the embodiment of the present disclosure is for training a neural network model. The input image includes multiple classes, each class corresponds to a positive class center and / or a negative class center, and each class has multiple samples. In the field of face recognition, a class can be a person, and a sample can be a picture of a person. The class center corresponding to a class can be understood as the average value of the features of all pictures of a person, the positive class center corresponding to a class can be understood as the average value of the features of the positive sample pictures of a person, and the negative class center corresponding to a class can be understood as the average value of the features of the negative sample pictures of a person. Samples whose features are greater than the preset threshold of face recognition are positive samples, and samples whose features are less than the preset threshold of face recognition are negative samples.

[0100] According to the technical solution provided by the embodiment of the present disclosure, because the embodiment of the present disclosure extracts the first feature, the second feature and the third feature of each sample in the input image through a neural network model, and performs feature fusion processing on the above three features to obtain the fusion feature of each sample, and then obtains the first mass score of each sample and the second mass score of the class center corresponding to each class by calculation; calculates the cosine value of the deviation angle of each sample and the class center corresponding to the class to which each sample belongs through the neural network model; trains the neural network model through a loss function based on the first mass score of each sample, the second mass score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle of each sample and the class center corresponding to the class to which each sample belongs; and then the trained neural network model can be used for face recognition. Therefore, the above technical means can solve the problem in the prior art that, in the training of the face recognition model, image feature extraction and image feature quality estimation cannot promote each other, thereby improving the accuracy of the face recognition model in recognizing faces.

[0101] Optionally, the feature fusion module 402 is also configured to input the input image into the neural network model, and output the first feature, the second feature and the third feature of each sample in the input image through the second stage, the third stage and the fourth stage of the neural network model respectively, wherein the neural network model has four stages; input the first feature of each sample into the first preset convolution layer, and output the fourth feature of each sample; input the second feature of each sample into the second preset convolution layer, and output the fifth feature of each sample; input the third feature of each sample into the third preset convolution layer, and output the sixth feature of each sample; perform feature splicing processing on the fourth feature, the fifth feature and the sixth feature of each sample to obtain the spliced ​​feature of each sample, wherein the feature fusion processing includes feature splicing processing; input the spliced ​​feature of each sample into the fourth preset convolution layer, and output the fused feature of each sample.

[0102] The four stages of the neural network model are well known to those skilled in the art and will not be described in detail here.

[0103] Specifically, the first preset convolution layer performs convolution on the first feature of the input with a convolution kernel of the first preset size, a downsampling step of the first preset step, and a number of channels of the first preset number, to obtain a fourth feature with a matrix dimension of the first preset dimension; by way of example, the first feature is convolved with a convolution kernel of 3x3, a downsampling step of 2, and a number of channels of 128 to obtain a fourth feature, and the fourth feature or the dimension of the matrix corresponding to the fourth feature is (14, 14, 128). The second preset convolution layer performs convolution on the second feature of the input with a convolution kernel of the second preset size and a number of channels of the second preset number to obtain a fifth feature with a matrix dimension of the second preset dimension; by way of example, the second feature is convolved with a convolution kernel of 3x3 and 128 channels to obtain a fifth feature, and the dimension of the fifth feature or the dimension of the matrix corresponding to the fifth feature is (14, 14, 128). The third preset convolution layer performs convolution with a third preset size and a third preset number of channels on the third feature of the input, and then performs a preset multiple of linear interpolation upsampling on the convolution result to obtain a sixth feature with a matrix dimension of the third preset dimension; for example, the third feature is convolved with a 3x3 convolution kernel and 128 channels, and then the convolution result is upsampled by 2 times linear interpolation to obtain the sixth feature, and the dimension of the sixth feature or the dimension of the matrix corresponding to the sixth feature is (14,14,128). The fourth preset convolution layer performs convolution with a fourth preset size and a fourth preset number of channels on the fourth feature of the input, and obtains a fusion feature with a matrix dimension of the fourth preset dimension; for example, the fourth feature is convolved with a 1x1 convolution kernel and 128 channels to obtain a convolution calculation result, and the convolution calculation result is used as a fusion feature. It should be noted that the fourth preset convolution layer can also perform normalization on the convolution calculation result, and the normalized result is used as a fusion feature.

[0104] According to the technical solution provided by the embodiment of the present disclosure, the first feature, the second feature and the third feature are processed by different preset convolutional layers respectively to obtain the fourth feature, the fifth feature and the sixth feature, and then the fused feature is obtained based on the fourth feature, the fifth feature and the sixth feature, thereby improving the similarity between the fused feature and the first feature, the second feature and the third feature.

[0105] Optionally, the second calculation module 403 is also configured to perform interactive calculations on the first feature and the fused feature of each sample to obtain a shallow quality score for each sample, wherein the first quality score includes a shallow quality score, including: inputting the first feature into the fifth preset convolutional layer and outputting the seventh feature; performing dimensionality flattening on the first matrix corresponding to the seventh feature and the second matrix corresponding to the fused feature, respectively, and multiplying the first matrix after the dimensionality flattening by the transpose of the second matrix after the dimensionality flattening to obtain a first product; calculating the first product using a normalized exponential function to obtain a first calculation result, and multiplying the first calculation result by the second matrix to obtain a second product; multiplying the second product by the first parameter matrix and then by the second parameter matrix to obtain a third product; and calculating the third product using a sigmoid function to obtain a shallow quality score.

[0106] The feature of the sample in the embodiment of the present disclosure may be a feature map. Since the feature map may correspond to a matrix, for ease of understanding, each feature in the embodiment of the present disclosure may be directly understood as a matrix.

[0107] Specifically, the fifth preset convolution layer performs convolution on the first input feature with a convolution kernel of the fifth preset size and a fifth preset number of channels to obtain the seventh feature. For example, the fifth feature is convolved with a convolution kernel of 1x1 and 128 channels to obtain the seventh feature. Then the dimension of the seventh feature is flattened into a matrix A of (28x28, 128), and the dimension of the first matrix is ​​flattened into a matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the number of channels; A is multiplied by the transpose of B; the product result, that is, the first product, is subjected to a normalized exponential function softmax operation; the operation result is then multiplied by B; and the product result, that is, the second product, is externally connected to the parameter matrix W 11 (128, 64) and matrix W 12 (64, 1), and obtain the third product; the third product is calculated using the sigmoid function to obtain the shallow quality score. It should be noted that after the third product is calculated using the sigmoid function, the calculation results can be averaged and the average value is used as the shallow quality score.

[0108] The above steps can be understood as the following formula:

[0109]

[0110] is the shallow quality score, and T is the symbol of the transposition operation. 11 is the first parameter matrix, W 12 is the second parameter matrix, and mean is the mean function.

[0111] Because the shallow quality score is the output of the second stage of the neural network model, the shallow quality score can be understood as the influence weight of the sample on the class center corresponding to the sample class.

[0112] Optionally, the second calculation module 403 is also configured to perform interactive calculations on the third feature and the fused feature of each sample to obtain a middle-level quality score for each sample, wherein the first quality score includes the middle-level quality score, including: inputting the third feature into the fifth preset convolutional layer and outputting the eighth feature; performing dimensionality flattening on the third matrix corresponding to the eighth feature and the second matrix corresponding to the fused feature, respectively, and multiplying the third matrix after the dimensionality flattening by the transpose of the second matrix after the dimensionality flattening to obtain a fourth product; calculating the fourth product using a normalized exponential function to obtain a second calculation result, and multiplying the second calculation result by the second matrix to obtain a fifth product; multiplying the fifth product by the third parameter matrix and then by the fourth parameter matrix to obtain a sixth product; and calculating the sixth product using a sigmoid function to obtain a middle-level quality score.

[0113] For example, the third feature is convolved with a kernel of 1x1 and 128 channels to obtain the eighth feature. Then the dimensions of the eighth feature are flattened into a matrix C of (7x7, 128), and the dimensions of the first matrix are flattened into a matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the number of channels; C is multiplied by the transpose of B; the product result, that is, the fourth product, is subjected to a normalized exponential function softmax operation; the result of the operation is then multiplied by B; and the product result, that is, the fifth product, is externalized to the parameter matrix W 21 (128, 64) and matrix W 22 (64, 1), and the sixth product is obtained; the sixth product is calculated using the sigmoid function to obtain the middle-layer quality score. It should be noted that after the sixth product is calculated using the sigmoid function, the calculation results can be averaged and the average value is used as the middle-layer quality score.

[0114] The above steps can be understood as the following formula:

[0115]

[0116] is the middle layer mass fraction, W 21 is the third parameter matrix, W 22 is the fourth parameter matrix.

[0117] Optionally, the second calculation module 403 is also configured to perform interactive calculations on the fused features of each sample to obtain a high-level quality score for each sample, wherein the first quality score includes a high-level quality score, including: inputting the fused feature into the fifth preset convolutional layer and outputting a ninth feature; performing dimensionality flattening on the fourth matrix corresponding to the ninth feature and the second matrix corresponding to the fused feature, respectively, and multiplying the fourth matrix after the dimensionality flattening by the transpose of the second matrix after the dimensionality flattening to obtain a seventh product; calculating the seventh product using a normalized exponential function to obtain a third calculation result, and multiplying the third calculation result by the second matrix to obtain an eighth product; multiplying the eighth product by the fifth parameter matrix and then by the sixth parameter matrix to obtain a ninth product; and calculating the ninth product using a sigmoid function to obtain a high-level quality score.

[0118] For example, the fusion feature is convolved with a convolution kernel of 1x1 and 128 channels to obtain the ninth feature. Then the dimension of the ninth feature is flattened into a matrix D of (14x14, 128), and the dimension of the first matrix is ​​flattened into a matrix B of (14x14, 128), where 14x14 is the size of the convolution kernel and 128 is the number of channels; D is multiplied by the transpose of B; the product result, that is, the seventh product, is subjected to a normalized exponential function softmax operation; the result of the operation is then multiplied by B; and the product result, that is, the eighth product, is externalized to the parameter matrix W 31 (128, 64) and matrix W 32 (64, 1), and the ninth product is obtained; the ninth product is calculated using the sigmoid function to obtain the high-level quality score. It should be noted that after the ninth product is calculated using the sigmoid function, the calculation results can be averaged and the average value is used as the high-level quality score.

[0119] The above steps can be understood as the following formula:

[0120]

[0121] is the high-level quality score, W 31 is the third parameter matrix, W 32 is the fourth parameter matrix.

[0122] Optionally, the third calculation module 404 is also configured to calculate the first queue score corresponding to each sample based on the middle-level quality score and the high-level quality score of each sample, wherein the first quality score includes: a shallow quality score, a middle-level quality score and a high-level quality score; calculate the second queue score corresponding to each sample based on the shallow quality score of each sample; calculate the second quality score of the class center corresponding to each class based on the first queue scores and the second queue scores corresponding to multiple samples in each class.

[0123] Specifically, the first queue score q is calculated according to the following formula: i,1 :

[0124]

[0125] The second queue score q is calculated according to the following formula i,2 :

[0126] q i,2 =norm(φ1)

[0127] The second quality score γ of the class center corresponding to each class is calculated according to the following formula:

[0128]

[0129] i is the serial number of samples in a class, R is the number of samples in a class, α is the adjustment parameter of the neural network model, which can be set to 0.2, and max(R) represents the number of samples of the class with the most samples among all classes.

[0130] Optionally, the model training module 405 is also configured to train the neural network model through a cross entropy loss function based on the first quality score of each sample and the cosine value of the deviation angle of each sample and the class center corresponding to the class to which each sample belongs, wherein the loss function includes a cross entropy loss function; wherein the first quality score includes: a shallow quality score, a middle quality score, and a high quality score; wherein the class center includes: a positive class center and a negative class center.

[0131] Specifically, the cross entropy loss function L1 is:

[0132]

[0133] s is the magnification factor, which can be set to 64, θ yi Represents the angle between the sample and the positive class center, θ j It represents the angle between the sample and the center of the negative class. i is the serial number of the positive sample in a class, j is the serial number of the negative sample in a class, m0 can be 0.35, m1 can be 0.25, N is the total number of all samples, and n is the total number of all negative samples.

[0134] Optionally, the model training module 405 is also configured to calculate the first sum of the inverse of the shallow quality score, the inverse of the middle quality score and the inverse of the high-level quality score of each sample, wherein the first quality score includes: the shallow quality score, the middle quality score and the high-level quality score; calculate the neighbor optimization result using the neighbor optimization function according to the second quality score of the class center corresponding to the class to which each sample belongs and the cosine value of the deviation angle between each sample and the negative class center corresponding to the class to which each sample belongs, wherein the class center includes: the positive class center and the negative class center; calculate the second sum of the shallow quality score, the middle quality score and the high-level quality score of each sample, and multiply the second sum corresponding to each sample by the neighbor optimization result corresponding to each sample to obtain the tenth product; add the first sum corresponding to each sample to the tenth product corresponding to each sample to obtain the third sum; train the neural network model through the third sum corresponding to each sample.

[0135] Specifically, the nearest neighbor optimization loss function L2 is:

[0136]

[0137] t can be set to 0.2, ∑ top10 cos(θ j -γt) is the nearest neighbor optimization result calculated by the nearest neighbor optimization function, z is the preset optimization number, assuming z is 10, ∑ top10 cos(θ j -γt) means that in a class, the top 10 negative samples with the highest similarity to the negative class center corresponding to this class are used to optimize the neural network model.

[0138] Optionally, the model training module 405 is further configured to update or modify the cosine value cosθ of the deviation angle between each sample and the positive class center corresponding to the class to which each sample belongs when training the neural network model. p , this step can be updated by the following formula cosine value cosθ p :

[0139] or

[0140] or

[0141] or

[0142]

[0143] m2 and m1, like m0, are parameters that can be set in advance.

[0144] Cosine value cosθ pThe value of is related to whether the neural network model determines whether the sample is a positive sample or a negative sample. When a sample is input into the neural network model, the neural network model can determine whether it is a positive sample or a negative sample.

[0145] Figure 5 FIG. 1 is a schematic diagram of the structure of the face recognition module provided by the embodiment of the present disclosure. Figure 5 As shown, the face recognition module includes:

[0146] The detection unit 501 is configured to acquire a face image of the target user through an image acquisition device upon detecting that the target user enters a preset area, and acquire a face prototype image corresponding to the face image from a face detection database;

[0147] The first calculation unit 502 is configured to extract the tenth feature and the eleventh feature corresponding to the face prototype image and the face image respectively through the neural network model, and calculate the first score and the second score corresponding to the tenth feature and the eleventh feature respectively;

[0148] A second calculation unit 503 is configured to calculate the Euclidean distance between the tenth feature and the eleventh feature;

[0149] A third calculation unit 504 is configured to calculate a Euclidean transformation distance according to the first score, the second score and the Euclidean distance;

[0150] The confirmation unit 505 is configured to confirm that the face recognition is successful when the Euclidean transformation distance is greater than a preset threshold.

[0151] According to the technical solution provided by the embodiment of the present disclosure, because the embodiment of the present disclosure can respectively extract the tenth feature and the eleventh feature corresponding to the face prototype image and the face image through the neural network model, and respectively calculate the first score and the second score corresponding to the tenth feature and the eleventh feature, and then calculate the Euclidean transformation distance, when the Euclidean transformation distance is greater than the preset threshold, the face recognition is confirmed to be successful. Therefore, the above-mentioned technical means can solve the problem in the prior art that the threshold of face recognition is fixed in face recognition, and thus provide a face recognition solution based on a dynamic threshold.

[0152] Specifically, the Euclidean transformation distance is:

[0153] D'=D+Euclidean transformation (s1, s2)

[0154] f1 and f2 represent the tenth and eleventh features respectively, s1 and s2 represent the first and second scores respectively, D is the Euclidean distance between f1 and f2, D' is the Euclidean transformation distance, β can be adjusted, generally taking 1.2, and f can take the min() function.

[0155] It can be seen from the above formula that the Euclidean transformation distance changes according to the facial image of the target user acquired or detected by the image acquisition device, so the threshold of face recognition in the embodiment of the present disclosure changes dynamically.

[0156] It should be understood that the order of execution of the steps in the above embodiments does not imply a sequence of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0157] Figure 6 Schematic diagram of an electronic device 6 provided in an embodiment of the present disclosure. Figure 6 As shown, the electronic device 6 of this embodiment includes: a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 601 executes the computer program 603, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0158] Exemplarily, the computer program 603 may be divided into one or more modules / units, which are stored in the memory 602 and executed by the processor 601 to complete the present disclosure. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 603 in the electronic device 6.

[0159] The electronic device 6 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 6 may include, but is not limited to, a processor 601 and a memory 602. Those skilled in the art will appreciate that Figure 6 It is only an example of the electronic device 6 and does not constitute a limitation of the electronic device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0160] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0161] The memory 602 may be an internal storage unit of the electronic device 6, for example, a hard disk or memory of the electronic device 6. The memory 602 may also be an external storage device of the electronic device 6, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 6. Further, the memory 602 may also include both an internal storage unit and an external storage device of the electronic device 6. The memory 602 is used to store computer programs and other programs and data required by the electronic device. The memory 602 may also be used to temporarily store data that has been output or is to be output.

[0162] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0163] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0164] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0165] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. Multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0166] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0167] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or software functional unit.

[0168] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0169] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.

Claims

1. A face recognition method, characterized in that: include: Acquire an input image, and extract a first feature, a second feature, and a third feature of each sample in the input image through a neural network model, and calculate a cosine value of a deviation angle between each sample and a class center corresponding to a class to which each sample belongs through the neural network model, wherein the input image has multiple classes, each class has a class center corresponding to it, and each class has multiple samples; Performing feature fusion processing on the first feature, the second feature, and the third feature of each sample to obtain a fused feature of each sample; Interactively calculating at least one of the following features of each sample: the first feature, the third feature, and the fusion feature, to obtain a first quality score of each sample; Calculating a second mass score of the class center corresponding to each class according to the first mass scores of a plurality of samples in each class in the input image; Training the neural network model by using a loss function according to the first mass score of each sample, the second mass score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs; Use the trained neural network model for face recognition; Wherein, calculating the second mass score of the class center corresponding to each class according to the first mass scores of multiple samples in each class in the input image includes: calculating the first queue score corresponding to each sample according to the middle-level mass score and the high-level mass score of each sample, wherein the first mass score includes: a shallow quality score, the middle-level mass score and the high-level mass score; calculating the second queue score corresponding to each sample according to the shallow quality score of each sample; calculating the second mass score of the class center corresponding to each class according to the first queue score and the second queue score corresponding to the multiple samples in each class.

2. The method according to claim 1, characterized in that The performing feature fusion processing on the first feature, the second feature, and the third feature of each sample to obtain a fusion feature of each sample includes: Inputting the input image into the neural network model, and outputting the first feature, the second feature, and the third feature of each sample in the input image through the second stage, the third stage, and the fourth stage of the neural network model, respectively, wherein the neural network model has four stages; Inputting the first feature of each sample into a first preset convolutional layer, and outputting the fourth feature of each sample; Inputting the second feature of each sample into a second preset convolutional layer, and outputting the fifth feature of each sample; Inputting the third feature of each sample into a third preset convolutional layer, and outputting the sixth feature of each sample; Performing feature splicing processing on the fourth feature, the fifth feature and the sixth feature of each sample to obtain a spliced ​​feature of each sample, wherein the feature fusion processing includes the feature splicing processing; The concatenated features of each sample are input into a fourth preset convolutional layer, and the fused features of each sample are output.

3. The method according to claim 1, characterized in that: The interactive calculation of at least one of the following features of each sample: the first feature, the third feature, and the fusion feature to obtain a first quality score of each sample includes: Performing the interactive calculation on the first feature and the fused feature of each sample to obtain a shallow quality score of each sample, wherein the first quality score includes the shallow quality score, including: Input the first feature into a fifth preset convolutional layer, and output a seventh feature; Performing dimensionality flattening processing on the first matrix corresponding to the seventh feature and the second matrix corresponding to the fusion feature respectively, and multiplying the first matrix after the dimensionality flattening processing by the transpose of the second matrix after the dimensionality flattening processing to obtain a first product; Using a normalized exponential function to calculate the first product to obtain a first calculation result, and multiplying the first calculation result by the second matrix to obtain a second product; Multiplying the second product by the first parameter matrix and then by the second parameter matrix to obtain a third product; The third product is calculated using a sigmoid function to obtain the shallow quality score.

4. The method according to claim 1, characterized in that: The interactive calculation of at least one of the following features of each sample: the first feature, the third feature, and the fusion feature to obtain a first quality score of each sample includes: Performing the interactive calculation on the third feature and the fusion feature of each sample to obtain a middle-level quality score of each sample, wherein the first quality score includes the middle-level quality score, including: Input the third feature into a fifth preset convolutional layer, and output an eighth feature; Performing dimensionality flattening processing on the third matrix corresponding to the eighth feature and the second matrix corresponding to the fusion feature respectively, and multiplying the third matrix after the dimensionality flattening processing by the transpose of the second matrix after the dimensionality flattening processing to obtain a fourth product; Using a normalized exponential function to calculate the fourth product to obtain a second calculation result, and multiplying the second calculation result by the second matrix to obtain a fifth product; Multiplying the fifth product by the third parameter matrix and then by the fourth parameter matrix to obtain a sixth product; The sixth product is calculated using the sigmoid function to obtain the middle layer quality score.

5. The method according to claim 1, characterized in that The interactive calculation of at least one of the following features of each sample: the first feature, the third feature, and the fusion feature to obtain a first quality score of each sample includes: Performing the interactive calculation on the fusion feature of each sample to obtain a high-level quality score of each sample, wherein the first quality score includes the high-level quality score, including: Input the fused feature into the fifth preset convolutional layer, and output a ninth feature; Performing dimensionality flattening processing on the fourth matrix corresponding to the ninth feature and the second matrix corresponding to the fusion feature respectively, and multiplying the fourth matrix after the dimensionality flattening processing by the transpose of the second matrix after the dimensionality flattening processing to obtain a seventh product; Calculating the seventh product using a normalized exponential function to obtain a third calculation result, and multiplying the third calculation result by the second matrix to obtain an eighth product; Multiply the eighth product by the fifth parameter matrix and then by the sixth parameter matrix to obtain a ninth product; The ninth product is calculated using a sigmoid function to obtain the high-level quality score.

6. The method according to claim 1, characterized in that The training of the neural network model by a loss function according to the first mass score of each sample, the second mass score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs comprises: Training the neural network model through a cross entropy loss function according to the first quality score of each sample and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs, wherein the loss function includes the cross entropy loss function; The first quality score includes: a shallow quality score, a middle quality score and a high quality score; The class centers include positive class centers and negative class centers.

7. The method according to claim 1, characterized in that The training of the neural network model by a loss function according to the first mass score of each sample, the second mass score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs comprises: The neural network model is trained by a nearest neighbor optimization loss function, wherein the loss function includes the nearest neighbor optimization loss function, including: Calculate the first sum of the reciprocal of the shallow layer quality score, the reciprocal of the middle layer quality score and the reciprocal of the high layer quality score of each sample, wherein the first quality score includes: the shallow layer quality score, the middle layer quality score and the high layer quality score; Calculate a neighbor optimization result using a neighbor optimization function according to the second mass score of the class center corresponding to the class to which each sample belongs and the cosine value of the deviation angle between each sample and the negative class center corresponding to the class to which each sample belongs, wherein the class center includes: a positive class center and a negative class center; Calculating a second sum of the shallow quality score, the middle quality score and the high quality score of each sample, and multiplying the second sum corresponding to each sample by the nearest neighbor optimization result corresponding to each sample to obtain a tenth product; Add the first sum corresponding to each sample to the tenth product corresponding to each sample to obtain a third sum; The neural network model is trained by corresponding to the third sum of each sample.

8. The method according to claim 1, characterized in that The method of using the trained neural network model to perform face recognition includes: When a target user is detected to enter a preset area, a facial image of the target user is acquired through an image acquisition device, and a facial prototype image corresponding to the facial image is acquired from a face detection database; Extracting the tenth feature and the eleventh feature corresponding to the face prototype image and the face image respectively through the neural network model, and calculating the first score and the second score corresponding to the tenth feature and the eleventh feature respectively; Calculating the Euclidean distance between the tenth feature and the eleventh feature; Calculating a Euclidean transformed distance according to the first score, the second score and the Euclidean distance; When the Euclidean transformation distance is greater than a preset threshold, it is confirmed that the face recognition is successful.

9. A face recognition device, characterized in that: include: A first calculation module is configured to obtain an input image, and extract a first feature, a second feature, and a third feature of each sample in the input image through a neural network model, and calculate a cosine value of a deviation angle between each sample and a class center corresponding to a class to which each sample belongs through the neural network model, wherein the input image has multiple classes, each class corresponds to a class center, and each class has multiple samples; a feature fusion module, configured to perform feature fusion processing on the first feature, the second feature and the third feature of each sample to obtain a fused feature of each sample; A second calculation module is configured to interactively calculate at least one of the following features of each sample: the first feature, the third feature and the fusion feature, so as to obtain a first quality score of each sample; A third calculation module is configured to calculate a second mass score of the class center corresponding to each class according to the first mass scores of the plurality of samples in each class in the input image; a model training module, configured to train the neural network model through a loss function according to the first quality score of each sample, the second quality score of the class center corresponding to the class to which each sample belongs, and the cosine value of the deviation angle between each sample and the class center corresponding to the class to which each sample belongs; A face recognition module is configured to perform face recognition using a trained neural network model; The third calculation module is also configured to calculate the first queue score corresponding to each sample according to the middle quality score and the high quality score of each sample, wherein the first quality score includes: a shallow quality score, the middle quality score and the high quality score; calculate the second queue score corresponding to each sample according to the shallow quality score of each sample; calculate the second quality score of the class center corresponding to each class according to the first queue scores and the second queue scores corresponding to multiple samples in each class.