A face recognition method, system and device
By simplifying the initial image features and category features of the face recognition model, selecting reference category features, and adjusting the model parameters, the problem of high resource consumption during training is solved, achieving efficient face recognition training and improving recognition accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- IFLYTEK CO LTD
- Filing Date
- 2023-03-22
- Publication Date
- 2026-06-05
AI Technical Summary
Existing facial recognition methods consume a lot of computing and storage resources during training, resulting in low training efficiency.
By simplifying the initial image features and category features, simplified image features and simplified category features are obtained. Reference category features are selected based on the similarity between the simplified features. The model parameters are then adjusted based on the category label and the initial image features until the convergence condition is met.
While saving training costs, it improves the stability and accuracy of the face recognition model.
Smart Images
Figure CN116486450B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of facial recognition technology, and in particular to a facial recognition method, system and device. Background Technology
[0002] With the increasing maturity of artificial intelligence-related technologies, facial recognition has become an important human-computer interaction method and is widely used in scenarios such as device login and identity authentication. Facial recognition technology is based on a trained facial recognition model.
[0003] Current facial recognition methods primarily involve first building a facial recognition model and then training it to learn the differences and similarities between extracted facial features and features from all categories within the model. This method consumes significant computational and storage resources during training, resulting in low training efficiency. Summary of the Invention
[0004] The main technical problem addressed by this application is to provide a face recognition method, system, and apparatus that can improve the accuracy of face recognition and save computational costs.
[0005] To solve the above-mentioned technical problems, this application adopts a technical solution as follows: providing a face recognition method, comprising: performing face recognition on a face image to be recognized based on a trained face recognition model, and obtaining face image features corresponding to the face image to be recognized; the training process of the face recognition model includes: constructing a training sample set containing multiple sample images, inputting the sample images into the face recognition model, and obtaining initial image features corresponding to each sample image; wherein, each sample image corresponds to a category label; obtaining multiple initial category features, processing the initial values of the initial category features into simplified values to obtain simplified category features, and processing the initial values of the initial image features into simplified values. Simplified values are obtained to obtain simplified image features; wherein the simplified values are either a first feature value or a second feature value; the initial image features and the initial category features are binarized to obtain simplified image features corresponding to the initial image features and simplified category features corresponding to the initial category features; a first similarity is obtained between the simplified image features and each of the simplified category features, and multiple reference category features corresponding to the sample image are obtained from all the initial category features based on the first similarity; the parameters in the face recognition model are adjusted based on the category labels corresponding to all the sample images, the initial image features, and the reference category features until the convergence condition is met.
[0006] To address the aforementioned technical problems, another technical solution adopted in this application is: providing a face recognition system, comprising: a face recognition module, used to perform face recognition on a face image to be recognized based on a trained face recognition model, and obtain face image features corresponding to the face image to be recognized; a training module, used to train the face recognition model, the training process including: constructing a training sample set containing multiple sample images, inputting the sample images into the face recognition model, and obtaining initial image features corresponding to each sample image; wherein, each sample image corresponds to a category label; obtaining multiple initial category features, processing the initial values of the initial category features into simplified values to obtain simplified category features, processing the initial values of the initial image features into simplified values to obtain simplified image features; wherein, the simplified values are first feature values or second feature values; obtaining a first similarity between the simplified image features and each of the simplified category features, obtaining multiple reference category features corresponding to the sample image from all the initial category features based on the first similarity; adjusting the parameters in the face recognition model based on the category labels corresponding to all the sample images, the initial image features, and the reference category features, until the convergence condition is met.
[0007] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a face recognition device, the system including a memory and a processor coupled to each other, the processor storing program instructions, the processor being used to execute the program instructions to implement the method mentioned in the above technical solution.
[0008] The beneficial effects of this application are as follows: Unlike existing technologies, this application proposes a face recognition model training method, comprising: after obtaining initial image features corresponding to multiple sample images and multiple initial category features for classifying the initial image features, simplifying the initial image features and initial category features to obtain corresponding simplified image features and simplified category features. Based on the first similarity between the simplified image features and each simplified category feature, obtaining multiple reference category features corresponding to the sample images from all initial category features, and calculating the model loss value using at least the category label corresponding to the sample images, the initial image features, and the reference category features to adjust the model parameters. By performing simplification first and then selecting some reference category features for training, the calculation of the model loss value using the less pervasive initial category features can be avoided, saving model training costs while improving the stability of the trained model. Furthermore, using the trained face recognition model to perform face recognition on the face image to be recognized can improve the face recognition accuracy. Attached Figure Description
[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:
[0010] Figure 1 This is a flowchart illustrating one implementation method of the applicant's face recognition model training method;
[0011] Figure 2 This is a flowchart illustrating step S103 corresponding to one implementation method;
[0012] Figure 3 This is a flowchart illustrating step S104 corresponding to another implementation method;
[0013] Figure 4 This is a flowchart illustrating step S1041 corresponding to one embodiment.
[0014] Figure 5 This is a structural schematic diagram of one embodiment of the applicant's facial recognition system;
[0015] Figure 6 This is a structural schematic diagram of one embodiment of the applicant's face recognition device. Detailed Implementation
[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0017] In response to the face recognition method proposed in this application, a face recognition model needs to be pre-trained before performing face recognition on the image to be recognized. The training process of the face recognition model is explained in detail below. Please refer to... Figure 1 , Figure 1 This is a flowchart illustrating one implementation method of the applicant's face recognition model training method, which includes:
[0018] S101: Construct a training sample set containing multiple sample images, input the sample images into the face recognition model, and obtain the initial image features corresponding to each sample image. Each sample image has a corresponding category label.
[0019] In one embodiment, step S101 includes: collecting facial information of multiple individuals to obtain multiple facial images for training a facial recognition model. Each individual corresponds to multiple different facial images, and the different facial images corresponding to the same individual can be obtained from different angles or in different scenes.
[0020] Furthermore, at least a portion of the aforementioned individuals are selected, and for each selected individual, at least two face images are chosen as sample images to be added to the training sample set.
[0021] Furthermore, a face recognition model is constructed, and all sample images from the training sample set are input into the face recognition model. The feature extraction module in the face recognition model extracts features from each sample image to obtain the initial image features corresponding to each sample image.
[0022] S102: Obtain multiple initial category features, process the initial values of the initial category features into simplified values to obtain simplified category features, and process the initial values of the initial image features into simplified values to obtain simplified image features. The simplified values are either the first feature value or the second feature value.
[0023] In one embodiment, step S102 includes: responding to the fact that the constructed face recognition model contains a fully connected layer, and the fully connected layer contains multiple initial category features for classifying the features extracted by the face recognition model. In this embodiment, each initial category feature and each initial image feature obtained above contains multiple initial values, and the initial values at each position of the initial category features and initial image features are greater than or equal to a first value and less than or equal to a second value.
[0024] Specifically, in this embodiment, the first value is -1 and the second value is 1. The initial image features and initial category features can be processed using feature normalization to ensure that the initial values of the initial category features and initial image features are greater than or equal to -1 and less than or equal to 1. The specific implementation process of feature normalization can be found in existing technologies and will not be described in detail here.
[0025] Furthermore, the initial image features and initial category features are processed to obtain simplified image features corresponding to the initial image features, and simplified category features corresponding to the initial category features.
[0026] Specifically, a first threshold is set, and each initial value in the initial image features and initial category features is compared with the first threshold. If the initial value of the initial image features and initial category features at the current position is less than or equal to the first threshold, the initial value at the current position is updated to a first feature value. Otherwise, the initial value at the current position is updated to a second feature value. Once the initial values at all positions in the initial image features and initial category features have been updated, simplified category features and simplified image features are obtained. Simplifying the initial image features and initial category features helps save computational costs in subsequent training and improves training efficiency.
[0027] In this embodiment, the first threshold is 0, the first feature value is -1, and the second feature value is 1. That is, the initial image features and initial category features can be simplified using the following formula:
[0028]
[0029] Where z represents the initial value of the initial image feature or initial category feature at the current position, and B represents the simplified value of the initial value.
[0030] Optionally, in other embodiments, step S102 may not perform feature normalization on the initial image features and initial category features, and instead set corresponding first thresholds based on the actual values of the initial values at each position in the initial image features and initial category features. Initial values in the initial image features and initial category features that are less than or equal to the first threshold are updated as first feature values; initial values that are greater than the first threshold are updated as second feature values.
[0031] S103: Obtain the first similarity between the simplified image features and each simplified category feature, and obtain multiple reference category features corresponding to the sample image from all initial category features based on the first similarity.
[0032] In one implementation, please refer to Figure 2 , Figure 2 This is a flowchart illustrating step S103 in one embodiment. Specifically, step S103 includes:
[0033] S1031: Obtain the first feature distance between the simplified image features and the simplified category features, and obtain the first similarity based on the first feature distance. The first similarity is inversely proportional to the corresponding first feature distance.
[0034] In one embodiment, step S1031 includes: in response to obtaining simplified category features corresponding to each initial category feature and simplified image features corresponding to each initial image feature, calculating a first feature distance between the simplified image features and each simplified category feature.
[0035] Specifically, in this embodiment, the first feature distance between simplified image features and simplified class features can be calculated using the Hamming distance method. By using simplified image features and simplified class features to calculate the Hamming distance, computational resources are saved, and it helps to filter out the initial class features with strong interference based on the Hamming distance.
[0036] In one specific implementation, in response to the fact that both the simplified image features and the simplified category features contain n-bit initial values, the formula for calculating the first feature distance is as follows:
[0037]
[0038] Where d(x,y) represents the first feature distance between the simplified image feature x and the simplified class feature y, x i Let y represent the initial value at the i-th bit in the simplified image feature x. i This represents the initial value at the i-th position in the simplified categorical feature y; The value indicates an XOR operation, meaning that if the initial values of the simplified image features and the simplified category features at the corresponding positions are different, the distance is incremented by one.
[0039] Alternatively, in other embodiments, the first feature distance between the simplified image features and the simplified category features can also be calculated using other commonly used methods for calculating feature distance.
[0040] Furthermore, based on the calculated first feature distance, a first similarity is determined between the corresponding simplified image features and simplified category features. The smaller the value of the first feature distance, the higher the corresponding first similarity. A smaller first similarity corresponds to a stronger interference between the initial category features and the corresponding initial image features. This allows the face recognition model constructed in this application to learn the differences between the two, which helps improve the model's stability.
[0041] S1032: Sort all the initial category features according to the first similarity value from smallest to largest to obtain the initial category feature queue.
[0042] In one embodiment, step S1032 includes: sorting all initial category features according to the first similarity between the current simplified image feature and all simplified category features in ascending order of the corresponding values of the first similarity, thereby obtaining the initial category feature queue corresponding to the current simplified image feature.
[0043] S1033: Select at least a portion of the initial category features from the initial category feature queue as reference category features.
[0044] In one embodiment, step S1033 includes: based on the arrangement order of the initial category features in the initial category feature queue obtained above, selecting the first number of initial category features as candidate category features.
[0045] Specifically, by selecting the top number of initial category features as candidate category features, the less disruptive initial category features are removed from all initial category features while saving computational costs, thereby improving the stability of the trained face recognition model. The aforementioned top number can be set according to actual needs.
[0046] Alternatively, in other embodiments, candidate category features can be selected based on a ratio. For example, based on the order of the initial category features in the obtained initial category feature queue, the initial category features ranked at the top by a preset ratio can be used as candidate category features. The preset ratio can be 20%, 30%, or others.
[0047] Furthermore, in response to obtaining multiple candidate category features by using the Hamming distance between the simplified image features corresponding to the sample image and multiple simplified category features, this method easily overlooks the subtle differences between the corresponding initial image features and the initial category features. For example, if feature A is (0.04, 0.21, -0.15, 0.11...) and feature B is (0.21, 0.11, -0.01, 0.04...), then the simplified feature A' obtained after the simplification process mentioned in step S102 is (1, 1, 0, 1...), and the simplified feature B' obtained after the simplification process of feature B is also (1, 1, 0, 1...). That is, different initial features may result in the same simplified feature after simplification. Therefore, in order to combine the subtle differences between the initial image features and the initial category features, after obtaining multiple candidate category features, the multiple candidate category features are further refined.
[0048] The specific process of the above-mentioned fine screening includes: obtaining the second feature distance between the initial image features and each candidate category feature; and, based on the magnitude of the second feature distance, selecting at least some category features with high similarity to the initial image features from multiple candidate category features as reference category features.
[0049] Specifically, after obtaining multiple candidate category features through screening, the Euclidean distance between the initial image features and each candidate category feature is calculated, and this Euclidean distance is used as the corresponding second feature distance. All candidate category features are sorted in ascending order of their second feature distances to obtain a candidate category feature queue. Based on the order of the candidate category features in the queue, the second-highest number of candidate category features is used as the reference category feature. The specific value of this second number can be set according to actual needs.
[0050] Optionally, in other embodiments, the second feature distance described above can also be obtained by other feature distance calculation methods.
[0051] Alternatively, in other embodiments, step S1033 may directly use the obtained multiple candidate category features as reference category features, that is, the fine screening step is not performed after obtaining multiple candidate category features.
[0052] S104: Based on the category labels, initial image features, and reference category features corresponding to all sample images, adjust the parameters in the face recognition model until the convergence condition is met.
[0053] In one embodiment, step S104 includes: in response to obtaining the category label, initial image features, and reference category features corresponding to the sample image through the above steps, calculating the total loss corresponding to the face recognition model constructed in this application based on the contrastive loss function and the cross-entropy loss function. The obtained total loss is then used to adjust the model parameters to obtain the trained face recognition model.
[0054] This application proposes a face recognition model training method, comprising: after obtaining initial image features corresponding to multiple sample images and multiple initial category features for classifying the initial image features, simplifying the initial image features and initial category features to obtain corresponding simplified image features and simplified category features. Based on the first similarity between the simplified image features and each simplified category feature, obtaining multiple reference category features corresponding to the sample images from all initial category features, and calculating the model loss value using at least the category label corresponding to the sample images, the initial image features, and the reference category features to adjust the model parameters. By performing simplification first and then selecting some reference category features for training, the calculation of the model loss value using the less pervasive initial category features can be avoided, saving the model training cost while improving the stability of the model obtained after training. In addition, using the face recognition model obtained after training to perform face recognition on the face image to be recognized can improve the face recognition accuracy.
[0055] In another embodiment, prior to step S104, the method further includes obtaining the class center feature corresponding to each current sample image. Specifically, in response to inputting each sample image from the training sample set into the face recognition model, class center extraction is performed on the feature information of the category to which the current sample image belongs, to obtain the class center feature corresponding to the current sample image. This application does not limit the method for obtaining the class center feature.
[0056] Furthermore, a first comparison image and a second comparison image corresponding to each current sample image are obtained from the training sample set. The first comparison image has the same category label as the sample image, while the second comparison image has a different category label than the sample image.
[0057] Specifically, for each current sample image, a sample image with the same category label as the current sample image is randomly selected from the corresponding training sample set as the first comparison image corresponding to the current sample image; and a sample image with a different category label than the current sample image is randomly selected from the corresponding training sample set as the second comparison image corresponding to the current sample image.
[0058] Furthermore, the initial image features of the first comparison image are used as the first comparison features corresponding to the current sample image, and the initial image features of the second comparison image are used as the second comparison features corresponding to the current sample image.
[0059] It should be noted that the process of obtaining the class center feature, the first contrast feature, and the second contrast feature corresponding to each sample image described above can be performed on [the following text is incomplete and likely refers to a different process]. Figure 1 The process is executed after step S101; that is, to construct a training sample set, input the sample images into the face recognition model, obtain the initial image features of each sample image, and then obtain the class center features, first contrast features, and second contrast features corresponding to each sample image.
[0060] Further, in this embodiment, in response to obtaining the class center feature, the first contrast feature, and the second contrast feature corresponding to the current sample image, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating step S104 in another embodiment. In this embodiment, step S104 includes:
[0061] S1041: Based on the initial image features, first contrast features, second contrast features, class center features, and at least some reference class features corresponding to all sample images, obtain the contrast loss corresponding to the face recognition model.
[0062] In this embodiment, a training sample set is constructed in response to step S101 above. This training sample set corresponds to training triplet pairs, which contain a first element, a second element, and a third element. The second element and the first element are correlated features, and the third element and the first element are dissimilar features. Furthermore, in this embodiment, the first element is an initial image feature or a first contrast feature. Any two of the initial image feature, the first contrast feature, and the class center feature are correlated features, and the initial image feature and the first contrast feature are dissimilar features to the second contrast feature or a reference class feature, respectively. That is, when the first element is an initial image feature, the second element can be the first contrast feature or the class center feature corresponding to the initial image feature, and the third element can be the second contrast feature or any reference class feature; when the first element is the first contrast feature, the second element can be the initial image feature or the class center feature corresponding to the initial image feature, and the third element can be the second contrast feature or any reference class feature.
[0063] It should be noted that, in order to save computational costs in training the model and to enable the model to learn the differences between the most disruptive reference class features and the sample images, in step S1041, all reference class features are sorted in ascending order of the second feature distance between the candidate class features and the initial image features corresponding to the sample images, resulting in a reference class feature queue. Based on the order of the reference class features in the queue, the third-ranked reference class features are selected to construct the aforementioned training triplet pairs. In this embodiment, the method for obtaining the second feature distance between the candidate class features and the initial image features corresponding to the sample images can refer to the corresponding implementation described above, and the third number is 10. Of course, in other embodiments, the third number can also be other.
[0064] Alternatively, in other embodiments, training triplet pairs can be constructed based solely on the initial image features, the first contrast feature, and the second contrast feature; that is, the first element is the initial image feature, the second element is the first contrast feature, and the third element is the second contrast feature. Furthermore, after inputting the training sample set into the face recognition model, multiple training triplet pairs corresponding to the training sample set are generated to aid in the subsequent calculation of the model loss value.
[0065] Further, please refer to Figure 4 , Figure 4 This is a flowchart illustrating step S1041 in one embodiment. In this embodiment, the specific implementation process of step S1041 includes:
[0066] S201: Obtain the contrastive loss for each training triplet pair. This contrastive loss includes the first loss and the second loss.
[0067] First, obtain the third feature distance between the first element and the second element and the fourth feature distance between the first element and the third element in each training triplet pair. Based on the third feature distance and the fourth feature distance, obtain the first loss for each training triplet pair.
[0068] Specifically, the first loss for each training triplet pair can be calculated using the triplet loss calculation function. This is in response to the first element of the current training triplet pair being 'a'. i The second element is p i The third element is n i The first loss L of the current training triplet pair fine The calculation formula is as follows:
[0069] L fine =max[dist(a i p i )-dist(a i n i +margin, 0]
[0070] Among them, dist(a i p i ) represents the third feature distance, dist(a i n i ) represents the fourth feature distance; margin is an adjustable parameter, which can be estimated by relevant technical personnel or obtained by back-calculation after a number of experiments.
[0071] Furthermore, the initial values at each position of the first, second, and third elements are constrained so that the initial values at each position of the first, second, and third elements are between the third and fourth values. In this embodiment, the third value is 0 and the fourth value is 1.
[0072] Specifically, the sigmoid function is used to constrain the initial values of each element in the current training triplet pair.
[0073] For example, when there is a feature S(x), the specific calculation formula for constraining the feature S(x) using the sigmoid function is as follows:
[0074] S(x)=[sigmoid(kx1), sigmoid(kx2),..., sigmoid(kx n )]
[0075] Among them, x1 to x nThe initial value represents the value at each position in feature S(x); k is an adjustment coefficient used to amplify the initial value at each position in feature S(x), and the specific value can be set according to actual needs. Preferably, in this embodiment, k > 10.
[0076] Furthermore, the fifth feature distance between the first constrained element and the second constrained element, and the sixth feature distance between the first constrained element and the third constrained element are obtained, and the second loss for each training triplet pair is obtained based on the fifth feature distance and the sixth feature distance.
[0077] Specifically, the formula for calculating the second loss of the current training triplet pair is as follows:
[0078] L coarse =max[dist(S(a i ), S(p i ))-dist(S(a i ), S(n i ))+margin,0]
[0079] Among them, L coarse S(a) represents the second loss of the current training triple pair. i S(p) represents the first element after the constraint. i S(n) represents the second element after the constraint. i ) represents the third element after the constraint, and margin represents the adjustable parameter.
[0080] By constraining the initial values of each element in the training triplet pair and calculating the second loss of the training triplet pair, the face recognition model obtained after subsequent training has a better recognition ability for the binarized features, thereby further improving the stability of the model obtained after training.
[0081] Furthermore, based on the first loss and the second loss, the contrast loss for the corresponding training triplet pair is obtained.
[0082] Specifically, in response to obtaining the first loss and the second loss of the training triplet pairs, a corresponding weight value is assigned to the second loss. The product of the second loss and the aforementioned weight value is obtained, and the sum of this product and the corresponding first loss is used as the comparison sub-loss for the corresponding training triplet pair. The specific calculation formula is as follows:
[0083] l = L fine +λL coarse
[0084] Where l represents the contrast loss of the current training triple pair, L fine L represents the first loss of the current training triplet pair. coarseLet λ represent the second loss of the current training triplet pair, and let λ represent the weight value corresponding to the second loss. Preferably, in this embodiment, λ = 0.1, but in other embodiments, λ can also be other values.
[0085] S202: The sum of all contrastive losses is used as the contrastive loss.
[0086] Specifically, given that there are M training triplet pairs in the training sample set, the sum of the contrastive losses of all training triplet pairs is used as the contrastive loss. The specific calculation formula is as follows:
[0087]
[0088] Among them, l i L represents the contrast loss corresponding to the i-th training triple pair. TL This represents the contrast loss corresponding to the face recognition model.
[0089] S1042: Based on the class labels, class center features, and at least some reference class features corresponding to all sample images, obtain the cross-entropy loss corresponding to the face recognition model.
[0090] In one embodiment, step S1042 includes: in response to the training sample set containing multiple sample images, calculating the cross-entropy sub-loss corresponding to each sample image based on the corresponding class label, class center feature, and at least a portion of all reference class features. In this embodiment, to learn the difference between the initial image features corresponding to the sample image and the multiple reference class features, based on the order of the reference class features in the reference class feature queue mentioned in step S1041, the fourth-ranked reference class features are used to calculate the cross-entropy loss corresponding to the face recognition model. This fourth-ranked number can be 1000. Of course, in other embodiments, the fourth-ranked number can be other values.
[0091] Specifically, the formula for calculating the cross-entropy sub-loss for each sample image is as follows:
[0092]
[0093] Among them, L j K represents the cross-entropy sub-loss corresponding to the j-th sample image. CE y represents the total number of the fourth selected reference category features and the class center features. j Let q represent the category label corresponding to the j-th sample image. c This represents the predicted value corresponding to the Cth category. The detailed process for calculating the cross-entropy loss can be found in existing technologies, and will not be elaborated upon here.
[0094] Furthermore, in response to obtaining the cross-entropy sub-loss corresponding to all sample images in the training sample set, the cross-entropy loss corresponding to the face recognition model is calculated, and the specific calculation formula is as follows:
[0095]
[0096] Among them, L CE denoted by , where represents the cross-entropy loss corresponding to the face recognition model, and N represents the number of sample images in the training sample set.
[0097] S1043: The sum of the contrast loss and the cross-entropy loss is used as the total loss of the face recognition model, and the parameters in the face recognition model are adjusted based on the total loss.
[0098] In one embodiment, step S1043 includes: taking the sum of the contrast loss calculated in step S1041 and the cross-entropy loss calculated in step S1042 as the total loss of the applicant's face recognition model, and using the total loss to adjust the parameters in the face recognition model.
[0099] Furthermore, a training sample set containing multiple sample images is reconstructed, and the above training process is performed until the convergence condition is met, resulting in a trained face recognition model.
[0100] In one embodiment, this application also proposes a face recognition method, which includes: obtaining a face image to be recognized, wherein the face image to be recognized may be obtained by real-time shooting by a camera device or by other means, such as the network.
[0101] Furthermore, based on the face recognition model obtained after training, face recognition is performed on the face image to be recognized, that is, the face image to be recognized is input into the face recognition model after training to obtain the face image features corresponding to the face image to be recognized.
[0102] Please see Figure 5 , Figure 5 This is a schematic diagram of one embodiment of the applicant's face recognition system. The face recognition system includes a face recognition module 10 and a training module 20 that are coupled to each other.
[0103] Specifically, the face recognition module 10 is used to perform face recognition on the face image to be recognized based on the trained face recognition model, and obtain the face image features corresponding to the face image to be recognized.
[0104] The training module 20 is used to train the face recognition model. The training process includes: constructing a training sample set containing multiple sample images; inputting the sample images into the face recognition model to obtain the initial image features corresponding to each sample image; wherein each sample image corresponds to a category label; obtaining multiple initial category features; processing the initial values of the initial category features into simplified values to obtain simplified category features; processing the initial values of the initial image features into simplified values to obtain simplified image features; wherein the simplified values are either the first feature value or the second feature value; obtaining the first similarity between the simplified image features and each simplified category feature; obtaining multiple reference category features corresponding to the sample images from all initial category features based on the first similarity; and adjusting the parameters in the face recognition model based on the category labels corresponding to all sample images, the initial image features, and the reference category features until the convergence condition is met.
[0105] The face recognition system proposed in this application also includes a simplification module 21 coupled to the training module 20. The initial values of the initial category features and initial image features are greater than or equal to a first value and less than or equal to a second value. The simplification module 21 updates the initial value at the current position to a first feature value in response to the initial value of the initial image features and initial category features at the current position being less than or equal to a first threshold; otherwise, it updates the initial value at the current position to a second feature value. After the initial values at all positions in the initial image features and initial category features are updated, simplified category features and simplified image features are obtained.
[0106] The face recognition system proposed in this application also includes a screening module 23 coupled to the training module 20. This screening module 23 is used to obtain a first feature distance between simplified image features and simplified category features, and to obtain a first similarity based on the first feature distance; wherein the first similarity is inversely proportional to the corresponding first feature distance; sorting all initial category features in ascending order of the first similarity values to obtain an initial category feature queue; and selecting at least a portion of the initial category features from the initial category feature queue as reference category features.
[0107] The process of selecting at least a portion of the initial category features from the initial category feature queue as reference category features includes: selecting the first number of initial category features as candidate category features based on the order of the initial category features in the initial category feature queue; obtaining the second feature distance between the initial image features and each candidate category feature; sorting all candidate category features in ascending order of the second feature distance to obtain a candidate category feature queue; and selecting the second number of candidate category features as reference category features based on the order of the candidate category features in the candidate category feature queue.
[0108] The face recognition system proposed in this application also includes an extraction module 25 coupled to the training module 20, used to adjust the parameters in the face recognition model based on the category labels, initial image features and reference category features corresponding to all sample images until the convergence condition is met. This includes: obtaining the class center features corresponding to each current sample image; obtaining a first comparison image and a second comparison image corresponding to each current sample image from the training sample set; wherein the first comparison image has the same category label as the sample image, and the second comparison image has a different category label than the sample image; using the initial image features of the first comparison image as the first comparison feature corresponding to the current sample image, and using the initial image features of the second comparison image as the second comparison feature corresponding to the current sample image.
[0109] Specifically, the parameters in the face recognition model are adjusted based on the category labels, image features, and reference category features corresponding to all sample images until the convergence condition is met. This includes: obtaining the contrast loss corresponding to the face recognition model based on the initial image features, first contrast features, second contrast features, class center features, and at least some reference category features corresponding to all sample images; obtaining the cross-entropy loss corresponding to the face recognition model based on the category labels, class center features, and at least some reference category features corresponding to all sample images; and using the sum of the contrast loss and the cross-entropy loss as the total loss of the face recognition model, and adjusting the parameters in the face recognition model based on the total loss.
[0110] The training sample set corresponds to training triplet pairs, each containing a first element, a second element, and a third element. The second element and the first element are correlated features, and the third element and the first element are dissimilar features. The first element is either an initial image feature or a first contrast feature. Any two of the initial image feature, the first contrast feature, and the class center feature are correlated features. The initial image feature and the first contrast feature are dissimilar features to the second contrast feature or the reference class feature, respectively. Based on the initial image feature, the first contrast feature, the second contrast feature, the class center feature, and at least some of the reference class features corresponding to the sample image, the contrast loss corresponding to the face recognition model is obtained, including: obtaining the contrast sub-loss of each training triplet pair, and summing all contrast sub-losses as the contrast loss.
[0111] The contrastive loss includes a first loss and a second loss. The contrastive sub-loss for each training triplet pair is obtained by: obtaining the third feature distance between the first and second elements and the fourth feature distance between the first and third elements in each training triplet pair; obtaining the first loss for each training triplet pair based on the third and fourth feature distances; constraining the initial values at each position of the first, second, and third elements so that the initial values at each position are between the third and fourth values; obtaining the fifth feature distance between the constrained first and second elements and the sixth feature distance between the constrained first and third elements; obtaining the second loss for each training triplet pair based on the fifth and sixth feature distances; and obtaining the contrastive sub-loss for the corresponding training triplet pair based on the first and second losses.
[0112] Please see Figure 6 , Figure 6 This is a schematic diagram of one embodiment of the applicant's face recognition device. The video switching system includes a memory 40 and a processor 50 coupled to each other. The processor 50 stores program instructions and executes these instructions to implement the face recognition method in any of the above embodiments. Specifically, the processor 50 can also be called a CPU (Central Processing Unit). The processor 50 may be an integrated circuit chip with signal processing capabilities. The processor 50 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. Furthermore, the processor 50 can be implemented using integrated circuit chips.
[0113] It should be noted that the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0114] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0115] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0116] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A face recognition method, characterized in that, include: Face recognition is performed on the face image to be recognized based on the trained face recognition model to obtain the face image features corresponding to the face image to be recognized; The training process of the face recognition model includes: A training sample set containing multiple sample images is constructed, and the sample images are input into the face recognition model to obtain the initial image features corresponding to each sample image; wherein, each sample image corresponds to a category label; Multiple initial category features are obtained, and the initial values of the initial category features are processed into simplified values to obtain simplified category features. The initial values of the initial image features are processed into simplified values to obtain simplified image features. The simplified values are either first feature values or second feature values. A first similarity is obtained between the simplified image features and each of the simplified category features, and multiple reference category features corresponding to the sample image are obtained from all the initial category features based on the first similarity; Obtain the class center feature corresponding to each current sample image; obtain a first comparison image and a second comparison image corresponding to each current sample image from the training sample set; wherein, the first comparison image has the same class label as the sample image, and the second comparison image has a different class label than the sample image; use the initial image features of the first comparison image as the first comparison feature corresponding to the current sample image, and use the initial image features of the second comparison image as the second comparison feature corresponding to the current sample image; Based on the category labels, initial image features, and reference category features corresponding to all the sample images, the parameters in the face recognition model are adjusted until the convergence condition is met. Specifically: based on the initial image features, first contrast feature, second contrast feature, class center feature, and at least some of the reference category features corresponding to all the sample images, the contrast loss corresponding to the face recognition model is obtained; based on the category labels, class center features, and at least some of the reference category features corresponding to all the sample images, the cross-entropy loss corresponding to the face recognition model is obtained; the sum of the contrast loss and the cross-entropy loss is used as the total loss of the face recognition model, and the parameters in the face recognition model are adjusted based on the total loss. The training sample set corresponds to training triplet pairs, each training triplet pair containing a first element, a second element, and a third element. The second element and the first element are correlated features, and the third element and the first element are dissimilar features. The first element is either the initial image feature or the first contrast feature. Any two of the initial image feature, the first contrast feature, and the class center feature are correlated features. The initial image feature and the first contrast feature are dissimilar features to the second contrast feature or the reference class feature, respectively. The step of obtaining the contrast loss corresponding to the face recognition model based on the initial image feature, the first contrast feature, the second contrast feature, the class center feature, and at least some of the reference class features corresponding to all the sample images includes: obtaining the contrast sub-loss of each training triplet pair, and summing all the contrast sub-losses as the contrast loss.
2. The method according to claim 1, characterized in that, The initial values of the initial category features and the initial image features are greater than or equal to a first value and less than or equal to a second value. The process of processing the initial values of the initial category features into simplified values to obtain simplified category features, and processing the initial values of the initial image features into simplified values to obtain simplified image features, includes: In response to the initial value at the current position of the initial image feature and the initial category feature being less than or equal to a first threshold, the initial value at the current position is updated to a first feature value; otherwise, the initial value at the current position is updated to a second feature value. Upon completion of the initial value update at all locations in the initial image features and the initial category features, the simplified category features and the simplified image features are obtained.
3. The method according to claim 1, characterized in that, The step of obtaining a first similarity between the simplified image features and each of the simplified category features, and obtaining multiple reference category features corresponding to the sample image from all the initial category features based on the first similarity, includes: A first feature distance is obtained between the simplified image features and the simplified category features, and a first similarity is obtained based on the first feature distance; wherein, the first similarity is inversely proportional to the corresponding first feature distance; Sort all the initial category features in ascending order of the first similarity values to obtain the initial category feature queue; At least a portion of the initial category features are selected from the initial category feature queue as the reference category features.
4. The method according to claim 3, characterized in that, The step of selecting at least a portion of the initial category features from the initial category feature queue as the reference category features includes: Based on the order of the initial category features in the initial category feature queue, the first number of initial category features in the queue are selected as candidate category features; Obtain the second feature distance between the initial image features and each of the candidate category features; Sort all the candidate category features in ascending order of the second feature distance to obtain a candidate category feature queue; Based on the order of the candidate category features in the candidate category feature queue, the second-highest number of the candidate category features is taken as the reference category feature.
5. The method according to claim 1, characterized in that, The contrastive loss includes a first loss and a second loss, and obtaining the contrastive loss for each of the training triplet pairs includes: Obtain the third feature distance between the first element and the second element and the fourth feature distance between the first element and the third element in each training triplet pair, and obtain the first loss for each training triplet pair based on the third feature distance and the fourth feature distance; The initial values at each position of the first element, the second element, and the third element are constrained so that the initial values at each position of the first element, the second element, and the third element are between the third value and the fourth value; The fifth feature distance between the first element and the second element after constraint and the sixth feature distance between the first element and the third element after constraint are obtained. The second loss of each training triple pair is obtained based on the fifth feature distance and the sixth feature distance. Based on the first loss and the second loss, the contrast sub-loss corresponding to the training triplet pair is obtained.
6. A face recognition system, characterized in that, include: The face recognition module is used to perform face recognition on the face image to be recognized based on the trained face recognition model, and obtain the face image features corresponding to the face image to be recognized. The training module is used to train the face recognition model. The training process includes: constructing a training sample set containing multiple sample images; inputting the sample images into the face recognition model to obtain initial image features corresponding to each sample image; wherein each sample image corresponds to a category label; obtaining multiple initial category features; processing the initial values of the initial category features into simplified values to obtain simplified category features; processing the initial values of the initial image features into simplified values to obtain simplified image features; wherein the simplified values are a first feature value or a second feature value; obtaining a first similarity between the simplified image features and each of the simplified category features; obtaining multiple reference category features corresponding to the sample image from all the initial category features based on the first similarity; obtaining a class center feature corresponding to each current sample image; obtaining a first comparison image and a second comparison image corresponding to each current sample image from the training sample set; wherein the first comparison image and the sample image have the same category label, and the second comparison image... The comparison image and the sample image have different category labels; the initial image features of the first comparison image are used as the first comparison feature corresponding to the current sample image, and the initial image features of the second comparison image are used as the second comparison feature corresponding to the current sample image; based on the category labels, the initial image features, and the reference category features corresponding to all the sample images, the parameters in the face recognition model are adjusted until the convergence condition is met; specifically: based on the initial image features, the first comparison feature, the second comparison feature, the class center feature, and at least some of the reference category features corresponding to all the sample images, the comparison loss corresponding to the face recognition model is obtained; based on the category labels, the class center feature, and at least some of the reference category features corresponding to all the sample images, the cross-entropy loss corresponding to the face recognition model is obtained; the sum of the comparison loss and the cross-entropy loss is used as the total loss of the face recognition model, and the parameters in the face recognition model are adjusted based on the total loss; The training sample set corresponds to training triplet pairs, each training triplet pair containing a first element, a second element, and a third element. The second element and the first element are correlated features, and the third element and the first element are dissimilar features. The first element is either the initial image feature or the first contrast feature. Any two of the initial image feature, the first contrast feature, and the class center feature are correlated features. The initial image feature and the first contrast feature are dissimilar features to the second contrast feature or the reference class feature, respectively. The step of obtaining the contrast loss corresponding to the face recognition model based on the initial image feature, the first contrast feature, the second contrast feature, the class center feature, and at least some of the reference class features corresponding to all the sample images includes: obtaining the contrast sub-loss of each training triplet pair, and summing all the contrast sub-losses as the contrast loss.
7. A face recognition device, characterized in that, The apparatus includes a memory and a processor coupled to each other, the processor storing program instructions for executing the program instructions to implement the method of any one of claims 1-5.