Model training method, face recognition method, device, equipment, medium and product

Through feature coding and relation matrix processing of face image sample sets, meta-models are generated, which solves the problem of low recognition accuracy in existing face recognition methods, and achieves fast and accurate face recognition.

CN119027756APending Publication Date: 2024-11-26CHINA MOBILE COMM GRP CHONGQING CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410918488.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-09
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing facial recognition methods have shortcomings in recognition accuracy, which is fast but low in recognition speed, and has limited processing capabilities for multiple samples.

Method used

By obtaining the sample set of face images, input it into the feature encoder to extract embedded features, calculate the relationship matrix and task attention matrix, eliminate noise, and generate a weighted relationship matrix, which is used for face recognition metamodel training, improve recognition accuracy and efficiency.

Benefits of technology

It realizes the accuracy and efficiency of face recognition while maintaining the fast recognition speed, and enhances the discriminative feature extraction ability of face images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027756A_ABST
    Figure CN119027756A_ABST
Patent Text Reader

Abstract

The invention discloses a model training method, a face recognition method, devices, equipment, a medium and a product, and the method comprises the steps: inputting a first sample set into a first feature encoder and a second feature encoder, and obtaining a first embedded feature corresponding to a support set and a second embedded feature corresponding to a first query image, the third embedding feature corresponds to the support set, and the fourth embedding feature corresponds to the first query image; calculating a first relation matrix according to the first embedding feature and the second embedding feature, and calculating a second relation matrix according to the third embedding feature and the fourth embedding feature; determining a task attention matrix based on the first relation matrix; merging the task attention matrix into the first relation matrix to obtain a weighted relation matrix; obtaining a prediction score based on the weighted relation matrix and the first target face image; and according to the prediction score, obtaining a meta-model so as to quickly and accurately recognize the face.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and in particular to a model training method, face recognition method, device, equipment, medium and product. Background Art

[0002] There are currently a variety of methods for face recognition. The most commonly used method is the face recognition method based on geometric features. Specifically, it can be a geometric description of the shape and structural relationship of facial features such as eyes, nose, mouth, chin, etc. in the face. As an important feature of face recognition, the geometric features here can be the geometric relationship between features such as eyes, nose, mouth, etc., such as the distance and angle between them.

[0003] The above face recognition method has a fast recognition speed and requires a small amount of memory, but has a low recognition accuracy. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a model training method, face recognition method, device, equipment, medium and product to quickly and accurately recognize faces.

[0005] The technical solution of this application is as follows:

[0006] In a first aspect, a model training method is provided, the method comprising:

[0007] Acquire a first sample set, the first sample set including first face images of N persons, the first sample set including a support set and a query set, where N is a positive integer;

[0008] Inputting the first sample set into a first feature encoder to obtain a first embedded feature corresponding to the support set and a second embedded feature corresponding to the first query image, and inputting the first sample set into a second feature encoder to obtain a third embedded feature corresponding to the support set and a fourth embedded feature corresponding to the first query image, wherein the first query image is a first face image of a first object in the query set;

[0009] Calculating a first relationship matrix between the support set and the first query image according to the first embedding feature and the second embedding feature, and calculating a second relationship matrix between the support set and the first query image according to the third embedding feature and the fourth embedding feature;

[0010] Based on the second relationship matrix, determining a task attention matrix;

[0011] Merging the task attention matrix into the first relationship matrix by element-by-element multiplication to obtain a weighted relationship matrix;

[0012] Based on the weighted relationship matrix and a first target facial image, obtaining a prediction score that the first target facial image is the first object, wherein the first target facial image is any first facial image in the first sample set;

[0013] According to the prediction scores, a meta-model for performing face recognition is obtained.

[0014] In a second aspect, a face recognition method is provided, the method comprising:

[0015] Obtaining a face image to be recognized;

[0016] Inputting the face image to be recognized into the meta-model to obtain a target recognition result of the face image to be recognized;

[0017] Among them, the meta-model is the meta-model obtained by the model training method described in the first aspect.

[0018] In a third aspect, a model training device is provided, the device comprising:

[0019] A first acquisition module is used to acquire a first sample set, wherein the first sample set includes first face images of N persons, the first sample set includes a support set and a query set, and N is a positive integer;

[0020] a first determination module, configured to input the first sample set into a first feature encoder to obtain first embedded features corresponding to the support set and second embedded features corresponding to the first query image, and input the first sample set into a second feature encoder to obtain third embedded features corresponding to the support set and fourth embedded features corresponding to the first query image, wherein the first query image is a first face image of a first object in the query set;

[0021] a second determining module, configured to calculate a first relationship matrix between the support set and the first query image according to the first embedding feature and the second embedding feature, and to calculate a second relationship matrix between the support set and the first query image according to the third embedding feature and the fourth embedding feature;

[0022] A third determination module, configured to determine a task attention matrix based on the second relationship matrix;

[0023] a fourth determination module, configured to merge the task attention matrix into the first relationship matrix by element-by-element multiplication to obtain a weighted relationship matrix;

[0024] a fifth determination module, configured to obtain a prediction score of the first target facial image being the first object based on the weighted relationship matrix and a first target facial image, wherein the first target facial image is any first facial image in the first sample set;

[0025] The sixth determination module is used to obtain a meta-model for performing face recognition according to the prediction score.

[0026] In a fourth aspect, a face recognition device is provided, the device comprising:

[0027] A first acquisition module is used to acquire a face image to be recognized;

[0028] The first determination module is used to input the face image to be recognized into the meta-model to obtain the target recognition result of the face image to be recognized; wherein the meta-model is the meta-model obtained by the model training method described in the first aspect.

[0029] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the model training method described in any one of the embodiments of the present application, and / or the steps of the face recognition method.

[0030] In a sixth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the model training method described in any one of the embodiments of the present application and / or the steps of the face recognition method are implemented.

[0031] In the seventh aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device can perform the steps of the model training method described in any embodiment of the present application, and / or the steps of the face recognition method.

[0032] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0033] In an embodiment of the present application, the first sample set is input into the first feature encoder to obtain the first embedding feature corresponding to the support set and the second embedding feature corresponding to the first query image, and the first sample set is input into the second feature encoder to obtain the third embedding feature corresponding to the support set and the fourth embedding feature corresponding to the first query image, and then the first relationship matrix between the support set and the first query image is calculated according to the first embedding feature and the second embedding feature, and the second relationship matrix between the support set and the first query image is calculated according to the third embedding feature and the fourth embedding feature, and the task attention matrix is ​​determined based on the second relationship matrix. The task attention matrix can adaptively extract the discriminative embedded features that support a certain query embedded feature in the focus, and merge the task attention matrix into the first relationship matrix through element-by-element multiplication. In this way, a series of noises such as the relationship formed by similar embedded features in the task can be eliminated, and the embedded features with discriminability can be strengthened to obtain a weighted relationship matrix. Based on the weighted relationship matrix and the first target face image, the prediction score of the first target face image as the first object is obtained. According to the prediction score, a meta-model for face recognition with noise eliminated is obtained. In this way, face recognition can be accurately performed based on the meta-model, thereby improving the accuracy and efficiency of human recognition.

[0034] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application, and do not constitute an improper limitation on the present application.

[0036] Figure 1 It is a flowchart of a model training method provided in an embodiment of the present application;

[0037] Figure 2 is a schematic diagram of the structure of a first neural network provided in an embodiment of the present application;

[0038] Figure 3 It is a schematic diagram of determining a task attention matrix provided in an embodiment of the present application;

[0039] Figure 4 It is a flowchart of a face recognition method provided in an embodiment of the present application;

[0040] Figure 5 It is a structural schematic diagram of a model training device provided in an embodiment of the present application;

[0041] Figure 6 is a structural schematic diagram of a face recognition device provided in an embodiment of the present application;

[0042] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make those of ordinary skill in the art better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are only examples consistent with some aspects of the present application as detailed in the attached claims.

[0045] As described in the background technology section, the current face recognition method has low recognition accuracy. In order to solve this problem, the embodiments of the present application provide a model training method, a face recognition method, an apparatus, a device, a medium and a product. The first sample set obtained is input into a first feature encoder to obtain a first embedded feature corresponding to the support set and a second embedded feature corresponding to the first query image, and the first sample set is input into a second feature encoder to obtain a third embedded feature corresponding to the support set and a fourth embedded feature corresponding to the first query image, and then a first relationship matrix between the support set and the first query image is calculated based on the first embedded feature and the second embedded feature, and a relationship matrix between the support set and the first query image is calculated based on the third embedded feature and the fourth embedded feature. A second relationship matrix between the query images is obtained, and a task attention matrix is ​​determined based on the second relationship matrix. The task attention matrix can adaptively extract the discriminative embedded features that support a certain query embedded feature in the set, and the task attention matrix is ​​merged into the first relationship matrix by element-by-element multiplication. In this way, a series of noises such as the relationship formed by similar embedded features in the task can be eliminated, and the embedded features with discriminability can be strengthened to obtain a weighted relationship matrix. Based on the weighted relationship matrix and the first target face image, a prediction score that the first target face image is the first object is obtained. According to the prediction score, a meta-model for face recognition with noise eliminated is obtained, so that face recognition can be accurately performed based on the meta-model, thereby improving the accuracy and efficiency of human recognition.

[0046] The model training method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0047] Figure 1 It is a flow chart of a model training method provided in an embodiment of the present application. The executor of the model training method may be a server. It should be noted that the above-mentioned executor does not constitute a limitation on the embodiment of the present application.

[0048] like Figure 1 As shown, the model training method provided in the embodiment of the present application may include steps 110 to 170.

[0049] Step 110: Obtain a first sample set, where the first sample set includes first face images of N persons, and the first sample set includes a support set and a query set.

[0050] The first sample set may be a sample set used for meta-training, and the first sample set may include first face images of N persons, where the first face image may be a face image of a certain object.

[0051] The first facial images of the N persons mentioned above may include K facial images of each person at different angles. For example, there are 10 persons in total, and each person includes K facial images at different angles. The first sample set here may also include label images of the K facial images of each person at different angles, that is, used to mark who each facial image corresponds to.

[0052] The first sample set includes a support set and a query set. The support set here may be composed of K face images and label images of each person at different angles, and the query set may be composed of samples in the first sample set except the support set. Here, N and K are both positive integers.

[0053] It should be noted that, in addition to K face images at different angles, each person may also have other face images, and the other face images may constitute a query set.

[0054] Step 120: input the first sample set into the first feature encoder to obtain the first embedded features corresponding to the support set and the second embedded features corresponding to the first query image, and input the first sample set into the second feature encoder to obtain the third embedded features corresponding to the support set and the fourth embedded features corresponding to the first query image.

[0055] Among them, the first feature encoder can be an encoder used to extract features in the first sample set. The function of the first feature encoder can be to convert the input sample image into an embedded feature of size 5×5×64. The specific structure of the first feature encoding will be described in detail in subsequent embodiments.

[0056] The second feature encoder may be a feature encoder different from the first feature encoder. Specifically, the second feature encoder may be a convolution block f consisting of two 1×1 convolution layers. v composed of.

[0057] The first embedded feature may be an embedded feature corresponding to a support set obtained based on the first feature encoder after the first sample set is input into the first feature encoder.

[0058] The first query image may be a first face image of a first object in the query set, where the first object may be any one of the N persons.

[0059] The second embedded feature may be an embedded feature corresponding to the first query image obtained based on the first feature encoder after the first sample set is input into the first feature encoder.

[0060] The third embedded feature may be an embedded feature corresponding to a support set obtained based on the second feature encoder after the first sample set is input into the second feature encoder.

[0061] The fourth embedded feature may be an embedded feature corresponding to the first query image obtained based on the second feature encoder after the first sample set is input into the second feature encoder.

[0062] It should be noted that after each input sample is input into the first feature encoder, an embedded feature can be obtained, which is expressed as Usually, each input sample is represented by a WH C-dimensional embedded feature, where WH is the convolution layer parameter of the first feature encoder. The convolution layer parameter of the first feature encoder determines the embedded feature. That is, WH is also the size information of the embedded feature, and C is the object of the input sample C.

[0063] When each input sample is represented by a WHC-dimensional embedding feature, the entire support set can be identified as the NKWHC-dimensional support embedding feature, that is, Each query image can be represented as a WH C-dimensional query embedding feature, i.e.

[0064] In some embodiments of the present application, the first sample set can be input into a first feature encoder to obtain a first embedded feature corresponding to the support set and a second embedded feature corresponding to the first query image, and the first sample set can be input into a second feature encoder to obtain a third embedded feature corresponding to the support set and a fourth embedded feature corresponding to the first query image.

[0065] In some embodiments of the present application, in order to accurately obtain the first feature encoder, before step 110, the above method may further include:

[0066] obtaining a second sample set;

[0067] Using the second sample set to train the first neural network to obtain a second neural network;

[0068] The third feature encoder is used as the first feature encoder.

[0069] The second sample set may include second facial images of Q persons and a label image corresponding to each second facial image, where Q is a positive integer and Q>N.

[0070] The first neural network can be a neural network to be trained, and the first neural network can include a third feature encoder and a classifier. Therefore, after the first neural network is trained to obtain the second neural network, the second neural network also includes a third feature encoder and a classifier. However, the parameters of the third feature encoder and the parameters of the classifier in the first neural network have not been optimized, while the parameters of the third feature encoder and the parameters of the classifier in the second neural network have been optimized and can be used directly.

[0071] In some embodiments of the present application, after training the first neural network to obtain the second neural network, the classifier in the second neural network can be removed, leaving only the third feature encoder, and the third feature encoder is used as the first feature encoder, or the parameters of the third feature encoder are applied to the first feature encoder.

[0072] In some embodiments of the present application, the specific process of training the first neural network using the second sample set may be: for each sample group in the second sample set, the sample group includes a face image and a label image corresponding to the face image, after the sample group is input into the first neural network, the features of the face image in the sample group are extracted using the third feature encoder in the first neural network, and then the extracted features are input into the classifier to obtain the classification corresponding to the face image, the classification is compared with the label image to obtain the loss function, and based on the loss function, the parameters of the third feature encoder and the parameters of the classifier are optimized until the loop stop condition is met, and the second neural network is obtained.

[0073] It should be noted that the above loss function can be a cross entropy loss function.

[0074] It should be noted that the structural diagram of the first neural network provided in the embodiment of the present application is as follows Figure 2 As shown, the first neural network is composed of a third feature encoder and a classifier, the third feature encoder includes 4 convolution blocks, each convolution block includes 64 convolution kernels of 3×3 size, followed by BatchNorm with a momentum value of 0.1, the activation function can be LeakyReLU, and the pooling can use the maximum value pooling with a window size of 2×2. In addition, the classifier can be composed of a fully connected layer and a normalization layer.

[0075] The reason why the third feature encoder includes four identical convolution kernels is that by using multiple convolution kernels of the same size to share parameters in the convolution layer, the number of parameters can be reduced, overfitting can be avoided, and the generalization ability of the model can be improved.

[0076] It should be noted that the third feature encoder in the second neural network obtained above can generate an embedded feature with a shape of 5×5×64.

[0077] It should be noted that, since the first neural network has 84×84 channels, the second sample set can be adjusted to a size of 84×84 before being input into the first neural network.

[0078] In an embodiment of the present application, the first neural network is trained using the acquired second sample set to obtain the second neural network, and then the third feature encoder in the second neural network is used as the first feature encoder. In this way, the precise first feature encoder can be used to better extract features from the first sample set, thereby improving the accuracy of face recognition.

[0079] In some embodiments of the present application, the existing face recognition methods may also include the following methods:

[0080] (1) Support vector machine. Support vector machine is a new hot topic in the field of statistical pattern recognition. It attempts to make the learning machine reach a compromise between empirical risk and generalization ability, thereby improving the performance of the learning machine. Support vector machine mainly solves a two-classification problem. Its basic idea is to try to transform a low-dimensional linear inseparable problem into a high-dimensional linear separable problem. However, general experimental results show that support vector machine has a good recognition rate, but it requires a large number of training samples, which is often unrealistic in practical applications. In addition, the training time of support vector machine is long, the implementation method is complex, and there is no unified theory for the function.

[0081] (2) Face recognition method based on eigenface: The eigenface method is an explicit principal component analysis face modeling, while some linear autoassociation and linear compression networks are implicit principal component analysis methods. They all represent the face as the weighted sum of some vectors, which are the main eigenvectors of the cross product matrix of the training set. In short, the eigenface method is a simple, fast and practical algorithm based on transformation coefficient features, but it has great limitations because it requires more training samples and is completely based on the statistical characteristics of image grayscale.

[0082] (3) Neural network face recognition method: This method has a high recognition rate, but requires more samples for training. In many applications, the number of samples is very limited, and the training of large neural networks consumes high computing and storage hardware performance.

[0083] (4) Elastic graph matching face recognition method: This method combines grayscale characteristics and geometric factors, allowing elastic deformation of the image during comparison, and has achieved good results in overcoming the impact of expression changes on recognition. At the same time, it no longer requires multiple samples for training a single person. The disadvantage of this method is that the training model requires a large amount of calculation and is slow.

[0084] (5) Face recognition method based on line segment Hausdorff distance: Experimental results show that LHD performs very well under different lighting conditions and postures, but it does not perform well in the case of large expressions.

[0085] In order to solve the above problem, in the embodiment of the present application, the second sample set can be divided into sample sets of multiple different scenes, and then a sample set of a scene is selected from the sample sets of multiple different scenes as the first sample set. That is, a group of episodes are randomly sampled from the second sample set, where the episode represents a sub-training set in a small sample scene sampled from the second sample set each time, which is used to simulate the training in a small sample scene. The small sample scene here is to train the model with fewer images so that the model can distinguish who a certain face image belongs to. In this way, fewer training samples can be used to train the model, thereby improving the generalization ability of the meta-model in the small sample image classification task.

[0086] In some embodiments of the present application, the process of training the first neural network to obtain the second neural network is a pre-training stage. After the pre-training stage, a meta-learning stage is performed, that is, the process of performing steps 110 to 170. The goal of the meta-learning is to improve the generalization performance by learning meta-knowledge from multiple tasks. In the N-way K-shot setting, the meta-learning model H(·|S) is trained by minimizing the N-way loss. To this end, a group of episodes need to be randomly sampled from the base class set (i.e., the second sample set). Each group of episodes contains K labeled samples of each class, that is, a total of N×K support samples for training and N×M query samples for testing. Even though each set contains only a small number of training support samples, the model H(·|S) shares the same parameters across multiple episodes. Therefore, training H(·|S) in multiple episodes helps reduce the model's demand for data. During the meta-training process, the validation set Dval is used to select the hyperparameters of H(·|S).

[0087] Step 130: Calculate a first relationship matrix between the support set and the first query image based on the first embedded feature and the second embedded feature, and calculate a second relationship matrix between the support set and the first query image based on the third embedded feature and the fourth embedded feature.

[0088] The first relationship matrix may be a relationship matrix between the support set and the first query image calculated based on the first embedded feature and the second embedded feature.

[0089] The first relationship matrix may be a relationship matrix between the support set and the first query image calculated based on the third embedding feature and the fourth embedding feature.

[0090] It should be noted that the relationship matrix represents the pixel correlation between the support image and the query image. For example, by calculating the similarity between the i-th pixel in the support image and the j-th pixel in the query image, the value of the (i, j) position in the relationship matrix can be obtained.

[0091] In some embodiments of the present application, in order to accurately determine the first relationship matrix, step 130 may specifically include:

[0092] According to the first embedded feature and the second embedded feature, the first relationship matrix between the support set and the first query image is calculated according to the following formula (1):

[0093]

[0094] Wherein, s(·,·) is a similarity function used to characterize the similarity between the first query image and each first face image in the support set, i∈{1,2,3…WH}, WH is the size information of the second embedded feature, and j∈{1,2,3…NKWH}, NKWH is the size information of the first embedded feature.

[0095] In the embodiment of the present application, according to the first embedded feature and the second embedded feature, the first relationship matrix between the support set and the first query image can be accurately obtained through the above formula (1).

[0096] In some embodiments of the present application, in order to accurately determine the second relationship matrix, step 130 may specifically include:

[0097] According to the third embedding feature and the fourth embedding feature, the second relationship matrix between the support set and the first query image is calculated according to the following formula (2):

[0098]

[0099] Among them, f v Convolutional layers of the convolutional block used to characterize the second feature encoder.

[0100] In some embodiments of the present application, in the second relationship matrix, each row element identifies an adaptive subspace relationship between each spatial position in the query image and all spatial positions in all support images.

[0101] In the embodiment of the present application, according to the third embedding feature and the fourth embedding feature, the second relationship matrix between the support set and the first query image can be accurately obtained through the above formula (2).

[0102] Step 140: Determine a task attention matrix based on the second relationship matrix.

[0103] In some embodiments of the present application, in order to eliminate noise, a threshold λ may be set for the second relationship matrix, and then based on the second relationship matrix, the task attention matrix may be obtained according to the following formulas (3) and (4):

[0104]

[0105] Where x is A i,j is an element in , and λ is the adaptive threshold of x.

[0106] In some embodiments of the present application, as shown in the above formula (3), in the entire task, most embedded features are often shared by multiple categories, which will greatly reduce the attention they receive. Therefore, their corresponding attention values ​​will also be relatively small. In addition, although the impact of each irrelevant noise is small, the number is large, and it still has a considerable impact on the distribution of task attention. For this reason, the above formula (4) is used to filter the task attention, which can screen out the relationships that can provide useful information and eliminate the negligible relationships.

[0107] Since the threshold λ in the above formula (4) is fixed, it cannot flexibly handle different query features. Therefore, the embodiment of the present application proposes a task encoder with an adaptive threshold λ', such as Figure 3 As shown, unlike the above formula (4), a converter consisting of three fully connected layers is used, which can adaptively generate a threshold λ' for each C-dimensional embedding feature of the query image. At the same time, considering that the above formula (4) is not differentiable, a variant of the sigmoid function M' can be used for fitting in the implementation of this application, as shown in the following formula (5):

[0108]

[0109] In the above formula (5), when β is large enough, M'(x) is equivalent to M(x) in the above formula (4).

[0110] In the embodiment of the present application, based on the second relationship matrix, according to the above formula (3), the task attention matrix can be accurately obtained.

[0111] Step 150: Merge the task attention matrix into the first relationship matrix by element-by-element multiplication to obtain a weighted relationship matrix.

[0112] In some embodiments of the present application, the task attention matrix can be merged into the first relationship matrix by element-by-element multiplication to obtain a weighted relationship matrix, that is, to obtain a weighted relationship matrix

[0113] Step 160: Based on the weighted relationship matrix and the first target facial image, obtain a prediction score that the first target facial image is the first object.

[0114] The first target face image may be any first face image in the first sample set.

[0115] In some embodiments of the present application, in order to accurately obtain the prediction score that the first target object face image is the first object, step 160 may specifically include:

[0116] Based on the weighted relationship matrix and the first target face image, the prediction score of the first target face image as the first object is obtained according to the following formula (6):

[0117]

[0118] Among them, A is the task attention matrix, R is the first relationship matrix, c is the category corresponding to the first object, It represents the tth relation of KWH relations belonging to class c among the NKWH relations in the support set, that is, the face image at the tth angle among the K face images at different angles belonging to the first object. i and j are the position information of each pixel in the first target face image.

[0119] In some embodiments of the present application, represents the tth relation of the KWH relations belonging to class c among the NKWH relations in the support set. Therefore, the final prediction probability of the first target face image can be expressed as shown in the following formula (7):

[0120]

[0121] In the above formula (7), N is the number of all categories under the N-way K-shot setting, that is, the number of N people.

[0122] In the embodiment of the present application, based on the weighted relationship matrix and the first target face image, according to the above formula (6), the prediction score of the first target face image being the first object can be accurately obtained.

[0123] Step 170: Obtain a meta-model for face recognition based on the prediction scores.

[0124] In some embodiments of the present application, after obtaining the prediction score, a meta-model for face recognition can be obtained based on the prediction score. Specifically, when the prediction score is greater than a certain threshold, for example, when the prediction score is greater than 0.6, it can be determined that the meta-model training is completed and the meta-model for face recognition is obtained.

[0125] In some embodiments of the present application, after obtaining the meta-model, that is, after the meta-training is completed, a new set Dnovel is usually used to evaluate the generalization performance of the trained model H(·|Sbase). All categories in Dnovel have not appeared in the meta-training stage. In the meta-testing stage, a new set of episodes is randomly extracted from Dnovel, also called the meta-testing set DtestT={Ti}I i=1. Here T is composed of Snovel and Qnovel. Then, given the new support set Snovel, the trained model H(·|Sbase) can be used to predict the category of the samples in Qnovel.

[0126] In some embodiments of the present application, after the meta-model is obtained, the meta-model can be used to recognize the face. Therefore, after step 170, the embodiment of the present application also provides a face recognition method, which can use the meta-model provided in the previous embodiment, such as Figure 4 As shown, the face recognition method provided in the embodiment of the present application may include steps 410 to 420.

[0127] Step 410: Obtain a face image to be recognized;

[0128] The face image to be identified may be an image to be subjected to face recognition.

[0129] Step 420: Input the face image to be recognized into the meta-model to obtain a target recognition result of the face image to be recognized.

[0130] The meta-model here may be a meta-model obtained by the model training method provided in the above embodiment.

[0131] The target recognition result may be a recognition result of a face image to be recognized, and specifically may be who the face image to be recognized belongs to.

[0132] In an embodiment of the present application, by inputting the acquired facial image to be identified into a pre-trained meta-model, a target recognition result of the facial image to be identified can be obtained based on the meta-model, so that the facial image can be accurately and quickly identified, thereby improving the efficiency and accuracy of facial recognition.

[0133] It should be noted that the model training method provided in the embodiment of the present application can be executed by a model training device or a control module in the model training device for executing the model training method. Correspondingly, the face recognition method provided in the embodiment of the present application can be executed by a face recognition device or a control module in the face recognition device for executing the face recognition method.

[0134] Based on the same inventive concept as the above-mentioned model training method, the present application also provides a model training device. Figure 5 The model training device provided in the embodiment of the present application is described in detail.

[0135] Figure 5 It is a structural schematic diagram of a model training device according to an exemplary embodiment.

[0136] like Figure 5 As shown, the model training device 500 may include:

[0137] A first acquisition module 510 is used to acquire a first sample set, where the first sample set includes first face images of N persons, and the first sample set includes a support set and a query set, where N is a positive integer;

[0138] A first determination module 520 is configured to input the first sample set into a first feature encoder to obtain first embedded features corresponding to the support set and second embedded features corresponding to the first query image, and input the first sample set into a second feature encoder to obtain third embedded features corresponding to the support set and fourth embedded features corresponding to the first query image, wherein the first query image is a first face image of a first object in the query set;

[0139] a second determining module 530, configured to calculate a first relationship matrix between the support set and the first query image according to the first embedding feature and the second embedding feature, and to calculate a second relationship matrix between the support set and the first query image according to the third embedding feature and the fourth embedding feature;

[0140] A third determination module 540 is used to determine a task attention matrix based on the second relationship matrix;

[0141] A fourth determination module 550 is used to merge the task attention matrix into the first relationship matrix by element-by-element multiplication to obtain a weighted relationship matrix;

[0142] a fifth determination module 560, configured to obtain a prediction score of the first target facial image being the first object based on the weighted relationship matrix and the first target facial image, wherein the first target facial image is any first facial image in the first sample set;

[0143] The sixth determination module 570 is used to obtain a meta-model for performing face recognition according to the prediction score.

[0144] In an embodiment of the present application, the first sample set is input into the first feature encoder to obtain the first embedding feature corresponding to the support set and the second embedding feature corresponding to the first query image, and the first sample set is input into the second feature encoder to obtain the third embedding feature corresponding to the support set and the fourth embedding feature corresponding to the first query image, and then the first relationship matrix between the support set and the first query image is calculated according to the first embedding feature and the second embedding feature, and the second relationship matrix between the support set and the first query image is calculated according to the third embedding feature and the fourth embedding feature, and the task attention matrix is ​​determined based on the second relationship matrix. The task attention matrix can adaptively extract the discriminative embedded features that support a certain query embedded feature in the focus, and merge the task attention matrix into the first relationship matrix through element-by-element multiplication. In this way, a series of noises such as the relationship formed by similar embedded features in the task can be eliminated, and the embedded features with discriminability can be strengthened to obtain a weighted relationship matrix. Based on the weighted relationship matrix and the first target face image, the prediction score of the first target face image as the first object is obtained. According to the prediction score, a meta-model for face recognition with noise eliminated is obtained. In this way, face recognition can be accurately performed based on the meta-model, thereby improving the accuracy and efficiency of human recognition.

[0145] In some embodiments of the present application, the above-mentioned device may further include:

[0146] A second acquisition module is used to acquire a second sample set, wherein the second sample set includes second face images of Q persons and a label image corresponding to each second face image, where Q is a positive integer;

[0147] a seventh determination module, configured to train the first neural network using the second sample set to obtain a second neural network, wherein the second neural network includes a third feature encoder and a classifier;

[0148] An eighth determining module is configured to use the third feature encoder as the first feature encoder.

[0149] In some embodiments of the present application, the first facial images of N persons include K facial images of each person at different angles, and the first sample set also includes label images of the K facial images of each person at different angles. The K facial images of each person at different angles and the label images are used as a support set, and the samples in the first sample set except the support set are used as a query set, and K is a positive integer.

[0150] In some embodiments of the present application, the second determining module 530 may specifically include:

[0151] A first determining unit is configured to calculate a first relationship matrix between the support set and the first query image according to the first embedding feature and the second embedding feature according to the following formula:

[0152]

[0153] Wherein, s(·,·) is a similarity function used to characterize the similarity between the first query image and each first face image in the support set, i∈{1,2,3…WH}, WH is the size information of the second embedded feature, and j∈{1,2,3…NKWH}, NKWH is the size information of the first embedded feature.

[0154] In some embodiments of the present application, the second determining module 530 may specifically include:

[0155] A second determining unit is configured to calculate a second relationship matrix between the support set and the first query image according to the third embedding feature and the fourth embedding feature according to the following formula:

[0156]

[0157] Among them, f v The convolutional layers of the convolutional block of the second feature encoder are used to characterize the second feature.

[0158] In some embodiments of the present application, the third determination module 540 may be specifically used to:

[0159] Based on the second relationship matrix, the task attention matrix is ​​determined according to the following formula:

[0160]

[0161] Where x is A i,j is an element in , λ is the adaptive threshold of x, When β is large enough, M'(x) is equivalent to M(x).

[0162] In some embodiments of the present application, the fifth determination module 560 may be specifically used to:

[0163] Based on the weighted relationship matrix and the first target face image, a prediction score that the first target face image is the first object is obtained according to the following formula:

[0164]

[0165] Wherein, A is the task attention matrix, R is the first relationship matrix, c is the category corresponding to the first object, It represents the tth relationship of the KWH relationships belonging to class c among the NKWH relationships in the support set, that is, the face image at the tth angle among the K face images at different angles belonging to the first object, and i and j are the position information of each pixel point in the first target face image.

[0166] The model training device provided in the embodiment of the present application can be used to execute the model training methods provided in the above-mentioned method embodiments. The implementation principles and technical effects are similar and will not be described here for the sake of brief introduction.

[0167] Based on the same inventive concept as the above-mentioned face recognition method, the present application also provides a face recognition device. Figure 6 The face recognition device provided in the embodiment of the present application is described in detail.

[0168] Figure 6 The figure is a schematic diagram showing the structure of a face recognition device according to an exemplary embodiment.

[0169] like Figure 6 As shown, the face recognition device 600 may include:

[0170] A first acquisition module 610 is used to acquire a face image to be recognized;

[0171] The first determination module 620 is used to input the face image to be recognized into the meta-model to obtain the target recognition result of the face image to be recognized; wherein the meta-model is the meta-model obtained by the model training method described in any one of claims 1-7.

[0172] In an embodiment of the present application, by inputting the acquired facial image to be identified into a pre-trained meta-model, a target recognition result of the facial image to be identified can be obtained based on the meta-model, so that the facial image can be accurately and quickly identified, thereby improving the efficiency and accuracy of facial recognition.

[0173] The face recognition device provided in the embodiment of the present application can be used to execute the face recognition methods provided in the above-mentioned method embodiments. The implementation principles and technical effects are similar and will not be described in detail here for the sake of simplicity.

[0174] Based on the same inventive concept, an embodiment of the present application also provides an electronic device.

[0175] Figure 7 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device may include a processor 701 and a memory 702 storing computer programs or instructions.

[0176] Specifically, the processor 701 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0177] The memory 702 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 702 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 702 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 702 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 702 is a non-volatile solid-state memory. The memory may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described in the model training method provided in the above-mentioned embodiments, and / or the operations described in the face recognition method.

[0178] The processor 701 implements any one of the model training methods and / or face recognition methods in the above embodiments by reading and executing computer program instructions stored in the memory 702.

[0179] In one example, the electronic device may further include a communication interface 703 and a bus 710. Figure 7 As shown, the processor 701, the memory 702, and the communication interface 703 are connected via a bus 710 and communicate with each other.

[0180] The communication interface 703 is mainly used to implement the communication between the modules, devices, units and / or devices in the embodiment of the present invention.

[0181] Bus 710 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 710 may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0182] The electronic device can execute the model training method and / or the face recognition method in the embodiment of the present invention, thereby realizing Figure 1 A description of the model training method, and / or, Figure 4 Describe the face recognition method.

[0183] In addition, in combination with the model training method and / or the face recognition method in the above embodiments, an embodiment of the present invention may provide a readable storage medium for implementation. The readable storage medium stores program instructions, and when the program instructions are executed by the processor, any one of the model training methods and / or the face recognition method in the above embodiments is implemented.

[0184] In addition, in combination with the model training method and / or face recognition method in the above embodiments, an embodiment of the present invention may provide a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes any one of the model training methods and / or face recognition methods in the above embodiments.

[0185] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0186] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0187] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0188] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0189] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A model training method, characterized in that: The method comprises: Acquire a first sample set, the first sample set including first face images of N persons, the first sample set including a support set and a query set, where N is a positive integer; Inputting the first sample set into a first feature encoder to obtain a first embedded feature corresponding to the support set and a second embedded feature corresponding to the first query image, and inputting the first sample set into a second feature encoder to obtain a third embedded feature corresponding to the support set and a fourth embedded feature corresponding to the first query image, wherein the first query image is a first face image of a first object in the query set; Calculating a first relationship matrix between the support set and the first query image according to the first embedding feature and the second embedding feature, and calculating a second relationship matrix between the support set and the first query image according to the third embedding feature and the fourth embedding feature; Based on the second relationship matrix, determining a task attention matrix; Merging the task attention matrix into the first relationship matrix by element-by-element multiplication to obtain a weighted relationship matrix; Based on the weighted relationship matrix and a first target facial image, obtaining a prediction score that the first target facial image is the first object, wherein the first target facial image is any first facial image in the first sample set; According to the prediction scores, a meta-model for performing face recognition is obtained.

2. The method according to claim 1, characterized in that: Before obtaining the first sample set, the method further includes: Acquire a second sample set, wherein the second sample set includes second facial images of Q persons and a label image corresponding to each of the second facial images, where Q is a positive integer; Using the second sample set to train the first neural network to obtain a second neural network, wherein the second neural network includes a third feature encoder and a classifier; The third feature encoder is used as the first feature encoder.

3. The method according to claim 1, characterized in that The first facial images of N persons include K facial images of each person at different angles. The first sample set also includes label images of the K facial images of each person at different angles. The K facial images of each person at different angles and the label images are used as a support set, and the samples in the first sample set except the support set are used as a query set, and K is a positive integer.

4. The method according to claim 3, characterized in that The step of calculating a first relationship matrix between the support set and the first query image according to the first embedding feature and the second embedding feature comprises: According to the first embedding feature and the second embedding feature, a first relationship matrix between the support set and the first query image is calculated according to the following formula: Wherein, s(·,·) is a similarity function used to characterize the similarity between the first query image and each first face image in the support set, i∈{1,2,3…WH}, WH is the size information of the second embedded feature, and j∈{1,2,3…NKWH}, NKWH is the size information of the first embedded feature.

5. The method according to claim 4, characterized in that The step of calculating a second relationship matrix between the support set and the first query image according to the third embedding feature and the fourth embedding feature comprises: According to the third embedding feature and the fourth embedding feature, a second relationship matrix between the support set and the first query image is calculated according to the following formula: Among them, f v The convolutional layers of the convolutional block of the second feature encoder are used to characterize the second feature.

6. The method according to claim 5, characterized in that The determining of the task attention matrix based on the first relationship matrix includes: Based on the second relationship matrix, the task attention matrix is ​​determined according to the following formula: Where x is A i,j is an element in , λ is the adaptive threshold of x, When β is large enough, M′(x) is equivalent to M(x).

7. The method according to claim 6, characterized in that The obtaining, based on the weighted relationship matrix and the first target face image, a prediction score of the first target face image being the first object includes: Based on the weighted relationship matrix and the first target face image, a prediction score that the first target face image is the first object is obtained according to the following formula: Wherein, A is the task attention matrix, R is the first relationship matrix, c is the category corresponding to the first object, It represents the tth relationship of the KWH relationships belonging to class c among the NKWH relationships in the support set, that is, the face image at the tth angle among the K face images at different angles belonging to the first object, and i and j are the position information of each pixel point in the first target face image.

8. A face recognition method, characterized in that: The method comprises: Obtain a face image to be recognized; Inputting the face image to be recognized into the meta-model to obtain a target recognition result of the face image to be recognized; Wherein, the metamodel is a metamodel obtained by the model training method described in any one of claims 1-7.

9. A model training device, characterized in that: The device comprises: A first acquisition module is used to acquire a first sample set, wherein the first sample set includes first face images of N persons, the first sample set includes a support set and a query set, and N is a positive integer; a first determination module, configured to input the first sample set into a first feature encoder to obtain first embedded features corresponding to the support set and second embedded features corresponding to the first query image, and input the first sample set into a second feature encoder to obtain third embedded features corresponding to the support set and fourth embedded features corresponding to the first query image, wherein the first query image is a first face image of a first object in the query set; a second determining module, configured to calculate a first relationship matrix between the support set and the first query image according to the first embedding feature and the second embedding feature, and to calculate a second relationship matrix between the support set and the first query image according to the third embedding feature and the fourth embedding feature; A third determination module, configured to determine a task attention matrix based on the second relationship matrix; a fourth determination module, configured to merge the task attention matrix into the first relationship matrix by element-by-element multiplication to obtain a weighted relationship matrix; a fifth determination module, configured to obtain a prediction score of the first target facial image being the first object based on the weighted relationship matrix and a first target facial image, wherein the first target facial image is any first facial image in the first sample set; The sixth determination module is used to obtain a meta-model for performing face recognition according to the prediction score.

10. A face recognition device, characterized in that: The device comprises: A first acquisition module is used to acquire a face image to be recognized; The first determination module is used to input the face image to be identified into the meta-model to obtain the target recognition result of the face image to be identified; wherein the meta-model is the meta-model obtained by the model training method described in any one of claims 1-7.

11. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of the model training method as described in any one of claims 1 to 7 and / or the steps of the face recognition method as described in claim 8 are implemented.

12. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps of the model training method described in any one of claims 1 to 7 and / or the steps of the face recognition method described in claim 8 are implemented.

13. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device performs the steps of the model training method described in any one of claims 1 to 7, and / or the steps of the face recognition method described in claim 8.