Model training method, face recognition method, electronic device, and storage medium

Abstract

Embodiments of the present application relate to the field of face recognition, and disclose a model training method, a face recognition method, an electronic device and a storage medium. The model training method comprises: constructing a feature extraction model for extracting face features from face image samples; an attention mechanism module is arranged between adjacent network layers of the feature extraction model; the attention mechanism module is used to form a weight matrix according to a first feature map output by a previous network layer in the adjacent network layers, and output a second feature map obtained by processing the first feature map using the weight matrix to a subsequent network layer in the adjacent network layers; a third feature map obtained by processing the first feature map using a weight matrix complementary to the weight matrix is taken as input to construct a feature classification model, and an output of the feature classification model is a preset information category affecting identity recognition of the face image samples; and the feature extraction model and the feature classification model are jointly trained.

Description

Technical Field

[0001] This invention relates to the field of face recognition, and particularly to a model training method, a face recognition method, an electronic device, and a storage medium. Background Technology

[0002] With the development and implementation of facial recognition technology, the requirements for facial recognition systems are becoming increasingly stringent, such as accurate facial recognition under various facial changes, including wearing masks, glasses, accessories, makeup, and lighting conditions. Currently, there are generally two methods for facial recognition of changes caused by various factors: The first is cooperative detection, which first assesses the integrity of the face. If incomplete facial information is detected, prompts (turn left, turn right, look down, look up, remove mask, adjust hat, etc.) guide the face to adjust to a state that meets the recognition criteria before recognition. The second is non-cooperative detection, which involves building a facial recognition model and then switching to different models based on the current facial recognition scenario.

[0003] However, the first method requires user cooperation to adjust the face to a normal, suitable form for detection, and cannot directly recognize face images in various scenarios, limiting its applicability and resulting in a poor user experience. The second method has high data acquisition costs and long training times, requiring multiple face recognition models from different scenarios to collaboratively solve face recognition under different facial changes, making the overall recognition model more complex and limiting its generalization ability. Summary of the Invention

[0004] The purpose of this invention is to provide a model training, face recognition method, electronic device, and storage medium. Based on distillation loss, first classification loss, and second classification loss, a feature extraction model and a feature classification model with an attention mechanism module are jointly trained. This results in a feature extraction model that is more robust to factors that change facial appearance but not identity, such as glasses, pose, lighting, makeup, and facial accessories. The model is also more lightweight and has better generalization ability.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a model training method, comprising: constructing a feature extraction model for extracting facial features from facial image samples; setting an attention mechanism module between adjacent network layers of the feature extraction model; the attention mechanism module being used to form a weight matrix based on a first feature map output by the preceding network layer in the adjacent network layers, and outputting a second feature map, processed by the weight matrix, of the first feature map to the following network layer in the adjacent network layers; using a third feature map, processed by a weight matrix complementary to the weight matrix, as input to construct a feature classification model, wherein the output of the feature classification model is a preset information category affecting the identity recognition of the facial image sample; the two complementary weight matrices are of the same type, and the sum of corresponding elements is 1; jointly training the feature extraction model and the feature classification model, wherein the loss function of the joint training is constructed based on the distillation loss between the facial features output by the pre-trained teacher model and the facial features output by the feature extraction model, a first classification loss corresponding to the predicted facial category output by the classifier of the facial features output by the feature extraction model, and a second classification loss corresponding to the predicted information category output by the feature classification model.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a face recognition method, comprising: inputting a face image to be tested into a pre-trained feature extraction model to obtain face features of the face image to be tested; wherein the feature extraction model is obtained through the model training method described above; and recognizing the face image to be tested based on the obtained face features to obtain a recognition result.

[0007] Embodiments of the present invention also provide an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the model training method mentioned in the above embodiments, or to perform the face recognition method mentioned in the above embodiments.

[0008] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the model training method mentioned in the above embodiments, or is capable of executing the incomplete face recognition method mentioned in the above embodiments.

[0009] The model training method provided in this invention jointly trains a feature extraction model and a feature classification model equipped with an attention mechanism module based on distillation loss, a first classification loss, and a second classification loss. During training, the input to the feature extraction model is a face image sample, and the output is face features. The input to the feature classification model is a third feature map obtained by processing a matrix of the same type complementary to the weight matrix of the second feature map, and the output is a preset information category affecting the identification of face image samples. In other words, this application extracts face features related to the identification of face image samples through the feature extraction model, and extracts and identifies face features that affect the identification of face image samples through the feature classification model. For features related to sample identity recognition and their categories (e.g., glasses, accessories, bangs, masks, facial pose, lighting, makeup), during joint training, the feature extraction model learns the classification boundaries between features related to identity recognition and features affecting identity recognition more accurately through distillation loss, first classification loss, and second classification loss. This improves the robustness of the feature extraction model to factors that change appearance but not identity, such as glasses, accessories, bangs, masks, facial pose, lighting, and makeup. Furthermore, since the feature extraction model is jointly trained with the teacher model and the feature classification model, the model has lower complexity and better generalization ability.

[0010] Furthermore, the model training method provided in this embodiment of the invention includes multiple attention mechanism modules located between adjacent network layers of the feature extraction model. The feature classification model includes multiple processing modules corresponding one-to-one with the attention mechanism modules, a fusion network, and a prediction network. Each attention mechanism module inputs the third feature map, processed by a weight matrix complementary to the weight matrix generated by the attention mechanism module, into the corresponding processing module to obtain a fourth feature map of the same preset size. The fourth feature maps output by each processing module are input into the fusion network to obtain a fused feature map. The fused feature map is then input into the prediction network to obtain the information category corresponding to the face image sample. The feature classification model of this application includes multiple processing modules corresponding one-to-one with the multiple attention mechanism modules, a fusion network, and a prediction network. By performing multi-label learning on features affecting identity recognition through the feature classification model, the face features output by the feature extraction model are highly correlated only with personal identity information during joint training, reducing the influence of other factors on identity recognition.

[0011] Furthermore, the model training method provided in this embodiment includes a first-stage joint training and a second-stage joint training. During the first-stage joint training of the feature extraction model and the feature classification model, the weight of the distillation loss is greater than the weight of the first classification loss, and the weight of the first classification loss is greater than the weight of the second classification loss. During the second-stage joint training of the feature extraction model and the feature classification model, the weight of the first classification loss is greater than the weight of the second classification loss, and the weight of the second classification loss is greater than the weight of the distillation loss. In the first-stage joint training, the weight of the distillation loss is set to the maximum to improve the network convergence speed in the initial training stage, enabling the feature extraction model to possess initial feature extraction capabilities. In the second stage, the weight of the first classification loss is set to the maximum, and the weight of the second classification loss is set to the second largest, thereby optimizing the feature extraction model towards the combined effect of the first and second classification losses. This results in better robustness and generalization ability of the feature extraction model to factors that alter facial appearance without changing identity, such as glasses, accessories, bangs, masks, facial pose, lighting, and makeup. Attached Figure Description

[0012] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0013] Figure 1 This is a flowchart of the model training method provided by an embodiment of the present invention;

[0014] Figure 2 This is a schematic diagram of the feature extraction model training process provided by an embodiment of the present invention;

[0015] Figure 3 This is a flowchart of a face recognition method provided by an embodiment of the present invention;

[0016] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of the present invention to enable the reader to better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments.

[0018] The following provides an example illustrating the implementation details of model training in this embodiment. The following implementation details are provided for ease of understanding only and are not essential for implementing this solution.

[0019] The embodiments of the present invention relate to a model training method, such as... Figure 1 As shown, it includes:

[0020] Step 101: Construct a feature extraction model for extracting facial features from facial image samples; an attention mechanism module is set between adjacent network layers of the feature extraction model; the attention mechanism module is used to form a weight matrix based on the first feature map output by the previous network layer in the adjacent network layer, and output the second feature map after the first feature map is processed by the weight matrix to the next network layer in the adjacent network layer.

[0021] In this embodiment, the feature extraction model is mainly built based on an extraction module and an attention mechanism module. The extraction module includes multiple cascaded network blocks of various types, such as ResidualNet-Block (a network block in a residual network) and DenseNet-Block (a network block in a convolutional neural network). That is, this application can build the network architecture of the extraction module based on the feature extraction part of various types of networks. The extraction module is used to extract a first feature map from face image samples. The attention mechanism module is connected after the extraction module and is used to set corresponding weights for each channel of the first feature map to obtain a second feature map. The weights of each channel of the first feature map constitute a weight matrix. The network structure of the attention mechanism module can be any of the existing attention mechanism network architectures such as Squeeze-and-Excitation (SE module) in SENet, Convolutional Block Attention Module (CBAM module), Pyramid Split Attention Module (PSA module), etc.

[0022] It should be noted that this embodiment only describes the modules that play a major role and are creative in the feature extraction model, but does not mean that it does not include other modules or network layers used to assist the extraction module and attention mechanism module in completing feature extraction, such as activation layers, pooling layers, fully connected layers, batch normalization layers, etc.

[0023] Step 102: The third feature map, processed by the first feature map with a weight matrix complementary to the weight matrix, is used as input to construct a feature classification model. The output of the feature classification model is a preset information category that affects the identity recognition of face image samples. The two complementary weight matrices are matrices of the same type, and the sum of their corresponding elements is 1.

[0024] In this embodiment, a weight W is set for each channel of the first feature map F through the attention mechanism module to obtain the second feature map F+ (i.e., F*W=F+), and the second feature map is input into the next network layer. In this way, during the training process, the features extracted by the subsequent network are more highly related to the face recognition task (i.e., features highly related to identity information are extracted). At the same time, the third feature map F- (i.e., F*(1-W)=F-) after the first feature map is processed by a weight matrix complementary to the weight matrix is ​​input into the feature classification model. In this way, the feature classification model performs multi-label learning on features that are not highly related to the face recognition task (i.e., features that are not related to face identity information).

[0025] Specifically, pre-defined information affecting the identification of facial image samples can include factors such as wearing glasses, makeup, lighting, facial pose, bangs, and accessories. Of course, before inputting facial image samples into the feature extraction model for training, each facial image needs to be labeled to obtain a labeled image. The label information includes identity information and pre-defined information (i.e., information that affects facial image identification or information unrelated to identity). Identity information can be facial feature information, facial bone structure information, iris information, etc.

[0026] For example, when the preset information affecting the identification of a facial image sample includes glasses, makeup, and accessories, the pre-defined rule for setting the label information is: wearing glasses is labeled as 1, not wearing glasses as 0; makeup is labeled as 1, not wearing makeup as 0; wearing accessories is labeled as 1, not wearing accessories as 0. If the face in facial image A wears glasses, is not wearing makeup, and is wearing accessories, then the label information for facial image A is 101. Of course, the specific representation method and setting rules of the label information can be flexibly adjusted according to the recognition needs. Furthermore, the method for labeling facial images can be manual or automatic labeling through machine learning, which will not be elaborated upon here.

[0027] In one embodiment, processing the first feature map using the weight matrix includes: processing the feature maps of each channel in the first feature map using the weight coefficients corresponding to each channel in the weight matrix to obtain a second feature map; processing the first feature map using a weight matrix complementary to the weight matrix includes: processing the feature maps of each channel in the first feature map using the weight coefficients corresponding to each channel in the complementary weight matrix to obtain a third feature map.

[0028] In this embodiment, the number of weight coefficients in the weight matrix is ​​equal to the number of channels in the first feature map. For example, assuming the number of channels in the first feature map is 6, the weight matrix of the second feature map obtained by processing the first feature map is [0.8, 0.75, 0.7, 0.3, 0.2, 0.6], and the weight matrix of the third feature map obtained by processing the first feature map is [0.2, 0.25, 0.3, 0.7, 0.8, 0.4]. Each weight coefficient corresponds to one channel. A large weight coefficient indicates that the features of that channel are highly correlated with the face recognition task (i.e., highly correlated with identity information), while a small weight coefficient indicates that the features of that channel are less correlated with the face recognition task (i.e., affect the identity recognition of the face image).

[0029] Step 103: Jointly train the feature extraction model and the feature classification model. The loss function for joint training is constructed based on the distillation loss between the face features output by the pre-trained teacher model and the face features output by the feature extraction model, the first classification loss corresponding to the predicted face category output by the classifier of the face features output by the feature extraction model, and the second classification loss corresponding to the predicted information category output by the feature classification model.

[0030] In this embodiment, the teacher model is essentially a pre-trained, complete feature extraction model. Generally, the teacher model is larger, more complex, and takes longer to train than the feature extraction model used as the student model. The network structure of the teacher model can be any network structure that can be used for feature extraction, such as the vision transformer structure, the traditional CNN structure, or the ResidualNet network.

[0031] Understandably, current facial recognition systems or models typically compare the facial features extracted from the image of the person being tested with the facial features stored in the system to determine the identity of the person. However, changes in the face of the person being tested can easily lead to a failure in comparing the extracted facial features with the stored features. For example, if the person has changed their hairstyle, shaved, is wearing glasses, or is in a large-angle pose, these scenarios can cause a decrease in the accuracy of existing facial recognition systems, or even prevent them from recognizing the person. Therefore, a feature extraction model is needed that extracts only features highly correlated with identity information, thus ensuring that subsequent facial recognition is not affected by information unrelated to identity recognition. Features highly correlated with identity information can include facial features, facial skeletal structure, iris features, and so on.

[0032] In the process of jointly training the feature extraction model and the feature classification model, this application uses distillation loss to make the facial features output by the feature extraction model as close as possible to the facial features output by the teacher model, uses the first classification loss to make the predicted face category output by the classifier of the facial features output by the feature extraction model as close as possible to the real face category of the face image, and uses the second classification loss to make the predicted information category output by the feature classification model as close as possible to the real information category of the face image.

[0033] In other words, this application enables the feature extraction model to quickly acquire initial feature extraction capabilities through a teacher model and distillation loss. Through a feature classification model, first classification loss, and second classification loss, the feature extraction model learns more accurately the classification boundaries between features related to identity recognition and features that affect identity recognition, so that the final facial features obtained are only highly correlated with factors that can represent personal identity.

[0034] In other words, the training process of the feature extraction model can be understood as both direct supervised training through labels representing identity information and indirect supervised training through labels representing preset information (information that affects identity recognition). The combination of the two makes facial features more robust and reduces interference from other factors.

[0035] In another embodiment, multiple attention mechanism modules are arranged between adjacent network layers of multiple groups in the feature extraction model. The feature classification model includes multiple processing modules corresponding one-to-one with the attention mechanism modules, a fusion network, and a prediction network. The first feature map is processed by a third feature map (complemented by a weight matrix complementary to the weight matrix) as input, and a preset information category affecting the identification of face image samples is used as output to construct the feature classification model. This includes: each attention mechanism module inputs its corresponding first feature map, processed by a third feature map (complemented by a weight matrix complementary to the weight matrix generated by the attention mechanism module), into the corresponding processing module to obtain a fourth feature map of the same preset size; the fourth feature maps output by each processing module are input into the fusion network to obtain a fused feature map; and the fused feature map is input into the prediction network to obtain the information category corresponding to the face image sample.

[0036] In this embodiment, as Figure 2 As shown, a feature extraction model can include multiple extraction modules and multiple attention mechanism modules. In a feature classification model, the processing modules correspond one-to-one with the attention mechanism modules. It can be understood that the multiple extraction modules are cascaded; the more cascaded modules, the deeper the features can be extracted. Therefore, the feature maps output by different extraction modules have different sizes. Figure 3Taking a feature extraction model as an example, assuming the size of a face image is 224*224, the feature map output by the first extraction module is 28*28, and the feature map output by the second extraction module is 14*14. This means that feature maps at different levels have different sizes; shallower feature maps are larger, and deeper feature maps are smaller. Therefore, before fusing or stitching feature maps of different sizes, the processing module needs to unify the size of multiple third feature maps of different sizes to obtain a fourth feature map of the same preset size. Then, each processing module inputs the fourth feature map into the fusion network to obtain a fused feature map, and the prediction network processes the fused feature map to obtain the corresponding preset information category.

[0037] Specifically, the processing module includes: a 3*3 convolutional layer, a batch normalization layer, a ReLU layer, a global average pooling layer, a 1*1 convolutional layer, and a fully connected layer.

[0038] Additionally, it should be noted that the location of the attention mechanism module is highly related to the input and structure of the feature extraction model, and is not unique. Figure 3 The feature extraction model shown is merely an exemplary structure. Figure 2 For a 224*224 input face, the optimal placement is after the last cascaded network block of the feature map modules of sizes 28*28, 14*14, and 7*7. Considering the weak feature extraction capability of shallow layers of the network, it is not recommended to connect the attention mechanism module after a feature map of size 56*56 or larger, otherwise it may affect the multi-label classification effect.

[0039] In another embodiment, joint training includes: a first-stage joint training and a second-stage joint training; when performing the first-stage joint training on the feature extraction model and the feature classification model, the weight of the distillation loss is greater than the weight of the first classification loss, and the weight of the first classification loss is greater than the weight of the second classification loss; when performing the second-stage joint training on the feature extraction model and the feature classification model, the weight of the first classification loss is greater than the weight of the second classification loss, and the weight of the second classification loss is greater than the weight of the distillation loss.

[0040] Specifically, the joint loss function is calculated as follows: First weight * Distillation loss + First weight * First classification loss + Third weight * Second classification loss. During the first training phase, the weight of the distillation loss is greater than the weight of the first classification loss, and the weight of the first classification loss is greater than the weight of the second classification loss. The purpose of adding the distillation loss is to enable the network to converge quickly and acquire initial feature extraction capabilities. The first classification loss can be achieved using arcface_loss, cosface_loss, etc.

[0041] In the second training phase, the weight of the first classification loss is greater than the weight of the second classification loss, and the weight of the second classification loss is greater than the weight of the distillation loss. It should be noted that at this time, there is a significant difference between the facial features output by the feature extraction model and the facial features output by the teacher model. In order not to reduce the effectiveness of joint training, the weight of the distillation loss should be significantly reduced in this phase, or even set to 0, so that the network optimizes in the direction of the combined effect of the first classification loss and the second classification loss.

[0042] Furthermore, the distillation loss in this embodiment is constructed using the following formula:

[0043]

[0044] Among them, y i The facial features output by the teacher model, y i * Let N be the facial features output by the feature extraction model, and N be the number of facial image samples.

[0045] Furthermore, the first classification loss is constructed using the following formula:

[0046]

[0047] Where, p im y represents the predicted probability of each face category to which the face features output by the classifier belong. im denoted as k, which represents the true probability of a face feature belonging to each face category, and k represents the number of face categories.

[0048] Furthermore, the second classification loss is constructed using the following formula:

[0049]

[0050] Where N represents the number of sample face images, K represents the number of categories of the preset information, and p ic y represents the predicted probability that the i-th face image sample belongs to the preset information category c. ic Let be the true probability that the i-th face image sample belongs to the preset information category c.

[0051] The model training method provided in this invention jointly trains a feature extraction model and a feature classification model equipped with an attention mechanism module based on distillation loss, a first classification loss, and a second classification loss. During training, the input to the feature extraction model is a face image sample, and the output is face features. The input to the feature classification model is a third feature map obtained by processing a matrix of the same type complementary to the weight matrix of the second feature map, and the output is a preset information category affecting the identification of face image samples. In other words, this application extracts face features related to the identification of face image samples through the feature extraction model, and extracts and identifies face features that affect the identification of face image samples through the feature classification model. For features related to sample identity recognition and their categories (e.g., glasses, accessories, bangs, masks, facial pose, lighting, makeup), during joint training, the feature extraction model learns the classification boundaries between features related to identity recognition and features affecting identity recognition more accurately through distillation loss, first classification loss, and second classification loss. This improves the robustness of the feature extraction model to factors that change appearance but not identity, such as glasses, accessories, bangs, masks, facial pose, lighting, and makeup. Furthermore, since the feature extraction model is jointly trained with the teacher model and the feature classification model, the model has lower complexity and better generalization ability.

[0052] The embodiments of the present invention relate to a face recognition method, such as... Figure 3 As shown, it includes:

[0053] Step 301: Input the face image to be tested into the pre-trained feature extraction model to obtain the face features of the face image to be tested; wherein, the feature extraction model is obtained by the model training method described in the above implementation method.

[0054] In this embodiment, the face image to be tested can be a face image from any scene. The feature extraction model obtained through the above training method is more robust to factors that change facial appearance but not identity, such as glasses, pose, lighting, makeup, and facial accessories. The extracted facial features are only highly correlated with factors that can represent an individual's identity, thus improving the accuracy and attack security of the face recognition system.

[0055] Step 302: Recognize the face image to be tested based on the acquired facial features, and obtain the recognition result.

[0056] Specifically, after extracting facial features, these features are input into a pre-trained face classifier to recognize the face image and obtain the recognition result. The face classifier can be a nearest neighbor classifier, a linear classifier, etc., and the specific model structure of the classifier is not limited here.

[0057] The face recognition method provided in this embodiment improves the accuracy and attack security of the face recognition system because facial features are more robust to factors that alter facial appearance without changing identity, such as glasses, pose, lighting, makeup, and facial accessories. Furthermore, the overall model has low complexity, a small number of parameters, and is universally applicable, especially for lightweight face recognition models deployed on embedded platforms. In addition, the model of this application can be used not only for face recognition tasks but also transferred to other face-related tasks, such as liveness detection methods.

[0058] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0059] Embodiments of the present invention relate to an electronic device, such as... Figure 4 As shown, it includes:

[0060] At least one processor 401; and a memory 402 communicatively connected to at least one processor 401; wherein the memory 402 stores instructions executable by at least one processor 401, the instructions being executed by at least one processor 401 to enable at least one processor 401 to perform the model training method mentioned in the above embodiments, or to perform the model training method mentioned in the above embodiments, or to perform the face recognition method mentioned in the above embodiments.

[0061] The electronic device includes: one or more processors 401 and memory 402. Figure 4 Taking a processor 401 as an example, the processor 401 and the memory 402 can be connected via a bus or other means. Figure 4 Taking a bus connection as an example, memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. For example, in this embodiment, the algorithms corresponding to each processing strategy in the strategy space are stored in memory 402. Processor 401 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in memory 402, thereby implementing the above-mentioned model training method or face recognition method.

[0062] Memory 402 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store an option list, etc. Furthermore, memory 402 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 402 may optionally include memory remotely located relative to processor 401, and these remote memories can be connected to external devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0063] One or more modules are stored in memory 402 and, when executed by one or more processors 401, can perform the model training method mentioned in the above embodiments, or can perform the face recognition method mentioned in the above embodiments.

[0064] The above-mentioned products can perform the methods provided in the embodiments of this application, and have the corresponding functional modules and beneficial effects of performing the methods. For technical details not described in detail in this embodiment, please refer to the methods provided in the embodiments of this application.

[0065] Embodiments of the present invention relate to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the above-described method embodiments.

[0066] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0067] Those skilled in the art will understand that the above embodiments are specific examples of implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.

Claims

1. A model training method, characterized in that, include: Construct a feature extraction model to extract facial features from facial image samples; An attention mechanism module is set between adjacent network layers of the feature extraction model; The attention mechanism module is used to form a weight matrix based on the first feature map output by the previous network layer in the adjacent network layer, and output the second feature map after the first feature map is processed by the weight matrix to the next network layer in the adjacent network layer. The first feature map is processed by a third feature map, which is a weight matrix complementary to the first weight matrix, and then used as input to construct a feature classification model. The output of the feature classification model is a preset information category that affects the identity recognition of the face image sample. The two complementary weight matrices are matrices of the same type, and the sum of their corresponding elements is 1. The feature classification model is equipped with a processing module for processing the input third feature map, and the feature classification model outputs predicted information category based on the processing result; The feature extraction model and the feature classification model are jointly trained. The loss function of the joint training is constructed based on the distillation loss between the face features output by the pre-trained teacher model and the face features output by the feature extraction model, the first classification loss corresponding to the predicted face category output by the classifier of the face features output by the feature extraction model, and the second classification loss corresponding to the predicted information category output by the feature classification model.

2. The model training method according to claim 1, characterized in that, Multiple attention mechanism modules are set between adjacent network layers of multiple groups in the feature extraction model. The feature classification model includes multiple processing modules that correspond one-to-one with the attention mechanism modules, as well as a fusion network and a prediction network. The step of using a third feature map, processed by a weight matrix complementary to the first feature map and then processed by the weight matrix, as input to construct a feature classification model includes: Each attention mechanism module inputs the corresponding first feature map, processed by a weight matrix complementary to the weight matrix generated by the attention mechanism module, into the corresponding processing module to obtain a fourth feature map of the same preset size. The fourth feature map output by each of the processing modules is input into the fusion network to obtain a fused feature map; The fused feature map is input into the prediction network to obtain the preset information category corresponding to the face image sample.

3. The model training method according to claim 1 or 2, characterized in that, The joint training includes: a first-stage joint training and a second-stage joint training; when performing the first-stage joint training on the feature extraction model and the feature classification model, the weight of the distillation loss is greater than the weight of the first classification loss, and the weight of the first classification loss is greater than the weight of the second classification loss. When performing the second-stage joint training of the feature extraction model and the feature classification model, the weight of the first classification loss is greater than the weight of the second classification loss, and the weight of the second classification loss is greater than the weight of the distillation loss.

4. The model training method according to claim 1 or 2, characterized in that, The step of processing the first feature map using the weight matrix includes: The feature maps of each channel in the first feature map are processed using the weight coefficients corresponding to each channel in the weight matrix to obtain the second feature map; The step of processing the first feature map with a weight matrix complementary to the weight matrix includes: The feature maps of each channel in the first feature map are processed using the weight coefficients corresponding to each channel in the complementary weight matrix to obtain the third feature map.

5. The model training method according to claim 1, characterized in that, The distillation loss is constructed using the following formula: Among them, y i The facial features output by the teacher model, y i * The facial features output by the feature extraction model are N, where N is the number of facial image samples.

6. The model training method according to claim 1, characterized in that, The first classification loss is constructed using the following formula: Where, p im y represents the predicted probability of the face feature belonging to each face category output by the classifier. im denoted as k, which represents the true probability of each face category to which the face feature belongs, and k represents the number of face categories.

7. The model training method according to claim 1, characterized in that, The second classification loss is constructed using the following formula: Where N represents the number of sample face images, K represents the number of categories of the preset information, and p ic y represents the predicted probability that the i-th face image sample belongs to the preset information category c. ic Let be the true probability that the i-th face image sample belongs to the preset information category c.

8. A face recognition method, characterized in that, include: The face image to be tested is input into a pre-trained feature extraction model to obtain the face features of the face image to be tested; wherein, the feature extraction model is obtained by the model training method as described in any one of claims 1 to 7 above; The face image to be tested is identified based on the acquired facial features, and the identification result is obtained.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the model training method as described in any one of claims 1 to 7, or to perform the face recognition method as described in claim 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the model training method of any one of claims 1 to 7, or the face recognition method of claim 8.

Images