Training methods, recognition methods, devices and equipment for heterogeneous face recognition models

Abstract

This invention relates to a training method, recognition method, apparatus, and device for a heterogeneous face recognition model, belonging to the field of image processing. The method involves inputting at least two modalities of face images of various target objects from a sample set into a pre-trained recognition model, outputting feature vectors corresponding to at least two modalities of the face images; determining a first prototype vector based on the feature vectors and initializing classifier parameters to obtain an initial heterogeneous face recognition model; iteratively executing the following steps until a final recognition model is obtained: inputting a preset number of cross-modal difficult sample pairs into a feature extractor obtained in the previous iteration, outputting two feature vectors for each cross-modal difficult sample pair; determining a second prototype vector based on any feature vector and the first prototype vector; constructing a model loss function based on the other feature vector and the second prototype vector; and adjusting the model parameters obtained in the previous iteration. In other words, during model training, cross-modal information is added to the prototypes to improve model accuracy.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a training method, recognition method, apparatus, and device for a heterogeneous face recognition model. Background Technology

[0002] In face recognition scenarios, near-infrared light is often used for spectral imaging to cope with the impact of environmental changes such as lighting on face recognition performance. However, pre-registered face images are usually composed only of visible light images. Therefore, how to achieve heterogeneous face recognition between visible light images and near-infrared light images has become a very important issue.

[0003] In related technologies, face recognition is usually performed using trained face recognition models. However, on the one hand, there is a lack of sufficient paired cross-modal face image training data, and collecting large-scale visible light face images and near-infrared face images is almost an impossible task. On the other hand, visible light face images and near-infrared face images come from different modalities and have inherent differences in data distribution, resulting in low accuracy of the trained heterogeneous face recognition models. Summary of the Invention

[0004] The embodiments of the present invention provide a training method, recognition method, apparatus and device for a heterogeneous face recognition model, so as to solve the technical problem of low training accuracy of heterogeneous face recognition models in related technologies.

[0005] In a first aspect, embodiments of the present invention provide a training method for a heterogeneous face recognition model. The method includes: acquiring a pre-trained heterogeneous face recognition model, the pre-trained heterogeneous face recognition model including a feature extractor and a classifier; determining at least two modal face images corresponding to various target objects in a first face image sample set, and inputting the at least two modal face images into the feature extractor of the pre-trained heterogeneous face recognition model, outputting face feature vectors corresponding to the at least two modal face images; determining a first prototype vector of the target object based on the face feature vectors corresponding to the at least two modal face images, and initializing classifier parameters based on the first prototype vector, the feature extractor and the initialized classifier constituting an initial heterogeneous face recognition model; iteratively executing the following steps for the initial heterogeneous face recognition model until a preset condition is met, and determining the current heterogeneous face recognition model. The final heterogeneous face recognition model is constructed as follows: A preset number of cross-modal hard sample pairs are input into the feature extractor of the heterogeneous face recognition model obtained in the previous iteration, and the first and second face feature vectors of each cross-modal hard sample pair are output. The cross-modal hard sample pair refers to the two modal face images with the lowest similarity for the same target object in the first face image sample set. A second prototype vector is determined based on any face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the first prototype vector. A model loss function is constructed based on the other face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the second prototype vector. The feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration are adjusted based on the model loss function to obtain the current heterogeneous face recognition model.

[0006] In some embodiments, before obtaining the pre-trained heterogeneous face recognition model, the method further includes: training the heterogeneous face recognition model to be trained based on a second face image sample set to obtain the pre-trained heterogeneous face recognition model; wherein the second face image sample set includes single-modal face images of the target object.

[0007] In some embodiments, before inputting a preset number of cross-modal hard sample pairs into the feature extractor of the heterogeneous face recognition model obtained in the previous iteration, the method further includes: randomly selecting a first modal face image of the target object from the first face image sample set, and selecting at least one second modal face image corresponding to the target object; determining the second modal face image with the smallest similarity to the first modal face image, wherein the first modal face image and the second modal face image with the smallest similarity constitute the cross-modal hard sample pair.

[0008] In some embodiments, determining the second modality face image with the lowest similarity to the first modality face image includes: inputting the first modality face image and at least one second modality face image into a pre-trained heterogeneous face recognition model, outputting corresponding first modality face feature vectors and at least one second modality face feature vector; calculating the cosine distance between the first modality face feature vector and each second modality face feature vector, and determining the second modality face image with the lowest similarity to the first modality face image based on the cosine distance.

[0009] In some embodiments, determining the second prototype vector based on any face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the first prototype vector includes: weighted summing of any face feature vector from each cross-modal hard sample pair and the first prototype variable to obtain the second prototype vector.

[0010] In some embodiments, the step of constructing a model loss function based on another face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the second prototype vector, and adjusting the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration based on the model loss function includes: determining a single-sample loss function based on another face feature vector and the second prototype vector of each cross-modal hard sample pair; calculating the mean of the single-sample loss functions of a preset number of cross-modal hard sample pairs, and determining the mean as the model loss function; and adjusting the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration based on the model loss function using stochastic gradient descent to obtain the current heterogeneous face recognition model.

[0011] In some embodiments, the formula for the single-sample loss function is as follows:

[0012]

[0013] Where L represents the single-sample loss function, x′ i x i Let y represent any face feature vector and the other face feature vector of the i-th cross-modal hard sample pair, respectively. i This represents the target object category corresponding to the i-th cross-modal hard sample pair. Indicates that the target object category is y i The corresponding first prototype variable, T represents the number of target object categories, N represents the preset quantity, and λ1 and λ2 represent weighting parameters.

[0014] Secondly, embodiments of the present invention provide a face recognition method, the method being based on a final heterogeneous face recognition model obtained by the training method of the heterogeneous face recognition model described in the first aspect; the method includes: acquiring a face image to be recognized; inputting the face image to be recognized into a feature extractor of the final heterogeneous face recognition model, and outputting a face feature vector to be recognized; comparing the face feature vector to be recognized with a face feature vector of a pre-registered identity, and identifying the identity information corresponding to the face image to be recognized.

[0015] Thirdly, embodiments of the present invention provide a training apparatus for a heterogeneous face recognition model, comprising an initial model module and a model fine-tuning module; wherein, the initial model module is used to acquire a pre-trained heterogeneous face recognition model, the pre-trained heterogeneous face recognition model comprising a feature extractor and a classifier; determine at least two modal face images corresponding to various target objects in a first face image sample set, and input the at least two modal face images into the feature extractor of the pre-trained heterogeneous face recognition model, outputting face feature vectors corresponding to the at least two modal face images; determine a first prototype vector of the target object based on the face feature vectors corresponding to the at least two modal face images, and initialize classifier parameters based on the first prototype vector, the feature extractor and the initialized classifier constituting the initial heterogeneous face recognition model; the model fine-tuning module is used to iteratively execute the following steps for the initial heterogeneous face recognition model until the following conditions are met. Preset conditions and determine the current heterogeneous face recognition model as the final heterogeneous face recognition model: Input a preset number of cross-modal hard sample pairs into the feature extractor of the heterogeneous face recognition model obtained in the previous iteration, and output the first face feature vector and the second face feature vector of each cross-modal hard sample pair. The cross-modal hard sample pair refers to the two modal face images with the lowest similarity for the same target object in the first face image sample set. Determine the second prototype vector based on any face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the first prototype vector. Construct a model loss function based on the other face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the second prototype vector. Adjust the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration based on the model loss function to obtain the current heterogeneous face recognition model.

[0016] Fourthly, embodiments of the present invention provide a face recognition device, the device being based on a final heterogeneous face recognition model obtained by the training method of the heterogeneous face recognition model described in the first aspect; the device includes an acquisition module and a recognition module; the acquisition module is used to acquire a face image to be recognized; the recognition module is used to input the face image to be recognized into the feature extractor of the final heterogeneous face recognition model and output a face feature vector to be recognized; the face feature vector to be recognized is compared with a face feature vector of a pre-registered identity to identify the identity information corresponding to the face image to be recognized.

[0017] Fifthly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor is used to execute the program stored in the memory to implement the steps of the training method for the heterogeneous face recognition model described in the first aspect or the face recognition method described in the second aspect.

[0018] In a sixth aspect, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the training method for the heterogeneous face recognition model described in the first aspect or the face recognition method described in the second aspect.

[0019] The embodiments of the present invention provide a training method, recognition method, apparatus, and device for a heterogeneous face recognition model. This involves acquiring a pre-trained heterogeneous face recognition model, which includes a feature extractor and a classifier; determining at least two modal face images corresponding to various target objects in a first face image sample set, and inputting these at least two modal face images into the feature extractor of the pre-trained heterogeneous face recognition model, outputting face feature vectors corresponding to the at least two modal face images; determining a first prototype vector of the target object based on the face feature vectors corresponding to the at least two modal face images, and initializing the classifier parameters based on the first prototype vector; the feature extractor and the initialized classifier constituting an initial heterogeneous face recognition model; iteratively executing the following steps for the initial heterogeneous face recognition model until a preset condition is met, and determining the current heterogeneous face recognition model as the final heterogeneous face recognition model: inputting a preset number of cross-modal difficult sample pairs into the heterogeneous face recognition model obtained in the previous iteration. In the feature extractor of the model, the first and second face feature vectors of each cross-modal hard sample pair are output. The cross-modal hard sample pair refers to the two modal face images with the lowest similarity for the same target object in the first face image sample set. A second prototype vector is determined based on any one of the first and second face feature vectors of each cross-modal hard sample pair and a first prototype vector. A model loss function is constructed based on the other face feature vector of each cross-modal hard sample pair and the second prototype vector. The feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration are adjusted based on the model loss function to obtain the current heterogeneous face recognition model. That is, in the training phase of the heterogeneous face recognition model, the prototype used in this embodiment of the invention considers face images under at least two modalities and further improves the prototype using cross-modal hard sample pairs, thereby improving the recognition accuracy of the trained heterogeneous face recognition model. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a training method for a heterogeneous face recognition model provided in an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of the structure of a heterogeneous face recognition model provided in an embodiment of the present invention;

[0024] Figure 3 A flowchart illustrating another training method for a heterogeneous face recognition model provided in an embodiment of the present invention;

[0025] Figure 4 This is a flowchart illustrating a face recognition method provided in an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of a process for obtaining a preset number of cross-modal hard sample pairs of facial feature vectors, provided by an embodiment of the present invention.

[0027] Figure 6 A flowchart illustrating a calculation model loss function provided in an embodiment of the present invention;

[0028] Figure 7 A schematic diagram of prototype fine-tuning provided for an embodiment of the present invention;

[0029] Figure 8 A schematic diagram of the structure of a training device for a heterogeneous face recognition model provided in an embodiment of the present invention;

[0030] Figure 9 This is a schematic diagram of the structure of a face recognition device provided in an embodiment of the present invention;

[0031] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] First, the terms used in this invention will be explained:

[0034] Heterogeneous face recognition: Different modalities, such as visible light images as one modality and near-infrared light images as another, are studied in heterogeneous face recognition as a problem of comparative learning between face images of two different modalities.

[0035] Vision is a crucial medium for humans to understand, learn, and perceive the world. Facial recognition uses visual technology to acquire facial information, which is then processed and analyzed to distinguish and identify different people. However, in practical applications, facial images are affected not only by the individual's expression, age, posture, and accessories, but also by changes in ambient lighting and the background of the shooting environment. These factors pose significant challenges to facial recognition. To address the impact of environmental changes such as lighting on facial recognition performance, near-infrared light is now widely used for spectral imaging. This allows for the acquisition of high-quality images under various environmental conditions, including low brightness. For example, in the security field, near-infrared surveillance cameras can provide clear images both day and night, enhancing security. However, since current databases contain only visible light images, while detection images are typically in near-infrared light mode, researching the heterogeneous facial recognition problem between visible light and near-infrared light images is crucial.

[0036] In related technologies, pre-trained face recognition models are typically used for identification. However, on the one hand, there is a lack of sufficient paired cross-modal face image training data, and large-scale collection of visible light and near-infrared face image pairs is almost an impossible task. On the other hand, visible light and near-infrared face images come from different modalities and have inherent differences in data distribution, resulting in low accuracy of the ultimately trained heterogeneous face recognition models.

[0037] To address the aforementioned technical problems, the technical concept of this invention is as follows: based on a pre-trained heterogeneous face recognition model, a prototype is obtained using a small-scale paired cross-modal dataset, and the prototype is further improved using cross-modal difficult sample face images, thereby improving the prototype learning method and increasing the accuracy of the finally trained model.

[0038] Figure 1 This is a flowchart illustrating a training method for a heterogeneous face recognition model provided in an embodiment of the present invention. The execution entity of this embodiment is a training device for the heterogeneous face recognition model, which can be implemented by any software and / or hardware. Figure 1 As shown, the training method for this heterogeneous face recognition model includes:

[0039] Step S101: Obtain a pre-trained heterogeneous face recognition model, wherein the pre-trained heterogeneous face recognition model includes a feature extractor and a classifier.

[0040] Figure 2 This is a schematic diagram of the structure of a heterogeneous face recognition model provided in an embodiment of the present invention, as shown below. Figure 2As shown, the heterogeneous face recognition model includes a feature extractor (or feature extraction model) and a classifier (or classification model). The feature extractor extracts deep-level features from the face image, i.e., outputs face features, including but not limited to: lightweight convolutional neural network models (LightCNN), such as LightCNN-9 and LightCNN-29, residual network models (ResNet), etc. The classifier performs identity recognition based on the face features extracted by the feature extractor, outputting the target category corresponding to the face image, i.e., identity information, including but not limited to the ArcFace classifier. In other words, this embodiment of the invention applies most feature extraction models and loss functions.

[0041] In this step, we directly obtain a heterogeneous face recognition model that has already been trained on a large scale of data. Using a pre-trained heterogeneous face recognition model can speed up subsequent training, make the model converge faster, and achieve better recognition results.

[0042] In some embodiments, before step S101, the method further includes: training the heterogeneous face recognition model to be trained based on a second face image sample set to obtain the pre-trained heterogeneous face recognition model; wherein the second face image sample set includes single-modal face images of the target object.

[0043] Specifically, the second face image sample set includes multiple target objects, each corresponding to multiple face images in a single modality. For example, the MS-Celeb-1M dataset includes visible light face images of the target objects, and has a large number of sample images. In this embodiment, the MS-Celeb-1M dataset can be used to pre-train an untrained heterogeneous face recognition model using stochastic gradient descent, thereby obtaining a corresponding pre-trained heterogeneous face recognition model. This initially solves the problem of the small number of face image sample sets currently available. The loss functions used during pre-training include, but are not limited to, the following: softmax loss function and CosFace loss function.

[0044] For example, the softmax loss function is shown in Equation (1):

[0045]

[0046] Where, x i Let y represent the face feature vector corresponding to the i-th sample image. i Let T represent the target category (or identity label) corresponding to the i-th sample image, T represent the number of target object categories, and N represent the number of sample images.

[0047] Step S102: Determine at least two modal face images corresponding to various target objects in the first face image sample set, and input the at least two modal face images into the feature extractor of the pre-trained heterogeneous face recognition model, and output the face feature vectors corresponding to the at least two modal face images.

[0048] Specifically, the first face image sample set includes multiple target objects, each target object includes multiple modalities, and each modality corresponds to multiple face images. For example, the CASIA NIR-VIS 2.0 dataset includes visible light face images and near-infrared light face images of the target objects, which may or may not overlap with the target objects in the MS-Celeb-1M dataset.

[0049] In this embodiment, multiple face images of each target object in various modalities from the first face image sample set are input into a pre-trained heterogeneous face recognition model, and the face feature vectors corresponding to the multiple face images of each target object in various modalities are output through the feature extractor.

[0050] Step S103: Determine the first prototype vector of the target object based on the face feature vectors corresponding to the at least two modal face images, and initialize the classifier parameters based on the first prototype vector. The feature extractor and the initialized classifier constitute the initial heterogeneous face recognition model.

[0051] Specifically, for each target object, the average of the facial feature vectors corresponding to multiple facial images under various modalities is calculated to obtain a unique facial feature vector corresponding to each target object. This uniquely corresponding facial feature vector can be determined as the prototype vector of the target object, i.e., the first prototype vector, which can also be called the class center. The classifier parameters are initialized based on the first prototype vector of each target object, i.e., the classifier parameters can be the first prototype variables. At this time, the feature extractor and the initialized classifier can constitute the initial heterogeneous face recognition model. Subsequently, the initial heterogeneous face recognition model will be fine-tuned and trained.

[0052] It should be noted that the determination of the first prototype vector takes into account face images in at least two modalities. Compared with the prototype vector determined by the traditional method using only the single modal image of visible light, this improves the training accuracy of heterogeneous face recognition.

[0053] Step S104: Input a preset number of cross-modal hard sample pairs into the feature extractor of the heterogeneous face recognition model obtained in the previous iteration, and output the first face feature vector and the second face feature vector of each cross-modal hard sample pair.

[0054] The cross-modal difficult sample pair refers to the two modal face images with the lowest similarity for the same target object in the first face image sample set.

[0055] Specifically, the preset quantity can be understood as the batch size, which is the number of cross-modal difficult sample pairs input in each iteration. Those skilled in the art can set it based on experience. The cross-modal difficult sample pair is formed by randomly selecting a visible light face image for each type of target object in the first face image sample set, and then selecting a near-infrared face image that belongs to the same person as the visible light face image and has the lowest similarity.

[0056] In this embodiment, when the initial heterogeneous face recognition model is trained for the first iteration, a preset number of cross-modal hard sample pairs are input into the initial heterogeneous face recognition model; during subsequent iterations, a preset number of cross-modal hard sample pairs are input into the heterogeneous face recognition model obtained in the previous iteration; the feature extractor outputs the first face feature vector and the second face feature vector for each cross-modal hard sample pair.

[0057] Step S105: Determine the second prototype vector based on any face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the first prototype vector.

[0058] Specifically, the corresponding second prototype vector can be determined based on any one of the two face feature vectors in each cross-modal hard sample pair and the first prototype vector, that is, the prototype is further improved by using cross-modal hard samples.

[0059] In some embodiments, step S105, which involves determining the second prototype vector based on any face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the first prototype vector, includes: weighted summing of any face feature vector from each cross-modal hard sample pair and the first prototype variable to obtain the second prototype vector.

[0060] Specifically, a weighted sum is taken of any one face feature vector from the two face feature vectors of the cross-modal hard sample pair and the first prototype vector to obtain a new prototype vector, namely the second prototype vector. In this embodiment, the prototype is further fine-tuned using cross-modal hard samples, which further improves the training accuracy of the heterogeneous face recognition model. It should be noted that the sum of the weighting parameters of any face feature vector and the weighting parameters of the first prototype vector is 1, and its specific value can be determined by those skilled in the art through experiments.

[0061] Step S106: Construct a model loss function based on the first face feature vector and the second face feature vector of each cross-modal hard sample pair, along with the second prototype vector.

[0062] Specifically, the model loss function of the heterogeneous face recognition model is constructed based on another face feature vector from the two face feature vectors of the cross-modal hard sample pair and a new prototype vector.

[0063] Step S107: Adjust the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration based on the model loss function to obtain the current heterogeneous face recognition model.

[0064] Specifically, based on the loss function of this model, the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration are adjusted, that is, the parameters of the heterogeneous face recognition model are adjusted to obtain the current heterogeneous face recognition model, which is used for the next round of iteration training.

[0065] Step S108: Determine whether the preset conditions are met.

[0066] If yes, proceed to step S109; otherwise, return to step S104.

[0067] Step S109: Determine the current heterogeneous face recognition model as the final heterogeneous face recognition model.

[0068] Specifically, the preset conditions include, but are not limited to, the following: the number of iterations reaches a preset number, and the model loss function converges. If the current number of iterations has reached the preset number, or the model loss function has converged, then the current heterogeneous face recognition model is determined to be the final trained heterogeneous face recognition model; if the preset number of iterations has not yet been reached, or the model loss function has not yet converged, then the next round of iteration training continues.

[0069] In some embodiments, step S106 includes the following steps: determining a single-sample loss function based on another face feature vector and a second prototype vector for each cross-modal hard sample pair; calculating the mean of the single-sample loss functions for a preset number of cross-modal hard sample pairs, and determining the mean as the model loss function; step S107 includes: adjusting the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration using stochastic gradient descent based on the model loss function, to obtain the current heterogeneous face recognition model.

[0070] Specifically, for each cross-modal hard sample pair, the other feature vector and the corresponding second prototype vector in the two face feature vectors can be used to calculate the corresponding single-sample loss function using the ArcFace loss function; repeat the above steps to obtain a preset number of single-sample loss functions corresponding to a preset number of cross-modal hard sample pairs; calculate their mean, and use the mean as the model loss function of the heterogeneous face recognition model; use stochastic gradient descent to adjust the feature extractor parameters and classifier parameters in the heterogeneous face recognition model obtained in the previous iteration; finally, after several rounds of parameter updates, the final heterogeneous face recognition model is obtained, and the feature extractor in the final heterogeneous face recognition model is used for subsequent face recognition.

[0071] In some embodiments, the single-sample loss function is as shown in Equation (2):

[0072]

[0073] Where L represents the single-sample loss function, x′ i x i Let y represent any face feature vector and the other face feature vector of the i-th cross-modal hard sample pair, respectively. i This represents the target object category corresponding to the i-th cross-modal hard sample pair. Indicates that the target object category is y i The corresponding first prototype variable, i.e., the initialized classifier parameters, is where T represents the number of target object categories and N represents the preset quantity. Let λ1 and λ2 represent the second prototype vector, and let λ1 and λ2 represent the weighting parameters.

[0074] The training method for a heterogeneous face recognition model provided in this embodiment of the invention involves obtaining a pre-trained heterogeneous face recognition model, which includes a feature extractor and a classifier; determining at least two modal face images corresponding to various target objects in a first face image sample set, and inputting the at least two modal face images into the feature extractor of the pre-trained heterogeneous face recognition model to output face feature vectors corresponding to the at least two modal face images; determining a first prototype vector of the target object based on the face feature vectors corresponding to the at least two modal face images, and initializing the classifier parameters based on the first prototype vector; the feature extractor and the initialized classifier constitute an initial heterogeneous face recognition model; iteratively executing the following steps for the initial heterogeneous face recognition model until a preset condition is met, and determining the current heterogeneous face recognition model as the final heterogeneous face recognition model: inputting a preset number of cross-modal hard sample pairs into the feature extractor of the heterogeneous face recognition model obtained in the previous iteration. In the device, the first and second face feature vectors of each cross-modal hard sample pair are output. The cross-modal hard sample pair refers to the two modal face images with the lowest similarity for the same target object in the first face image sample set. A second prototype vector is determined based on any one of the first and second face feature vectors of each cross-modal hard sample pair and a first prototype vector. A model loss function is constructed based on the other face feature vector of the first and second face feature vectors of each cross-modal hard sample pair and the second prototype vector. The feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration are adjusted based on the model loss function to obtain the current heterogeneous face recognition model. That is, in the training phase of the heterogeneous face recognition model, the prototype used in this embodiment considers images under multiple modalities and is further fine-tuned based on cross-modal hard sample pairs, improving the prototype learning method and increasing the accuracy of the trained heterogeneous face recognition model.

[0075] The following describes in detail the selection process of cross-modal difficult sample pairs using a specific example.

[0076] Figure 3 A flowchart illustrating another training method for a heterogeneous face recognition model provided in an embodiment of the present invention is shown below. Figure 3 As shown, based on the above embodiments, the method of this embodiment may further include S301 and S302 before S104.

[0077] It should be noted that S301 and S302 in this embodiment can be executed before the training of the initial heterogeneous face recognition model begins to obtain cross-modal difficult sample pairs of target objects of all categories in the first sample image set, or they can be executed before each iteration to obtain a preset number of cross-modal difficult sample pairs of categories in the first sample image set.

[0078] Step S301: Randomly select a first modal face image of the target object from the first face image sample set, and select at least one second modal face image corresponding to the target object.

[0079] Step S302: Determine the second modal face image with the smallest similarity to the first modal face image. The first modal face image and the second modal face image with the smallest similarity constitute the cross-modal difficult sample pair.

[0080] Specifically, taking the CASIA NIR-VIS 2.0 dataset as an example, a visible light face image is first randomly selected, and then a near-infrared face image belonging to the same person as the visible light face image and having the lowest similarity is selected to form a cross-modal hard sample face image pair.

[0081] In some embodiments, the step S202 of determining the second modality face image with the smallest similarity to the first modality face image includes: inputting the first modality face image and at least one second modality face image into a pre-trained heterogeneous face recognition model, outputting corresponding first modality face feature vectors and at least one second modality face feature vector; calculating the cosine distance between the first modality face feature vector and each second modality face feature vector, and determining the second modality face image with the smallest similarity to the first modality face image based on the cosine distance.

[0082] Taking the CASIA NIR-VIS 2.0 dataset as an example, a visible light face image is randomly selected first, and then all near-infrared face images belonging to the same person as the visible light face image are selected. Then, the visible light face image and all near-infrared face images are input into a pre-trained heterogeneous face recognition model, which outputs the first face feature vector corresponding to the visible light face image and the second face feature vector corresponding to each near-infrared face image. By calculating the cosine distance between the first face feature vector and each second face feature vector, the similarity between the visible light face image and each near-infrared face image is measured, and the near-infrared face image and the visible light image with the smallest similarity to the visible light image are determined to form a cross-modal hard sample pair.

[0083] Based on the aforementioned embodiments, a first modal face image of the target object is randomly selected from the first face image sample set, and at least one second modal face image corresponding to the target object is selected; the second modal face image with the smallest similarity to the first modal face image is determined, and the first modal face image and the second modal face image with the smallest similarity constitute the cross-modal hard sample pair, realizing the construction of a small-scale paired cross-modal dataset, and the prototype can be further fine-tuned using the cross-modal dataset, making full use of small batches of data and improving the model training accuracy.

[0084] Figure 4 This is a flowchart illustrating a face recognition method provided in an embodiment of the present invention. The method is based on the final heterogeneous face recognition model obtained by the training method of the heterogeneous face recognition model described above. Figure 4 As shown, the face recognition method includes:

[0085] Step S401: Obtain the face image to be identified.

[0086] Step S402: Input the face image to be recognized into the feature extractor of the final heterogeneous face recognition model, and output the face feature vector to be recognized.

[0087] Step S403: Compare the facial feature vector to be identified with the facial feature vector of the pre-registered identity to identify the identity information corresponding to the facial image to be identified.

[0088] Specifically, the heterogeneous face recognition model trained using the above embodiments can be used for face recognition, especially for near-infrared face image recognition, where it shows better results. The near-infrared face image to be recognized can be input into the feature extractor of the final heterogeneous face recognition model to obtain the face feature vector to be recognized. The face feature vector to be recognized is then compared with pre-registered face feature vectors to match the corresponding identity information.

[0089] The face recognition method provided in this embodiment of the invention is applied to the final heterogeneous face recognition model obtained in the aforementioned embodiments. Because the final heterogeneous face recognition model uses cross-modal data during training, the facial features extracted from the near-infrared face image are more accurate when applied, thus improving the accuracy of heterogeneous face recognition. Furthermore, compared to the heterogeneous face recognition stage in traditional methods, many cumbersome steps have been added, such as the inclusion of attention mechanisms and adversarial domain adaptation modules, resulting in faster recognition speed.

[0090] To provide a detailed understanding of the embodiments of the present invention, the implementation process of the embodiments of the present invention will now be described in detail. The embodiments of the present invention can be divided into a training stage and an application stage. The training stage includes the following steps:

[0091] Step 1: Use the MS-Celeb-1M visible light face image set to pre-train the untrained heterogeneous face recognition model using the SoftMax loss function to obtain the corresponding pre-trained model. The feature extractor in the heterogeneous face recognition model adopts the LightCnn-29 network model.

[0092] Step 2: Input each face image from the CASIA NIR-VIS 2.0 dataset into the pre-trained model obtained in Step 1, and output the face feature vector corresponding to each face image;

[0093] Step 3: Calculate the average of the facial feature vectors of all facial images belonging to the same person, and use it as the first prototype vector of this person;

[0094] Step 4: Initialize the classifier parameters of the pre-trained model obtained in Step 1 using the first prototype vector of each person;

[0095] Step 5: Randomly select one or a batch of cross-modal hard sample pairs from the CASIA NIR-VIS 2.0 dataset and input them into the pre-trained model to obtain two face feature vectors for each cross-modal hard sample pair.

[0096] Specifically, Figure 5 This is a flowchart illustrating a process for obtaining a preset number of cross-modal hard sample pairs of facial feature vectors, as provided in an embodiment of the present invention. Figure 5 As shown, four cross-modal hard sample pairs are used, such as 1A and 1B forming the first cross-modal hard sample pair, 2A and 2B forming the second cross-modal hard sample pair, 3A and 3B forming the third cross-modal hard sample pair, and 4A and 4B forming the fourth cross-modal hard sample pair. These four cross-modal hard sample pairs are input into a pre-trained heterogeneous face recognition model, and the corresponding face feature vectors are output through the feature extractor.

[0097] Step 6: Add one of the two feature vectors of each cross-modal hard sample pair obtained in Step 5 to the first prototype of the corresponding person by a certain coefficient to obtain the cross-modal prototype, i.e., the second prototype vector.

[0098] Step 7: Using the second prototype vector and another feature vector, calculate the corresponding single-sample loss using the ArcFace loss function until a preset number of single-sample losses are obtained, and then calculate the mean as the model loss function.

[0099] Figure 6 This is a flowchart illustrating a calculation model loss function provided in an embodiment of the present invention, as shown below. Figure 6 As shown, any one of the two feature vectors of a cross-modal hard sample pair (such as...) Figure 6The feature vector located below in the model is added to the prototype vector by a certain coefficient to obtain the second prototype vector; then another feature vector (such as...) Figure 6 The model loss function is calculated by multiplying the feature vector located above the second prototype vector by the feature vector located above the second prototype vector. Figure 7 A schematic diagram of prototype fine-tuning provided for an embodiment of the present invention, such as... Figure 7 As shown, the traditional method uses the feature vector of a visible light face image to determine its prototype, while this embodiment uses cross-modal hard sample pairs of the feature vectors of visible light face images and near-infrared light face images to adjust the prototype.

[0100] Step 8: Based on the model loss function, use stochastic gradient descent to update the LightCnn-29 network model parameters. After several rounds of parameter updates, the final LightCnn-29 network model is obtained.

[0101] After training the final heterogeneous face recognition model, the application phase primarily utilizes the trained LightCNN-29 network model, which demonstrates significant effectiveness in recognizing near-infrared light and other modalities. The application phase includes the following steps:

[0102] Step 1: Obtain the near-infrared face image to be identified.

[0103] Step 2: Input the near-infrared face image into the LightCnn-29 network model and output the face feature vector to be identified;

[0104] Step 3: Compare the facial feature vector of the person to be identified with the facial feature vector of the person with the pre-registered identity to identify the identity information corresponding to the facial image of the person to be identified.

[0105] In summary, the beneficial effects of this embodiment are as follows:

[0106] (1) Generalization: In this embodiment of the invention, there are no requirements for the specific feature extraction model and the loss function of the classifier involved in the training of the recognition model. That is to say, the feature extraction model is not limited to the LightCNN model. Other models such as ResNet or loss functions such as CosFace can also be used in this embodiment, and the accuracy will be improved.

[0107] (2) Fast recognition speed. The embodiments of the present invention only change the model training process and do not add parameters to the normal use process of the model, so it will not affect its recognition speed.

[0108] (3) High recognition accuracy: The embodiments of the present invention adopt the pre-training and cross-modal difficult sample method. Cross-modal information is added during the fine-tuning process to change the original prototype and improve the model accuracy.

[0109] Figure 8 This is a schematic diagram of the structure of a training device for a heterogeneous face recognition model provided in an embodiment of the present invention, as shown below. Figure 8 As shown, the training device includes an initial model module 801 and a model fine-tuning module 802:

[0110] The initial model module 801 is used to acquire a pre-trained heterogeneous face recognition model, which includes a feature extractor and a classifier; determine at least two modal face images corresponding to various target objects in a first face image sample set, and input the at least two modal face images into the feature extractor of the pre-trained heterogeneous face recognition model, outputting face feature vectors corresponding to the at least two modal face images; determine the first prototype vector of the target object based on the face feature vectors corresponding to the at least two modal face images, and initialize the classifier parameters based on the first prototype vector. The feature extractor and the initialized classifier constitute the initial heterogeneous face recognition model. The model fine-tuning module is used to iteratively execute the following steps for the initial heterogeneous face recognition model until a preset condition is met, and determine the current heterogeneous face recognition model as the final heterogeneous face recognition model. The heterogeneous face recognition model is constructed by inputting a preset number of cross-modal hard sample pairs into the feature extractor of the heterogeneous face recognition model obtained in the previous iteration, and outputting a first face feature vector and a second face feature vector for each cross-modal hard sample pair. The cross-modal hard sample pair refers to the two modal face images with the lowest similarity for the same target object in the first face image sample set. A second prototype vector is determined based on any one of the first and second face feature vectors of each cross-modal hard sample pair and a first prototype vector. A model loss function is constructed based on the other face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the second prototype vector. The feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration are adjusted based on the model loss function to obtain the current heterogeneous face recognition model.

[0111] In some embodiments, the apparatus further includes a pre-training module 803, which is used to train the heterogeneous face recognition model to be trained based on a second face image sample set to obtain the pre-trained heterogeneous face recognition model; wherein the second face image sample set includes single-modal face images of the target object.

[0112] In some embodiments, the model fine-tuning module 802 is further configured to: randomly select a first modal face image of the target object from the first face image sample set, and select at least one second modal face image corresponding to the target object; determine the second modal face image with the smallest similarity to the first modal face image, wherein the first modal face image and the second modal face image with the smallest similarity constitute the cross-modal difficult sample pair.

[0113] In some embodiments, the model fine-tuning module 802 is specifically configured to: input the first modal face image and at least one second modal face image into a pre-trained heterogeneous face recognition model, and output the corresponding first modal face feature vector and at least one second modal face feature vector; calculate the cosine distance between the first modal face feature vector and each second modal face feature vector, and determine the second modal face image with the smallest similarity to the first modal face image based on the cosine distance.

[0114] In some embodiments, the model fine-tuning module 802 is specifically used to: weight and sum any face feature vector and the first prototype variable in each cross-modal hard sample pair to obtain the second prototype vector.

[0115] In some embodiments, the model fine-tuning module 802 is specifically used to: determine a single-sample loss function based on another face feature vector and a second prototype vector for each cross-modal hard sample pair; calculate the mean of the single-sample loss function for a preset number of cross-modal hard sample pairs, and determine the mean as the model loss function; and adjust the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration using stochastic gradient descent based on the model loss function to obtain the current heterogeneous face recognition model.

[0116] In some embodiments, the formula for the single-sample loss function is as follows:

[0117]

[0118] Where L represents the single-sample loss function, x′ i x i Let y represent any face feature vector and the other face feature vector of the i-th cross-modal hard sample pair, respectively. i This represents the target object category corresponding to the i-th cross-modal hard sample pair. Indicates that the target object category is y i The corresponding first prototype variable, T represents the number of target object categories, N represents the preset quantity, and λ1 and λ2 represent weighting parameters.

[0119] The training device for the heterogeneous face recognition model provided in this embodiment of the invention has a similar implementation principle and technical effect to the above embodiments, and will not be described again here.

[0120] Figure 9 This is a schematic diagram of a face recognition device provided in an embodiment of the present invention. The face recognition device is based on the final heterogeneous face recognition model obtained by the training method of the heterogeneous face recognition model described above; as follows: Figure 9 As shown, the face recognition device includes an acquisition module 901 and a recognition module 902;

[0121] The acquisition module 901 is used to acquire a face image to be identified; the recognition module 902 is used to input the face image to be identified into the feature extractor of the final heterogeneous face recognition model and output the face feature vector to be identified; the face feature vector to be identified is compared with the face feature vector of a pre-registered identity to identify the identity information corresponding to the face image to be identified.

[0122] The face recognition device provided in this embodiment of the invention has a similar implementation principle and technical effect to the above embodiments, and will not be described again here.

[0123] like Figure 10 As shown, this embodiment of the invention provides an electronic device, including a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other via the communication bus 1004.

[0124] Memory 1003 is used to store computer programs;

[0125] In one embodiment of the present invention, when the processor 1001 executes the program stored in the memory 1003, it implements the steps of the training method for the heterogeneous face recognition model or the face recognition method provided in any of the aforementioned method embodiments.

[0126] The electronic device provided in this embodiment of the invention has a similar implementation principle and technical effect to the above embodiments, and will not be described again here.

[0127] The aforementioned memory 1003 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 1003 has storage space for program code used to perform any of the method steps described above. For example, the storage space for program code may include individual program codes for implementing the various steps in the methods described above. This program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, optical discs (CDs), memory cards, or floppy disks. Such computer program products are typically portable or fixed storage units. The storage unit may have storage segments or storage spaces arranged similarly to the memory 1003 in the aforementioned electronic device. The program code may be compressed, for example, in a suitable form. Typically, the storage unit includes programs for performing the method steps according to embodiments of the invention, i.e., code that can be read by a processor such as 1001, which, when run by the electronic device, causes the electronic device to perform the various steps in the methods described above.

[0128] Embodiments of the present invention also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the training method for the heterogeneous face recognition model or the face recognition method described above.

[0129] The computer-readable storage medium may be included in the device / apparatus described in the above embodiments; or it may exist independently and not assembled into the device / apparatus. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of the present invention.

[0130] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0131] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0132] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A training method for a heterogeneous face recognition model, characterized in that, The method includes: Obtain a pre-trained heterogeneous face recognition model, wherein the pre-trained heterogeneous face recognition model includes a feature extractor and a classifier; Determine at least two modal face images corresponding to various target objects in the first face image sample set, and input the at least two modal face images into the feature extractor of the pre-trained heterogeneous face recognition model, and output the face feature vectors corresponding to the at least two modal face images; The first prototype vector of the target object is determined based on the facial feature vectors corresponding to the at least two modal facial images, and the classifier parameters are initialized based on the first prototype vector. The feature extractor and the initialized classifier constitute an initial heterogeneous face recognition model. For the initial heterogeneous face recognition model, the following steps are iteratively executed until a preset condition is met, and the current heterogeneous face recognition model is determined to be the final heterogeneous face recognition model: A preset number of cross-modal hard sample pairs are input into the feature extractor of the heterogeneous face recognition model obtained in the previous iteration, and the first face feature vector and the second face feature vector of each cross-modal hard sample pair are output. The cross-modal hard sample pair refers to the two modal face images with the lowest similarity for the same target object in the first face image sample set. The second prototype vector is determined based on any face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the first prototype vector; The model loss function is constructed based on the first face feature vector and the second face feature vector of each cross-modal hard sample pair, along with the second prototype vector. Based on the model loss function, adjust the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration to obtain the current heterogeneous face recognition model.

2. The method according to claim 1, characterized in that, Before obtaining the pre-trained heterogeneous face recognition model, the process also includes: The heterogeneous face recognition model to be trained is trained based on the second set of face image samples to obtain the pre-trained heterogeneous face recognition model. The second face image sample set includes single-modal face images of the target object.

3. The method according to claim 1, characterized in that, Before inputting a preset number of cross-modal hard sample pairs into the feature extractor of the heterogeneous face recognition model obtained in the previous iteration, the method further includes: Randomly select a first modal face image of the target object from the first face image sample set, and select at least one second modal face image corresponding to the target object; A second modal face image with the lowest similarity to the first modal face image is determined, and the first modal face image and the second modal face image with the lowest similarity constitute the cross-modal difficult sample pair.

4. The method according to claim 3, characterized in that, The step of determining the second modality face image with the lowest similarity to the first modality face image includes: The first modal face image and at least one second modal face image are input into a pre-trained heterogeneous face recognition model, and the corresponding first modal face feature vector and at least one second modal face feature vector are output. Calculate the cosine distance between the first modality face feature vector and each second modality face feature vector, and determine the second modality face image with the smallest similarity to the first modality face image based on the cosine distance.

5. The method according to claim 1, characterized in that, The step of determining the second prototype vector based on any one of the first and second face feature vectors from each cross-modal hard sample pair and the first prototype vector includes: The second prototype vector is obtained by weighted summing of any face feature vector in each cross-modal hard sample pair with the first prototype variable.

6. The method according to any one of claims 1-5, characterized in that, The process involves constructing a model loss function based on another face feature vector from the first and second face feature vectors of each cross-modal hard sample pair, and the second prototype vector; adjusting the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration based on the model loss function to obtain the current heterogeneous face recognition model, including: The single-sample loss function is determined based on another face feature vector and a second prototype vector for each cross-modal hard sample pair; Calculate the mean of the single-sample loss function for a predetermined number of cross-modal hard sample pairs, and determine the mean as the model loss function; Based on the model loss function, the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration are adjusted using the stochastic gradient descent method to obtain the current heterogeneous face recognition model.

7. The method according to claim 6, characterized in that, The formula for the single-sample loss function is as follows: Where L represents the single-sample loss function, x′ i x i Let y represent any face feature vector and the other face feature vector of the i-th cross-modal hard sample pair, respectively. i This represents the target object category corresponding to the i-th cross-modal hard sample pair. Indicates that the target object category is y i The corresponding first prototype variable, T represents the number of target object categories, N represents the preset quantity, and λ1 and λ2 represent weighting parameters.

8. A face recognition method, characterized in that, The method obtains the final heterogeneous face recognition model based on the training method of the heterogeneous face recognition model according to any one of claims 1-7; the method includes: Acquire the image of the face to be identified; The image of the face to be identified is input into the feature extractor of the final heterogeneous face recognition model, and the feature vector of the face to be identified is output. The facial feature vector to be identified is compared with the facial feature vector of a pre-registered identity to identify the identity information corresponding to the facial image to be identified.

9. A training device for a heterogeneous face recognition model, characterized in that, Includes an initial model module and a model fine-tuning module; The initial model module is used to obtain a pre-trained heterogeneous face recognition model, which includes a feature extractor and a classifier. Determine at least two modal face images corresponding to various target objects in the first face image sample set, and input the at least two modal face images into the feature extractor of the pre-trained heterogeneous face recognition model, and output the face feature vectors corresponding to the at least two modal face images; The first prototype vector of the target object is determined based on the facial feature vectors corresponding to the at least two modal facial images, and the classifier parameters are initialized based on the first prototype vector. The feature extractor and the initialized classifier constitute an initial heterogeneous face recognition model. The model fine-tuning module is used to iteratively execute the following steps for the initial heterogeneous face recognition model until a preset condition is met, and to determine the current heterogeneous face recognition model as the final heterogeneous face recognition model: A preset number of cross-modal hard sample pairs are input into the feature extractor of the heterogeneous face recognition model obtained in the previous iteration, and the first face feature vector and the second face feature vector of each cross-modal hard sample pair are output. The cross-modal hard sample pair refers to the two modal face images with the lowest similarity for the same target object in the first face image sample set. The second prototype vector is determined based on any face feature vector from the first and second face feature vectors of each cross-modal hard sample pair and the first prototype vector; The model loss function is constructed based on the first face feature vector and the second face feature vector of each cross-modal hard sample pair, along with the second prototype vector. Based on the model loss function, adjust the feature extractor parameters and classifier parameters of the heterogeneous face recognition model obtained in the previous iteration to obtain the current heterogeneous face recognition model.

10. A face recognition device, characterized in that, The device is based on the training method of the heterogeneous face recognition model according to any one of claims 1-7 to obtain the final heterogeneous face recognition model; the device includes an acquisition module and a recognition module; The acquisition module is used to acquire the face image to be identified; The recognition module is used to input the face image to be recognized into the feature extractor of the final heterogeneous face recognition model and output the face feature vector to be recognized. The facial feature vector to be identified is compared with the facial feature vector of a pre-registered identity to identify the identity information corresponding to the facial image to be identified.

11. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the training method for the heterogeneous face recognition model as described in any one of claims 1-7 or the face recognition method as described in claim 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the training method for the heterogeneous face recognition model as described in any one of claims 1-7 or the face recognition method as described in claim 8.

Images