Explanatable face recognition method based on anti-fact inference

Abstract

The invention provides an interpretable face recognition method based on anti-fact inference. The interpretable face recognition method based on anti-fact inference effectively solves the problem that an existing face recognition model interpretable method has defects. The method comprises the following steps: the anti-fact module processes a face image to be recognized to obtain an anti-fact image, and outputs the anti-fact image to the face recognition module; the face recognition module extracts face features in the to-be-recognized face image, the reference image and the anti-fact image to obtain corresponding depth face features so as to obtain a corresponding face recognition result based on the depth face features; the recognition and interpretation module reconstructs a corresponding reconstructed image so as to obtain corresponding face attribute data based on the reconstructed image; and the recognition interpretation module calculates a face attribute difference value based on the face attribute data so as to interpret a face recognition result obtained by the face recognition module based on the face attribute difference value.

Description

Technical Field

[0001] The present application relates to the technical fields of computer vision and face recognition, and in particular to an explainable face recognition method based on counterfactual inference. Background Art

[0002] In recent years, the advantage of deep learning models is that they can map facial images into complex high-dimensional feature spaces, thereby performing effective similarity assessments, which has enabled deep learning models to make significant progress in the field of face recognition technology. However, human observers cannot understand this complex feature space, and deep learning models do not provide specific explanations. This inexplicability not only limits humans' understanding of the model's decision-making process, but also affects the credibility and controllability of face recognition technology in practical applications. Therefore, improving the interpretability of face recognition technology has become an urgent problem to be solved.

[0003] Existing methods for improving the interpretability of face recognition models mainly attribute the recognition results to local areas or pixels of the input image through methods such as gradient backpropagation. The main disadvantages of such methods are:

[0004] 1. The explanation provided by this type of method for the recognition results does not necessarily have a causal effect. The salient regions obtained by gradient back propagation only have a more reliable causal effect in linear models. However, the face recognition model based on deep learning is highly nonlinear.

[0005] 2. This type of method cannot quantitatively describe the causes of the recognition results, but can only provide qualitative descriptions;

[0006] 3. The reasons for the recognition results provided by this type of method are rough. For example, it may attribute the recognition result to the eye area of ​​the input face image, but the key factor may actually be the shape of the eye, or the relative position of the eye on the face, or the color of the iris. Existing methods cannot describe the reasons in a more detailed and accurate manner. Summary of the invention

[0007] In view of this, the purpose of the present application is to provide an explainable face recognition method based on counterfactual inference, which effectively solves the problems of defects in the existing methods of interpretability of face recognition models.

[0008] In a first aspect, an embodiment of the present application provides an explainable face recognition method based on counterfactual inference, which is applied to a face recognition system, wherein the face recognition system includes a counterfactual module, a face recognition module, and a recognition interpretation module, and the method includes:

[0009] The counterfactual module processes the face image to be recognized to obtain a counterfactual image, and outputs the counterfactual image to the face recognition module;

[0010] The face recognition module extracts face features from the face image to be recognized, the reference image and the counterfactual image respectively to obtain corresponding deep face features, so as to obtain corresponding face recognition results based on the deep face features;

[0011] The recognition and interpretation module reconstructs corresponding reconstructed images based on the face recognition result for the deep face features corresponding to the face image to be recognized, the reference image and the counterfactual image, so as to obtain corresponding face attribute data based on the reconstructed images;

[0012] The recognition and interpretation module calculates facial attribute differences based on the facial attribute data corresponding to the facial image to be recognized, the reference image and the counterfactual image, so as to interpret the facial recognition result obtained by the facial recognition module based on the facial attribute differences.

[0013] In combination with the first aspect, an embodiment of the present application provides a first possible implementation of the first aspect, wherein the recognition and interpretation module calculates a facial attribute difference based on the facial attribute data corresponding to the facial image to be recognized, the reference image, and the counterfactual image, respectively, so as to interpret the facial recognition result obtained by the facial recognition module based on the facial attribute difference, including:

[0014] The recognition and interpretation module calculates the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image respectively based on a preset attribute calculation network to obtain a facial attribute difference;

[0015] The recognition interpretation module confirms the reason why the face recognition module recognizes the face based on the attribute corresponding to the face attribute difference.

[0016] In combination with the first aspect, the embodiment of the present application provides a second possible implementation of the first aspect, wherein the recognition and interpretation module calculates the facial attribute data corresponding to the face image to be recognized, the reference image, and the counterfactual image based on a preset attribute calculation network to obtain a facial attribute difference, including:

[0017] The recognition and interpretation module outputs the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image to the attribute calculation network;

[0018] The attribute calculation network calculates the facial attribute data corresponding to the face image to be identified, the reference image and the counterfactual image respectively to obtain facial attribute differences.

[0019] In combination with the first aspect, the embodiment of the present application provides a third possible implementation of the first aspect, wherein the recognition interpretation module confirms the reason why the face recognition module recognizes the face based on the attribute corresponding to the face attribute difference, including:

[0020] The face recognition module determines whether the face attribute difference meets a preset recognition threshold range to determine whether there is a target attribute difference;

[0021] If so, the recognition and interpretation module confirms that the target attribute corresponding to the target attribute difference is the reason why the face recognition module recognizes the face.

[0022] In combination with the first aspect, the embodiment of the present application provides a fourth possible implementation of the first aspect, wherein the counterfactual module processes the face image to be recognized to obtain a counterfactual image, and outputs the counterfactual image to the face recognition module, including:

[0023] The counterfactual module calls a preset facial image processing method based on the facial image to be identified;

[0024] The counterfactual module processes the face image to be identified based on the face image processing method to obtain the counterfactual image.

[0025] In combination with the first aspect, an embodiment of the present application provides a fifth possible implementation of the first aspect, wherein the recognition and interpretation module reconstructs corresponding reconstructed images for the deep face features corresponding to the face image to be recognized, the reference image, and the counterfactual image respectively based on the face recognition result, including:

[0026] The recognition interpretation module determines whether the face recognition results corresponding to the face image to be recognized, the reference image and the counterfactual image respectively meet the preset reconstruction conditions;

[0027] If so, the recognition and interpretation module reconstructs corresponding reconstructed images based on the deep facial features corresponding to the face image to be recognized, the reference image and the counterfactual image respectively.

[0028] In combination with the first aspect, the embodiment of the present application provides a sixth possible implementation of the first aspect, wherein obtaining corresponding facial attribute data based on the reconstructed image includes:

[0029] The recognition and interpretation module generates an attribute call instruction based on the reconstructed image, and sends the attribute call instruction to the face attribute estimator;

[0030] The face attribute estimator extracts features from the reconstructed image in response to the attribute calling instruction to obtain corresponding face attribute data.

[0031] In a second aspect, an embodiment of the present application provides an explainable face recognition device based on counterfactual inference, which is applied to a face recognition system. The face recognition system includes a counterfactual module, a face recognition module, and a recognition interpretation module. The device includes:

[0032] A processing module, used for the counterfactual module to process the face image to be recognized to obtain a counterfactual image, and output the counterfactual image to the face recognition module;

[0033] An extraction module, used for the face recognition module to respectively extract the face features in the face image to be recognized, the reference image and the counterfactual image to obtain corresponding deep face features, so as to obtain a corresponding face recognition result based on the deep face features;

[0034] A reconstruction module, used for the recognition and interpretation module to reconstruct corresponding reconstructed images based on the face recognition result for the deep face features corresponding to the face image to be recognized, the reference image and the counterfactual image, so as to obtain corresponding face attribute data based on the reconstructed images;

[0035] An interpretation module is used for the recognition interpretation module to calculate the facial attribute difference based on the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image, so as to interpret the face recognition result obtained by the face recognition module based on the facial attribute difference.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, any step of an explainable face recognition method based on counterfactual inference is performed.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes the steps of any one of the methods for interpretable face recognition based on counterfactual inference.

[0038] The embodiment of the present application provides an explainable face recognition method based on counterfactual inference, which is applied to a face recognition system. The face recognition system includes a counterfactual module, a face recognition module and a recognition and interpretation module. The method first processes a face image to be recognized by the counterfactual module to obtain a counterfactual image, and outputs the counterfactual image to the face recognition module; secondly, the face recognition module extracts face features from the face image to be recognized, a reference image and the counterfactual image to obtain corresponding deep face features, so as to obtain corresponding face recognition results based on the deep face features; then, the recognition and interpretation module respectively processes the face image to be recognized, the reference image and the counterfactual image based on the face recognition results. The corresponding deep facial features are used to reconstruct the corresponding reconstructed image, so as to obtain the corresponding facial attribute data based on the reconstructed image; finally, the facial attribute difference is calculated by the recognition and interpretation module based on the facial attribute data corresponding to the facial image to be identified, the reference image and the counterfactual image, so as to explain the facial recognition result obtained by the facial recognition module based on the facial attribute difference, so as to give the facial recognition module an effect with causal effect, detailed explanation and qualitative description, thereby improving the accuracy and effectiveness of facial recognition, and solving the problem that the existing methods of interpretability of facial recognition models do not have necessary causal effects, cannot be qualitatively described, and the reasons for the recognition results provided are rough. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A schematic diagram of the process of a first explainable face recognition method based on counterfactual inference provided in an embodiment of the present application is shown;

[0041] Figure 2 A schematic diagram of a process for obtaining a reconstructed image provided in an embodiment of the present application is shown;

[0042] Figure 3 A flow chart showing the reasons corresponding to the face recognition results provided in the embodiment of the present application;

[0043] Figure 4 A schematic diagram of the structure of a first interpretable face recognition device based on counterfactual inference provided in an embodiment of the present application is shown;

[0044] Figure 5A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0045] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0046] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0047] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0048] At present, the existing methods for improving the interpretability of face recognition models are mainly to attribute the recognition results to local areas or some pixels of the input image through methods such as gradient back propagation. The main disadvantages of this type of method are: the explanation provided by this type of method for the recognition results does not necessarily have a causal effect. The significant area obtained by gradient back propagation only has a more reliable causal effect in linear models, but the face recognition model based on deep learning is highly nonlinear; this type of method cannot quantitatively describe the cause of the recognition result, but can only provide a qualitative description; the cause of the recognition result provided by this type of method is rough. For example, it may attribute the recognition result to the eye area of ​​the input face image, but the key role may actually be played by the shape of the eye, or the relative position of the eye on the face, or the color of the iris. The existing methods cannot provide a more detailed and accurate description of the cause.

[0049] Based on this, an embodiment of the present application provides an explainable face recognition method based on counterfactual inference, which is described below through an embodiment.

[0050] Example 1

[0051] To facilitate understanding of this embodiment, firstly, a method for interpretable face recognition based on counterfactual inference disclosed in the embodiment of this application is described in detail. Figure 1 The flowchart of an explainable face recognition method based on counterfactual inference is shown in the figure. The explainable face recognition method based on counterfactual inference provided by the present application is applied to a face recognition system. The face recognition system includes a counterfactual module, a face recognition module and a recognition and interpretation module. The method includes:

[0052] S101, the counterfactual module processes the face image to be recognized to obtain a counterfactual image, and outputs the counterfactual image to the face recognition module;

[0053] S102, the face recognition module extracts face features from the face image to be recognized, the reference image and the counterfactual image respectively to obtain corresponding deep face features, so as to obtain a corresponding face recognition result based on the deep face features;

[0054] S103, the recognition and interpretation module reconstructs corresponding reconstructed images based on the face recognition result for the deep face features corresponding to the face image to be recognized, the reference image and the counterfactual image, so as to obtain corresponding face attribute data based on the reconstructed images;

[0055] S104, the recognition and interpretation module calculates a facial attribute difference based on the facial attribute data corresponding to the facial image to be recognized, the reference image and the counterfactual image, so as to interpret the facial recognition result obtained by the facial recognition module based on the facial attribute difference.

[0056] In step S101, the counterfactual module is a module that processes the face image to be identified based on image processing technology and obtains an image that is different from the face image to be identified. The processing includes image transformation, noise addition, data enhancement, etc. The image transformation includes rotation, scaling, flipping, etc. The data enhancement is to add noise, change lighting conditions, etc. to generate an image that is different from the original image but can be used for specific testing or analysis, that is, a "counterfactual image". In the present application, the counterfactual module processes the face image to be identified to obtain a counterfactual image, that is, the counterfactual image is an image that is different from the face image to be identified, and outputs the counterfactual image to the face recognition module. The robustness of the face recognition module is improved by the counterfactual module, and the performance of the face recognition module under different conditions is improved by generating a variety of possible image variants.

[0057] In the specific implementation process of step S101, there is an embodiment in which the counterfactual module processes the face image to be recognized to obtain a counterfactual image, and outputs the counterfactual image to the face recognition module, including:

[0058] S1011, the counterfactual module calls a preset facial image processing method based on the facial image to be identified;

[0059] S1012: The counterfactual module processes the face image to be identified based on the face image processing method to obtain the counterfactual image.

[0060] In steps S1011-S1012, the counterfactual module calls a preset face image processing method based on the face image to be identified, and the face processing method is to add adversarial noise to the face image to be identified, which may be a PGD method or other methods of adding adversarial noise, which are not limited here. That is, this method enables the counterfactual module to process the face image to be identified based on the PGD method to obtain a counterfactual image that is different from the face image to be identified, and the counterfactual image retains the key features of the original image, but there are differences in some aspects such as the eyes or nose of the character, such as the color of the iris on the eyes, the size of the eye distance, the size of the nose wing, etc., which are different from the original image.

[0061] In step S102, when the face recognition module receives the counterfactual image, it also receives the face image to be recognized, and calls the reference image preset in the face recognition module. Since the face recognition module needs to perform face recognition, the reference image also needs to be a face image, and the identity is known. The face image to be recognized is denoted as x, and the counterfactual image is denoted as x. adv , the reference image is denoted as x refIn the face recognition module, based on the stored face recognition model, the face features in the face image to be recognized, the reference image and the counterfactual image are respectively extracted to obtain corresponding deep face features. The face recognition model is based on a deep learning module, so the obtained face features are deep face features, that is, the deep face features of the face image to be recognized, the deep face features in the reference image, and the deep face features in the counterfactual image are obtained to obtain the corresponding face recognition result based on the deep face features. The recognition result is the face recognition result obtained by whether the face recognition module recognizes the deep face features based on the face image to be recognized, the deep face features in the reference image and the deep face features in the counterfactual image. The face recognition results include two types, namely recognized and not recognized, that is, the face recognition module recognizes the face image to be recognized, the reference image and the counterfactual image respectively to obtain the corresponding face recognition results.

[0062] In step S103, after receiving the recognition results corresponding to the face image to be recognized, the reference image and the counterfactual image respectively generated by the face recognition module, the recognition interpretation module judges the recognition results corresponding to the face image to be recognized, the reference image and the counterfactual image respectively. If the recognition results corresponding to the face image to be recognized, the reference image and the counterfactual image respectively meet the conditions, the deep face features corresponding to the face image to be recognized, the reference image and the counterfactual image respectively are reconstructed into corresponding reconstructed images, and the reconstructed images are the deep face features corresponding to the face image to be recognized, the reference image and the counterfactual image respectively generated by the deep neural network. The invention relates to a method for realizing facial features, wherein the deep neural network comprises a convolutional layer, a fully connected layer, and an activation layer. During the training process of the deep neural network, three types of loss functions, namely, reconstruction loss, implicit coding guided loss, and regularization loss, are used to achieve a relatively good reconstruction effect. Based on the reconstructed image, attribute estimation is performed via a face attribute estimator to obtain face attribute data corresponding to the face image to be identified, the reference image, and the counterfactual image respectively. The face attribute estimator is based on theories and technologies in the fields of deep learning, computer vision, and artificial intelligence, and can detect and analyze various attributes of the face in the picture, such as gender, age, race, expression, whether the mouth is open, whether there is a beard, etc.

[0063] In the specific implementation process of step S103, there is an embodiment: Figure 2 As shown, the recognition and interpretation module reconstructs corresponding reconstructed images based on the face recognition result for the deep face features corresponding to the face image to be recognized, the reference image and the counterfactual image, respectively, including:

[0064] S10311, the recognition and interpretation module determines whether the face recognition results corresponding to the face image to be recognized, the reference image and the counterfactual image respectively meet the preset reconstruction conditions;

[0065] S10312: If yes, the recognition and interpretation module reconstructs corresponding reconstructed images based on the deep facial features corresponding to the facial image to be recognized, the reference image and the counterfactual image respectively.

[0066] In steps S10311-S10312, the recognition and interpretation module determines whether the face recognition results corresponding to the face image to be recognized, the reference image and the counterfactual image respectively meet the preset reconstruction conditions, and the preset reconstruction conditions are that the face recognition results of the face image to be recognized and the reference image are both recognized, that is, the recognition results of the reference image and the face image to be recognized are recognized and the recognition result of the counterfactual image is not recognized. If the recognition and interpretation module determines that the face recognition results corresponding to the face image to be recognized, the reference image and the counterfactual image respectively meet the preset reconstruction conditions, the recognition and interpretation module reconstructs the corresponding reconstructed images for the deep face features corresponding to the face image to be recognized, the reference image and the counterfactual image respectively, otherwise, no reconstruction is performed.

[0067] In the specific implementation process of step S103, there is another embodiment: obtaining corresponding facial attribute data based on the reconstructed image includes:

[0068] S10321, the recognition and interpretation module generates an attribute call instruction based on the reconstructed image, and sends the attribute call instruction to the face attribute estimator;

[0069] S10322. The facial attribute estimator performs feature extraction on the reconstructed image in response to the attribute calling instruction to obtain corresponding facial attribute data.

[0070] In steps S10321-S10322, after reconstructing the reconstructed image, the recognition and interpretation module generates an attribute call instruction and sends the attribute call instruction to the face attribute estimator. There are multiple sub-estimators in the face attribute estimator, and the sub-estimators correspond to each attribute on the face. Based on the attribute call instruction, the face attribute estimator calls all sub-estimators to extract features from the reconstructed image to obtain corresponding face attribute data, that is, to obtain the face attributes corresponding to the face image to be identified, the reference image and the counterfactual image respectively, and to parameterize the face attributes to obtain the corresponding face attribute data. Based on a certain attribute, the face attribute data corresponding to the face image to be identified, the reference image and the counterfactual image respectively are expressed as a, a ref 、aadv .

[0071] In step S104, after obtaining the facial attribute data corresponding to the face image to be identified, the reference image and the counterfactual image respectively, in order to explain the reason why the face recognition module recognizes the face, the recognition and interpretation module calculates the facial attribute difference based on the facial attribute data corresponding to the face image to be identified, the reference image and the counterfactual image respectively, so as to explain the face recognition result obtained by the face recognition module based on the facial attribute difference. By comparing the attribute difference between the face image to be identified and the counterfactual image, it can be analyzed which facial attributes have a significant impact on the recognition result. For example, if the counterfactual image significantly changes the recognition result by changing the lighting conditions, then it can be inferred that the lighting conditions are an important influencing factor, thereby forming the interpretability of the face recognition module.

[0072] In the specific implementation process of step S104, there is an embodiment: Figure 3 As shown, the recognition and interpretation module calculates the face attribute difference based on the face attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image, so as to interpret the face recognition result obtained by the face recognition module based on the face attribute difference, including:

[0073] S1041, the recognition and interpretation module calculates the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image respectively based on a preset attribute calculation network to obtain a facial attribute difference;

[0074] S1042: The recognition and interpretation module confirms the reason why the face recognition module recognizes the face based on the attribute corresponding to the face attribute difference.

[0075] In steps S1041-S1042, the recognition and interpretation module obtains the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image, i.e., a, a ref 、a adv Then, the preset attribute calculation network is called to calculate the facial attribute data corresponding to the face image to be identified, the reference image and the counterfactual image respectively to obtain the facial attribute difference. The recognition and interpretation module confirms the reason why the face recognition module identifies the face based on the attribute corresponding to the facial attribute difference. That is, if a facial attribute difference is significantly different from other facial attribute differences, then the attribute corresponding to this facial attribute difference is the reason why the face recognition module identifies the corresponding face. For example, if the facial attribute difference corresponding to the iris is significantly different from other facial attribute differences, then the iris is the reason why the face recognition module recognizes the face.

[0076] In the specific implementation process of step S1041, there is an embodiment in which the recognition and interpretation module calculates the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image respectively based on a preset attribute calculation network to obtain a facial attribute difference, including:

[0077] S10411, the recognition and interpretation module outputs the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image respectively to the attribute calculation network;

[0078] S10412. The attribute calculation network calculates the facial attribute data corresponding to the face image to be identified, the reference image and the counterfactual image respectively to obtain facial attribute differences.

[0079] In steps S10411-S10412, the recognition and interpretation module outputs the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image to the attribute calculation network, and the attribute calculation network is represented by formula (1):

[0080]

[0081] Where, formula (1) is s a is the causal effect of the attribute, ||·|| represents the Euclidean distance, and through formula (1), the recognition and interpretation module calculates the face attribute difference based on the causal effect representation on a certain attribute.

[0082] In the specific implementation process of step S1042, there is an embodiment in which the recognition interpretation module confirms the reason why the face recognition module recognizes the face based on the attribute corresponding to the face attribute difference, including:

[0083] S10421, the face recognition module determines whether the face attribute difference meets a preset recognition threshold range to determine whether there is a target attribute difference;

[0084] S10422: If yes, the recognition and interpretation module confirms that the target attribute corresponding to the target attribute difference is the reason why the face recognition module recognizes the face.

[0085] In steps S10421-S10422, the face recognition module determines whether the face attribute difference satisfies a preset recognition threshold range to determine whether there is a target attribute difference. The preset recognition threshold range is greater than 1, that is, the face attribute difference>1, then the attribute corresponding to this face attribute difference has a significant effect as the target attribute difference, and the attribute corresponding to the target attribute difference is the reason why the face recognition module recognizes the face. Since there are multiple attributes on the face, the number of target attribute differences may not be unique, and there may be multiple, then the target attribute is also not unique, and there may be multiple, and if all face attribute differences are ≤1, that is, there is no face attribute difference that satisfies the preset recognition threshold range, then the recognition and interpretation module cannot generate a corresponding interpretation, and will improve and adjust its own algorithm to obtain a specific interpretation.

[0086] Example 2

[0087] The present application also provides an explainable face recognition device based on counterfactual inference, such as Figure 4 The block diagram of an interpretable face recognition device based on counterfactual inference is shown. The functions implemented by the interpretable face recognition device based on counterfactual inference correspond to the steps of executing an interpretable face recognition method based on counterfactual inference on the terminal device. The device can be understood as a component of a server including a processor, which is applied to a face recognition system. The face recognition system includes a counterfactual module, a face recognition module and a recognition and interpretation module. The device includes:

[0088] Processing module 401, used for the counterfactual module to process the face image to be recognized to obtain a counterfactual image, and output the counterfactual image to the face recognition module;

[0089] An extraction module 402 is used for the face recognition module to respectively extract face features from the face image to be recognized, the reference image and the counterfactual image to obtain corresponding deep face features, so as to obtain a corresponding face recognition result based on the deep face features;

[0090] The reconstruction module 403 is used for the recognition and interpretation module to reconstruct corresponding reconstructed images based on the face recognition result for the deep face features corresponding to the face image to be recognized, the reference image and the counterfactual image, so as to obtain corresponding face attribute data based on the reconstructed images;

[0091] The interpretation module 404 is used for the recognition interpretation module to calculate the facial attribute difference based on the facial attribute data corresponding to the facial image to be recognized, the reference image and the counterfactual image, so as to interpret the facial recognition result obtained by the facial recognition module based on the facial attribute difference.

[0092] In a feasible implementation, the interpretation module includes:

[0093] A first calculation module, used for the recognition and interpretation module to calculate the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image respectively based on a preset attribute calculation network to obtain a facial attribute difference;

[0094] The first confirmation module is used for the recognition and interpretation module to confirm the reason why the face recognition module recognizes the face based on the attribute corresponding to the face attribute difference.

[0095] In a feasible implementation manner, the interpretation module further includes:

[0096] An output module, used for the recognition and interpretation module to output the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image respectively to the attribute calculation network;

[0097] The second calculation module is used for the attribute calculation network to calculate the facial attribute data corresponding to the face image to be identified, the reference image and the counterfactual image respectively to obtain the facial attribute difference.

[0098] In a feasible implementation, the interpretation module further includes:

[0099] A first judgment module, used for the face recognition module to judge whether the face attribute difference meets a preset recognition threshold range to judge whether there is a target attribute difference;

[0100] The second confirmation module is used for, if yes, the recognition and interpretation module confirms that the target attribute corresponding to the target attribute difference is the reason why the face recognition module recognizes the face.

[0101] In a feasible implementation manner, the processing module includes:

[0102] A calling module, used for the counterfactual module to call a preset facial image processing method based on the facial image to be identified;

[0103] The first processing module is used for the counterfactual module to process the face image to be identified based on the face image processing method to obtain the counterfactual image.

[0104] In a feasible implementation, the reconstruction module includes:

[0105] A second judgment module, used by the recognition and interpretation module to judge whether the face recognition results corresponding to the face image to be recognized, the reference image and the counterfactual image respectively meet the preset reconstruction conditions;

[0106] The first reconstruction module is used to reconstruct corresponding reconstructed images based on the deep facial features corresponding to the face image to be identified, the reference image and the counterfactual image respectively.

[0107] In a feasible implementation manner, the reconstruction module further includes:

[0108] A generating module, used for the recognition and interpretation module to generate an attribute calling instruction based on the reconstructed image, and send the attribute calling instruction to the face attribute estimator;

[0109] A response module is used for the face attribute estimator to extract features from the reconstructed image in response to the attribute call instruction to obtain corresponding face attribute data.

[0110] Example 3

[0111] The present application also provides an electronic device, such as Figure 5 As shown, it includes: a processor 501, a memory 502 and a bus 503, the memory 502 stores machine-readable instructions executable by the processor 501, when the electronic device is running, the processor 501 and the memory 502 communicate through the bus 503, and when the machine-readable instructions are executed by the processor 501, any step of an explainable face recognition method based on counterfactual inference is performed.

[0112] Example 4

[0113] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes any one of the steps of an explainable face recognition method based on counterfactual inference.

[0114] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0115] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0117] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a platform server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0118] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An explainable face recognition method based on counterfactual inference, characterized in that: Applied to a face recognition system, the face recognition system includes a counterfactual module, a face recognition module and a recognition interpretation module, and the method includes: The counterfactual module processes the face image to be recognized to obtain a counterfactual image, and outputs the counterfactual image to the face recognition module; The face recognition module extracts face features from the face image to be recognized, the reference image and the counterfactual image respectively to obtain corresponding deep face features, so as to obtain corresponding face recognition results based on the deep face features; The recognition and interpretation module reconstructs corresponding reconstructed images based on the face recognition result for the deep face features corresponding to the face image to be recognized, the reference image and the counterfactual image, so as to obtain corresponding face attribute data based on the reconstructed images; The recognition and interpretation module calculates facial attribute differences based on the facial attribute data corresponding to the facial image to be recognized, the reference image and the counterfactual image, so as to interpret the facial recognition result obtained by the facial recognition module based on the facial attribute differences.

2. The method according to claim 1, characterized in that The recognition and interpretation module calculates a facial attribute difference based on the facial attribute data corresponding to the facial image to be recognized, the reference image, and the counterfactual image, so as to interpret the facial recognition result obtained by the facial recognition module based on the facial attribute difference, including: The recognition and interpretation module calculates the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image respectively based on a preset attribute calculation network to obtain a facial attribute difference; The recognition interpretation module confirms the reason why the face recognition module recognizes the face based on the attribute corresponding to the face attribute difference.

3. The method according to claim 2, characterized in that The recognition and interpretation module calculates the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image respectively based on a preset attribute calculation network to obtain facial attribute differences, including: The recognition and interpretation module outputs the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image to the attribute calculation network; The attribute calculation network calculates the facial attribute data corresponding to the face image to be identified, the reference image and the counterfactual image respectively to obtain facial attribute differences.

4. The method according to claim 2, characterized in that: The recognition interpretation module confirms the reason why the face recognition module recognizes the face based on the attribute corresponding to the face attribute difference, including: The face recognition module determines whether the face attribute difference meets a preset recognition threshold range to determine whether there is a target attribute difference; If so, the recognition and interpretation module confirms that the target attribute corresponding to the target attribute difference is the reason why the face recognition module recognizes the face.

5. The method according to claim 1, characterized in that The counterfactual module processes the face image to be recognized to obtain a counterfactual image, and outputs the counterfactual image to the face recognition module, including: The counterfactual module calls a preset facial image processing method based on the facial image to be identified; The counterfactual module processes the face image to be identified based on the face image processing method to obtain the counterfactual image.

6. The method according to claim 1, characterized in that The recognition and interpretation module reconstructs corresponding reconstructed images based on the face recognition result for the deep face features corresponding to the face image to be recognized, the reference image, and the counterfactual image, respectively, including: The recognition interpretation module determines whether the face recognition results corresponding to the face image to be recognized, the reference image and the counterfactual image respectively meet the preset reconstruction conditions; If so, the recognition and interpretation module reconstructs corresponding reconstructed images based on the deep facial features corresponding to the face image to be recognized, the reference image and the counterfactual image respectively.

7. The method according to claim 1, characterized in that The obtaining corresponding facial attribute data based on the reconstructed image includes: The recognition and interpretation module generates an attribute call instruction based on the reconstructed image, and sends the attribute call instruction to the face attribute estimator; The face attribute estimator extracts features from the reconstructed image in response to the attribute calling instruction to obtain corresponding face attribute data.

8. An explainable face recognition device based on counterfactual inference, characterized in that: Applied to a face recognition system, the face recognition system includes a counterfactual module, a face recognition module and a recognition interpretation module, and the device includes: A processing module, used for the counterfactual module to process the face image to be recognized to obtain a counterfactual image, and output the counterfactual image to the face recognition module; An extraction module, used for the face recognition module to respectively extract the face features in the face image to be recognized, the reference image and the counterfactual image to obtain corresponding deep face features, so as to obtain a corresponding face recognition result based on the deep face features; A reconstruction module, used for the recognition and interpretation module to reconstruct corresponding reconstructed images based on the face recognition result for the deep face features corresponding to the face image to be recognized, the reference image and the counterfactual image, so as to obtain corresponding face attribute data based on the reconstructed images; An interpretation module is used for the recognition interpretation module to calculate the facial attribute difference based on the facial attribute data corresponding to the face image to be recognized, the reference image and the counterfactual image, so as to interpret the face recognition result obtained by the face recognition module based on the facial attribute difference.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of an explainable face recognition method based on counterfactual inference are performed as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of an explainable face recognition method based on counterfactual inference as described in any one of claims 1 to 7.

Images