Facial recognition model training method and device, electronic equipment and storage medium

Abstract

The invention discloses a face recognition model training method and device, electronic equipment and a storage medium, and the method comprises the steps: training a first face recognition model and an initial white box model based on a first sample image, and enabling the initial white box model to be a Transform model; after training of the first face recognition model and the initial white box model is completed, a second face recognition model corresponding to the first face model and a target white box model corresponding to the initial white box model are obtained; determining an initial sample image, and inputting the initial sample image into the target white box model to extract a first attention weight map corresponding to the initial sample image; executing attention sorting interference, data interference and addition disturbance on the first attention weight map, and superposing the first attention weight map with the initial sample image to obtain an adversarial sample image; and training a second face recognition model based on the adversarial sample image to obtain a target face recognition model. The risk of misjudgment caused by hostile attacks can be reduced, and high reliability and safety are ensured.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, and more particularly to training methods, devices, electronic devices, and storage media for facial recognition models. Background Technology

[0002] In financial institutions and mobile banking systems, facial recognition technology has become one of the core means of user authentication and transaction authorization. However, with the widespread application of artificial intelligence technology, financial institutions face increasingly complex cybersecurity threats, such as adversarial attacks and data breaches. Adversarial attacks, by generating carefully designed adversarial examples, can mislead facial recognition models, leading to misjudgments or unauthorized authorizations, seriously threatening user funds and system stability. Especially in black-box attack scenarios, attackers do not need to understand the internal structure of the model; they can launch attacks simply by observing the inputs and outputs, further increasing the difficulty of defense. Therefore, developing a technology that can effectively improve the robustness and security of facial recognition models has become an urgent problem for financial institutions. Summary of the Invention

[0003] This invention provides a training method, apparatus, electronic device, and storage medium for a facial recognition model, in order to reduce the risk of misjudgment caused by malicious attacks and ensure its high reliability and security in financial institutions and mobile banking systems.

[0004] According to one aspect of the present invention, a method for training a facial recognition model is provided, comprising:

[0005] A first face recognition model and an initial white-box model are trained based on the first sample image, wherein the initial white-box model is a Transformer model;

[0006] After the first face recognition model and the initial white-box model are trained, a second face recognition model corresponding to the first face model and a target white-box model corresponding to the initial white-box model are obtained.

[0007] An initial sample image is determined and input into the target white-box model to extract the first attention weight map corresponding to the initial sample image;

[0008] Attention ranking interference, data interference, and perturbation are applied to the first attention weight map, and then superimposed with the initial sample image to obtain an adversarial sample image.

[0009] The second face recognition model is trained based on the adversarial sample images to obtain the target face recognition model.

[0010] According to another aspect of the present invention, a training apparatus for a facial recognition model is provided, comprising:

[0011] An initial training module is used to train a first face recognition model and an initial white-box model based on a first sample image, wherein the initial white-box model is a Transformer model;

[0012] The model determination module is used to obtain a second face recognition model corresponding to the first face model and a target white box model corresponding to the initial white box model after the first face recognition model and the initial white box model have been trained.

[0013] The weight extraction module is used to determine the initial sample image and input the initial sample image into the target white-box model to extract the first attention weight map corresponding to the initial sample image;

[0014] The sample generation module is used to perform attention ranking interference, data interference and perturbation on the first attention weight map, and superimpose it with the initial sample image to obtain an adversarial sample image;

[0015] The target training module is used to train the second face recognition model based on the adversarial sample image to obtain the target face recognition model.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor;

[0018] and memory that is communicatively connected to at least one processor;

[0019] The memory stores a computer program that can be executed by at least one processor, which enables the at least one processor to execute the training method of the face recognition model according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute a training method for a facial recognition model according to any embodiment of the present invention.

[0021] The technical solution of this invention improves existing defense techniques by combining self-attention mechanisms and data augmentation. The self-attention mechanism of the Transformer model analyzes the impact of the input image sequence on the output of the original model (the second facial recognition model), identifies regions that play a crucial role in the original model's decisions (such as key facial feature points), and adds minor noise to these regions to generate high-quality adversarial examples. These adversarial examples simulate potential black-box attack scenarios, helping the original model better identify and resist malicious attacks. By adding the generated adversarial examples to the original training dataset in an appropriate proportion and retraining the original model using the expanded dataset, it can better identify and resist malicious attacks. Finally, by generating adversarial examples multiple times and retraining the model, the model's performance and robustness are gradually optimized, reducing the risk of misjudgment due to malicious attacks and ensuring its high reliability and security in financial institutions and mobile banking systems.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating a training method for a facial recognition model provided in an embodiment of the present invention;

[0025] Figure 2 A flowchart illustrating another method for training a facial recognition model provided in an embodiment of the present invention;

[0026] Figure 3 This is an attention flow graph provided in Embodiment 2 of the present invention;

[0027] Figure 4 This is a weight information exchange diagram provided in Embodiment 2 of the present invention;

[0028] Figure 5 This is a principle block diagram of weight ranking interference provided in Embodiment 2 of the present invention;

[0029] Figure 6 This is a data interference flowchart provided in Embodiment 2 of the present invention;

[0030] Figure 7This is a block diagram illustrating the principle of adversarial sample generation provided in Embodiment 2 of the present invention;

[0031] Figure 8 A flowchart of the overall adversarial training process provided for a preferred embodiment of the present invention;

[0032] Figure 9 A schematic diagram of the structure of a training device for a facial recognition model provided in an embodiment of the present invention;

[0033] Figure 10 A schematic diagram of the structure of an electronic device for implementing the training method of the facial recognition model in this embodiment of the invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] Before introducing specific embodiments of the present invention, the relevant terms are explained as follows:

[0037] Black-box attacks: Black-box attacks refer to attacks launched by attackers who do not understand the internal structure and parameters of a model, but only by observing its inputs and outputs. In facial recognition scenarios in financial institutions, black-box attacks can bypass facial recognition systems by modifying input images. This invention specifically addresses defenses against this type of attack.

[0038] Adversarial attacks are techniques used to mislead machine learning models using carefully crafted input samples (i.e., adversarial examples). Attackers make small but specific modifications to the input data, causing the model to produce incorrect outputs. In facial recognition scenarios at financial institutions, adversarial attacks may lead to the system misidentifying users, thereby posing security risks.

[0039] Adversarial Training: Adversarial training is a training method to improve the robustness of a model. By adding adversarial examples to the training dataset, the model can learn how to identify and resist these malicious examples, thereby improving its performance in the face of adversarial attacks. This invention utilizes adversarial training to enhance the robustness of a face recognition model.

[0040] Data augmentation is a technique that generates new training samples by transforming or adding noise to the original data. Its purpose is to improve the generalization ability of a model, enabling it to better adapt to different input scenarios. In this invention, data augmentation techniques are used to generate adversarial examples and expand the training dataset.

[0041] The Transformer model is a deep learning architecture based on the self-attention mechanism, primarily used for natural language processing (NLP) tasks such as machine translation and text generation. In recent years, the Transformer model has also been widely applied in computer vision (CV), such as image classification, object detection, and face recognition.

[0042] Self-Attention Mechanism: A core technique in Transformer models, self-attention captures the relationships between different locations in an input sequence. In face recognition, it helps the model identify key regions in an image (such as eyes, nose, and mouth), thus generating adversarial examples more accurately.

[0043] Figure 1 This is a flowchart illustrating a training method for a facial recognition model provided in an embodiment of the present invention. This embodiment is applicable to improving the defense capabilities of facial recognition models against adversarial attacks, reducing the risk of misjudgment caused by malicious attacks, and thus ensuring the secure operation of financial institutions and mobile banking systems. This method can be executed by a facial recognition model training device, which can be implemented in hardware and / or software and can be configured in a computer device. Figure 1 As shown, the method specifically includes the following steps:

[0044] S110. Train the first face recognition model and the initial white-box model based on the first sample image.

[0045] In this embodiment of the invention, the first sample image refers to the original sample used to train the first facial recognition model, and the first facial recognition model refers to the original, untrained facial recognition model. Data annotation can be performed on the first sample image, which can then be used as training samples to train the first facial recognition model, enabling it to have facial recognition functionality. The initial white-box model is a Transformer model, and the first sample image can also be used to train the initial white-box model, enabling it to also have facial recognition functionality.

[0046] S120. After the first face recognition model and the initial white box model are trained, a second face recognition model corresponding to the first face model and a target white box model corresponding to the initial white box model are obtained.

[0047] The second facial recognition model and the target white-box model are both models obtained after training.

[0048] In some embodiments, training a first face recognition model and an initial white-box model based on a first sample image includes: training a first face recognition model based on the first sample image and training an initial white-box model based on the first sample image; determining whether the first face recognition model and the initial white-box model meet preset fitting conditions; if both the first face recognition model and the initial white-box model meet the preset fitting conditions, determining the first face recognition model as a second face recognition model and the initial white-box model as a target white-box model; if the first face recognition model and / or the initial white-box model do not meet the preset fitting conditions, then continuing to train the first face recognition model and the initial white-box model until the preset fitting conditions are met.

[0049] Specifically, a first face recognition model and an initial white-box model can be trained in parallel based on the same batch of first sample images, and their performance can be evaluated separately to see if they meet preset fitting conditions, such as validation set accuracy and loss convergence. If both models meet the criteria, the first face recognition model is named the second face recognition model, and the initial white-box model is named the target white-box model; if either model fails to meet the criteria, training continues until the conditions are met.

[0050] S130. Determine the initial sample image and input it into the target white-box model to extract the first attention weight map corresponding to the initial sample image.

[0051] The initial sample image refers to the sample image used to generate adversarial examples. By processing the initial sample image, adversarial examples can be obtained. The first attention weight map can be used to represent the attention weight values ​​corresponding to different regions in the initial sample image.

[0052] It is understandable that both the target white-box model and the second face recognition model are trained based on the first sample image. Therefore, the first attention weight map can reflect the degree of attention the second face recognition model pays to different regions of the image during actual face recognition. For example, if the attention to facial key points such as eyes and nose is high, the corresponding attention weight value of that region in the image will also be higher.

[0053] S140. Perform attention ranking perturbation, data perturbation and add perturbation on the first attention weight map, and superimpose it with the initial sample image to obtain the adversarial sample image.

[0054] Among them, attention ranking interference refers to adjusting the weights of different regions in the first attention weight map, such as reducing the weight of facial key points and strengthening the weight of background regions.

[0055] Specifically, operations such as attention ranking perturbation, data perturbation, and adding disturbances are performed on the first attention weight map. The perturbed weight map is then superimposed on the original image to generate adversarial example images. Adversarial examples refer to images that can mislead the second face recognition model.

[0056] S150. Train the second face recognition model based on adversarial sample images to obtain the target face recognition model.

[0057] Specifically, adversarial sample images can be labeled, and a second facial recognition model can be trained based on the labeled adversarial samples to obtain the target facial recognition model. The target facial recognition model can better identify and resist malicious attacks.

[0058] In some embodiments, after obtaining the target face recognition model, a face image to be recognized can also be obtained, and the face image to be recognized can be input into the target face recognition model to obtain the target recognition result output by the target face recognition model.

[0059] The facial image to be recognized refers to the facial image that needs to be recognized in practical applications. Inputting it into the target facial recognition model can yield a relatively accurate target recognition result.

[0060] The technical solution of this invention improves existing defense techniques by combining self-attention mechanisms and data augmentation. The self-attention mechanism of the Transformer model analyzes the impact of the input image sequence on the output of the original model (the second facial recognition model), identifies regions that play a crucial role in the original model's decisions (such as key facial feature points), and adds minor noise to these regions to generate high-quality adversarial examples. These adversarial examples simulate potential black-box attack scenarios, helping the original model better identify and resist malicious attacks. By adding the generated adversarial examples to the original training dataset in an appropriate proportion and retraining the original model using the expanded dataset, it can better identify and resist malicious attacks. Finally, by generating adversarial examples multiple times and retraining the model, the model's performance and robustness are gradually optimized, reducing the risk of misjudgment due to malicious attacks and ensuring its high reliability and security in financial institutions and mobile banking systems.

[0061] Figure 2 This is a flowchart illustrating another training method for a face recognition model provided by an embodiment of the present invention. Based on the above embodiments, this embodiment further optimizes the process of generating adversarial examples. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here. Figure 2 As shown, the method specifically includes the following steps:

[0062] S210. Train the first face recognition model and the initial white-box model based on the first sample image.

[0063] S220. After the first face recognition model and the initial white box model are trained, a second face recognition model corresponding to the first face model and a target white box model corresponding to the initial white box model are obtained.

[0064] S230. Determine the initial sample image and input the initial sample image into the target white-box model to extract the first attention weight map corresponding to the target block layer of the target white-box model.

[0065] Among them, the block layer responsible for high-level semantic feature extraction in the target white-box model can be selected as the target block layer.

[0066] For example, since the embodiment of the present invention adopts a relatively novel ViT model, and based on its characteristics, the block with a higher proportion of decision weights in the later layers of the model is selected as the target block layer.

[0067] Furthermore, after inputting the initial sample image into the target white-box model, the Attention Rollout method can be used to extract attention from the initial sample image to obtain multiple attention weight maps. For example, each target block layer corresponds to a first attention weight map.

[0068] Here, a target patch can be understood as multiple image tiles obtained after segmenting the initial sample image. The first attention weight map includes the first attention weight sub-map corresponding to each target patch, and the target patches together form the initial sample image. For each first attention weight map, based on the characteristics of ViT patch partitioning, each first attention weight map includes the first attention weight map of 14*14 patches.

[0069] S240. For the first attention weight graph corresponding to the target block layer, sort the first attention weight subgraphs of each target patch.

[0070] S250. Based on the sorting results, perform attention weight exchange between each target patch to obtain the second attention weight subgraph of each target patch.

[0071] S260. Combine the second attention weight subgraphs of each target patch to obtain the second attention weight graph corresponding to the first attention weight graph.

[0072] The second attention weight map is the weight map obtained after attention ranking interference.

[0073] Specifically, the weight information of the high-weight target patch and the weight information of the low-weight target patch are exchanged to generate a new weight map, namely the second attention weight map.

[0074] S270. Perform data perturbation on the second attention weight map to obtain the third attention weight map, and add perturbation to the third attention weight map to obtain the fourth attention weight map.

[0075] S280. The fourth attention weight map is superimposed on the initial sample image to obtain the adversarial sample image.

[0076] Specifically, data perturbation is applied to the second attention weight map to obtain the third attention weight map, and perturbation is added to the third attention weight map to obtain the fourth attention weight map. The fourth attention weight map is then superimposed on the initial sample image to obtain the adversarial example image. This process includes the following three parts:

[0077] (1) Attention weight ranking based on ViT:

[0078] Attention Rollout is used to obtain the attention weights of each block in the ViT model for the entire input image. Due to the characteristics of the ViT architecture, the input image is divided into 196 patches of size 16*16. Therefore, after obtaining the attention weight information, the attention weights of each patch are weighted and averaged to obtain the attention patch weight map, i.e., the first attention weight map. Then, the weights between different patches are swapped. For example, the attention weights can be sorted and the weight information of the patches can be swapped from the beginning to the end to obtain the second attention weight map.

[0079] (2) Data interference:

[0080] To ensure information consistency, data interference is performed on all patches, namely by scaling and padding and then pooling back to the original patch size, thereby changing the position of weight information in the patch and obtaining the third attention weight map.

[0081] (3) Adversarial example generation:

[0082] Multiplying the third attention weight map by an adversarial perturbation induces erroneous information in class tokens layer by layer by disrupting inter-layer information, ultimately causing the model to make incorrect decisions.

[0083] After obtaining the generated adversarial examples, the adversarial examples are labeled and added to the existing face recognition model training set. Then, the white-box model is used to fit the trained existing model, and the model performance and robustness are iteratively optimized in the form of adversarial training.

[0084] S290. Train the second face recognition model based on adversarial sample images to obtain the target face recognition model.

[0085] To further clarify the technical effects of the present invention, the research process of the present invention will be further explained.

[0086] First, this invention studies the attention mechanism of ViT: ViT possesses unique attention weights, and the class token used for classification by ViT is influenced by both the attention generated by the preceding blocks and the class token. Therefore, describing the attention weights of the ViT model using a single attention weight map is extremely difficult. Second, the visualization of the weight map cannot be simply done using Grad-CAM. The main reason is that in CNN models, feature maps from multiple channels are weighted, with the weights determined by the gradients assigned to each. However, in the ViT model, following the Grad-CAM approach, feature maps from multiple patches are weighted, with the weights also determined by the gradients assigned to each. Since this would change the feature maps of the patches to the required input size of 224*224 pixels, it is unreasonable. Therefore, based on this characteristic of the ViT architecture, existing technologies directly use attention weight matrices and propose an attention flow graph format. In this format, the attention of each layer is residually connected to the attention of the previous layer, and self-attention weights are considered. Thus, the attention weight map of each layer can be extracted and utilized. In this process, by considering the class token information of each layer, a relatively complete heatmap is gradually generated. Under these conditions, exploratory experiments were conducted. For a ViT architecture model (ViT-B / 16) with L layers, the computation was performed from l... i Layer to l j All attentions of layer (j < i), for a given layer in l i The node v of the layer is calculated to l j The attention information passed to node u in the layer is calculated by multiplying it by the weights of all edges in the path, and all possible paths between the two nodes are calculated by recursively calling the attention weight matrices of all layers below, as shown in Equation (1).

[0087]

[0088] After calculating the attention weights for all layers, a threshold is set to draw a heatmap for each layer, and the extracted attention flow is as follows: Figure 3 As shown, Figure 3 This is an attention flow graph provided in Embodiment 2 of the present invention.

[0089] As can be seen from the attention flow graph of the model, the weight ratio of each block in the decision of the input image is different. Therefore, it is feasible to carry out a guided attack on the attention weight graph generated by different blocks. That is, the idea of ​​guiding feature information is introduced into the process of generating class token (classification decision head) of each block, so that it is accumulated into the class token that ultimately determines the decision, thereby achieving the goal of making the judgment incorrect.

[0090] In a preferred embodiment, the attention weight swapping process is optimized as follows:

[0091] After extracting attention from the input image using the Attention Rollout method, we obtain the attention weight map for each block in ViT. However, unlike CNN networks, ViT cannot directly segment attention weight regions by setting a threshold like Grad-CAM. Therefore, we choose to average the weight information contained in the pixels of each patch to represent the weight information of each patch. Finally, we sort and compare the weight values ​​of all patches to obtain a sequence of patch weight information sorted by size in each block. The sorted sequence reveals the attention weight of each patch in the input image, allowing us to swap the weight information of patches with higher weights with those of lower weights. Figure 4 The weight information exchange graph provided in Embodiment 2 of the present invention has the following specific exchange method: Figure 4 As shown.

[0092] Finally, the mask generated from the total weight map is multiplied by the adversarial perturbation for subsequent attacks. By influencing the class token decisions at each layer, this effect gradually accumulates in later block layers, ultimately causing the model to misclassify. The process is as follows: Figure 5 The diagram shown is a principle block diagram of the weight ranking interference provided in Embodiment 2 of the present invention.

[0093] The specific approach involves sorting all patches by weight within each layer, recording the corresponding patch numbers, and then swapping high-weight and low-weight information. This swapping follows a one-to-one correspondence: the weight of a patch with a higher attention weight is added to the weight of a patch with a lower attention weight, and vice versa, thus distributing the feature information. For a 224*224 input image, we can obtain 196 patches. We can then sort these 196 patches by attention weight, perform 98 information swaps (corresponding to the first and last patches), and then re-embed the patch weight information into the original weight map to generate a new overall weight map.

[0094] In some embodiments, performing data perturbation on the second attention weight map to obtain the third attention weight map includes: performing upsampling, zero-filling, and pooling operations sequentially on each second attention weight subgraph in the second attention weight map to obtain the third attention weight subgraph; and combining the third attention weight subgraphs corresponding to all the second attention weight subgraphs to obtain the third attention weight map.

[0095] Specifically, such as Figure 6 The diagram shown is a data interference flowchart provided in Embodiment 2 of the present invention. To further increase randomness, reduce the "overfitting" of the source model, and improve the transferability of adversarial examples, a data interference module is designed on the extracted patch weight information. This module takes the patch weight information map as input and performs a series of operations such as upsampling, zero-padding, and pooling to further interfere with the patch's own weight information. Because the attention weight distribution in different layers changes during the iteration process, to maintain consistency in each iteration, data interference is performed on all patch weight information, as follows: Figure 6 As shown.

[0096] Figure 6 As shown, after obtaining the input image in each iteration, a total of 196 patch weight information can be obtained. Then, bilinear interpolation is performed on each patch weight information, which will increase the size of the 16*16 patch weight information. However, the result still needs to be embedded into the original feature information weight map. Therefore, for the patch weight information with increased size, the size is finally restored to 16*16 by pooling to achieve better results.

[0097] In some embodiments, adding a perturbation to the third attention weight map to obtain a fourth attention weight map includes: multiplying the third attention weight map by an initial perturbation and superimposing it with an initial sample image to obtain a candidate sample image; calculating a loss function based on the candidate sample image and updating the initial perturbation based on the loss function to obtain a target perturbation; and multiplying the target perturbation by the third attention weight map to obtain a fourth attention weight map.

[0098] In this embodiment of the invention, it is noted that traditional feature attacks utilize attention to attack high-level features. While this method performs well on CNN-based image classifiers, it performs poorly on ViT-based image classifiers that focus more on global information. Therefore, an attention mechanism is used to extract attention weight maps from deeper blocks, guiding the attack on the feature information of each block, ultimately accumulating influence on the class token used for decision-making in the last layer, achieving the attack objective and improving the transferability of adversarial examples. Figure 7The diagram shown is a principle block diagram of adversarial sample generation provided in Embodiment 2 of the present invention.

[0099] Depend on Figure 7 It can be seen that by introducing an attention mechanism and combining it with the characteristics of the ViT architecture itself, the deeper block attention weight information is extracted. By dispersing the attention weight information of each layer, a total weight map is generated and multiplied by an adversarial perturbation. Finally, it is superimposed on the input image, causing ViT's extraction of global information to make incorrect judgments on the final accumulated class token. This is used to create adversarial examples, giving them stronger generalization ability. The attack algorithm's objective function and iterative solution formula can be defined as equations (2), (3), and (4):

[0100]

[0101] δ0=0 (3)

[0102] δ i+1 =Clip ε {δ i +α·sign(▽ δ J((X+AT i (X)⊙δ),y gt ;θ)+λ||δ||2)} (4)

[0103] X adv =X+δ N (5)

[0104] In equation (2), X represents the input image; ⊙ represents pixel-level multiplication; δ represents the generated adversarial perturbation, which is restricted to ε; when N represents the number of iterations, AT i (X) represents the total weight map obtained after the input image has undergone a series of operations such as attention extraction, data perturbation, and 98 weight swaps on the (12-N+i)th block of the white-box model. A larger l2 norm is added here, which increases the distance between the input image and the original image, thus resulting in adversarial examples with a higher attack success rate. In handling this norm, a hyperparameter λ is chosen to control the balance between the loss function and the regularization term. Equations (3) and (4) are the iterative update formulas for solving the optimization formula, and equation (5) represents the adversarial example generation formula after iteration.

[0105] Figure 8 This is a flowchart illustrating the overall adversarial training process, provided as a preferred embodiment of the present invention. Figure 8As shown, this method combines existing face recognition models and uses a white-box ViT (Vision Transformer) model to fit the input and output. When the fit reaches a certain level, noise is added to generate adversarial examples using the reverse gradient descent method proposed in this paper. These examples are then added to the dataset of the existing model for adversarial training, ultimately improving the robustness of the model.

[0106] The technical solution of this invention improves existing defense techniques by combining self-attention mechanisms and data augmentation. The self-attention mechanism of the Transformer model analyzes the impact of the input image sequence on the output of the original model (the second facial recognition model), identifies regions that play a crucial role in the original model's decisions (such as key facial feature points), and adds minor noise to these regions to generate high-quality adversarial examples. These adversarial examples simulate potential black-box attack scenarios, helping the original model better identify and resist malicious attacks. By adding the generated adversarial examples to the original training dataset in an appropriate proportion and retraining the original model using the expanded dataset, it can better identify and resist malicious attacks. Finally, by generating adversarial examples multiple times and retraining the model, the model's performance and robustness are gradually optimized, reducing the risk of misjudgment due to malicious attacks and ensuring its high reliability and security in financial institutions and mobile banking systems.

[0107] Figure 9 This is a schematic diagram of a training device for a facial recognition model provided in an embodiment of the present invention. Figure 9 As shown, the device includes:

[0108] The initial training module 310 is used to train the first face recognition model and the initial white-box model based on the first sample image, wherein the initial white-box model is a Transformer model;

[0109] The model determination module 320 is used to obtain a second face recognition model corresponding to the first face model and a target white box model corresponding to the initial white box model after the first face recognition model and the initial white box model have been trained.

[0110] The weight extraction module 330 is used to determine the initial sample image and input the initial sample image into the target white-box model to extract the first attention weight map corresponding to the initial sample image;

[0111] The sample generation module 340 is used to perform attention ranking interference, data interference and add perturbation on the first attention weight map, and superimpose it with the initial sample image to obtain an adversarial sample image.

[0112] The target training module 350 is used to train a second face recognition model based on adversarial sample images to obtain the target face recognition model.

[0113] Optional, initial training module 310, specifically used for:

[0114] A first face recognition model is trained based on the first sample image, and an initial white-box model is trained based on the first sample image;

[0115] Determine whether the first facial recognition model and the initial white-box model meet the preset fitting conditions;

[0116] If both the first face recognition model and the initial white-box model meet the preset fitting conditions, the first face recognition model is determined as the second face recognition model, and the initial white-box model is determined as the target white-box model.

[0117] If the first face recognition model and / or the initial white-box model do not meet the preset fitting conditions, the first face recognition model and the initial white-box model will continue to be trained until the preset fitting conditions are met.

[0118] Optional, the weight extraction module 330 is specifically used for:

[0119] The initial sample image is input into the target white-box model to extract the first attention weight map corresponding to the target block layer of the target white-box model;

[0120] The first attention weight map includes the first attention weight sub-map corresponding to each target patch, and each target patch forms the initial sample image.

[0121] Optionally, the sample generation module 340 is specifically used for:

[0122] For the first attention weight graph corresponding to the target block layer, sort the first attention weight subgraphs of each target patch;

[0123] Based on the sorting results, attention weights are swapped between each target patch to obtain the second attention weight subgraph of each target patch.

[0124] The second attention weight subgraphs of each target patch are combined to obtain the second attention weight graph corresponding to the first attention weight graph. The second attention weight graph is the weight graph obtained after attention ranking interference.

[0125] Optionally, the sample generation module 340 is also used for:

[0126] The second attention weight map is perturbed to obtain the third attention weight map, and the third attention weight map is perturbed to obtain the fourth attention weight map.

[0127] The adversarial sample image is obtained by overlaying the fourth attention weight map with the initial sample image.

[0128] Optionally, the sample generation module 340 is also used for:

[0129] For each second attention weight subgraph in the second attention weight graph, perform upsampling, zero-padding, and pooling operations in sequence to obtain the third attention weight subgraph.

[0130] Combine all the third attention weight subgraphs corresponding to the second attention weight subgraphs to obtain the third attention weight graph.

[0131] Optionally, the sample generation module 340 is also used for:

[0132] The third attention weight map is multiplied by the initial perturbation and superimposed with the initial sample image to obtain the candidate sample image;

[0133] The loss function is calculated based on the candidate sample images, and the initial perturbation is updated based on the loss function to obtain the target perturbation;

[0134] Multiply the target perturbation by the third attention weight map to obtain the fourth attention weight map.

[0135] The training device for the facial recognition model provided in this embodiment of the invention can execute the training method for the facial recognition model provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0136] Figure 10 This is a schematic diagram of the structure of an electronic device for implementing the training method of the facial recognition model according to embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0137] like Figure 10As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0138] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0139] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the training methods for a facial recognition model.

[0140] In some embodiments, the training method for the facial recognition model can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the training method for the facial recognition model described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the training method for the facial recognition model by any other suitable means (e.g., by means of firmware).

[0141] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0145] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0146] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0147] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0148] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for training a facial recognition model, characterized in that, include: A first face recognition model and an initial white-box model are trained based on the first sample image, wherein the initial white-box model is a Transformer model; After the first face recognition model and the initial white-box model are trained, a second face recognition model corresponding to the first face model and a target white-box model corresponding to the initial white-box model are obtained. An initial sample image is determined and input into the target white-box model to extract the first attention weight map corresponding to the initial sample image; Attention ranking interference, data interference, and perturbation are applied to the first attention weight map, and then superimposed with the initial sample image to obtain an adversarial sample image. The second face recognition model is trained based on the adversarial sample images to obtain the target face recognition model.

2. The method according to claim 1, characterized in that, The training of the first face recognition model and the initial white-box model based on the first sample image includes: The first face recognition model is trained based on the first sample image, and the initial white-box model is trained based on the first sample image; Determine whether the first facial recognition model and the initial white-box model meet the preset fitting conditions; If both the first facial recognition model and the initial white-box model meet the preset fitting conditions, the first facial recognition model is determined as the second facial recognition model, and the initial white-box model is determined as the target white-box model. If the first face recognition model and / or the initial white-box model do not meet the preset fitting conditions, then the first face recognition model and the initial white-box model continue to be trained until the preset fitting conditions are met.

3. The method according to claim 1, characterized in that, The step of inputting the initial sample image into the target white-box model to extract the first attention weight map corresponding to the initial sample image includes: The initial sample image is input into the target white-box model to extract the first attention weight map corresponding to the target block layer of the target white-box model; The first attention weight map includes a first attention weight sub-map corresponding to each target patch, and each target patch constitutes the initial sample image.

4. The method according to claim 3, characterized in that, The process of performing attention ranking perturbation on the first attention weight graph includes: For the first attention weight map corresponding to the target block layer, sort the first attention weight sub-maps of each target pattern; Based on the sorting results, attention weights are exchanged between each target patch to obtain a second attention weight subgraph for each target patch. The second attention weight subgraphs of each target patch are combined to obtain the second attention weight graph corresponding to the first attention weight graph, wherein the second attention weight graph is a weight graph obtained after attention ranking interference.

5. The method according to claim 4, characterized in that, The step of performing data perturbation and adding disturbances to the first attention weight map, and then superimposing it with the initial sample image to obtain an adversarial sample image, includes: Data perturbation is performed on the second attention weight map to obtain a third attention weight map, and perturbation is added to the third attention weight map to obtain a fourth attention weight map; The fourth attention weight map is superimposed on the initial sample image to obtain the adversarial sample image.

6. The method according to claim 5, characterized in that, The step of performing data perturbation on the second attention weight map to obtain the third attention weight map includes: For each second attention weight subgraph in the second attention weight graph, upsampling, zero-filling, and pooling operations are performed sequentially to obtain the third attention weight subgraph. The third attention weight subgraph is obtained by combining all the third attention weight subgraphs corresponding to the second attention weight subgraphs.

7. The method according to claim 5, characterized in that, The step of adding perturbations to the third attention weight map to obtain the fourth attention weight map includes: The third attention weight map is multiplied by the initial perturbation and superimposed with the initial sample image to obtain the candidate sample image; The loss function is calculated based on the candidate sample image, and the initial perturbation is updated based on the loss function to obtain the target perturbation; The target perturbation is multiplied by the third attention weight map to obtain the fourth attention weight map.

8. A training device for a facial recognition model, characterized in that, include: An initial training module is used to train a first face recognition model and an initial white-box model based on a first sample image, wherein the initial white-box model is a Transformer model; The model determination module is used to obtain a second face recognition model corresponding to the first face model and a target white box model corresponding to the initial white box model after the first face recognition model and the initial white box model have been trained. The weight extraction module is used to determine the initial sample image and input the initial sample image into the target white-box model to extract the first attention weight map corresponding to the initial sample image; The sample generation module is used to perform attention ranking interference, data interference and perturbation on the first attention weight map, and superimpose it with the initial sample image to obtain an adversarial sample image; The target training module is used to train the second face recognition model based on the adversarial sample image to obtain the target face recognition model.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the training method for the face recognition model according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the training method of the facial recognition model according to any one of claims 1-7.

Images