An attack detection method, device and equipment in face recognition

Abstract

The embodiment of the specification discloses an attack detection method, device and equipment in face recognition. By acquiring a training sample containing a partially occluded image, the partially occluded image is reconstructed to generate a reconstructed image; a first model is trained according to the difference between the reconstructed image and the training sample; and a second model is trained according to the first model, wherein the second model is a lightweight model that is partially heterogeneous with the first model; a picture to be recognized containing a face is acquired, and attack detection is performed using the second model, so as to realize the training of the lightweight second model based on hierarchical image reconstruction and model construction distillation.

Description

Technical Field

[0001] This specification relates to the field of Internet technology, and in particular to an attack detection method, apparatus, and device in facial recognition. Background Technology

[0002] With the development of mobile internet, facial recognition applications are becoming increasingly widespread. However, this also brings with it a surge in attack techniques. For example, deepfake attacks use algorithms to tamper with or generate facial videos, replacing A's face with B's face in a video to carry out risky attacks. Due to the diversity of these attacks and their invisibility to visual perception, they have become a major challenge for facial recognition systems. Currently, common deepfake detection methods are often too computationally complex to be deployed on the client side.

[0003] Therefore, a solution is needed that can accurately detect attacks in facial recognition on the client side. Summary of the Invention

[0004] This specification provides an embodiment of an attack detection device, apparatus, and storage medium for face recognition, in order to solve the following technical problem: the need for a solution that can accurately detect attacks in face recognition on the client side.

[0005] To solve the above-mentioned technical problems, one or more embodiments of this specification are implemented as follows:

[0006] In a first aspect, embodiments of this specification provide an attack detection method for face recognition, comprising: acquiring training samples containing partially occluded images; reconstructing the partially occluded images to generate a reconstructed image; training and generating a first model based on the difference between the reconstructed image and the training samples; training and generating a second model based on the first model, wherein the second model is a lightweight model that is partially heterogeneous with the first model; acquiring a face-to-be-recognized image; and using the second model for attack detection.

[0007] In a second aspect, embodiments of this specification provide an attack detection device for face recognition, comprising: a sample acquisition module for acquiring training samples containing partially occluded images, reconstructing the partially occluded images, and generating a reconstructed image; a first model training module for training and generating a first model based on the differences between the reconstructed image and the training samples; a second model training module for training and generating a second model based on the first model, wherein the second model is a lightweight model that is partially heterogeneous with the first model; and an attack detection module for acquiring a face-to-be-recognized image and using the second model for attack detection.

[0008] In a third aspect, one or more embodiments of this specification provide an electronic device, comprising:

[0009] At least one processor; and,

[0010] A memory communicatively connected to the at least one processor; wherein,

[0011] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0012] In a fourth aspect, embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, which, when read by a computer from the storage medium, cause one or more processors to perform the method described in the first aspect.

[0013] The above-described at least one technical solution adopted in one or more embodiments of this specification can achieve the following beneficial effects: by acquiring training samples containing partially occluded images, reconstructing the partially occluded images to generate reconstructed images; training and generating a first model based on the difference between the reconstructed images and the training samples; training and generating a second model based on the first model, wherein the second model is a lightweight model that is partially heterogeneous with the first model; acquiring a face-to-be-recognized image, using the second model for attack detection, thereby realizing the training of a lightweight second model based on hierarchical image reconstruction and model distillation, and performing accurate attack detection in face recognition on the client side. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating an attack detection method in face recognition provided for one or more embodiments of this specification;

[0016] Figure 2 A schematic diagram of the training architecture of a first model provided in an embodiment of this specification;

[0017] Figure 3 A schematic diagram of the training architecture of a second model provided in the embodiments of this specification;

[0018] Figure 4 This is a schematic diagram of the structure of an attack detection device in face recognition provided in an embodiment of this specification;

[0019] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0020] This specification provides an attack detection method, apparatus, device, and storage medium for face recognition.

[0021] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0022] Currently, deepfake detection methods can be divided into two types. The first type is based on single-frame image detection methods. These methods use a single frame image as input to train a deep learning model for classification, determining whether the input sample is a deepfake attack. Due to the limited input information, these models have limited performance in deepfake detection. The second type is based on video data. These methods receive multiple frames of video data as input and combine temporal information such as optical flow and keypoint stability for deepfake detection. Although these methods offer improved accuracy, their overall computational complexity is too high, making them difficult to apply on the client side. Therefore, this specification provides an embodiment of a scheme that can accurately detect attacks in face recognition on the client side.

[0023] In the first aspect, such as Figure 1 As shown, Figure 1 A flowchart illustrating an attack detection method in face recognition provided for one or more embodiments of this specification, including:

[0024] S101, Obtain training samples containing partially occluded images, reconstruct the partially occluded images, and generate reconstructed images.

[0025] The training samples should include partial occlusion of the images. This occlusion can be achieved by using other similar images or by using blank images.

[0026] For example, the eye region in the original image can be occluded using a similar eye image (which can be generated based on an algorithm); or, the eye region in the original image can be occluded using a blank image.

[0027] In the training samples, occlusion markers can be applied to occluded areas to indicate the pixels that have been occluded.

[0028] The model to be trained may include modules for image reconstruction, such as a content reconstruction module. The content reconstruction module can perform image reconstruction based on the extracted feature vectors to obtain the reconstructed image.

[0029] During the reconstruction process, multiple reconstruction iterations can be used. For example, for the pixels indicated by the occlusion marker, the image corresponding to the pixels close to the occluded area in the face is first partially reconstructed (i.e., the pixels at the edge of the occluded area). After the partial reconstruction is completed, the partially reconstructed image is used as the input of the model to extract the corresponding content reconstruction module feature vector and reconstruct the image again. The entire reconstruction process is completed after multiple iterations.

[0030] S103, a first model is generated by training based on the difference between the reconstructed image and the training samples.

[0031] Specifically, if it is a normal training sample, after multiple reconstructions and feature extraction, the corresponding facial features (i.e., reconstructed features) should be as consistent as possible with the facial features corresponding to the original image. However, if it is an adversarial training sample, the opposite is true. The facial features corresponding to the reconstructed image will be inconsistent with the facial features of the original image. This inconsistency can be characterized by the similarity between features. The greater the similarity, the higher the consistency.

[0032] Furthermore, based on the reconstructed features extracted from the reconstructed image, the reconstructed feature consistency loss between the reconstructed features and the facial features can be determined. The first model is then trained based on this reconstructed feature consistency loss.

[0033] In a specific implementation, the first model may include three parts: the first part is a feature encoding module, the second part is a deepfakes classifier module, and the third part is a content reconstruction module. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of the training architecture of a first model provided in an embodiment of this specification.

[0034] During training, the feature encoding module takes as input the face images contained in the training samples and outputs the extracted facial features of the training samples; the deepfakes classifier module takes as input the extracted facial features and outputs the sample classification result; the content reconstruction module takes as input a partially reconstructed image and outputs the corresponding reconstructed image. The content reconstruction module can complete the entire reconstruction process through multiple iterations, making the hierarchical reconstruction more traceable and stable.

[0035] Correspondingly, the loss function can include three parts: the first part is to classify based on the facial features and determine the classification loss; the second part is to determine the reconstruction loss based on the difference between the reconstructed image and the original image (the image of the unoccluded training sample); and the third part is to extract the reconstructed features corresponding to the reconstructed image and determine the reconstruction feature consistency loss between the reconstructed features and the facial features of the original image. In this process, when there are multiple iterations of reconstruction, the final image from these iterations can be input into the feature encoder, compared with the features of the original image, and the reconstruction feature consistency loss can be determined.

[0036] Finally, the classification loss, reconstruction loss, and reconstruction feature consistency loss can be fused together, and the fused loss value can be backpropagated to train the aforementioned feature encoding module, deepfakes classifier module, and content reconstruction module. When the loss value converges, a usable first model is generated.

[0037] The trained first model can detect deepfakes attacks on the image to be identified in two ways. The first is to extract features from the input image to be identified and classify it based on a deepfakes classifier. The second is to perform multiple iterations of image reconstruction based on the content reconstruction module trained in the first model, extract facial features from the reconstructed image, and compare the consistency between the facial features in the reconstructed image and those in the original image (i.e., the input image to be identified). When the difference exceeds a threshold T, the image to be identified is judged to be a deepfakes attack.

[0038] It should be noted that the first model is usually a high-precision and high-complexity model, which is suitable for deployment on the server side with strong computing power, but generally not suitable for deployment on the client side with weak computing power.

[0039] S105, Based on the first model, a second model is trained and generated, wherein the second model is a lightweight model that is partially heterogeneous with the first model.

[0040] As mentioned earlier, the first model is not suitable for deployment on the client side. Therefore, model distillation is needed to obtain a lightweight model based on the first model, so that it can be deployed on the client side.

[0041] In model distillation, there can be isomorphic model distillation or partially heterogeneous model distillation. In the scenarios described in the embodiments of this specification, the encoder for facial features in the first model is a Transformer neural network; correspondingly, the encoder for facial features in the second model is a Convolutional Neural Network (CNN) model. This is because Transformer provides excellent detection capabilities, while CNN offers user-friendly edge deployment. Therefore, by changing the encoder to a heterogeneous CNN model, heterogeneous distillation can achieve model performance close to that of Transformer, while also being suitable for deployment on the client side, making it more suitable for practical situations.

[0042] Meanwhile, in the second model, in order to achieve model distillation based on the first model, other parts of the structure are still retained as a lightweight model with the same functions as the other structures of the first model.

[0043] Specifically, the second model, in addition to including a CNN-type feature encoder, also includes a deepfakes classification module and a hierarchical content reconstruction module, which have the same functions as the first model. For example... Figure 3 As shown, Figure 3 This is a schematic diagram of the training architecture of a second model provided in an embodiment of this specification. In this diagram, the eepfakes classification module and content reconstruction module included in the first model are omitted and not shown.

[0044] However, it should be noted that, compared with the first model, the deepfakes classification module and the hierarchical content reconstruction module in the second model have the same function, that is, the deepfakes classification module can also perform classification, and the hierarchical content reconstruction module can also perform image reconstruction as in the first model. However, the structure of the second model is more lightweight.

[0045] Lightweight refers to the lightweight nature of the model structure from several different perspectives: First, the same functional module contains fewer network layers and fewer channels. For example, in the first model, the deepfakes classification module might contain 10 hidden layers, while in the second model it might only have 5. Second, the same functional module requires less computation. For example, in the first model, a 9x9 convolution kernel might be used when convolving the input image, while in the second model it might use a 3x3 kernel. Third, the same functional module contains fewer parameters to be trained. For example, in the first model, the fully connected layer in the deepfakes classification module might contain 200 parameters to be trained, while in the second model, the fully connected layer in the deepfakes classification module might contain only 50 parameters to be trained.

[0046] In this process, since the first model and the second model are encoded in different ways on the feature encoder, a heterogeneous distillation module can be added to the second model in order to make the encoder trained by the second model closer to the first model.

[0047] The input to the heterogeneous distillation module is the feature map generated by the CNN feature extraction module in the second model mentioned above, and its output is the self-attention matrix generated by the Transformer neural network in the first model.

[0048] In other words, the first model, based on the Transformer neural network, can generate a first self-attention matrix for the training samples, including matrices Q, K, and V. Q, K, and V are obtained by linear transformation of the input feature vectors, where the weight matrix used for the linear transformation can be learned. This transformation improves the model's fitting ability. The obtained Q, K, and V can be regarded as follows: Q represents the information to be queried, K represents the queried vector, and V represents the value obtained from the query.

[0049] Since the first model is a pre-trained model, in order for the trained second model to approximate the first model, facial features of the training samples can be obtained based on a neural network (CNN). The aforementioned heterogeneous distillation module can then be used to generate a corresponding second self-attention matrix based on the facial features obtained by the CNN. Furthermore, the heterogeneous distillation loss L can be determined based on the difference between the first and second self-attention matrices. KD A second model is generated by training based on the heterogeneous distillation loss.

[0050] Specifically, when training the second model, since it also includes a lightweight deepfakes classification module and hierarchical content reconstruction module with the same functionality as the first model, it can also generate a classification loss L for the training samples. cls Reconstruction loss L rec And the consistency loss of reconstruction features L feat At this point, the classification loss, reconstruction loss, reconstruction feature consistency loss, and heterogeneous distillation loss of the second model for the training samples can be combined to train and generate the second model. That is, for the second model, the loss function calculation during its training process can be L2 = L... cls +L rec +L feat +L KD Based on the above model structure and loss function, the network is trained until the model converges and a usable second model is generated.

[0051] S107, Obtain the image to be identified containing a human face, and use the second model to perform attack detection.

[0052] After training the second model, it can be deployed to the client and used on the client side to perform attack detection on the acquired images to be identified.

[0053] Specifically, the second model can also perform deepfakes attack detection in two ways, similar to the first model.

[0054] The first method involves obtaining an image containing a human face to be identified, using the second model to extract facial features from the image, and then classifying the facial features of the image into attack categories.

[0055] The second method involves reconstructing the image based on facial features in the image to be identified, and then detecting attacks based on the differences between the facial features in the reconstructed image and those in the original image. If multiple iterations are used during the reconstruction process, attack detection can be performed based on the differences between the facial features in the reconstructed images and those in the original image. For example, if the difference is greater than a threshold T, it is identified as a deepfakes attack.

[0056] By acquiring training samples containing partially occluded images, the partially occluded images are reconstructed to generate reconstructed images; a first model is trained based on the differences between the reconstructed images and the training samples; a second model is trained based on the first model, wherein the second model is a lightweight model that is partially heterogeneous with the first model; an image to be recognized containing a face is acquired, and the second model is used for attack detection, thereby realizing the training of a lightweight second model based on hierarchical image reconstruction and model distillation, and performing accurate attack detection in face recognition on the client side.

[0057] In one implementation, the second model can be quantized before being used for attack detection. Quantization can be performed simultaneously with training the second model based on the first model, or it can be performed after the second model has been trained.

[0058] Quantization of the second model refers to converting some parameters in the second model from floating-point to integer (for example, converting a 32-bit floating-point number to an 8-bit integer int8), while expecting the accuracy of the quantized model to be similar to that before quantization.

[0059] Based on this, the embodiments of this specification provide a hybrid quantization method, namely, selecting a portion of layers in the second model for quantization. Specifically, during the training process of the second model, a quantization cost-effectiveness evaluation module is added to evaluate the quantization cost-effectiveness of each layer.

[0060] During training, the parameters of each layer in the second model change. At this point, the parameters of that layer can be mapped to a vector, and the L1 norm of this vector can be calculated. This L1 norm is then used as the local quantization sparsity loss for that layer to evaluate its quantization efficiency. Generally, the smaller the local quantization sparsity loss, the more parameters that can be quantized to zero in that layer, resulting in higher quantization efficiency. This method ensures that the quantizable parameters in any layer that needs to be quantized are sufficiently sparse, thus maintaining the model's accuracy.

[0061] Simultaneously, the parameters contained in all layers of the second model can be mapped to another vector, and then the L1 norm of this other vector can be calculated. The L1 norm of this other vector is then determined as the global quantization sparse loss corresponding to the model. The smaller the global quantization sparse loss, the more parameters that can be quantized to 0 in the global model. This method allows for as many layers as possible to be quantized in the second model, thereby maximizing the lightweight nature of the model and improving the computational efficiency of the quantized second model.

[0062] Under the aforementioned quantization method, the local quantization sparse loss and the global quantization sparse loss can be fused to train and generate a quantized second model. At this point, the loss value of the second model can be obtained from the aforementioned L2 = L... cls +L rec +L feat +L KD Based on this, we further add local quantization sparsity loss Lr-sparse and global quantization sparsity loss Lw-sparse. That is, L2 = L cls +L rec +L feat +L KD +Lr-spars+Lw-sparse. Based on the above model structure and loss function, the network is trained until the model converges. The resulting second model is the quantized second model. In this way, the quantization of the second model can be achieved simultaneously during the training process, while maintaining the performance of the second model close to that of the first model, and the training method is simpler.

[0063] Based on the same idea, one or more embodiments of this specification also provide apparatus and devices corresponding to the above methods, such as... Figure 4 , Figure 5 As shown.

[0064] In the second aspect, such as Figure 4 As shown, Figure 4 This is a schematic diagram of an attack detection device in face recognition provided in an embodiment of this specification. The device includes:

[0065] The sample acquisition module 401 acquires training samples containing partially occluded images, reconstructs the partially occluded images, and generates reconstructed images.

[0066] The first model training module 403 trains and generates a first model based on the differences between the reconstructed image and the training samples.

[0067] The second model training module 405 trains and generates a second model based on the first model, wherein the second model is a lightweight model that is partially heterogeneous with the first model.

[0068] The attack detection module 407 acquires an image containing a human face to be identified and uses the second model to perform attack detection.

[0069] Optionally, the first model training module 403 acquires training samples containing faces, extracts facial features from the training samples, classifies them according to the facial features, and determines a classification loss; determines a reconstruction loss based on the difference between the reconstructed image and the training samples; extracts the reconstruction features corresponding to the reconstructed image, determines the reconstruction feature consistency loss between the reconstruction features and the facial features; and trains and generates a first model by fusing the classification loss, reconstruction loss, and reconstruction feature consistency loss.

[0070] Optionally, in the device, the encoder for facial features in the first model is a Transformer neural network; correspondingly, the encoder for facial features in the second model is a Convolutional Neural Network (CNN), and the second model is a lightweight model with the same other structural functions as the first model.

[0071] Optionally, the second model training module 405 determines the first self-attention matrix of the first model for the training samples; uses the convolutional neural network (CNN) to obtain the facial features of the training samples, and generates a corresponding second self-attention matrix based on the facial features; determines the heterogeneous distillation loss based on the difference between the first self-attention matrix and the second self-attention matrix, and trains and generates the second model based on the heterogeneous distillation loss.

[0072] Optionally, the second model training module 405 determines the classification loss, reconstruction loss, and reconstruction feature consistency loss of the second model for the training samples; and trains the second model by fusing the classification loss, reconstruction loss, reconstruction feature consistency loss, and heterogeneous distillation loss of the second model for the training samples.

[0073] Optionally, the device further includes a quantization module 409, which performs quantization evaluation on the parameters of any layer included in the second model to determine the quantization cost-effectiveness of the layer; and trains and generates the quantized second model based on the quantization cost-effectiveness.

[0074] Optionally, the quantization module 409 determines the local quantization sparsity loss corresponding to any layer in the second model based on the quantization cost-effectiveness of that layer; determines the global quantization sparsity loss corresponding to the model based on the quantization cost-effectiveness of all layers in the second model; and trains and generates the quantized second model by fusing the local quantization sparsity loss and the global quantization sparsity loss.

[0075] Optionally, the attack detection module 407 acquires an image to be identified containing a face, extracts facial features of the image to be identified using the second model, classifies the facial features of the image to be identified into attack categories, or performs image reconstruction based on the facial features in the image to be identified, and performs attack detection based on the difference between the facial features corresponding to the reconstructed image and the facial features of the image to be identified.

[0076] In the third aspect, such as Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. The device includes:

[0077] At least one processor; and,

[0078] A memory communicatively connected to the at least one processor; wherein,

[0079] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0080] In a fourth aspect, based on the same idea, embodiments of this specification also provide a non-volatile computer storage medium corresponding to the above method, storing computer-executable instructions, which, when read by a computer from the storage medium, cause one or more processors to execute the method described in the first aspect.

[0081] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0082] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0083] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0084] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0085] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0089] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0090] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0091] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0092] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0093] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0094] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0095] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0096] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. An attack detection method in face recognition, comprising: Obtain training samples containing partially occluded images, reconstruct the partially occluded images, and generate reconstructed images; Based on the differences between the reconstructed image and the training samples, a first model is trained and generated through classification constraints, reconstruction quality constraints, and feature consistency constraints. Based on the first model, a second model is generated by training the alignment loss of the first self-attention feature corresponding to the first model and the second self-attention feature corresponding to the second model, wherein the second model is a lightweight model that is partially heterogeneous with the first model. Obtain an image containing a human face to be identified, and use the second model for attack detection.

2. The method as described in claim 1, wherein, A first model is generated by training based on the differences between the reconstructed image and the training samples, including: Obtain training samples containing human faces, and extract facial features from the training samples; Classify the facial features and determine the classification loss. The reconstruction loss is determined based on the difference between the reconstructed image and the training samples; Extract the reconstructed features corresponding to the reconstructed image, and determine the reconstructed feature consistency loss between the reconstructed features and the facial features; The first model is generated by training by fusing the classification loss, reconstruction loss, and reconstruction feature consistency loss.

3. The method as described in claim 1, wherein, The second model is a lightweight model that is partially heterogeneous with the first model, including: The encoder for facial features in the first model is a Transformer neural network; Accordingly, the encoder for facial features in the second model is a convolutional neural network (CNN), and the second model is a lightweight model with the same other structural functions as the first model.

4. The method of claim 3, wherein, Based on the first model, a second model is trained and generated, including: Determine the first self-attention matrix of the first model for the training samples; The facial features of the training samples are obtained using the convolutional neural network (CNN), and a corresponding second self-attention matrix is ​​generated based on the facial features. The heterogeneous distillation loss is determined based on the difference between the first self-attention matrix and the second self-attention matrix, and a second model is trained and generated based on the heterogeneous distillation loss.

5. The method of claim 4, wherein, A second model is generated based on the heterogeneous distillation loss training, including: Determine the classification loss, reconstruction loss, and reconstruction feature consistency loss of the second model for the training samples; The second model is generated by combining the classification loss, reconstruction loss, reconstruction feature consistency loss, and heterogeneous distillation loss of the training samples.

6. The method of claim 1, wherein, Before using the second model for attack detection, the method further includes: The parameters of any layer included in the second model are quantitatively evaluated to determine the quantization cost-effectiveness of that layer. Based on the quantization cost-effectiveness, a second quantized model is trained and generated.

7. The method of claim 6, wherein, Based on the aforementioned quantization cost-effectiveness, a second quantized model is trained and generated, including: The local quantization sparsity loss corresponding to a layer is determined based on the quantization cost-effectiveness of any layer in the second model. The global quantization sparsity loss of the model is determined based on the quantization cost-effectiveness of all layers in the second model. The local quantization sparse loss and the global quantization sparse loss are combined to train and generate a quantized second model.

8. The method of claim 1, wherein, Obtain the image to be identified, containing a face, and perform attack detection using the second model, including: Obtain an image containing a human face to be identified, and use the second model to extract the facial features of the image to be identified; Attack categories are classified based on the facial features of the image to be identified; Alternatively, image reconstruction can be performed based on the facial features in the image to be identified, and attack detection can be performed based on the difference between the facial features corresponding to the reconstructed image and the facial features of the image to be identified.

9. An attack detection device for face recognition, comprising: The sample acquisition module acquires training samples containing partially occluded images, reconstructs the partially occluded images, and generates reconstructed images. The first model training module generates a first model by training based on the differences between the reconstructed image and the training samples through classification constraints, reconstruction quality constraints, and feature consistency constraints. The second model training module generates a second model based on the first model by using the alignment loss of the first self-attention feature corresponding to the first model and the second self-attention feature corresponding to the second model. The second model is a lightweight model that is partially heterogeneous with the first model. The attack detection module acquires an image containing a human face to be identified and uses the second model to perform attack detection.

10. An electronic device, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1 to 8.

Images