Method and system for detecting adversarial attacks

Abstract

The present specification provides a detection method and system for detecting adversarial attacks, after obtaining an original image of a user, constructing at least one dimensional graph model based on the original image, inputting the at least one dimensional graph model into a graph neural network model, determining the associated region features of the biological features, and inputting the associated region features into an adversarial attack detection model to obtain the adversarial attack detection result of the user. The method and system use at least one dimensional graph model to establish the association relationship between different region features, which can not only detect adversarial attacks from the dimension of the original region features, but also detect adversarial attacks from the dimension of the association relationship between different original region features, thereby enriching the input information of the adversarial attack detection and improving the detection accuracy of the adversarial attack.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a method and system for detecting adversarial attacks. Background Technology

[0002] With the development of the internet, facial recognition has become widely used. Adversarial attacks have become one of the main security threats to facial recognition systems. By deliberately adding subtle interferences that are imperceptible to the human eye to the input samples, the model can be made to output an incorrect classification result. Because the attack patterns in adversarial attacks are small in area and difficult to detect, defending against adversarial attacks is quite difficult.

[0003] Currently, adversarial attack detection is mainly carried out in the pre-processing / detection stage of face recognition or during the face recognition stage. However, adversarial attacks have small attack patterns and are difficult to detect, making defense against them more challenging.

[0004] In summary, there is a need to provide a way to improve the detection performance against attacks. Summary of the Invention

[0005] This specification provides a method and system for detecting adversarial attacks, which can improve the detection performance of adversarial attacks.

[0006] In a first aspect, this specification provides a method for detecting adversarial attacks, comprising: acquiring an original image of a user, the original image including the user's biometric features; constructing a graph model of at least one dimension based on the original image, the graph model including the correlation relationships between different regions of the biometric features; inputting the graph model of at least one dimension into a graph neural network model to determine the correlation region features of the biometric features; and inputting the correlation region features into an adversarial attack detection model to obtain the adversarial attack detection result of the user.

[0007] In some embodiments, constructing a graph model with at least one dimension based on the original image includes: inputting the original image into a feature extraction model to obtain multiple original region features corresponding to the biometric features; establishing connections between at least some of the region features that are related in each of the multiple original region features in each of the at least one dimension to obtain the corresponding graph model; and performing cross-dimensional connections between the same original region features in the graph model of the at least one dimension.

[0008] In some embodiments, the graph model of the at least one dimension includes at least one of a neighborhood graph model, a key point graph model, and a salient region graph model.

[0009] In some embodiments, the neighborhood graph model is determined using the following steps: each of the plurality of original region features is taken as a target neighborhood node, and adjacent original region features are connected to the target neighborhood node.

[0010] In some embodiments, the key point graph model is determined using the following steps: taking the original region features representing key points of biological features from the plurality of original region features as target key nodes, and connecting the target key nodes whose similarity is greater than a preset similarity threshold.

[0011] In some embodiments, the saliency region map model is determined using the following steps: determining multiple saliencies corresponding to the multiple original region features, wherein the saliency represents the degree of attention paid to the original region features; determining the original region features among the multiple original region features whose saliency is greater than a preset saliency threshold as saliency nodes; and establishing pairwise connections between the saliency nodes.

[0012] In some embodiments, the feature extraction model is trained using the following steps: acquiring a first image sample of the biometric features of the training object; using a preset feature extraction network to extract features from multiple image regions corresponding to the first image sample, obtaining multiple feature maps corresponding to the multiple image regions and corresponding first loss information, wherein the first loss information characterizes the difference between the training identity category and the labeled identity category of the training object corresponding to the first image sample; and converging the preset feature extraction network based on the first loss information until training ends, thereby obtaining the feature extraction model. In some embodiments, the graph neural network model is trained using the following steps: acquiring a second image sample of the biometric features of the training object; inputting the second image sample into the feature extraction model to obtain multiple original region feature samples corresponding to the second image sample; constructing a graph model sample of at least one dimension corresponding to the second image sample based on the multiple original region feature samples; inputting the graph model of at least one dimension into a preset graph neural network to obtain the associated region feature samples of the biometric features and corresponding second loss information; and converging the preset graph neural network based on the second loss information until training ends, thereby obtaining the graph neural network model.

[0013] In some embodiments, the network structure of the preset graph neural network is the same as that of a graph model with at least one dimension of cross-layer connections. The second loss information is obtained by the following steps: adversarial attack classification is performed based on multiple new region feature samples output by the preset graph neural network according to the multiple original region feature samples to obtain multiple training adversarial attack categories corresponding to the second image sample; multiple adversarial attack classification loss information is determined based on the difference between the labeled adversarial attack category corresponding to the second image sample and the multiple training adversarial attack categories; and the second loss information is obtained based on the weighted sum of the multiple adversarial attack classification loss information.

[0014] In some embodiments, the adversarial attack detection model is trained using the following steps: acquiring a third image sample of the biometric features of the training object; constructing a graph model sample of at least one dimension with cross-dimensional connections based on the third image sample; inputting the graph model sample of at least one dimension corresponding to the third image sample into the graph neural network model to obtain the associated region feature sample corresponding to the third image sample; inputting the associated region feature sample corresponding to the third image sample into a preset adversarial attack detection network to obtain the training adversarial attack classification and the corresponding third loss information corresponding to the third image sample, wherein the third loss information characterizes the difference between the training adversarial attack category corresponding to the third image sample and the labeled adversarial attack category; and converging the preset adversarial attack detection network based on the third loss information until training ends to obtain the adversarial attack detection model.

[0015] Secondly, this specification also provides a detection system for counter-attacks, comprising: at least one storage medium storing at least one instruction set for detecting counter-attacks; and at least one processor communicatively connected to the at least one storage medium, wherein, when the detection system for counter-attacks is running, the at least one processor reads the at least one instruction set and executes the method described in any one of the first aspects according to the instructions of the at least one instruction set.

[0016] As can be seen from the above technical solutions, the adversarial attack detection method and system provided in this specification acquires the original image including the user's biometric features, constructs a graph model of at least one dimension based on the original image, the graph model includes the correlation relationships between different regions of the biometric features, and inputs the graph model of at least one dimension into a graph neural network model to determine the correlation region features of the biometric features, and inputs the correlation region features into an adversarial attack detection model to obtain the adversarial attack detection result for the user. Because this solution uses a graph model of at least one dimension, constructing the graph model from different perspectives and establishing the correlation relationships between the original region features, it enriches the input information of the adversarial attack detection model. This allows for adversarial attack detection not only from the original region features but also from the correlation relationships between the original region features, thereby improving the performance and detection accuracy of adversarial attacks.

[0017] Other functions of the methods and systems for detecting adversarial attacks provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the methods and systems for detecting adversarial attacks provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic diagram illustrating an application scenario of an anti-attack detection system provided according to an embodiment of this specification is shown.

[0020] Figure 2 A hardware structure diagram of a computing device provided according to an embodiment of this specification is shown;

[0021] Figure 3 A flowchart of a method for detecting adversarial attacks provided according to an embodiment of this specification is shown;

[0022] Figure 4 A flowchart is shown illustrating a method for constructing a graph model of at least one dimension according to embodiments of this specification;

[0023] Figure 5 A schematic diagram of the overall process for adversarial attack detection provided according to an embodiment of this specification is shown. Detailed Implementation

[0024] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0025] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0026] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0027] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0028] For ease of description, the terms used in this specification are explained as follows:

[0029] Hierarchical graphs: In this scheme, this refers to various graph structures with different nodes and connections, which can be used to construct different features related to adversarial attacks. For example, neighborhood graphs and keypoint graphs construct different feature relationships.

[0030] Graph models are models that contain nodes and connections. Unlike ordinary image models, the connections in graph models are sparse and irregular, so the training methods are also different from those of ordinary convolutional neural networks.

[0031] Adversarial attacks refer to attacks that involve pasting adversarial stickers (small in area, generally less than 50% of the face) onto the face area, thereby causing the face recognition system to misjudge (for example, user A is mistakenly identified as user B after pasting an adversarial sticker).

[0032] Adversarial attack detection: In this solution, it refers to various methods for detecting adversarial attacks on faces.

[0033] Before describing the specific embodiments in this specification, the application scenarios of this specification will be introduced as follows:

[0034] The adversarial attack detection method provided in this specification can be applied to any adversarial attack detection scenario in biometric processes. For example, in scenarios such as face payment or face recognition, the adversarial attack detection method can be used to detect adversarial attacks on the original images of the biometric features of the user to be paid or identified; in identity verification scenarios, the adversarial attack detection method can be used to detect adversarial attacks on the original images of the user's biometric features; and it can also be applied to any other adversarial attack detection scenario, which will not be elaborated here. The biometric features may include, but are not limited to, one or more of the following: facial image, iris, sclera, fingerprint, palm print, voiceprint, and skeletal projection. For ease of description, this application will use the application of the adversarial attack detection method in a face recognition scenario to describe adversarial attack detection on a face as an example.

[0035] Those skilled in the art should understand that the detection methods and systems for counter-attacks described in this specification are also within the scope of protection of this specification when applied to other use cases.

[0036] Figure 1 This diagram illustrates an application scenario of an adversarial attack detection system 001 provided according to an embodiment of this specification. The adversarial attack detection system 001 (hereinafter referred to as System 001) can be applied to adversarial attack detection in any scenario, such as adversarial attack detection in scenarios like facial recognition payment, identity verification, access control, and information query. Figure 1 As shown, system 001 may include user 100 and computing device 600.

[0037] User 100 is a user who needs to undergo biometric identification, or a user who is currently undergoing biometric identification. User 100 can be the target of system 001's detection. User 100 can initiate a biometric identification process, thereby triggering adversarial attack detection against User 100.

[0038] The computing device 600 may be a device for executing a method for detecting adversarial attacks. The computing device 600 may store data or instructions for executing the adversarial attack detection method described herein, and may execute or be used to execute said data or instructions. In some embodiments, the computing device 600 may include a hardware device with data processing capabilities and the necessary programs required to drive the hardware device. In some embodiments, the computing device 600 may acquire the original image used by the user 100 for biometric identification and execute the adversarial attack detection method provided herein to detect adversarial attacks on the original image.

[0039] In some embodiments, computing device 600 may include local device 200. The method for detecting adversarial attacks can be executed on local device 200. In this case, local device 200 may store data or instructions for executing the method for detecting adversarial attacks described herein, and may execute or be used to execute the data or instructions. In some embodiments, local device 200 may include hardware devices with data processing capabilities and necessary programs to drive the hardware devices. In some embodiments, local device 200 may include mobile devices, tablets, laptops, built-in devices in motor vehicles, or similar content, or any combination thereof. In some embodiments, the mobile device may include smart home devices, smart mobile devices, virtual reality devices, augmented reality devices, or similar devices, or any combination thereof. In some embodiments, the smart home device may include smart TVs, desktop computers, etc., or any combination thereof. In some embodiments, the smart mobile device may include smartphones, personal digital assistants, gaming devices, navigation devices, etc., or any combination thereof. In some embodiments, the virtual reality device or augmented reality device may include virtual reality headsets, virtual reality glasses, virtual reality controllers, augmented reality headsets, augmented reality glasses, augmented reality controllers, or similar content, or any combination thereof. For example, the virtual reality device or the augmented reality device may include Google Glass, head-mounted displays, VR, etc. In some embodiments, the built-in device in the motor vehicle may include an on-board computer, an on-board television, etc. In some embodiments, the local device 200 may include an image acquisition device for acquiring raw images of the user's biometric features. In some embodiments, the image acquisition device may be a two-dimensional image acquisition device (such as an RGB camera, an IR camera, etc.), or it may be a combination of a two-dimensional image acquisition device (such as an RGB camera, an IR camera, etc.) and a depth image acquisition device (such as a 3D structured light camera, a laser detector, etc.). In some embodiments, the local device 200 may be a device with positioning technology for locating the position of the local device 200.

[0040] In some embodiments, computing device 600 may include remote device 300. Remote device 300 may be a server providing various services, such as a backend server supporting adversarial attack detection of raw images of user biometrics collected on local device 200. In some embodiments, the adversarial attack detection method may be executed on remote device 300. In this case, remote device 300 may store data or instructions for executing the adversarial attack detection method described herein, and may execute or be used to execute said data or instructions. In some embodiments, remote device 300 may include hardware devices with data processing capabilities and the necessary programs required to drive the hardware devices. Remote device 300 may be communicatively connected to multiple local devices 200 and receive data sent by local devices 200.

[0041] like Figure 1 As shown, local device 200 can communicate with remote device 300. In some embodiments, remote device 300 can communicate with multiple local devices 200. In some embodiments, local device 200 can interact with remote device 300 through network 400 to receive or send messages, such as receiving or sending raw images.

[0042] Network 400 serves as a medium to provide a communication connection between local device 200 and remote device 300. Network 400 facilitates the exchange of information or data. For example... Figure 1 As shown, local device 200 and remote device 300 can connect to network 400 and transmit information or data to each other through network 400. In some embodiments, network 400 can be any type of wired or wireless network, or a combination thereof. For example, network 400 may include cable networks, wired networks, fiber optic networks, telecommunications networks, intranets, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), and Bluetooth. TM Network, ZigBee TM A network, a near field communication (NFC) network, or a similar network. In some embodiments, network 400 may include one or more network access points. For example, network 400 may include wired or wireless network access points, such as base stations or internet exchange points, through which one or more components of local device 200 and remote device 300 can connect to network 400 to exchange data or information.

[0043] It should be understood that Figure 1The number of local devices 200, remote devices 300, and networks 400 shown is merely illustrative. Any number of local devices 200, remote devices 300, and networks 400 can be used depending on implementation needs.

[0044] It should be noted that the method for detecting the anti-attack can be executed entirely on the local device 200, entirely on the remote device 300, or partially on the local device 200 and partially on the remote device 300.

[0045] Figure 2 A hardware structure diagram of a computing device 600 provided according to an embodiment of this specification is shown. The computing device 600 can execute the anti-attack detection method described in this specification. The anti-attack detection method is described in other parts of this specification. When the anti-attack detection method is executed on a local device 200, the computing device 600 can be the local device 200. When the anti-attack detection method is executed on a remote device 300, the computing device 600 can be the remote device 300. When the anti-attack detection method can be executed partly on the local device 200 and partly on the remote device 300, the computing device 600 can be both the local device 200 and the remote device 300.

[0046] like Figure 2 As shown, the computing device 600 may include at least one storage medium 630 and at least one processor 620. In some embodiments, the computing device 600 may also include a communication port 650 and an internal communication bus 610. Additionally, the computing device 600 may include I / O components 660.

[0047] The internal communication bus 610 can connect different system components, including storage medium 630, processor 620 and communication port 650.

[0048] I / O component 660 supports input / output between computing device 600 and other components.

[0049] Communication port 650 is used for data communication between computing device 600 and external sources. For example, communication port 650 can be used for data communication between computing device 600 and network 400. Communication port 650 can be a wired communication port or a wireless communication port.

[0050] Storage medium 630 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 632, a read-only storage medium (ROM) 634, or a random access storage medium (RAM) 636. Storage medium 630 also includes at least one instruction set stored in the data storage device. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the detection methods against attacks provided in this specification.

[0051] At least one processor 620 can be communicatively connected to at least one storage medium 630 and a communication port 650 via an internal communication bus 610. At least one processor 620 is used to execute at least one instruction set. When the computing device 600 is running, at least one processor 620 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the anti-attack detection method provided in this specification. Processor 620 can execute all steps included in the anti-attack detection method. Processor 620 can be in the form of one or more processors. In some embodiments, processor 620 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 620 is described in this specification for the computing device 600. However, it should be noted that the computing device 600 may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described herein, or they may be executed jointly by multiple processors. For example, if processor 620 of the computing device 600 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 620 (e.g., a first processor executes step A, a second processor executes step B, or the first and second processors jointly execute steps A and B).

[0052] Figure 3A flowchart of a counter-attack detection method P100 provided according to an embodiment of this specification is shown. As previously described, the computing device 600 can execute the counter-attack detection method P100 of this specification. Specifically, the processor 620 can read an instruction set stored in its local storage medium and then execute the counter-attack detection method P100 of this specification according to the provisions of the instruction set. Figure 3 As shown, method P100 may include:

[0053] S110: Obtain the user's original image.

[0054] The original image includes the user's biometrics. Here, the user is the one subject to adversarial attack detection. Biometrics are inherent physiological characteristics of the human body, which may include at least one of the following: face, iris, sclera, fingerprint, palm print, voiceprint, and skeletal projection, or other inherent physiological characteristics of the human body capable of facial recognition. For ease of description, this specification will use the face as an example of biometrics. Those skilled in the art should understand that other biometric features are also within the scope of this specification.

[0055] The original image can be an image including all or part of the user's body parts, acquired by an image acquisition module. The original image may contain the user's biometric features, i.e., the detection target for adversarial attacks. In some embodiments, the original image can be an image acquired by the image acquisition module, or an image obtained by cropping an image acquired by the image acquisition module. When user biometrics is triggered, the processor 620 can control the image acquisition module to directly acquire the initial image of the user's biometric features, or it can receive the initial image uploaded by the user 100 through the local device 200 or a terminal device, or it can retrieve the initial image from the storage address based on the received adversarial attack detection request carrying the storage address of the initial image containing the user's biometric features. Afterwards, the processor 620 preprocesses the initial image to obtain the original image. The preprocessor can be a face detection function, for example, performing face detection on the initial image to obtain the face regions (face regions in RGB images, NIR images, and Depth images) in the initial image.

[0056] S120: Construct a graph model with at least one dimension based on the original image.

[0057] The graph model can include the relationships between different regions of a biometric feature. This specification constructs a graph model with at least one dimension to express the relationships between different regions of a user's biometric feature. In some embodiments, different regions of a biometric feature have different structural features. Taking a face as an example, the structural features of different regions of a face, such as facial features and the face itself, are also different. In some embodiments, there may be some relationships between different regions of a biometric feature. The relationships can be any form of connection that can be established. Taking a face as an example, the left and right eyes may have a relationship, such as a positional relationship, a size relationship, or a similarity relationship, etc. In some embodiments, regions with relationships may influence each other. Method P100 can perform adversarial attack detection based on these relationships to improve detection performance. Specifically, method P100 can establish a graph model with at least one dimension to construct the relationships between different regions. The at least one dimension can be a dimension of different relationships, such as a positional dimension, a similarity dimension, a salience dimension, etc.

[0058] Figure 4 A flowchart illustrating a method for constructing a graph model with at least one dimension is shown. Figure 4 Corresponding to step S120. For example... Figure 4 As shown, step S120 may include:

[0059] S122: Input the original image into the feature extraction model to obtain multiple original region features corresponding to the biological features.

[0060] The feature extraction model here is configured to extract biometric features from the original image. After receiving the input original image, the feature extraction model can process multiple image regions corresponding to the biometric features in the original image.

[0061] Feature extraction is performed to obtain multiple raw region features corresponding to the biometric features. For example, feature extraction on a face can yield 14*14*256 dimensional features, representing the raw region features of 14*14=196 regions of the user's entire face.

[0062] The original region features of each region are 256-dimensional in length.

[0063] Before applying a feature extraction model for feature extraction, the model needs to be trained. The feature extraction model uses a first image sample as its training sample, and the labeled identity category corresponding to the first image sample as its training data.

[0064] The corresponding labels are obtained and trained using the classification loss of identity category as the loss function. Specifically, the feature extraction model 5 can be trained using the following steps: For example, after obtaining the first image sample of the biometric features of the training object,

[0065] The first image sample is input into a preset feature extraction network to obtain multiple feature maps corresponding to the first image sample and the corresponding first loss information. The preset feature extraction network is then converged based on the first loss information until the training ends, resulting in a feature extraction model.

[0066] During the training of the feature extraction model, the first image sample corresponds to a labeled identity category. Labeled identity category 0 represents pre-labeled information indicating the identity category of the training object. The pre-defined feature extraction network is then trained based on a face comparison task.

[0067] Training the model yields a feature extraction model. For example, the training dataset for this model might include 10,000 face images, each corresponding to one of 10,000 users. The labeled identity categories of these 10,000 face images can represent the identity information of those users. For instance, these 10,000 users could be numbered from 1 to 10,000.

[0068] Furthermore, each user's identity category is identified using a number. The first image sample is input into a preset feature extraction network, which then extracts features from multiple image regions of the first image sample, thereby obtaining multiple images.

[0069] The system obtains multiple feature maps corresponding to a region and the training identity category of the training object. Based on the difference between the training identity category and the labeled identity category, it determines the first loss information and performs backpropagation based on the first loss information to adjust the network parameters of the preset feature extraction network. The training continues until the first loss information is minimized or the preset number of training iterations is reached, and the training ends, thus obtaining the feature extraction model.

[0070] The preset feature extraction network can be a network structure such as ResNet101 used for feature extraction. The preset feature extraction network includes a global average pooling layer and a softmax layer. The global average pooling layer is configured to perform average pooling on the feature maps of the feature extraction layer across the entire image, forming a single feature point. These feature points are then combined to form a final feature vector, which is then output to the softmax layer for classification. For example, if the feature extraction layer outputs 10 6x6 feature maps, the global average pooling layer calculates the average of all pixels in each of the 10 feature maps, outputting a single data value. This results in 10 data values ​​from the 10 feature maps. These data values ​​are combined to form a 1x10 vector, resulting in a feature vector, which is then fed into the softmax layer for classification calculation, thereby obtaining the training identity category of the training object in the first image sample.

[0071] It should be noted that during training, identity categories are needed as supervisory signals, thus requiring a softmax layer for classification calculations. However, in inference applications, classification is not required. Therefore, during inference, the feature maps output by the network layers before the global average pooling layer can be used as multiple original region features. Taking a face image as an example, the network layers before the global average pooling layer output 14*14*256-dimensional features, representing the original region features of 196 regions of the entire face, with each original region feature having a length of 256 dimensions.

[0072] After training the feature extraction model, features can be extracted from multiple regions of the original image based on the feature extraction model to obtain multiple original region features in the original image.

[0073] S124: In each of at least one dimension, establish connections between at least some of the region features that are related among multiple original region features to obtain the corresponding graph model.

[0074] After obtaining multiple original region features, connections can be established between some or all of the related region features in each of the multiple original region features based on at least one pre-selected dimension, thereby obtaining the corresponding graphical model in that dimension. In other words, the graphical model for each dimension connects the related region features among multiple original region features to represent the original image. The connection relationship can be established by linking related region features pairwise with lines. During adversarial attack detection, not only can the original region features of the biometric feature be detected, but the correlation between the original region features can also be detected. The correlation between different regions within the same biometric feature can influence each other. When some original region features are subjected to adversarial attacks while their associated original region features are not, their correlation may change. Therefore, the processor 620 can perform adversarial attack detection by detecting the correlation between its associated original region features, thereby enriching the detection information and improving detection accuracy. In some embodiments, the selected at least one dimension includes at least one of a neighborhood dimension, a keypoint dimension, and a saliency dimension. A graph model with at least one dimension includes at least one of a neighborhood graph model, a keypoint graph model, and a salient region graph model. In the neighborhood dimension, a neighborhood graph is constructed based on multiple original region features. In the keypoint dimension, a keypoint graph is constructed based on multiple original region features. In the salient dimension, a salient region graph is constructed based on multiple original feature regions. At least one of a neighborhood graph, a keypoint graph, and a salient region graph constitutes a graph model with at least one dimension. Specifically, a graph model with at least one dimension can include any one of the following seven cases: (1) a neighborhood graph model; (2) a keypoint graph model; (3) a salient region graph model; (4) a neighborhood graph model and a keypoint graph model; (5) a neighborhood graph model and a salient region graph model; (6) a keypoint graph model and a salient region graph model; (7) a neighborhood graph model, a keypoint graph model, and a salient region graph model.

[0075] The construction process of the above three graph models will be described below:

[0076] (1) Neighborhood Graph Model. The neighborhood graph model is constructed by using each of the multiple original region features as a target neighborhood node and connecting adjacent original region features to that target neighborhood node. Adjacent original region features can be located in the four directions (up, down, left, right) of the target neighborhood node, or in the eight directions (up, down, left, right, upper left corner, upper right corner, lower left corner, and lower right corner). Of course, other directions or numbers of original region features are also possible; this specification does not impose any restrictions. Generally, adjacent original region features are more closely related due to their proximity, resulting in higher similarity. For example, adjacent regions may belong to the same structure, such as both belonging to the eyes, mouth, or nose, etc., thus exhibiting higher similarity. The purpose of the neighborhood graph is to establish connections between original region features that are close in location, thereby establishing connections between original region features with high similarity.

[0077] (2) Keypoint Graph Model. The construction of a keypoint graph model involves using the original region features representing key biological features from multiple original region features as target key nodes, and connecting target key nodes with a similarity greater than a preset similarity threshold, thereby directly establishing associations within the original feature regions with high similarity. Taking a face image as an example, the original region features representing key biological features can be features of areas such as eyes, nose, mouth, and eyebrows. The keypoint graph model aims to connect key points with high similarity. For example, if two original region features both belong to the eyes, and the similarity between eyes is high, the keypoint graph model aims to connect the original region features corresponding to the areas containing key points such as eyes, nose, mouth, and eyebrows to represent the features of the face image.

[0078] (3) Significance region map model.

[0079] Before constructing the salient region map model, it is necessary to determine multiple saliencies corresponding to multiple original region features. The saliency of each original region feature represents the degree of attention given to that original region feature, such as the degree of attention given to the eyes, the degree of attention given to the nose, and so on. Then, the original region features with saliencies greater than a preset saliency threshold are identified as salient nodes, and pairs of salient nodes are connected to establish a correlation between original region features with higher saliency.

[0080] Here, GradCAM can be used to calculate the responsivity of each original region feature among multiple original region features. For example, GradCAM can be used to generate a responsivity map for multiple original region features. This responsivity map includes the responsivity of each original region feature, which is the saliency. Then, the original region features with responsivity greater than the responsivity threshold can be connected pairwise to obtain the saliency region map model.

[0081] In some embodiments, the at least one dimension may also include other dimensions, such as row graph models, column graph models, and other graph models. For example, for each original region feature, K similar blocks are found within a set search range and extended into column vectors to construct a similar block group G. For the similar block group G, each column vector is treated as a node to construct a vector-based column graph model; at the same time, each row vector is treated as a node to construct a vector-based row graph model.

[0082] like Figure 4 As shown, step S120 may further include:

[0083] S126: Connect the same original region features across dimensions in a graphical model of at least one dimension.

[0084] After obtaining a graph model with at least one dimension, the same original region features in the graph model with at least one dimension can be connected across dimensions. For example, if the graph model with at least one dimension includes all three graph models mentioned above, the same original region features in the three graph models are connected, thus establishing a correlation between the three graph models and forming a large cross-dimensional graph model. Step S126 connects graph models with at least one dimension, thereby connecting graph models with different dimensions. This ensures that the graph model with at least one dimension includes not only correlations within different dimensions but also correlations between different dimensions. In other words, step S126 ensures that the graph model with at least one dimension includes not only intra-dimensional correlations but also inter-dimensional correlations, thereby enriching the input information for adversarial attack detection and improving the accuracy of adversarial attack detection.

[0085] S130: Input a graph model of at least one dimension into a graph neural network model to determine the associated region features of biofeatures.

[0086] The graph neural network model is configured to extract relationships between original region features in a graph model sample of at least one dimension. The pre-defined graph neural network structure is identical to that of the graph model with cross-dimensional connections of at least one dimension. In the graph model of at least one dimension, the transfer function between nodes (original region features) is a transformer block (i.e., an attention module), used to transfer information about the original region features between nodes. The input to the pre-defined graph neural network is the graph model of at least one dimension, and the output is the new region features after information transfer, i.e., associated region features. Associated region features are new feature representations of the original region features. Associated region features not only include the original region features but also the influence of other region features that are related to the original region features, thus obtaining a new fused region feature.

[0087] After inputting at least one dimension of graphical model samples into a graphical neural network model, the model learns the relationships between the features of each original region in the at least one dimension of the graphical model samples, thus obtaining the relationship features between the original region features. This can also be understood as the original region features, after passing through a pre-defined graphical neural network, obtaining a new feature representation.

[0088] Before applying a graph neural network (Graph Neural Network) model, it needs to be trained. After constructing a graph model with at least one dimension, a pre-defined graph neural network can be trained based on this model to obtain the graph neural network model. The graph neural network model is trained using a second image sample as the training sample, adversarial attack classification results as the labels corresponding to the training samples, and adversarial attack classification loss as the loss function. The graph neural network model is trained using the following steps: After obtaining second image samples of the biometric features of the training object, the second image samples are input into a feature extraction model to obtain multiple original region feature samples corresponding to the second image samples. Based on these original region feature samples, a graph model sample with at least one dimension corresponding to the second image samples is constructed. This graph model sample with at least one dimension is input into a pre-defined graph neural network to obtain the associated region feature samples of the biometric features and the corresponding second loss information. The pre-defined graph neural network is then converged based on the second loss information until training is complete, resulting in the graph neural network model.

[0089] During the training of the graph neural network model, a feature extraction model is used to extract features from multiple image regions in the second image sample, resulting in multiple original region feature samples. Based on these original region feature samples, a graph model sample with at least one dimension is constructed, such as at least one of a neighborhood graph model sample, a keypoint graph model sample, and a salient region graph model sample. Specifically, the graph model sample with at least one dimension can include any one of the following seven cases: 1) Neighborhood graph model sample; 2) Keypoint graph model sample; 3) Salient region graph model sample; 4) Neighborhood graph model sample and keypoint graph model sample; 5) Neighborhood graph model sample and salient region graph model sample; 6) Neighborhood graph model sample and salient region graph model sample; 7) Neighborhood graph model sample, keypoint graph model sample, and salient region graph model sample.

[0090] During the training of the pre-defined graph neural network, the relationships between the original region features in at least one dimension of the graph model with cross-dimensional connections are learned; that is, how each original region feature is influenced by other surrounding original region features. For example, when the original region feature 'a' input to node A is passed to another node B after one information transmission, node B's expression of the original region feature 'a' may become 0.9*a + 0.1*b, where b represents other original region features surrounding node A. After the information transmission between the nodes in the pre-defined neural network, the output of the last node is the final expression of the original region feature 'a', which is the new region feature 'a' of the original region feature 'a'.

[0091] The pre-defined graph neural network (Graph Neural Network) transmits information from multiple original region feature samples to obtain multiple new region feature samples corresponding to the original region features. These new region features are then input into a shared classifier for adversarial attack classification, resulting in multiple training adversarial attack categories. These training adversarial attack categories can be classification results indicating whether a region is an adversarial attack target, outputting the probability of belonging to an adversarial attack target, or other forms representing the adversarial attack classification results. After obtaining these training adversarial attack categories, the differences between these categories and the labeled adversarial attack categories are used to determine multiple adversarial attack classification loss information. Then, a second loss information is obtained by weighted summing these loss information. This second loss information is then used to backpropagate through the pre-defined graph neural network, updating its network parameters. Training continues until the second loss information is minimized or the preset number of training iterations is reached, at which point training ends, resulting in the graph neural network model. After obtaining the graph neural network model, new features can be extracted based on a graph model with at least one dimension, yielding associated region features. Associated region features can be understood as concatenated features obtained by concatenating the new region features obtained from multiple original region features after information transmission. The associated region features include the relationships between original region features and other original region features, as well as the connection relationships of the graph model.

[0092] S140: Input the associated region features into the adversarial attack detection model to obtain the user's adversarial attack detection results.

[0093] The adversarial attack detection model is configured to detect adversarial attacks based on associated region features. The network structure of the adversarial attack detection model can adopt existing network structures used for adversarial attack detection, such as an N-layer Multilayer Perceptron (MLP), where N is an integer greater than or equal to 1. The adversarial attack detection model can be trained using the following steps: After obtaining a third image sample of the biometric features of the training object, a graph model sample with at least one dimension and cross-dimensional connections is constructed based on the third image sample. The graph model sample with at least one dimension corresponding to the third image sample is input into the graph neural network model to obtain the associated region feature sample corresponding to the third image sample. This associated region feature sample is then input into the pre-defined adversarial attack detection network to obtain the training adversarial attack classification corresponding to the third image sample and the corresponding third loss information. The third loss information represents the difference between the training adversarial attack category and the labeled adversarial attack category corresponding to the third image sample. The pre-defined adversarial attack detection network is then converged based on the third loss information until training is complete, resulting in the adversarial attack detection model.

[0094] When constructing a graph model sample with at least one dimension of cross-dimensional connectivity based on a third image sample, the trained feature extraction model can be used to extract features from multiple image regions of the third image sample, obtaining multiple original region feature samples corresponding to the third image sample. Based on these original region feature samples, a graph model sample with at least one dimension can be constructed, such as at least one of a neighborhood graph model sample, a keypoint graph model sample, and a salient region graph model sample, thus obtaining a graph model sample with at least one dimension corresponding to the third image sample. Then, using this graph model sample with at least one dimension corresponding to the third image sample as training samples for a pre-defined adversarial attack detection network, using the labeled adversarial category corresponding to the third image sample as the label, and the adversarial attack classification loss as the loss function, the pre-defined adversarial attack detection network is iteratively trained until the third loss information is minimized, or until a pre-defined number of iterations is reached, thus obtaining the adversarial attack detection model.

[0095] After obtaining the adversarial attack detection model, it can be used to detect adversarial attacks on the original image. The adversarial attack detection model performs adversarial attack detection based on the relational region features corresponding to the original image and outputs the corresponding adversarial attack probability. If the adversarial attack probability is greater than a threshold T, it is determined to be an adversarial attack target; if the adversarial attack probability is less than the threshold T, it is determined to be a live target. If the adversarial attack probability is equal to T, it can be determined to be either an adversarial attack target or a live target. Those skilled in the art can set whether to determine whether to determine it as an adversarial attack target or a live target when the adversarial attack probability is equal to T according to actual needs in practical applications. This specification does not impose any restrictions on this.

[0096] For details on how to construct at least one of the neighborhood graph model sample, key point graph model sample, and salient region graph model sample corresponding to the third image sample, please refer to the foregoing introduction, which will not be repeated here.

[0097] To improve the detection capability of adversarial attacks, this solution proposes an adversarial attack detection scheme based on hierarchical graph modeling of region relationships. For example... Figure 5 As shown, this adversarial attack detection scheme based on hierarchical graph model region relationship modeling mainly consists of four parts: data acquisition and preprocessing, feature extraction and hierarchical graph construction, graph neural network (GNN) model training, and adversarial attack detection, which can be described as follows:

[0098] (1) Data Acquisition and Preprocessing: After the user begins face recognition, the original image is acquired and data preprocessing is performed. Data preprocessing may include face detection to obtain the face region.

[0099] (2) Feature extraction and hierarchical graph construction: Feature extraction of the face region is performed in the image region dimension, and a graph model of at least one dimension is constructed based on the extracted original region features.

[0100] (3) GNN model training: The GNN is trained based on the constructed graph model of at least one dimension to obtain new regional features.

[0101] (4) Adversarial attack detection: Adversarial attack detection is performed based on new region features containing graph information of at least one dimension.

[0102] In summary, the adversarial attack detection method P100 and system 001 provided in this specification acquire an original image including the user's biometric features, construct a graph model with at least one dimension based on the original image, the graph model including the correlation relationships between different regions of the biometric features, and input the graph model with at least one dimension into a graph neural network model to determine the associated region features of the biometric features. The associated region features are then input into an adversarial attack detection model to obtain the adversarial attack detection result for the user. Because this scheme uses a graph model with at least one dimension, it constructs the graph model from different perspectives, establishes the correlation relationships between the original region features, and enriches the input information of the adversarial attack detection model. This allows for adversarial attack detection not only from the original region features but also from the correlation relationships between the original region features, thereby improving the performance and detection accuracy of adversarial attacks.

[0103] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for detecting counter-attacks. When the executable instructions are executed by a processor, they instruct the processor to perform the steps of the counter-attack detection method P100 described in this specification. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing device 600, the program code causes the computing device 600 to perform the steps of the counter-attack detection method P100 described in this specification. The program product for implementing the above method may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on the computing device 600. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on computing device 900, partially on computing device 900, as a standalone software package, partially on computing device 900 and partially on a remote computing device, or entirely on a remote computing device.

[0104] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0105] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0106] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0107] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0108] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. All contents used for all purposes, except for any history of prosecution documents relating to it, that may be inconsistent with or conflict with this document, or any such history of prosecution documents that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0109] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A method for detecting adversarial attacks, comprising: Acquire the user's original image, which includes the user's biometric features; The original image is input into a feature extraction model to obtain multiple original region features corresponding to the biological features; In each of at least two dimensions, connections are established between at least some of the correlated regional features among the plurality of original regional features to obtain a corresponding graph model; the graph model includes the correlation between different regions of the biological feature. Connect the same original region features across dimensions in the graph model; The graph model is input into a graph neural network model to determine the associated region features of the biological features; as well as The associated region features are input into the adversarial attack detection model to obtain the adversarial attack detection results for the user.

2. The method according to claim 1, wherein, The graph model includes at least two of the following: neighborhood graph model, key point graph model, and salient region graph model.

3. The method according to claim 2, wherein, The neighborhood graph model is determined using the following steps: Each of the multiple original region features is used as a target neighborhood node, and adjacent original region features are connected to the target neighborhood node.

4. The method according to claim 2, wherein, The key point graph model is determined using the following steps: The original region features representing key biological features among the multiple original region features are used as target key nodes, and the target key nodes with a similarity greater than a preset similarity threshold are connected.

5. The method according to claim 2, wherein, The salient region map model is determined using the following steps: Determine multiple saliencies corresponding to the multiple original region features, wherein the saliency characterizes the degree of attention paid to the original region features; Among the multiple original region features, those with a saliency greater than a preset saliency threshold are identified as saliency nodes. as well as Connect each of the salient nodes in pairs.

6. The method according to claim 1, wherein, The feature extraction model is trained using the following steps: Obtain the first image sample of the biometric features of the training object; A preset feature extraction network is used to extract features from multiple image regions corresponding to the first image sample, resulting in multiple feature maps and corresponding first loss information for the multiple image regions. The first loss information represents the difference between the training identity category and the labeled identity category of the training object corresponding to the first image sample. as well as The preset feature extraction network is converged based on the first loss information until training ends, thus obtaining the feature extraction model.

7. The method according to claim 1, wherein, The graph neural network model is trained using the following steps: Obtain second image samples of the biometric features of the training objects; The second image sample is input into the feature extraction model to obtain multiple original region feature samples corresponding to the second image sample; Based on the multiple original region feature samples, construct the graph model sample corresponding to the second image sample; The graph model samples are input into a preset graph neural network to obtain the associated region feature samples of the biological features and the corresponding second loss information; The preset graph neural network is converged based on the second loss information until the training ends, thus obtaining the graph neural network model.

8. The method according to claim 7, wherein, The network structure of the preset graph neural network is the same as the structure of the graph model with cross-dimensional connections. The second loss information is obtained using the following steps: Based on the preset graph neural network, multiple new region feature samples output from the multiple original region feature samples are used to classify adversarial attacks, thereby obtaining multiple training adversarial attack categories corresponding to the second image sample. Based on the difference between the labeled adversarial attack category corresponding to the second image sample and the multiple trained adversarial attack categories, multiple adversarial attack classification loss information is determined; as well as The second loss information is obtained by weighted summation of the loss information from the multiple adversarial attack classifications.

9. The method according to any one of claims 1-8, wherein, The adversarial attack detection model is trained using the following steps: Obtain third image samples of the biometric features of the training objects; Construct cross-dimensional connected graph model samples based on the third image samples; The graph model sample corresponding to the third image sample is input into the graph neural network model to obtain the associated region feature sample corresponding to the third image sample; The associated region feature samples corresponding to the third image sample are input into a preset adversarial attack detection network to obtain the training adversarial attack classification and the corresponding third loss information corresponding to the third image sample. The third loss information represents the difference between the training adversarial attack category and the labeled adversarial attack category corresponding to the third image sample. as well as The preset adversarial attack detection network is converged based on the third loss information until training ends, thus obtaining the adversarial attack detection model.

10. A detection system against attacks, comprising: At least one storage medium storing at least one instruction set for detecting adversarial attacks; as well as At least one processor is communicatively connected to the at least one storage medium. When the anti-attack detection system is running, the at least one processor reads the at least one instruction set and executes the method according to any one of claims 1-9 according to the instructions of the at least one instruction set.

Images