A method, device, processor and computer readable storage medium for implementing a gait recognition attack sample generation process

Abstract

The present application relates to a kind of methods for realizing gait recognition countermeasure sample generation process, comprising the following steps: constructing real gait image sequence;Constructing generative adversarial network;With the original real gait image input image sequence fusion module of generated countermeasure image;Construct gait countermeasure sample generation network;Using real gait image sequence training set and target gait recognition model feedback result to train network;Generate corresponding gait recognition countermeasure sample.The present application also relates to a kind of device for realizing gait recognition countermeasure sample generation, processor and its storage medium.Using the method for realizing gait recognition countermeasure sample generation process, device, processor and its computer readable storage medium of the present application can obtain higher attack success rate, the network structure is simple, execution efficiency is high, the gait countermeasure sample generated is real, clear, has broad application prospect in gait countermeasure sample data set construction and gait recognition security field.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to the field of gait recognition security, and more particularly to a method and device for generating gait recognition attack samples, a processor and a computer readable storage medium thereof. BACKGROUND

[0002] Gait recognition has a wide range of applications in the field of public security due to its advantages in long-distance identity recognition. Although gait recognition systems have high accuracy, their adversarial robustness has been proven to be at risk. Many studies have shown that deep gait recognition neural networks are vulnerable to adversarial attacks, which can affect the model output results by adding small perturbations to the input samples. Moreover, very advanced gait recognition models are also very vulnerable to various types of adversarial samples. At present, most gait recognition devices on the market are black-box encapsulated and only provide simple input and output interfaces, so the specific model structure and parameters cannot be known. At this time, various gait recognition product testing and detection agencies need to use more black-box gait adversarial attack samples to detect and verify various models. Since gait recognition needs to be based on video or image sequences, the success rate of adversarial attacks on gait recognition models has always been low, which greatly increases the difficulty of detection and evaluation of various gait recognition products for various testing and detection agencies. SUMMARY

[0003] The present application overcomes the shortcomings of the prior art and provides a method and device for generating gait recognition attack samples, a processor and a computer readable storage medium thereof, which satisfy high attack success rate, simple network structure and wide application range.

[0004] To achieve the above-mentioned purpose, the method, device, processor and computer readable storage medium for generating gait recognition attack samples of the present application are as follows:

[0005] The method for generating gait recognition attack samples has the following steps:

[0006] (1) According to the target gait recognition model to be attacked, construct a real gait image sequence according to the requirements of the gait image sample sequence;

[0007] (2) Construct a generative adversarial network, select a single image sample to be attacked from the real gait image sequence, and input the single image sample into the generative network to generate a corresponding adversarial image;

[0008] (3) Input the generated adversarial image and the original real gait image into an image sequence fusion module to generate an image sequence according to the image frame number;

[0009] (4) Construct a gait adversarial attack sample generation network, input the generated image sequence into a target gait recognition model, and perform reverse optimization on the gait adversarial attack sample generation network model according to the characteristic value fed back by the target gait recognition model;

[0010] (5) Training the network using the real gait image sequence training set and the target gait recognition model feedback result to obtain the final gait adversarial attack sample generation network model;

[0011] (6) Input the original real gait image into the gait adversarial attack sample generation network model to generate corresponding gait recognition adversarial attack samples.

[0012] Preferably, the step (1) specifically comprises the following steps:

[0013] (1.1) Determine the gait recognition image size;

[0014] (1.2) Determine the length of the continuous gait recognition image sequence.

[0015] Preferably, the length of the real gait image sequence constructed in the step (1) is N, that is, it contains N consecutive images.

[0016] Preferably, the step (2) specifically comprises the following steps:

[0017] (2.1) Construct an image frame selection vector V with a length of N, each element value in V being 0 or 1;

[0018] (2.2) According to the number of images selected by the vector V, that is, selecting M frames from the N consecutive images, the elements in V corresponding to the selected frames are 1, and the others are 0.

[0019] Preferably, in the step (3), the image sequence fusion module fuses the generated adversarial images according to V in the step (2) with the original real gait images to obtain a new gait image sequence with a length of N.

[0020] Preferably, the reverse optimization in the step (4) uses a loss function that measures the cosine similarity between the real sequence and the adversarial sequence.

[0021] Preferably, the step (5) specifically comprises the following steps:

[0022] (5.1) Input the real gait image sequence into the gait adversarial attack sample generation network;

[0023] (5.2) Optimize the generated adversarial loss;

[0024] (5.3) Optimize the feature cosine loss of the generated gait image sequence and the real gait image sequence.

[0025] Preferably, the corresponding gait recognition adversarial attack sample generated in step (6) is consistent in size with the original real gait image, and is combined with the image sequence fusion module to form a corresponding gait recognition adversarial attack sample sequence.

[0026] The device for implementing the gait recognition adversarial attack sample generation process mainly comprises:

[0027] a processor configured to execute computer executable instructions;

[0028] a memory storing one or more computer executable instructions, which, when executed by the processor, implement each step of the above-mentioned method for implementing the gait recognition adversarial attack sample generation process.

[0029] The processor for implementing the gait recognition adversarial attack sample generation process mainly comprises: the processor is configured to execute computer executable instructions, and the computer executable instructions, when executed by the processor, implement each step of the above-mentioned method for implementing the gait recognition adversarial attack sample generation process.

[0030] The computer readable storage medium mainly comprises that a computer program is stored thereon, and the computer program can be executed by the processor to implement each step of the above-mentioned method for implementing the gait recognition adversarial attack sample generation process.

[0031] The method, device, processor and computer readable storage medium for implementing the gait recognition adversarial attack sample generation process of the present application can maximize the feature difference extracted by the target gait recognition model for real and generated gait adversarial attack sample sequences in the minimum perturbation range by using deep learning technology, can deceive the gait recognition model by using the generated adversarial attack sample or sample sequence, and can verify the security of the gait comparison / verification / recognition product and system. The core of the present application is to use a black box method to generate adversarial high-quality gait adversarial attack samples based on a GAN architecture from a semantic point of view, and to fuse the generated samples with original samples. By sparsely replacing or inserting a small number of adversarial gait samples, a higher attack success rate can be obtained. The network structure is simple, the execution efficiency is high, the generated gait adversarial attack samples are real and clear, and the network structure has broad application prospects in the fields of gait adversarial attack sample dataset construction and gait recognition security. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The flowchart of the method for implementing the gait recognition adversarial attack sample generation process of the present application.

[0033] Figure 2A schematic diagram of an image sequence fusion module for implementing the method for generating gait recognition adversarial attack samples of the present application.

[0034] Figure 3 A schematic diagram of a gait recognition adversarial attack sample generation network for implementing the method for generating gait recognition adversarial attack samples of the present application. DETAILED DESCRIPTION

[0035] In order to more clearly describe the technical content of the present application, further description will be made below in combination with specific embodiments.

[0036] The method for generating gait recognition adversarial attack samples of the present application includes the following steps:

[0037] (1) According to the target gait recognition model to be attacked, construct a real gait image sequence according to the requirements of the gait image sample sequence;

[0038] (2) Construct a generative adversarial network, select a single image sample to be subjected to adversarial attack from the real gait image sequence, and input the single image sample into the generative network in turn to generate a corresponding adversarial image;

[0039] (3) Input the generated adversarial image and the original real gait image into an image sequence fusion module, and generate an image sequence according to the image frame number;

[0040] (4) Construct a gait adversarial attack sample generation network, input the generated image sequence into the target gait recognition model, and perform backward tuning of the gait adversarial attack sample generation network model according to the characteristic value fed back by the target gait recognition model;

[0041] (5) Train the network using the real gait image sequence training set and the target gait recognition model feedback result to obtain a final gait adversarial attack sample generation network model;

[0042] (6) Input the original real gait image into the gait adversarial attack sample generation network model to generate a corresponding gait recognition adversarial attack sample.

[0043] As a preferred embodiment of the present application, the step (1) specifically includes the following steps:

[0044] (1.1) Determine the gait recognition image size;

[0045] (1.2) Determine the length of the continuous gait recognition image sequence.

[0046] As a preferred embodiment of the present application, the length of the real gait image sequence constructed in step (1) is N, i.e. it contains N frames of continuous images.

[0047] As a preferred embodiment of the present application, the step (2) specifically comprises the following steps:

[0048] (2.1) Construct an image frame selection vector V with length N, each element value in V being 0 or 1;

[0049] (2.2) According to the number M of images selected by the vector V, M frames are selected from the N consecutive images, and the elements corresponding to the selected frames in V are 1, and the others are 0.

[0050] As a preferred embodiment of the present application, in the step (3), the image sequence fusion module fuses the generated adversarial images according to V in the step (2) with the original real gait images to obtain a new gait image sequence with length N.

[0051] As a preferred embodiment of the present application, the reverse optimization in the step (4) utilizes a loss function that measures the cosine similarity between the real sequence and the adversarial sequence.

[0052] As a preferred embodiment of the present application, the step (5) specifically comprises the following steps:

[0053] (5.1) Input the real gait image sequence into the gait adversarial attack sample generation network;

[0054] (5.2) Optimize the generated adversarial loss;

[0055] (5.3) Optimize the feature cosine loss between the generated gait image sequence and the real gait image sequence.

[0056] As a preferred embodiment of the present application, in the step (6), the generated corresponding gait recognition adversarial attack sample is consistent in size with the original real gait image, and is combined with the image sequence fusion module to form a corresponding gait recognition adversarial attack sample sequence.

[0057] The device for implementing the gait recognition adversarial attack sample generation process of the present application, wherein the device comprises:

[0058] a processor configured to execute computer executable instructions;

[0059] a memory storing one or more computer executable instructions, which, when executed by the processor, implement each step of the above-mentioned method for implementing the gait recognition adversarial attack sample generation process.

[0060] The processor for implementing the gait recognition adversarial attack sample generation process of the present application, wherein the processor is configured to execute computer executable instructions, and the computer executable instructions, when executed by the processor, implement each step of the method for implementing the gait recognition adversarial attack sample generation process described above.

[0061] The computer readable storage medium of the present application, on which a computer program is stored, the computer program can be executed by a processor to implement each step of the method for implementing the gait recognition adversarial attack sample generation process described above.

[0062] In the detailed description of the present application, as shown in Figure 1 , when the target gait recognition model to be attacked is GaitSet, a gait recognition adversarial attack sample generation method specifically includes the following steps:

[0063] (1) Preprocess the real gait picture data set for training, first crop the contour area of the original picture containing the gait contour, and the size of the cropped human face picture is 60x240 pixels. The ratio of the area of the gait contour to the area of the background is about 5:1. The number of gait picture samples in the final real gait image sequence data set used for training is 50000, and there are 5000 sequences, and the number of consecutive gait pictures in each sequence is 10 (N=10);

[0064] (2) Construct a system architecture based on a generative adversarial network (GAN), select a vector V according to the image frame, and input a single image sample selected from the real gait image sequence to the generation network to generate a corresponding adversarial image; the length of the image frame selection vector V is 10, and each element value in V is 0 or 1; the number of images selected according to V is 3 (M=3), that is, 3 frames are selected from 10 consecutive images, and the elements in V corresponding to the selected frames are 1 and the others are 0; if the selected image corresponds to frames 1, 3, and 7, then V=[1 0 1 0 0 0 1 0 0 0]; wherein the loss function used by the adversarial attack network is L GAN , which consists of two parts:

[0065] L GAN =ɑL mse +βL WGAN-GP ;

[0066] Wherein, ɑ, β are weight parameters.

[0067] The adversarial loss L WGAN-GP and the pixel-by-pixel loss L mse ; L WGAN-GP is the adversarial loss of the generative adversarial network using WGAN-GP:

[0068]

[0069] wherein m is the number of gait images sampled per batch, x is the source gait image, G is the generated data, D(x i ) is the probability that the adversarial network judges the real data to be real, and D(G(x i )) is the probability that the adversarial network judges the generated data to be real.

[0070] L mse is the pixel-by-pixel MSE loss of the generated gait and the real gait, which is minimized to ensure that the generated gait image and the real gait image have the smallest pixel difference, so that they look indistinguishable, and thus can deceive the human eye, and its expression is as follows:

[0071]

[0072] wherein N2 is the number of image pixels, x and G(x) are the real face image and the generated face image, respectively.

[0073] (3) The generated adversarial image is fused into a new gait image sequence X with a length of 10 according to V with the original real gait image by using an image sequence fusion module, and a schematic diagram of the image sequence fusion module is shown in the accompanying drawings of the specification Figure 2 , and the calculation of X is as follows:

[0074] X = V·G(M) + (1-V)·X *

[0075] wherein X * is the original real image sequence, and G(M) is the generated image sequence corresponding to the selected M frames of original images.

[0076] (4) A gait adversarial attack sample generation network is constructed, the generated image sequence is input into a target gait recognition model, and the gait adversarial attack sample generation network model is inversely optimized according to the recognition feature values fed back by the target gait recognition model; a schematic diagram of the gait adversarial attack sample generation network is shown in the accompanying drawings of the specification Figure 3 , wherein the inverse optimization utilizes a loss function L cos that measures the cosine similarity between the real sequence and the adversarial sequence, and the purpose is to maximize the distance between the features extracted by the target model GaitSet from the real sample sequence and the generated sample sequence, and the expression of L cos is as follows:

[0077] L cos = ζ cos (f(X), f(X * ))

[0078] wherein X *For the original real image sequence, X is the generated new gait image sequence, f is the target model GaitSet feature extraction function.

[0079] (5) The network is trained using the real gait image sequence training set until the result converges, and the final gait adversarial attack sample generation network model can be obtained, wherein the total loss function L total The expression is as follows:

[0080] L total = L GAN + εL cos = αL mse + βL WGAN-GP + εL cos

[0081] Wherein, α, β, ε are weight parameters.

[0082] (6) The original real gait image is input into the gait adversarial attack sample generation network model, and the corresponding gait recognition adversarial attack sample can be generated. The generated corresponding gait recognition adversarial attack sample is consistent in size with the original real gait image, and combined with the image sequence fusion module C, a black box gait recognition adversarial attack sample sequence corresponding to each type of gait recognition model can be further generated, which can make the target gait recognition model misidentify the sample.

[0083] The specific implementation scheme of the embodiment can be referred to the related description in the above embodiment, which will not be repeated here.

[0084] It can be understood that the same or similar parts in the above embodiments can be mutually referred to, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0085] It should be noted that in the description of the present application, the terms "first", "second" and the like are only for the purpose of description and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" is at least two.

[0086] Any process or method descriptions in the flowchart or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions or steps, and the scope of the preferred embodiments of the present application includes additional implementation in which the functions described are performed in a different order, including substantially simultaneously, or in reverse order, or in an order that is different from the order shown or discussed, including as an outcome of the functions being performed in a different order, as will be understood by those skilled in the art of the technical field of embodiments of the present application.

[0087] It should be understood that each part of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if realized by hardware, and as in another embodiment, any one or a combination of the following technologies known in the art can be used: discrete logic circuit with logic gate circuit for implementing logic functions on data signals, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA), etc.

[0088] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by a program instructing the relevant hardware, and the corresponding program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0089] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can exist physically, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. The integrated module, if realized in the form of a software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0090] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0091] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0092] The method, device, processor and computer readable storage medium for realizing gait recognition attack sample generation processing adopt deep learning technology to maximize the feature difference of a target gait recognition model for real and generated gait attack sample sequences in a minimum disturbance range, and can use the generated attack samples or sample sequences to deceive the gait recognition model and pass the security verification of gait comparison / verification / recognition products and systems. The core of the application is to use a black box to generate high-quality gait attack samples from semantics based on a GAN architecture, and to fuse the generated samples with the original samples. By sparsely replacing or inserting a small number of gait attack samples, a high attack success rate can be achieved. The network structure is simple, the execution efficiency is high, the generated gait attack samples are real and clear, and the network structure has broad application prospects in the fields of gait attack sample dataset construction and gait recognition security.

[0093] In this specification, the application has been described with reference to its particular embodiments. It is clear, however, that various modifications and changes can be made without departing from the spirit and scope of the application. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A method for implementing a gait recognition attack sample generation process, characterized in that, The method comprises the following steps: (1) According to the gait recognition model of the target to be attacked, a real gait image sequence is constructed according to the requirements of the gait image sample sequence; (2) A generative adversarial network is constructed, a single image sample to be subjected to adversarial attack is selected from the real gait image sequence, and the single image sample is sequentially input into the generative network to generate a corresponding adversarial image; (3) The generated adversarial image and the original real gait image are input into an image sequence fusion module, and an image sequence is generated according to the image frame number; (4) A gait adversarial attack sample generation network is constructed, the generated image sequence is input into the target gait recognition model, and the gait adversarial attack sample generation network model is inversely optimized according to the characteristic value fed back by the target gait recognition model; (5) The network is trained using the real gait image sequence training set and the target gait recognition model feedback result to obtain a final gait adversarial attack sample generation network model; (6) The original real gait image is input into the gait adversarial attack sample generation network model to generate a corresponding gait recognition adversarial attack sample.

2. The method of claim 1, wherein the method is implemented by a computer system. The step (1) specifically comprises the following steps: (1.1) Determine the gait recognition image size; (1.2) Determine the length of the continuous gait recognition image sequence.

3. The method of claim 1, wherein the method comprises: The real gait image sequence constructed in step (1) has a length of N, that is, it contains N consecutive images.

4. The method of claim 1, wherein the method comprises: The step (2) specifically comprises the following steps: (2.1) Construct an image frame selection vector V with a length of N, each element value in V being 0 or 1; (2.2) According to the number of images selected by the vector V, M frames are selected from the N consecutive images, and the elements in V corresponding to the selected frames are 1 and the others are 0.

5. The method of claim 4, wherein the method comprises: In step (3), the image sequence fusion module fuses the generated adversarial image with the original real gait image according to V in step (2) to obtain a new gait image sequence with a length of N.

6. The method of claim 1, wherein the method is implemented by a computer system, and the method comprises: obtaining a plurality of gait recognition attack sample generation processes; and determining a gait recognition attack sample generation process from the plurality of gait recognition attack sample generation processes. The reverse optimization in step (4) uses a loss function that measures the cosine similarity between the real sequence and the adversarial sequence.

7. The method of claim 1, wherein the method is implemented by a computer system, and the method comprises: determining a target gait recognition attack sample; and generating a gait recognition attack sample based on the target gait recognition attack sample. The step (5) specifically comprises the following steps: (5.1) Input the real gait image sequence into the gait adversarial attack sample generation network; (5.2) Optimize the generative adversarial loss; (5.3) Optimize the feature cosine loss of the generated gait image sequence and the real gait image sequence.

8. The method of claim 1, wherein the method is implemented by a computer system, and the method comprises: obtaining a plurality of gait recognition attack sample generation processes; and determining a gait recognition attack sample generation process from the plurality of gait recognition attack sample generation processes. The step (6) generates a corresponding gait recognition adversarial attack sample consistent with the original real gait image size, and combines the image sequence fusion module to form a corresponding gait recognition adversarial attack sample sequence.

9. An apparatus for implementing a gait recognition adversarial attack sample generation process, characterized in that, The device comprises: a processor configured to execute computer executable instructions; a memory storing one or more computer executable instructions, which, when executed by the processor, implement each step of the method for generating a gait recognition adversarial attack sample as claimed in any one of claims 1 to 8.

10. A processor for implementing a gait recognition adversarial attack sample generation process, characterized in that, The processor is configured to execute computer executable instructions, which, when executed by the processor, implement each step of the method for generating a gait recognition adversarial attack sample as claimed in any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, A computer program product, comprising a computer readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the method of any one of claims 1 to 8. A computer program product, comprising a computer readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the method of any one of claims 1 to 8.

Images