Gait identity information desensitization method and system based on sparse contour hopping

Abstract

The invention provides a gait identity information desensitization method and system based on sparse contour hopping. The method comprises the steps that S1, a gait video is deframed into an image sequence, and binarization processing is conducted on the image sequence; s2, extracting identity features in the binary image sequence by using a gait recognition model, and calculating an adversarial disturbance matrix of each image according to a loss function; s3, binary image sequence key frames are selected at equal intervals, and adversarial disturbance meeting the pixel jump constraint is added to the human body contour edge in the key frames; s4, embedding the finally added disturbance value into the binary image sequence to obtain a reversible human body contour image sequence; and S5, pixel color backfilling is carried out on the modified binary image sequence, and a desensitized gait video is obtained. According to the method, the loss function of the image sequence before and after desensitization is maximized by means of the gait recognition model, so that the pixel directly generates binary hopping towards the gradient rising direction, non-directional or directional interference on the gait recognition result is realized, and the identity information privacy is protected.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method and system for desensitizing gait identity information based on sparse contour jumps. Background Art

[0002] Gait identity recognition is a technology that identifies people from walking videos. Gait videos contain variables such as environment, clothing, shooting distance and angle. After preprocessing operations such as human body contour extraction, segmentation, and time alignment, a standard gait sequence can be extracted. After obtaining the standard gait sequence, gait features can be extracted, which are mainly divided into human morphological features and gait motion features. Identity matching is to identify the extracted gait features through the gait model constructed by machine learning or deep learning. Gait identity recognition technology can be mainly divided into model-based methods and non-model methods. Model-based gait recognition technology analyzes human limbs, such as arms, knees, calves and thighs, and obtains a series of posture features through modeling and tracking, thereby realizing the analysis and recognition of identity information; gait recognition technology based on non-model methods does not consider the structure of the human body, generally directly extracts the human body contour and further extracts the target identity information. The main purpose of gait identity information desensitization is to keep the human body in the video natural while protecting the gait features in the video from being analyzed and synthesized by intelligent algorithms, preventing the algorithm from correctly extracting identity-related visual features such as appearance and movement in the picture, thereby protecting the identity information privacy of the people in the video.

[0003] Patent document CN115527252A uses the Mask Transfiner model to infer the gait sequence to obtain the gait mask sequence, and uses the gait recognition model to perform target detection training, and then uses the trained model to infer the gait identity; this application directly uses the recognition model to identify the gradient information of the loss function of the image, with the goal of maximizing the loss and thus interfering with the recognition result, so that the pixel directly jumps to the direction of the gradient rise, and can be directly protected by the inference stage of the pre-trained model without network training. There are obvious differences between the two technical routes.

[0004] A video gait privacy protection algorithm based on contour sparse adversarial[J]. Information Network Security, 24(1):48-59. The method in this paper modifies the contour edge, calculates the gradient direction of the loss function within the range of pixels around the contour, and adds adversarial perturbations along the direction of the gradient change on these pixels. This method adds noise to each frame in the image sequence, and the image sequence after adding noise cannot be restored. This method selects key frames for adversarial, ensures sparsity in time and space, and increases the embedding of perturbation information, making the image after adding noise reversible. After noise removal, gait information can be correctly identified. There are obvious differences between the two technical routes. Summary of the invention

[0005] In view of the defects in the prior art, the object of the present invention is to provide a gait identity information desensitization method and system based on sparse contour jump.

[0006] According to the present invention, a gait identity information desensitization method based on sparse contour jump is provided, comprising:

[0007] Step S1: Obtain individual gait video and perform preprocessing;

[0008] The preprocessing includes de-framing the video into an image sequence and performing a binarization process of the human body contour using a background subtraction method;

[0009] Step S2: Use the gait recognition model to extract identity features from the binary image sequence, and calculate the adversarial perturbation matrix of each image according to the loss function;

[0010] Step S3: selecting key frames of the binary image sequence at equal intervals, and adding adversarial perturbations that satisfy pixel jump constraints to the edges of the human body contours in the key frames;

[0011] Step S4: embedding the disturbance value finally added into the binary image sequence to obtain a reversible human body contour image sequence;

[0012] Step S5: backfill the pixel colors of the modified binary image sequence to obtain a desensitized gait video.

[0013] Specifically, the step S1 includes:

[0014] Obtain gait video, deframe the original video into an image sequence I = [I1, I2, ...], and remove images that do not meet the preset requirements;

[0015] Using background subtraction, each image sequence I iSubtract the pure background image BG without human figures in the same scene to obtain the grayscale image sequence N = [N1, N2, ...] of the human foreground, and binarize the image to obtain the binary contour sequence B = [B1, B2, ...]. For the pixel value of the coordinate (a, b), it is processed according to the binarization threshold t:

[0016] B i (a,b)={255,if N i (a,b)≥t; 0, if N i (a,b) <t}

[0017] After binarization, the image is cropped according to the portrait height. After cropping, the portrait is located in the center of the image and scaled to a size of 64*64 to obtain a normalized binary image sequence x=[x1,x2,…].

[0018] Specifically, step S2 includes:

[0019] Use a self-developed or publicly available gait recognition model f, input the binary image sequence into the model, and obtain the network's loss function L and the corresponding gradient matrix for identity recognition of each image. The loss function L is used to measure the current unknown identity image x i The eigenvector f(x i ) is similar to the feature vector y of the labeled identity to determine whether the two samples belong to the same identity. The gradient of the model f is calculated inversely through the similarity loss to obtain the gradient value of the loss function of each frame image And calculate the gradient direction The upward jump value is 1, and the downward jump value is -1. In this method, only the upward jump value is retained, which is expressed as

[0020] Specifically, step S3 includes selecting a key frame x at intervals of 10 frames in a complete gait sequence x=[x1, x2, ...] p =[x1,x 11 ,x 21 ,...], and for each sample sequence, the actual position and interval of the selected samples are fixed. Then the canny edge detection algorithm is used to detect x p Each binary image x i The edge of the portrait is dilated by three pixels, and the position set after dilation is E(x i ), calculate the mask mask according to the following formula, and limit the range of pixel jump to this mask:

[0021]

[0022] In the formula, x i(a, b) is each human body contour edge pixel on each binary image. Finally, the perturbation value after the adversarial perturbation on the key frame passes through the mask is added to the key frame to obtain the desensitized binary image sequence.

[0023]

[0024] Input the desensitized image sequence x adv into the gait recognition model for gait identity recognition, and evaluate the protection effect according to the change of the identity recognition accuracy acc and the structural similarity SSIM(x, x adv ) before and after desensitization.

[0025] Specifically, step S4 includes embedding the added adversarial noise in the desensitized binary image sequence x adv . Since each pixel is represented by an 8-bit integer, set the lowest bit at the position of the adversarial noise in x adv to 1 and the lowest bit at the original pixel position to 0. Therefore, the adversarial noise pixel value 11111111₂ = 255 10 , the pixel value of the foreground part of the original image 11111110₂ = 254 10 , and the pixel value of the background part of the original image 00000000₂ = 0 10 . Subsequently, the clean image sample can be restored according to the embedding rule.

[0026] Specifically, step S5 includes: selecting a threshold t₁ close to the background threshold t, adding the modified binary image sequence x adv to the image sequence I = [I₁, I₂,...], so that the human body contour pixel value in the color image sequence exceeds the threshold t₁, and the adversarial noise part is greater than t and less than t₁, to obtain the desensitized image sequence I' = [I'₁, I'₂,...]: If the pixel position (a, b) is non-zero in x adv and belongs to the human body contour, if the original pixel value I(a, b) is greater than the threshold t₁, no modification is made, and if it is less than the threshold t₁, it is changed to

[0027] I′(a, b) = t₁ + 1

[0028] Similarly, if the pixel position (a, b) is non-zero in x adv and belongs to the adversarial noise, if the original pixel value I(a, b) is greater than the threshold t, no modification is made, and if it is less than the threshold t, it is changed to

[0029] I′(a, b) = t + 1

[0030] According to a gait identity information desensitization system based on sparse contour jump provided by the present invention, it includes:

[0031] Module M1: Preprocessing module

[0032] Deframe the video into image sequences and use background subtraction to binarize the human body contours;

[0033] Module M2: Reversible adversarial sample generation module

[0034] The identity features in the binary image sequence are extracted using the gait recognition model, and the adversarial perturbation matrix of each image is calculated according to the loss function; key frames of the binary image sequence are selected at equal intervals, and adversarial perturbations that satisfy pixel jump constraints are added to the edges of the human body contours in the key frames; the final added perturbation values ​​are embedded in the binary image sequence to obtain a reversible human body contour image sequence;

[0035] Module M3: Backfill Module

[0036] The modified binary image sequence is backfilled with pixel colors to obtain the desensitized gait video.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. In order to solve the problem that the protection ability of the existing methods is not strong enough, the present invention uses the gradient information of the loss function of the recognition model on the image, with the goal of maximizing the loss and thus interfering with the recognition result, so that the pixel directly undergoes a binary jump in the direction of the gradient increase. Such a jump can maximize the loss of the image in the recognition process, thereby causing identity recognition errors and protecting the privacy of identity information.

[0039] 2. In view of the problem that existing methods make large changes to images, this invention proposes the idea of ​​"sparseness" for the first time. On the one hand, it is spatial sparseness, which only performs sparse pixel jumps on the edges of human body contours, and determines the jump mode based on gradient information, rather than making the same changes to a large area of ​​the image; on the other hand, it is temporal sparseness, which only extracts equally spaced key frames in the complete gait sequence for processing, which is concealed and can also achieve a good identity information desensitization effect;

[0040] 3. The present invention directly interferes with the original knowledge of the gait recognition model, does not need to modify or retrain the gait recognition model, is reversible, reduces the amount of calculation and operation complexity, and has good application prospects.

[0041] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0043] Figure 1 The figure is a flow chart of the method of the present invention.

[0044] Figure 2 It is a schematic diagram of the device of the present invention. DETAILED DESCRIPTION

[0045] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0046] Reference Figure 1 As shown, a gait identity information desensitization method based on sparse contour jump includes the following steps:

[0047] Step S1: Obtain individual gait video and perform preprocessing;

[0048] The preprocessing includes de-framing the video into an image sequence and performing a binarization process of the human body contour using a background subtraction method;

[0049] Step S2: Use the gait recognition model to extract identity features from the binary image sequence, and calculate the adversarial perturbation matrix of each image according to the loss function;

[0050] Step S3: selecting key frames of the binary image sequence at equal intervals, and adding adversarial perturbations that satisfy pixel jump constraints to the edges of the human body contours in the key frames;

[0051] Step S4: embedding the disturbance value finally added into the binary image sequence to obtain a reversible human body contour image sequence;

[0052] Step S5: backfill the pixel colors of the modified binary image sequence to obtain a desensitized gait video.

[0053] Specifically, the step S1 includes:

[0054] Get the gait video and decode the original video into an image sequence I = [I 11 ,I2,…], remove images that do not meet the preset requirements;

[0055] Using background subtraction, each image sequence I iSubtract the pure background image BG without human figures in the same scene to obtain the grayscale image sequence N = [N1, N2, ...] of the human foreground, and binarize the image to obtain the binary contour sequence B = [B1, B2, ...]. For the pixel value of the coordinate (a, b), it is processed according to the binarization threshold t:

[0056] B i (a,b)={255,if N i (a,b)≥t; 0, if N i (a,b) <t}

[0057] After binarization, the image is cropped according to the portrait height. After cropping, the portrait is located in the center of the image and scaled to a size of 64*64 to obtain a normalized binary image sequence x=[x1,x2,…].

[0058] Specifically, step S2 includes:

[0059] Use a self-developed or publicly available gait recognition model f, input the binary image sequence into the model, and obtain the network's loss function L and the corresponding gradient matrix for identity recognition of each image. The loss function L is used to measure the current unknown identity image x i The eigenvector f(x i ) is similar to the feature vector y of the labeled identity to determine whether the two samples belong to the same identity. The gradient of the model f is calculated inversely through the similarity loss to obtain the gradient value of the loss function of each frame image And calculate the gradient direction The upward jump value is 1, and the downward jump value is -1. In this method, only the upward jump value is retained, which is expressed as

[0060] Specifically, step S3 includes selecting a key frame x at intervals of 10 frames in a complete gait sequence x=[x1, x2, ...] p =[x1,x 11 ,x 21 ,...], and for each sample sequence, the actual position and interval of the selected samples are fixed. Then the canny edge detection algorithm is used to detect x p Each binary image x i The edge of the portrait is dilated by three pixels, and the position set after dilation is E(x i ), calculate the mask mask according to the following formula, and limit the range of pixel jump to this mask:

[0061]

[0062] In the formula, x i(a, b) is each pixel on the edge of the human body contour in each binary image. Finally, the perturbation value after the adversarial perturbation on the key frame passes through the mask is added to the key frame to obtain the desensitized binary image sequence.

[0063]

[0064] The desensitized image sequence x adv is input into the gait recognition model for gait identity recognition. According to the changes in the identity recognition accuracy acc and the structural similarity SSIM(x, x adv ) before and after desensitization, the protection effect is evaluated, where acc evaluates the gait identity recognition result after desensitization and the recognition accuracy with the true identity label y i . SSIM(x, x adv ) evaluates the similarity μ of the samples before and after desensitization x , is the mean and variance of the image x, is the mean and variance of the image x adv , is the covariance of the images x and x adv , and c1 and c2 are constant terms to avoid a zero denominator.

[0065]

[0066] Specifically, step S4 includes embedding the added adversarial noise in the desensitized binary image sequence x adv . Since each pixel is represented by an 8-bit integer, the lowest bit at the position of the adversarial noise in x adv is set to 1, and the lowest bit at the original pixel position is 0. Therefore, the adversarial noise pixel value 11111111₂ = 255 10 , the pixel value of the foreground part of the original image 11111110₂ = 254 10 , and the pixel value of the background part of the original image 00000000₂ = 0 10 . Subsequently, the clean image sample can be restored according to the embedding rule.

[0067] Specifically, step S5 includes: selecting a threshold t1 close to the background threshold t, adding the modified binary image sequence x adv to the image sequence I = [I1, I2,...], so that the pixel values of the human body contour in the color image sequence exceed the threshold t1, and the adversarial noise part is greater than t and less than t1, to obtain the desensitized image sequence I' = [I'1, I'2,...]: If the pixel position (a, b) is in x advis not zero and belongs to the human body contour. If the original pixel value I(a,b) is greater than the threshold t1, no modification is made. If it is less than the threshold t1, it is changed to:

[0068] I′(a,b)=t1+1

[0069] Similarly, if the pixel position (a, b) is at x adv If the original pixel value I(a,b) is not zero and belongs to adversarial noise, it will not be modified if it is greater than the threshold t. If it is less than the threshold t, it will be changed to:

[0070] I′(a,b)=t+1

[0071] The present invention also provides a gait identity information desensitization system based on sparse contour jumps. The gait identity information desensitization system based on sparse contour jumps can be implemented by executing the process steps of the gait identity information desensitization method based on sparse contour jumps, that is, those skilled in the art can understand the gait identity information desensitization method based on sparse contour jumps as a preferred implementation of the gait identity information desensitization system based on sparse contour jumps.

[0072] Reference Figure 2 Specifically, a gait identity information desensitization system based on sparse contour jumps includes:

[0073] Module M1: Preprocessing module

[0074] Deframe the video into image sequences and use background subtraction to binarize the human body contours;

[0075] Module M2: Reversible adversarial sample generation module

[0076] The identity features in the binary image sequence are extracted using the gait recognition model, and the adversarial perturbation matrix of each image is calculated according to the loss function; key frames of the binary image sequence are selected at equal intervals, and adversarial perturbations that satisfy pixel jump constraints are added to the edges of the human body contours in the key frames; the final added perturbation values ​​are embedded in the binary image sequence to obtain a reversible human body contour image sequence;

[0077] Module M3: Backfill Module

[0078] The modified binary image sequence is backfilled with pixel colors to obtain the desensitized gait video.

[0079] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0080] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A gait identity information desensitization method based on sparse contour jump, characterized in that: include: Step S1: Obtain individual gait video and perform preprocessing; The preprocessing includes de-framing the video into an image sequence and performing binarization processing; Step S2: using the gait recognition model, input the preprocessed image sequence into the recognition network, and obtain the gradient matrix of the loss function of the network for identity recognition of each image; Step S3: randomly extracting a part of the continuous sequence from the complete sequence; Step S4: locating the gradient value corresponding to each human body contour edge pixel point on each binary image in the extracted partial continuous sequence; Step S5: Obtain the sign of the gradient according to the gradient matrix of the loss function of the network for identity recognition of each image, perform pixel jump on each edge pixel of the human body contour, and obtain a modified binary image sequence; Step S6: backfill the pixel colors of the modified binary image sequence to obtain a desensitized gait video.

2. The gait identity information desensitization method based on sparse contour jump according to claim 1 is characterized in that: The step S1 comprises: Use a camera to obtain gait videos of different individuals under different conditions, deframe the original walking video into an image sequence s = [s1, s2, ...], and remove images that do not meet the preset requirements; Using background subtraction, we use each image sequence s i Subtract the pure background image bg without the portrait at the same angle to get the grayscale image N of the human body and background i , processed according to the binary threshold t: B i (a,b)={255,if N i (a,b)≥t;0,if N i (a,b)<t} The background part is changed to black and the portrait part is changed to white, the portrait part is separated from the background, the picture is cropped into an image with the portrait height as the image height and the portrait in the center of the image, and it is scaled to a uniform size to obtain the normalized binary image sequence x=[x1,x2,…].

3. The gait identity information desensitization method based on sparse contour jump according to claim 2 is characterized in that: The step S2 comprises: Use the gait recognition model G to input the image into the recognition network f and obtain the gradient matrix of the loss function L of the network for identity recognition of each image. The processed image is sent to the inference stage of the gait recognition model G to extract gait features and calculate the loss function L; the gait recognition model is used to implement the mapping function f, and the image x i Mapped to its corresponding gait feature vector y; the reasoning process is: L=argmin{L(f(x i ),y)} The loss function L is used to measure the current unknown identity image x i The eigenvector f(x i ) is used to determine whether two samples belong to the same identity by comparing their similarity with the feature vector y of the labeled identity, and to obtain the gradient value of the loss function for each frame of the image 4. The gait identity information desensitization method based on sparse contour jump according to claim 3 is characterized in that: The step S3 comprises randomly selecting a portion of consecutive frames x in a complete gait sequence x=[x1, x2, ...] p =[x i ,x i+1 ,…,x i+k ], and for each sample sequence, the selected position and length are fixed.

5. The gait identity information desensitization method based on sparse contour jump according to claim 4 is characterized in that: The step S4 includes using an edge detection algorithm to detect x p Each binary image x i The edge of the portrait is expanded to a pixel width E(x i ), calculate the mask according to the following formula, and limit the range of pixel jump to within this mask: In the formula, x i (a, b) are the edge pixels of each human body contour on each binary image.

6. The gait identity information desensitization method based on sparse contour jump according to claim 5, characterized in that: The step S5 comprises: According to the sign of the gradient For edge pixel x i (a,b) pixel jumps to x i '(a,b), get the modified binary image sequence x' p =[x' i ,x' i+1 ,…,x' i+k ], so that it replaces x in x=[x1,x2,…] p =[x i ,x i+1 ,…,x i+k ] part, we get x'=[x1,x2,…,x' i ,x' i+1 ,…,x' i+k ,…,x n ]; at the same time, x' is input into the gait recognition model for gait identity recognition, and the protection effect is evaluated according to the identity recognition accuracy acc of the modified binary image sequence and the structural similarity SSIM(x,x') before and after modification.

7. The gait identity information desensitization method based on sparse contour jump according to claim 6, characterized in that: The step S6 comprises: According to the image before binarization, the modified black and white gait graph is refilled into a color image s' p =[s' i ,s' i+1 ,…], and combine them in their original order with the unmodified sequence in s=[s1,s2,…] to form the desensitized sequence s'=[s1,s2,…,s' i ,s' i+1 ,…s' i+k ,…,s n ] to obtain the desensitized gait video.

8. A gait identity information desensitization system based on sparse contour jump, characterized in that: include: Module M1: Obtain individual gait video and perform preprocessing; The preprocessing includes de-framing the video into an image sequence and performing binarization processing; Module M2: Using the gait recognition model, the preprocessed image sequence is input into the recognition network to obtain the gradient matrix of the loss function of the network for identity recognition of each image; Module M3: randomly extract a part of the continuous sequence from the complete sequence; Module M4: locate the gradient value corresponding to each human body contour edge pixel point on each binary image in the extracted partial continuous sequence; Module M5: Obtain the gradient sign according to the gradient matrix of the loss function of the network for identity recognition of each image, perform pixel jump on each edge pixel of the human body contour, and obtain a modified binary image sequence; Module M6: Perform pixel color backfill on the modified binary image sequence to obtain the desensitized gait video.

9. The gait identity information desensitization system based on sparse contour jump according to claim 8, characterized in that: The module M1 comprises: Use a camera to obtain gait videos of different individuals under different conditions, deframe the original walking video into an image sequence s = [s1, s2, ...], and remove images that do not meet the preset requirements; Using background subtraction, we use each image sequence s i Subtract the pure background image bg without the portrait at the same angle to get the grayscale image N of the human body and background i , processed according to the binary threshold t: B i (a,b)={255,if N i (a,b)≥t;0,if N i (a,b)<t} The background part is changed to black and the portrait part is changed to white, the portrait part is separated from the background, the picture is cropped into an image with the portrait height as the image height and the portrait in the center of the image, and it is scaled to a uniform size to obtain the normalized binary image sequence x=[x1,x2,…].

10. The gait identity information desensitization system based on sparse contour jump according to claim 9, characterized in that: The module M2 comprises: Use the gait recognition model G to input the image into the recognition network f and obtain the gradient matrix of the loss function L of the network for identity recognition of each image. The processed image is sent to the inference stage of the gait recognition model G to extract gait features and calculate the loss function L; the gait recognition model is used to implement the mapping function f, and the image x i Mapped to its corresponding gait feature vector y; the reasoning process is: L=argmin{L(f(x i ),y)} The loss function L is used to measure the current unknown identity image x i The eigenvector f(x i ) is used to determine whether two samples belong to the same identity by comparing their similarity with the feature vector y of the labeled identity, and to obtain the gradient value of the loss function for each frame of the image

Images