Keyframe adversarial sample generation method based on knowledge distillation
By decoupling video actions into pose and appearance features through knowledge distillation, and locating and replacing keyframes, this approach solves the scalability and attack control problems of existing video adversarial example generation methods, and improves the attack efficiency and effectiveness in the RGB image domain.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINESE PEOPLES LIBERATION ARMY UNIT 32801
- Filing Date
- 2025-09-05
- Publication Date
- 2026-06-09
Smart Images
Figure CN121146016B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of adversarial attack technology in computer vision, specifically relating to a method for generating adversarial examples of key video frames based on knowledge distillation. Background Technology
[0002] With the continuous development and evolution of artificial intelligence technology, it is necessary to use adversarial attack technology to test and explore the robustness of artificial intelligence models. At present, the main target of adversarial attack applications is image recognition tasks based on convolutional neural networks, such as image classification, object detection, face recognition, etc. Less attention is paid to recognition tasks based on time-series videos, such as gait recognition, action recognition, etc.
[0003] The technology most relevant to this invention is: adversarial gait contour generation technology based on adversarial generative networks. For example... Figure 1 As shown, in the adversarial attack process, the latent variable Z is the only parameter that needs to be optimized. First, Z is randomly sampled, and then it is optimized using gradient backpropagation (blue arrow part). Then, the optimized Z is used as the input of the gait contour generator G to generate a video sequence that can affect the accuracy of gait recognition. Then, the original sequence is randomly replaced using a mask M to complete the production of adversarial video samples. Summary of the Invention
[0004] (a) Technical problems to be solved
[0005] The technical problem this invention aims to solve is how to provide a key video frame adversarial example generation method based on knowledge distillation, in order to address the shortcomings of existing video adversarial example generation methods: 1) Poor scalability: the range of sample generation is limited to gait contours. These images are single-channel images with only black and white colors, which are a specific input for gait recognition tasks and cannot be extended to the RGB image domain or other temporal recognition tasks; 2) In RGB images, this example generation method based on generative adversarial networks cannot guarantee that its perturbation is focused only on the target's motion posture; 3) The adversarial video sequence generated by the method lacks selection strategy guidance when replacing interfering frames, and only randomly replaces them, which affects the effectiveness of the attack.
[0006] (II) Technical Solution
[0007] To address the aforementioned technical problems, this invention proposes a key video frame adversarial sample generation method based on knowledge distillation. This method employs a serialization recognition model and an adversarial video sample generation module based on temporal information destruction.
[0008] The serialization recognition model decouples video actions into two types: pose features and appearance features, and then aggregates them to obtain sequence features;
[0009] When generating adversarial perturbations of pose or appearance, the adversarial video sample generation module backpropagates the gradient to the pixel values of the target image to obtain adversarial pose or appearance features that are destroyed in each frame. Then, it locates key frames and replaces only the pose or appearance features corresponding to the key frames with adversarial pose or appearance features. Finally, it uses decoder D to generate adversarial video sequences.
[0010] Only one of the extracted pose features and appearance features is perturbed.
[0011] (III) Beneficial Effects
[0012] This invention proposes a key video frame adversarial example generation method based on knowledge distillation. Compared with existing technologies, this invention is geared towards general video recognition tasks, designs a more general adversarial attack example generation method, and improves the effectiveness of adversarial example generation. 1) The video adversarial example generation method is extended to the RGB domain, wherein the method has f a with f g Two features, where f a 1) It refers to appearance features, namely the RGB domain; 2) It can control the scope of the attack when generating adversarial examples for videos, and freely choose to change the appearance or the pose; 3) At the same time, it introduces a keyframe selective replacement algorithm during the iterative attack process to further improve the attack efficiency. Attached Figure Description
[0013] Figure 1 This is a schematic diagram of an adversarial gait contour generation technique based on adversarial generative networks in the prior art.
[0014] Figure 2 Here is a block diagram of a key video frame adversarial example generation method based on knowledge distillation;
[0015] Figure 3 A schematic diagram illustrating the selective replacement process used to counter video sample generation algorithms. Detailed Implementation
[0016] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.
[0017] This invention designs a method for generating adversarial examples from key video frames based on knowledge distillation. The adversarial video examples can be formalized as follows: Where N is the number of frames in the video sequence, and C, H, and W are the number of channels, height, and width of each frame, respectively. (The last part, "and X," appears to be a typo and can be left as is.) * correspond, This invention produces benign video sequences unaffected by adversarial attacks. The core innovations of this invention are: 1) Utilizing knowledge distillation, open-source frame feature extractors and temporal feature extractors are used as teachers, leveraging their prior knowledge to decouple video actions into pose and appearance features; 2) During sample generation, corresponding feature domains can be selected for perturbation as needed, achieving changes only to the sample's pose or appearance; 3) Furthermore, a keyframe replacement algorithm based on maximizing the effect of adversarial perturbation is designed to improve the quality and efficiency of adversarial attacks. The adversarial video sequence generation process of this invention is as follows:
[0018] like Figure 2 As shown, the present invention provides a method for generating adversarial examples of key video frames based on knowledge distillation. This method employs a serialization recognition model and an adversarial video example generation module based on temporal information destruction.
[0019] The serialization recognition model decouples video actions into two types: pose features and appearance features, and then aggregates them to obtain sequence features;
[0020] When generating adversarial perturbations of pose or appearance, the adversarial video sample generation module backpropagates the gradient to the pixel values of the target image to obtain adversarial pose or appearance features that are destroyed in each frame. Then, it locates key frames and replaces only the pose or appearance features corresponding to the key frames with adversarial pose or appearance features. Finally, it uses decoder D to generate adversarial video sequences.
[0021] Only one of the extracted pose features and appearance features is perturbed.
[0022] in,
[0023] The serialization recognition model includes: frame feature extractor and temporal feature extractor;
[0024] The frame feature extractor extracts features from each frame of the input sequence. The extracted features can be divided into pose features and appearance features. Then, the pose features are input into the temporal feature extractor, which aggregates the pose features and appearance features to obtain the sequence features.
[0025] The temporal feature extractor is implemented based on an LSTM structure.
[0026] To avoid unnecessary pixel-level perturbations in image frames, the adversarial video example generation module based on temporal information destruction backpropagates gradients to the pixel values of the target image when generating adversarial perturbations for pose or appearance. This operation can perturb only the extracted pose or appearance features, and compared to perturbing the entire image, this perturbation reduces some gradient backpropagation operations, resulting in lower computational overhead and more efficient attacks. The key video frame adversarial example generation method based on knowledge distillation of this invention maximizes the video sequence recognition loss and performs gradient backpropagation to attack pose or appearance features, obtaining adversarial pose or appearance features that are destroyed in each frame.
[0027] After generating a video sequence where the entire sequence is corrupted, we will selectively replace it. The specific replacement process is as follows: Figure 3 As shown, it selects key video frames for attack. The selection of key frames is based on the semantic distance between the pose or appearance features of each frame in the corrupted video sequence and the video sequence. Positions with larger distances are the key positions that the adversarial attack focuses on, thus allowing the location of key frames in the sequence. The semantic distance is calculated using the cosine distance of the semantic feature vectors.
[0028] Then, the adversarial video sample generation module based on temporal information destruction selectively replaces the original pose features or appearance features according to the selected keyframe positions, and replaces the adversarial pose features or appearance features into the corresponding positions, thereby generating the final selective adversarial features.
[0029] After the replacement is completed, the adversarial video sample generation module based on temporal information destruction uses decoder D to generate an adversarial video sequence X according to selective adversarial features. * .
[0030] The following section describes the pose characteristics:
[0031] When selecting keyframes, adversarial video example generation models that have disrupted temporal information will use the frame pose features f in the sequence. g ∈F g The frame is considered the target of the attack, and then the frame that maximizes the semantic distance of the pose features before and after the attack is selected as the key frame.
[0032] To obtain the desired adversarial perturbation δ, the adversarial video sample generation module based on temporal information destruction will maximize the loss function. The gradient direction is used to perform an iterative attack using the sign function in the PGD attack. For the i-th iteration in the iterative process, the update details of the perturbation δ are as follows:
[0033]
[0034] Where α is the iteration step size of the attack, ΩT (·)for Figure 2 Temporal feature extractor in the serialization recognition model Ω. This is a differentiation operation used to obtain the gradient. The function is a general loss function.
[0035] Let F g As pose features, after obtaining the final perturbation δ of each frame, the pose features f of each frame are... g By superimposing δ, we obtain the adversarial pose features f that are disrupted in every frame. ' g , forming F ' g Then based on F g With F ' g The semantic distance between each frame of the image is selected as top. k There are 1 keyframes, and the f corresponding to these keyframes is... g Finally replaced with f ' g Non-keyframes remain unchanged, resulting in the final adversarial pose features. Final adversarial posture characteristics The one with the largest semantic distance from the original pose features. Where f ' g =f g +δ.
[0036] Let Dis(·) be the distance measurement function. Then, the adversarial video sample generation module based on temporal information destruction constructs the final adversarial pose features. It can be formalized as:
[0037]
[0038] Let ⊕ be the concatenation operation of vectors, F a For appearance features. By splicing F a and Two features are used, and then decoder D is used to generate the adversarial video sequence for training:
[0039]
[0040] The decoder D is a CNN decoder.
[0041] Compared with existing technologies, this invention addresses general video recognition tasks, designs a more universal adversarial attack example generation method, and improves the effectiveness of adversarial example generation. 1) The video adversarial example generation method is extended to the RGB domain, where the method has two features, fa and fg, where fa is the appearance feature, i.e., the RGB domain; 2) The scope of the attack can be controlled when generating video adversarial examples, freely choosing to change the appearance or pose; 3) At the same time, a keyframe selective replacement algorithm is introduced during the iterative attack process to further improve attack efficiency.
[0042] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for generating adversarial examples from key video frames based on knowledge distillation, characterized in that, This method employs a serialization recognition model and an adversarial video sample generation module based on temporal information destruction; The serialization recognition model decouples video actions into two types: pose features and appearance features, and then aggregates them to obtain sequence features; The adversarial video sample generation module backpropagates gradients to the pixel values of the target image when generating adversarial perturbations for pose or appearance, resulting in adversarial pose or appearance features that are corrupted in each frame. This allows for the location of keyframes, and only the pose or appearance features corresponding to these keyframes are replaced with adversarial pose or appearance features. However, using the decoder... Generate adversarial video sequences; Only one of the extracted pose features and appearance features is perturbed; in, The serialization recognition model includes: frame feature extractor and temporal feature extractor; The frame feature extractor is used to extract features from each frame of the input sequence. The extracted features are divided into pose features and appearance features. The pose features are input into the temporal feature extractor, which aggregates the pose features and appearance features to obtain sequence features.
2. The method for generating adversarial examples of key video frames based on knowledge distillation as described in claim 1, characterized in that, The temporal feature extractor is implemented based on an LSTM structure.
3. The method for generating adversarial examples of key video frames based on knowledge distillation as described in claim 1, characterized in that, The selection of keyframes is based on the semantic distance between the pose or appearance features of each frame in the corrupted video sequence and the video sequence. Positions with larger distances are the key positions that adversarial attacks are more concerned with.
4. The method for generating adversarial examples of key video frames based on knowledge distillation as described in claim 3, characterized in that, Semantic distance is calculated using the cosine distance of semantic feature vectors.
5. The method for generating adversarial examples of key video frames based on knowledge distillation as described in claim 1, characterized in that, The adversarial video sample generation module based on temporal information destruction selectively replaces the original pose features or appearance features according to the selected keyframe positions, and replaces the adversarial pose features or appearance features into the corresponding positions, thereby generating the final selective adversarial features. After the replacement is completed, the adversarial video sample generation module based on temporal information destruction utilizes the decoder. Generate adversarial video sequences based on selective adversarial features.
6. The method for generating adversarial examples of key video frames based on knowledge distillation as described in any one of claims 1-5, characterized in that, Regarding pose features, when selecting keyframes, adversarial video example generation models, whose temporal information is disrupted, will use the pose features of frames in the sequence. The frame that maximizes the semantic distance of the pose features before and after the attack is considered the target of the attack and is then selected as the key frame. To obtain the desired counter-perturbation The adversarial video sample generation module based on temporal information destruction will follow the principle of maximizing the loss function. The gradient direction, utilizing the PGD attack Iterative attacks are carried out using the sign-deleting function; For the th iteration process Next iteration, perturbation The update details are as follows: in The iteration step size of the attack. For serialization recognition model Temporal feature extractor in This is a differentiation operation used to obtain the gradient.
7. The method for generating adversarial examples of key video frames based on knowledge distillation as described in claim 6, characterized in that, make As pose features, the final perturbation of each frame image is obtained. Then, the pose features of each frame are... Overlay This yields adversarial pose features that are destroyed in every frame. ,composition Then based on and semantic distance selection between each frame Each keyframe corresponds to a keyframe. Finally replaced with Non-keyframes remain unchanged, resulting in the final adversarial pose features. The final adversarial posture characteristics The semantic distance is largest with the original pose features, where .
8. The method for generating adversarial examples of key video frames based on knowledge distillation as described in claim 7, characterized in that, Let the distance measurement function be... Then, the adversarial pose feature is constructed based on the adversarial video sample generation module that destroys temporal information. It can be formalized as: 。 9. The method for generating adversarial examples of key video frames based on knowledge distillation as described in claim 8, characterized in that, make This is a concatenation operation for vectors. For appearance features, through splicing and Two features, then using the decoder Generate adversarial video sequences for training: 。