A black-box attack method for self-supervised video object segmentation

Abstract

The application discloses a kind of black box attack methods for self-supervised video target segmentation, and the self-supervised video target segmentation model based on affinity matrix learns the feature representation of video sequence to realize powerful pixel correspondence.First, based on the initialization randomly generated adversarial perturbation, construct the contrast loss for single frame, double frame and multi-frame, and carry out iterative optimization;Then design feature loss to enhance the transferability of the adversarial samples generated by black box attack;Adopt pixel-level loss to make the noise of generated adversarial sample imperceptible;Construct multi-path aggregation module to obtain iteratively optimized adversarial perturbation and add it to the original video frame to generate adversarial samples;Finally, input the adversarial video into the self-supervised video target segmentation network to obtain the final prediction mask.The research on the black box attack method of the self-supervised video target segmentation model identifies the vulnerability of the segmentation algorithm, which can further improve the security and robustness of the self-supervised video target segmentation task.

Description

TECHNICAL FIELD

[0001] The present application relates to a black box attack method for self-supervised video object segmentation, and belongs to the technical field of image processing. BACKGROUND

[0002] Self-supervised learning, which trains models in a supervised manner by utilizing self-generated labels from the data itself, has become the learning representation for deep neural networks. In recent years, self-supervised learning on videos has led to fruitful research. However, deep neural networks are very vulnerable to attacks and can easily fool the latest video object segmentation models by adding visual noise to the original images. The form of this attack is a small perturbation to the video frames that cannot be perceived by the human visual system. This attack causes the model to completely change its prediction of the video frames. Worse still, the attacked model reports a high degree of confidence in the wrong prediction. In addition, the same adversarial perturbation can deceive multiple neural network models.

[0003] Due to the large amount of online data, various ideas have been explored to learn to represent the correspondence by utilizing the spatio-temporal information in videos. The current self-supervised video object segmentation method is achieved by modeling the pairwise correspondence between the target frame and the reference frame, which maintains the spatio-temporal consistency. The self-supervised segmentation model based on affinity matrix learns the feature representation of the video sequence to achieve strong pixel correspondence. Therefore, the black box attack mainly targets the affinity matrix between the video frame sequence. Based on the initialization of the randomly generated adversarial perturbation, a contrastive loss for single-frame, double-frame and multi-frame is constructed, and iterative optimization is performed. Feature loss is used to enhance the transferability of the adversarial samples generated by the black box attack. In addition, in order to make the generated adversarial sample noise imperceptible, pixel-level loss is also used. Finally, a multi-path aggregation module is constructed to obtain the iterative optimized adversarial perturbation. SUMMARY

[0004] The present application relates to a black box attack method for self-supervised video object segmentation, and belongs to the technical field of image processing.

[0005] Technical scheme: In order to achieve the above purpose, the technical scheme adopted by the present application is:

[0006] A black-box attack method for self-supervised video target segmentation is presented. The self-supervised video target segmentation network employs an affinity matrix-based network. This method finds an adversarial sample with imperceptible differences from the self-supervised video target segmentation network by disrupting the affinity matrix between video frames, thus rendering the self-supervised video target segmentation network ineffective. Considering the two challenges of implementing adversarial attacks on video target segmentation: ① Unlike adversarial attacks on classification, which only require the classifier to misclassify, the conditions for successful segmentation attacks are more ambiguous; ② Considering the target segmentation within the video, adversarial perturbations generated based on single-frame features cannot be pasted into every video frame. Therefore, this invention considers generating adversarial perturbations frame-by-frame. The implementation of this invention includes the following steps:

[0007] (1) For the unannotated raw video sequence X={x 1 ,x 2 ,…,x n First, an initial adversarial perturbation ΔX = {Δx1, Δx2, ..., Δx} is randomly generated using a self-supervised noise generator. n};x i Let Δx represent the original image of the i-th frame. i Indicates the corresponding x i The adversarial perturbation, i = 1, 2, ..., n;

[0008] (2) Add the adversarial perturbation ΔX to the original video sequence X to obtain the adversarial video sequence X. adv ;

[0009] (3) Considering the consistency problem between video sequences, we construct contrast loss attacks for single frames, two frames and multiple frames respectively to obtain the total contrast loss;

[0010] (4) Design a feature loss function to make the original image semantically closer to the adversarial frame in the feature space, thereby further enhancing the transferability of the adversarial video sequence;

[0011] (5) Design a pixel-level loss function to make the noise in the adversarial video sequence imperceptible;

[0012] (6) Iteratively optimize the overall loss, construct a multi-path aggregation module to obtain the iteratively optimized adversarial perturbation ΔX′, and add the adversarial perturbation ΔX′ to the original video sequence X to obtain the final adversarial video sequence X′. adv ;

[0013] (7) Given an initial frame mask, the adversarial video sequence X′ adv The final predicted mask is obtained by inputting the self-supervised video object segmentation network.

[0014] Preferably, in the step (2), the adversarial perturbation ΔX is added to the original video sequence X to obtain the adversarial video sequence

[0015]

[0016] wherein: represents the adversarial sample corresponding to x i , and ε represents the maximum allowed adversarial perturbation threshold, ||·|| represents the infinity norm. ∞

[0017] Preferably, in the step (3), considering the consistency problem between video sequences, the contrast loss attacks for single frame, double frame and multi-frame are constructed respectively:

[0018]

[0019]

[0020]

[0021]

[0022] wherein: and respectively represent the contrast loss functions for single frame, double frame and multi-frame, L con represents the total contrast loss function, x i represents the original image of the i-th frame, x i+1 represents the original image of the i+1-th frame, sim(·,·) represents the cosine similarity function, and v represents the temperature parameter, {x neg} is the set of elements in the dynamic queue composed of adversarial samples, {x pos} is the set of elements in the dynamic queue composed of original images, and m represents the number of elements in each dynamic queue.

[0023] Because the black-box attack does not know the parameters and structure of the attack model, in order to further enhance the transferability of the generated adversarial samples, the feature loss function is designed to make the original image more close to the adversarial frame in the semantic feature space, that is, the predicted segmentation mask of the adversarial sample should be more close to the target mask between the feature spaces. Preferably, in the step (4), the feature loss function is designed based on the features extracted by the feature extractor F θ

[0024]

[0025] wherein: represents the feature map of x θ extracted by the feature extractor F ,​​ denotes the feature map extracted by the feature extractor F θ x i denotes the feature map extracted by the feature extractor F

[0026] Preferably, the feature extractor F θ adopts Resnet50 network, the feature map of the input image is output by the layer before the last fully connected layer of Resnet50 network, and θ denotes the to-be-learned parameter of Resnet50 network.

[0027] Preferably, in the step (5), a pixel-level loss function is designed to make the noise of the adversarial video sequence imperceptible.

[0028]

[0029] wherein x i denotes the original image of the i-th frame, denotes the adversarial sample corresponding to x i , and ||·||2 denotes the L2 norm.

[0030] Specifically, because the disturbance mode is similar to noise, smoothing the image helps to alleviate the adversarial effect. Therefore, in order to suppress the smoothness, a pixel-level loss is applied in the image pixel space. The pixel-level loss represents the L2 distance between the adversarial example and the original clean example. The purpose of minimizing the L2 distance is to constrain the difference between the adversarial video sequence and the clean video sequence at the pixel level to facilitate the visual perception of the adversarial sample.

[0031] Preferably, in the step (6), the overall loss function adopted by the iterative optimization of the overall loss is:

[0032]

[0033] wherein ΔX' denotes the adversarial disturbance after the iterative optimization, λ, μ and η are weight parameters of L con , and respectively; λ controls the relative importance of the noise and the contrast loss, μ controls the relative importance of each target feature in the video, and η controls the relative importance of the pixels of the video frame.

[0034] Preferably, in the step (6), the multi-path aggregation module is constructed to obtain the adversarial disturbance ΔX' after the iterative optimization, the multi-path aggregation module aims to integrate the common features from different path video frames to effectively generate the adversarial disturbance, and the input of the multi-path aggregation module is F t , F t-1 and F t+1, the output of the multi-path aggregation module is the iteratively optimized adversarial perturbation ΔX'; F t , F t-1 , and F t+1 represent the feature maps of the original images x t-1 , x t , and x t+1 , respectively, the sizes of F t , F t-1 , and F t+1 are all B×H×W×C=b×h×w×c, and B, H, W, and C represent the batch size, height, width, and channel number of the feature map, respectively, and b, h, w, and c represent the values of B, H, W, and C, respectively; the processing procedure of the multi-path aggregation module includes:

[0035] (61) projecting F t-1 into the feature space of F t , and the specific procedure is as follows: first, using 1×1 convolution to process F t and F t-1 , adjusting the size of F t to B×H×W×C=b×c×w×h, adjusting the size of F t-1 to B×H×W×C=b×w×c×h, performing matrix multiplication on the adjusted-size F t and F t-1 , and then performing normalization on the result, finally forming a projection P t-1 with a size of B×H×W×C=b×c×c×h:

[0036] P t-1 =Resize(BN(Resize(Conv(F t-1 )))×Resize(Conv(F t )))

[0037] wherein Resize(·) represents an image scaling function, BN(·) represents a normalization operation, and Conv(·) represents a convolution function;

[0038] (62) projecting F t+1 into the feature space of F t , and the specific procedure is as follows: first, using 1×1 convolution to process F t and F t+1 , adjusting the size of F t to B×H×W×C=b×c×w×h, adjusting the size of F t+1 to B×H×W×C=b×w×c×h, performing matrix multiplication on the adjusted-size F t and F t+1 , and then performing normalization on the result, finally forming a projection P t+1:

[0039] P t+1 = Resize(BN(Resize(Conv(F t+1 ))) * Resize(Conv(F t )))

[0040] (63) from F t , output the iteratively optimized adversarial perturbation ΔX':

[0041]

[0042] wherein: Concat(·) represents a merging function;

[0043] (64) add the adversarial perturbation ΔX' to the original video sequence X to obtain the final adversarial video sequence

[0044]

[0045] wherein: represents the overall loss function.

[0046] The multi-path aggregation module projects the feature maps of the previous frame and the next frame onto the current frame through transformation, and finally subtracts the aggregated features from the current frame. This image-level operation can effectively integrate noise.

[0047] Preferably, in the step (7), the affinity matrix based self-supervised video object segmentation is given a pair of video frames, based on the following assumptions: the content in two consecutive video frames is coherent; the frame reconstruction (pixel replication) operation can be represented by using a linear transformation of the affinity matrix, which describes the replication process from the reference frame to the target frame. The general choice of similarity measure in the affinity matrix is the dot product operation between feature maps. The goal of the black-box adversarial attack is to find an adversarial sample with imperceptible differences from the segmentation model by destroying the affinity matrix between video frames, so as to make the segmentation model fail.

[0048] Beneficial effects: the black-box attack method for self-supervised video object segmentation provided by the present application generates adversarial perturbations with the help of contrastive loss, which can perform adversarial attacks on the self-supervised video object segmentation task, and realizes the erroneous segmentation of all pixels of the target; the present application can be used for research in specific tasks or specific fields, and can help understand how black-box attacks affect model performance, which helps to reduce these influencing factors to enhance model security and robustness. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is the implementation flowchart of the method of the present application;

[0050] Figure 2 For the acquired adversarial samples and adversarial prediction masks, schematic diagrams are shown in the following figures:

[0051] Figure 3 For the multi-path aggregation module structure, a schematic diagram is shown in the following figure:

[0052] Figure 4 For a device for implementing the method of the present application, a schematic diagram of the structure is shown in the following figure. DETAILED DESCRIPTION

[0053] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0054] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first" and "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0055] As Figure 1 shown is an implementation flowchart of a black-box attack method for self-supervised video target segmentation, which will be described in detail below.

[0056] Step S01: For the original video sequence X = {x 1 ,x 2 ,…,x n}, first generate an initialized adversarial perturbation ΔX = {Δx1, Δx2, … Δx n} by a self-supervised trained noise generator; x i represents the original image of the i-th frame, Δx i represents the adversarial perturbation corresponding to x i , i = 1, 2, …, n.

[0057] Step S02: Add the adversarial perturbation ΔX to the original video sequence X to obtain the adversarial video sequence X adv .

[0058] The adversarial video sequence is represented as

[0059]

[0060] Where: represents the adversarial video sequence, and xi The adversarial examples, where ε represents the maximum permissible adversarial perturbation threshold, ||·|| ∞ It represents the infinite norm.

[0061] Step S03: Considering the consistency problem between video sequences, construct contrast loss attacks for single frames, two frames, and multiple frames respectively to obtain the total contrast loss.

[0062] Construct contrastive loss attacks targeting single-frame, two-frame, and multi-frame scenarios:

[0063]

[0064]

[0065]

[0066]

[0067] in: and Let L represent the contrast loss functions for single frame, two frames, and multiple frames, respectively. con Let x represent the overall contrastive loss function. i Let x represent the original image of the i-th frame. i+1 Let sim(·,·) represent the original image of frame i+1, ν represent the cosine similarity function, and {x} represent the temperature parameter. neg} is the set of elements in a dynamic queue composed of adversarial examples, {x pos} is a set of elements in a dynamic queue consisting of the original images, where m represents the number of elements in each dynamic queue.

[0068] Step S04: Design a feature loss function to make the original image semantically closer to the adversarial frame in the feature space, thereby further enhancing the transferability of the adversarial video sequence.

[0069] Based on feature extractor F θ Design the feature loss function for the extracted features; Feature extractor F θ The ResNet50 network is used, and the feature map of the input image is obtained from the output of the layer before the last fully connected layer of the ResNet50 network. θ represents the parameters to be learned in the ResNet50 network.

[0070]

[0071] in: Indicates the use of feature extractor F θ Extracted Feature map, Indicates the use of feature extractor F θ Extracted xi where H, W and C represent the height, width and channel number of the feature map, and ||·||2represents the L2 norm.

[0072] Step S05: design a pixel-level loss function to make the noise of the adversarial video sequence imperceptible.

[0073] The pixel-level loss function is represented as:

[0074]

[0075] where x i represents the original image of the i-th frame, represents the adversarial sample corresponding to x i , and ||·||2represents the L2 norm.

[0076] Because the disturbance pattern is similar to noise, smoothing the image helps to alleviate the adversarial effect. Therefore, in order to suppress the smoothness, a pixel-level loss is applied in the image pixel space. The pixel-level loss represents the L2 distance between the adversarial example and the original clean example. The purpose of minimizing the L2 distance is to constrain the difference between the adversarial video sequence and the clean video sequence at the pixel level to facilitate the visual perception of the adversarial sample.

[0077] Step S06: iteratively optimize the overall loss to construct a multi-path aggregation module to obtain the iteratively optimized adversarial disturbance ΔX' and add the adversarial disturbance ΔX' to the original video sequence X to obtain the final adversarial video sequence X' adv .

[0078] The multi-path aggregation module is constructed to obtain the iteratively optimized adversarial disturbance ΔX', as shown in Figure 3 , the input of the multi-path aggregation module is F t , F t-1 and F t+1 , and the output of the multi-path aggregation module is the iteratively optimized adversarial disturbance ΔX'; F t , F t-1 and F t+1 represent the feature maps of the original images x t-1 , x t and x t+1 , and the sizes of F t , F t-1 and F t+1 are all BxHxWxC=bxhxwxc, where B, H, W and C represent the batch size, height, width and channel number of the feature map, and b, h, w and c represent the values of B, H, W and C, respectively; the processing process of the multi-path aggregation module includes:

[0079] (61) projecting F t-1 to Ft The specific process is: first, using 1x1 convolution to process F t and F t-1 , adjusting the size of F t to BxHxWxC=bxcxwxh, adjusting the size of F t-1 to BxHxWxC=bwxcxh, performing matrix multiplication on the resized F t and F t-1 , and then normalizing the result, finally forming a projection P t-1 with a size of BxHxWxC=bxcxcxh.

[0080] P t-1 =Resize(BN(Resize(Conv(F t-1 )))xResize(Conv(F t )))

[0081] Wherein: Resize(·) represents an image scaling function, BN(·) represents a normalization operation, and Conv(·) represents a convolution function.

[0082] (62) Project F t+1 into the feature space of F t , the specific process is: first, using 1x1 convolution to process F t and F t+1 , adjusting the size of F t to BxHxWxC=bxcxwxh, adjusting the size of F t+1 to BxHxWxC=bwxcxh, performing matrix multiplication on the resized F t and F t+1 , and then normalizing the result, finally forming a projection P t+1 with a size of BxHxWxC=bxcxcxh.

[0083] P t+1 =Resize(BN(Resize(Conv(F t+1 )))xResize(Conv(F t )))

[0084] (63) Subtract the aggregated features from F t , output the iteratively optimized adversarial perturbation ΔX':

[0085]

[0086] Wherein: Concat(·) represents a merging function.

[0087] (64) adding the adversarial perturbation AX' to the original video sequence X to obtain a final adversarial video sequence

[0088]

[0089] wherein: represents the overall loss function.

[0090] (65) the overall loss function used for calculating the iterative optimization overall loss is:

[0091]

[0092] wherein: AX' represents the adversarial perturbation after iterative optimization, and λ, μ and η are weight parameters of L con , and .

[0093] (7) given the initial frame mask of the original video sequence X, the adversarial video sequence X adv is input into the self-supervised video object segmentation network to obtain the final predicted mask.

[0094] As shown in Figure 2 , it is a result visualization diagram of the black-box attack method for self-supervised video object segmentation provided by the present case, the first two rows are the original video sequence and its normal segmentation mask respectively, and the last two rows are the adversarial video sequence obtained by adding the adversarial perturbation to the original video sequence and its erroneous segmentation mask respectively; it can be seen from the drawing that after adding the adversarial perturbation which is difficult for human eyes to detect in the original video sequence, the self-supervised video object segmentation model estimates the target boundary position inaccurately, and the segmentation accuracy decreases significantly, so that the target object cannot be segmented correctly.

[0095] As shown in Figure 4 , it is a device for the black-box attack method for self-supervised video object segmentation provided by the present case, which comprises a generator, a Resnet network and a multi-path aggregation module, the generator is a noise generator trained by self-supervision, which is used for randomly generating an initialized adversarial perturbation AX, adding the adversarial perturbation AX to the original video sequence X to obtain an adversarial video sequence X adv , obtaining the pixel-level loss adv of the original video sequence X and the adversarial video sequence X The Resnet50 network is used to construct a feature extractor F θ , and the feature extractor F θ is used to extract the feature map of the original video sequence X and the adversarial video sequence X adv , so as to calculate the feature loss and the total contrastive loss L con; the multipath aggregation module is configured to obtain the overall loss the iteratively optimized adversarial perturbation ΔX′.

[0096] In the description of the present application, it is necessary to point out that, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "linking" should be understood in a broad sense, for example, can be fixed connection, can also be detachable connection, or integrally connected; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through intermediate medium, can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0097] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0098] The basic principles, main features and advantages of the present application are shown and described above. Those skilled in the art should understand that the above embodiments do not limit the present application in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present application.

Claims

1. A black-box attack method for self-supervised video target segmentation, characterized in that: The self-supervised video object segmentation network employs an affinity matrix-based self-supervised video object segmentation method, which includes the following steps: (1) For the unannotated raw video sequence X={x 1 ,x 2 ,…,x n First, an initial adversarial perturbation ΔX = {Δx1, Δx2, ..., Δx} is randomly generated using a self-supervised noise generator. n };x i Let Δx represent the original image of the i-th frame. i Indicates the corresponding x i The adversarial perturbation, i = 1, 2, ..., n; (2) Add the adversarial perturbation ΔX to the original video sequence X to obtain the adversarial video sequence X. adv ; (3) Considering the consistency problem between video sequences, we construct contrast loss attacks for single frames, two frames and multiple frames respectively to obtain the total contrast loss; (4) Design a feature loss function to make the original image semantically closer to the adversarial frame in the feature space, thereby further enhancing the transferability of the adversarial video sequence; (5) Design a pixel-level loss function to make the noise in the adversarial video sequence imperceptible; (6) Iteratively optimize the overall loss, construct a multi-path aggregation module to obtain the iteratively optimized adversarial perturbation ΔX′, and add the adversarial perturbation ΔX′ to the original video sequence X to obtain the final adversarial video sequence X′. adv ; (7) Given the initial frame mask of the original video sequence X, convert the adversarial video sequence X′ into a single frame mask. adv The final predicted mask is obtained by inputting the self-supervised video object segmentation network.

2. The black-box attack method for self-supervised video target segmentation according to claim 1, characterized in that: In step (2), an adversarial perturbation ΔX is added to the original video sequence X to obtain an adversarial video sequence. in: Indicates the corresponding x i The adversarial examples, where ε represents the maximum permissible adversarial perturbation threshold, ||·|| ∞ It represents the infinite norm.

3. The black-box attack method for self-supervised video target segmentation according to claim 2, characterized in that: In step (3), considering the consistency problem between video sequences, contrast loss attacks are constructed for single frames, two frames, and multiple frames respectively: in: and Let L represent the contrast loss functions for single frame, two frames, and multiple frames, respectively. con Let x represent the overall contrastive loss function. i Let x represent the original image of the i-th frame. i+1 Let sim(·,·) represent the original image of frame i+1, ν represent the cosine similarity function, and {x} represent the temperature parameter. neg } is the set of elements in a dynamic queue composed of adversarial examples, {x pos } is a set of elements in a dynamic queue consisting of the original images, where m represents the number of elements in each dynamic queue.

4. The black-box attack method for self-supervised video target segmentation according to claim 3, characterized in that: In step (4), based on feature extractor F θ Design feature loss function for extracted features ： in: Indicates the use of feature extractor F θ Extracted Feature map, Indicates the use of feature extractor F θ Extracted x i The feature map is defined by H, W, and C, which represent the height, width, and number of channels of the feature map, respectively, and ||·||2 represents the L2 norm.

5. The black-box attack method for self-supervised video target segmentation according to claim 4, characterized in that: The feature extractor F θ The ResNet50 network is used, and the feature map of the input image is obtained from the output of the layer before the last fully connected layer of the ResNet50 network. θ represents the parameters to be learned in the ResNet50 network.

6. The black-box attack method for self-supervised video target segmentation according to claim 5, characterized in that: In step (5), a pixel-level loss function is designed to make the noise in the adversarial video sequence imperceptible: Where: x i Represents the original image of the i-th frame. Indicates the corresponding x i The adversarial examples are denoted by ||·||2, which represents the L2 norm.

7. The black-box attack method for self-supervised video target segmentation according to claim 6, characterized in that: In step (6), the overall loss function used for iterative optimization of the overall loss is: Where: ΔX′ represents the adversarial perturbation after iterative optimization, and λ, μ, and η are respectively L con , and The weight parameters.

8. The black-box attack method for self-supervised video target segmentation according to claim 1, characterized in that: In step (6), a multi-path aggregation module is constructed to obtain the iteratively optimized adversarial perturbation ΔX′. The input of the multi-path aggregation module is F. t F t-1 and F t+1 The output of the multi-path aggregation module is the iteratively optimized adversarial perturbation ΔX′; F t F t-1 and F t+1 Representing the original image x t-1 x t and x t+1 Feature map, F t F t-1 and F t+1 The size of each feature map is B×H×W×C=b×h×w×c, where B, H, W, and C represent the batch size, height, width, and number of channels of the feature map, respectively, and b, h, w, and c represent the values ​​of B, H, W, and C, respectively. The processing steps of the multi-path aggregation module include: (61) F t-1 Projected onto F t The feature space is processed by first using a 1×1 convolution to process F. t and F t-1 , will F t The size is adjusted to B×H×W×C=b×c×w×h, and F is... t-1 The size is adjusted to B×H×W×C=b×w×c×h, and the size of F is adjusted accordingly. t and F t-1 Perform matrix multiplication, then normalize the results to form a projection P of size B×H×W×C=b×c×c×h. t-1 : P t-1 =Resize(BN(Resize(Conv(F t-1 )))×Resize(Conv(F t ))) Where: Resize(·) represents the image scaling function, BN(·) represents the normalization operation, and Conv(·) represents the convolution function; (62) F t+1 Projected onto F t The feature space is processed by first using a 1×1 convolution to process F. t and F t+1 , will F t The size is adjusted to B×H×W×C=b×c×w×h, and F is... t+1 The size is adjusted to B×H×W×C=b×w×c×h, and the size of F is adjusted accordingly. t and F t+1 Perform matrix multiplication, then normalize the results to form a projection P of size B×H×W×C=b×c×c×h. t+1 : P t+1 =Resize(BN(Resize(Conv(F t+1 )))×Resize(Conv(F t ))) (63) From F t Subtract the aggregated features from the output and output the iteratively optimized adversarial perturbation ΔX′: Where: Concat(·) represents the merge function; (64) Add the adversarial perturbation ΔX′ to the original video sequence X to obtain the final adversarial video sequence. in: This represents the overall loss function.

Images