A black-box video adversarial attack method based on style transfer

Abstract

The application discloses a kind of black box video confrontation attack methods based on style migration, including the following steps: design style data set, and select optimal style image for clean video;According to optimal style image, video style migration is carried out to clean video to obtain stylized video;Gradient estimation is carried out to stylized video, and iteration is updated, and output confrontation video.The application is obtained by video style migration to clean video according to optimal style image to stylized video, and the setting of gradient estimation and iteration update to stylized video is conducive to maintaining the sensory comfort of confrontation video and reducing the required query number;By considering the unrestricted confrontation disturbance of semantic invariance, the efficiency of black box attack is improved, and existing confrontation defense methods can be bypassed, which is a new type of video attack method with low cost, high efficiency and practicality.

Description

Technical Field

[0001] This invention relates to the field of adversarial security technology, and in particular to a black-box video adversarial attack method based on style transfer. Background Technology

[0002] Since AlexNet (a type of convolutional neural network) was proposed by Alex Krizhevsky et al. at the University of Toronto in 2012, deep neural networks (DNNs) have experienced rapid development due to their advantages over traditional methods. DNNs are now widely used in image recognition, video classification, sentiment analysis, and other fields, and are gradually permeating people's daily lives. However, research has shown that DNNs are highly vulnerable; attackers can design a subtle, imperceptible perturbation and superimpose it onto the input image, creating an adversarial example that can successfully mislead DNN-based classifiers. Because DNN classifiers are commonly deployed in automated equipment and unmanned monitoring scenarios, malicious attacks on these classifiers can cause incalculable losses.

[0003] In recent years, the demand for machine learning-assisted video classification systems has increased significantly. Damage to video moderation systems can have a series of serious consequences. Attackers can maliciously modify video classification systems to bypass the classifier, causing it to incorrectly classify videos as benign. Once these videos are released and spread online, they can cause public panic. On the other hand, the emergence of DeepFakes (a deepfake technology) in the field of artificial intelligence has greatly reduced the difficulty of generating fake videos, making video moderation extremely important. Therefore, the robustness of video classification systems urgently needs to be improved.

[0004] Early adversarial attack methods primarily focused on the image domain, but in recent years they have begun to extend to the video domain. Because attackers cannot access the structure and internal parameters of video classification models in real-world scenarios, recent research has shifted towards black-box attacks. Compared to images, videos contain temporal information, significantly increasing computational complexity and attack difficulty. However, to enhance the stealth of adversarial perturbations, current techniques only consider launching attacks within a limited perturbation range. Such attacks incur substantial query costs and are easily defended against by emerging adversarial defense methods.

[0005] In the field of adversarial attacks, existing video adversarial attack methods only consider perturbation optimization within perturbation constraints to improve the concealment of the perturbations. Although the generated adversarial examples are difficult for the human eye to detect, these methods lead to excessively high query counts, significantly increasing the cost of attacking video classification systems in real-world scenarios. Furthermore, most adversarial defense methods proposed in the image domain can be extended to the video domain and can defend against existing video adversarial examples, rendering the attack methods ineffective.

[0006] The scheme described in Chinese patent CN112819109B proposes a method to enhance the security of a video classification system against black-box adversarial example attacks. This method employs a region partitioning function to divide video pre-samples generated by three white-box video adversarial example generation systems into regions, and then uses a black-box gradient estimation algorithm to optimize the gradients of the pre-samples, thereby generating adversarial examples. Finally, the adversarial examples and clean samples are input into the video classification system and retrained. The trained model can effectively distinguish between adversarial examples and clean samples. This method adopts the idea of ​​adversarial training, adding different types of adversarial examples to the model training process to improve the model's robustness against adversarial examples. However, such a defense strategy can only defend against adversarial examples within the perturbation constraint range. For semantically invariant unconstrained adversarial perturbations, the model's search space becomes uncontrollable, and the adversarial training method therefore cannot defend against unconstrained perturbations.

[0007] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] The purpose of this invention is to solve the problems of high query count and weak resistance to video adversarial attacks, and to provide a black-box video adversarial attack method based on style transfer.

[0009] The technical problem of this invention is solved by the following technical solution:

[0010] A black-box video adversarial attack method based on style transfer includes the following steps:

[0011] S1. Design a style dataset and select the optimal style image for a clean video;

[0012] S2. Perform video style transfer on the clean video based on the optimal style image to obtain a stylized video;

[0013] S3. Perform gradient estimation on the stylized video and iteratively update it to output the adversarial video.

[0014] In some embodiments, in step S1, the style dataset includes, but is not limited to, images in the dataset. Any image downloaded from the network can be added to the style dataset as long as the category to which the image belongs is within the category range of the dataset.

[0015] In some embodiments, step S1 specifically involves: randomly selecting a certain proportion of videos in the video dataset and extracting video frames to form a style dataset, with the remaining videos forming a dataset to be attacked, and then selecting the optimal style image based on color theme similarity and / or target category confidence.

[0016] In some embodiments, during a targeted attack, the color theme similarity and target category confidence of all images in the clean video and style dataset are calculated, and the optimal style image is selected, specifically:

[0017] Given a clean video No. Style images randomly selected from a color theme and style dataset s The Each color theme is located in the HSV three-dimensional color coordinate system. , The color theme similarity is determined by Calculated;

[0018] Based on the input frame count requirements of the video classifier, the style images are... s By repeating this process to reach the required number of frames, a still video is obtained. Next, Input to classifier f And obtain the label with the highest score and its corresponding confidence score; given the target category y t The target category confidence score is calculated using the following formula:

[0019] ;

[0020] Compare the weighted sum of color theme similarity and target category confidence of images with different styles, and select the optimal style image. The optimal style image We obtain it from the following formula:

[0021] ;

[0022] in For style datasets, μ These are the weighting coefficients.

[0023] In some embodiments, during non-targeted attacks, the color theme similarity between the clean video and all images in the style dataset is calculated, and the optimal style image is selected. The optimal style image We obtain it from the following formula:

[0024] ;

[0025] in This is a style dataset.

[0026] In some embodiments, in step S2, the loss function for video style transfer is as follows:

[0027] ;

[0028] in For video frame rate, , , and These represent content loss, style loss, total variance loss, and time loss, respectively. , , and These are the weighting coefficients.

[0029] In some embodiments, in step S2, the stylized video's first... frame With the frame Time loss function between As shown in the following formula:

[0030]

[0031] in , , These are the height, width, and number of channels of a single frame of video. This represents a warping function that outputs a warped frame from the input frame using pre-computed optical flow. Represents the optical flow middle The occlusion mask matrix of the frame.

[0032] In some embodiments, in step S3, the gradient estimation of the stylized video specifically involves using dual sampling on the stylized video. Surrounding by standard deviation Gaussian sampling One point, estimated The surrounding adversarial gradient; the gradient estimate can be calculated by the following formula:

[0033] ;

[0034] in Standard Gaussian noise, As a unit array, The loss function is for misclassification.

[0035] Wherein, the misclassification loss function In targeted attacks, the model output is correlated with the target category. The cross-entropy is the negative of the cross-entropy of the true class in non-targeted attacks.

[0036] In some embodiments, in step S3, the stylized video is iteratively updated. The iterative update process can be represented by the following formula:

[0037] ;

[0038] in Step size, The perturbation threshold, Represents a symbolic function. This means that input will be restricted to a set range.

[0039] In some embodiments, in step S3, when the iteration stops, the adversarial video is output. The iteration stopping condition is that the updated stylized video meets the requirement of misclassification as either a targeted attack or a non-targeted attack. That is, confrontational video .

[0040] The beneficial effects of this invention compared to the prior art include:

[0041] This invention obtains stylized videos by performing video style transfer on clean videos based on the optimal style image. The stylized videos are then subjected to gradient estimation and iterative updates, which helps maintain the sensory comfort of the adversarial video and reduce the number of queries required. By considering color theme similarity and target category confidence to select the optimal style image, the stylized video retains its semantic information. At the same time, it adds semantically invariant unrestricted adversarial perturbations to the clean video, improving the efficiency of black-box attacks and bypassing existing adversarial defense methods. This is a low-cost, efficient, and powerful new video attack method. Attached Figure Description

[0042] Figure 1 This is a flowchart of a black-box video adversarial attack method based on style transfer in an embodiment of the present invention;

[0043] Figure 2 This is a flowchart of a black-box video adversarial attack method based on style transfer in another embodiment of the present invention;

[0044] Figure 3aThis is a schematic diagram of the initial clean video of the target attack in the experimental example of this invention (correctly classified as "sit-ups");

[0045] Figure 3b This is a schematic diagram of the optimal style image selected from the initial clean video as the target attack in the experimental example of this invention;

[0046] Figure 3c This is a schematic diagram of the adversarial video of the target attack in the experimental example of this invention (which was incorrectly classified as "upside down");

[0047] Figure 4a This is a schematic diagram of the initial clean video (correctly classified as "vault horse") of a non-targeted attack in the experimental example of this invention;

[0048] Figure 4b This is a schematic diagram of the optimal style image selected for the initial clean video by a non-targeted attack in the experimental example of this invention;

[0049] Figure 4c This is a schematic diagram of a non-targeted attack adversarial video (misclassified as "ping-pong ball hitting") in an experimental example of the present invention. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0051] It should be noted that the directional terms such as left, right, up, down, top, and bottom used in this embodiment are only relative concepts or are based on the normal use of the product, and should not be considered as restrictive.

[0052] This invention fully utilizes video style transfer technology, aiming to develop a novel black-box video adversarial attack method based on style transfer. This embodiment does not restrict the perturbation without altering the video's semantic information, and enables the generated adversarial examples to successfully deceive video classification models.

[0053] This invention discloses a style transfer-based adversarial attack method for black-box videos, belonging to the field of adversarial attacks. It addresses the problem that existing technologies in black-box video attacks only consider limited perturbations, resulting in a large number of queries required for the attack and an inability to bypass existing adversarial defense strategies. This invention first constructs a style dataset and selects the optimal style image from the input clean video based on color theme similarity and target category confidence. Next, image style transfer is extended to video, and a temporal loss function is introduced to perform video style transfer on the clean video, obtaining a stylized video. Finally, based on a black-box gradient optimization method, the stylized video is iteratively updated, ultimately generating video adversarial examples. This invention represents a semantically invariant, unrestricted perturbation attack, improving the efficiency of black-box video attacks, reducing attack costs, and enhancing the resilience of adversarial examples against adversarial defense methods. It is a preliminary exploration towards unrestricted and non-semantic adversarial perturbations. This embodiment differs significantly from existing technologies. Existing technologies only consider limited perturbations, resulting in inefficient attacks and weak defense capabilities. This embodiment considers semantically invariant, unrestricted perturbation attacks, consuming fewer queries and completing the attack in a shorter time. On the other hand, using style transfer for attacks expands the range of non-semantic pixels in the perturbation, which can resist existing defense methods. It can also bypass existing adversarial defenses, making it a low-cost, efficient, and practical new video attack method.

[0054] An overview of the embodiments of the present invention is as follows:

[0055] This invention provides a method for adversarial attacks on black-box videos based on style transfer. Figure 1 As shown, it includes the following steps:

[0056] S1. Design a style dataset and select the optimal style image for a clean video;

[0057] Specifically, the style dataset includes, but is not limited to, the images in the dataset. Any image downloaded from the network can be added to the style dataset, as long as the category to which the image belongs is within the category range of the dataset.

[0058] Furthermore, a certain proportion of videos are randomly selected from the video dataset and video frames are extracted to form a style dataset. The remaining videos form the attack dataset. Then, the optimal style image is selected based on color theme similarity and / or target category confidence.

[0059] Furthermore, in targeted attacks, the color theme similarity and target category confidence of all images in the clean video and style dataset are calculated, and the optimal style image is selected, specifically as follows:

[0060] Given a clean video No. Style images randomly selected from a color theme and style dataset s The Each color theme is located in the HSV three-dimensional color coordinate system. , Color theme similarity Calculated;

[0061] Based on the input frame count requirements of the video classifier, the style images are... s By repeating this process to reach the required number of frames, a still video is obtained. Next, Input to classifier f And obtain the label with the highest score and its corresponding confidence score; given the target category y t The target category confidence score is calculated using the following formula:

[0062] ;

[0063] Compare the weighted sum of color theme similarity and target category confidence of images with different styles, and select the optimal style image. Optimal style image We obtain it from the following formula:

[0064] ;

[0065] in For style datasets, μ These are the weighting coefficients.

[0066] Furthermore, in non-targeted attacks, the color theme similarity between the clean video and all images in the style dataset is calculated, and the optimal style image is selected. Optimal style image We obtain it from the following formula:

[0067] ;

[0068] in This is a style dataset.

[0069] S2. Perform video style transfer on the clean video based on the optimal style image to obtain a stylized video;

[0070] Specifically, the loss function for video style transfer is shown in the following equation:

[0071] ;

[0072] in For video frame rate, , , and These represent content loss, style loss, total variance loss, and time loss, respectively. , , and These are the weighting coefficients.

[0073] Specifically, the first stylized video frame With the frame Time loss function between As shown in the following formula:

[0074]

[0075] in , , These are the height, width, and number of channels of a single frame of video. This represents a warping function that outputs a warped frame from the input frame using pre-computed optical flow. Represents the optical flow middle The occlusion mask matrix of the frame.

[0076] S3. Perform gradient estimation on the stylized video, specifically by using dual sampling on the stylized video. Surrounding by standard deviation Gaussian sampling One point, estimated The surrounding adversarial gradient; the gradient estimate can be calculated by the following formula:

[0077] ;

[0078] in Standard Gaussian noise, As a unit array, The loss function is for misclassification.

[0079] Among them, the misclassification loss function In targeted attacks, the model output is correlated with the target category. The cross-entropy is the negative of the cross-entropy of the true class in non-targeted attacks.

[0080] Specifically, iteratively update stylized videos The iterative update process can be represented by the following formula:

[0081] ;

[0082] in Step size, The perturbation threshold, Represents a symbolic function. This means that input will be restricted to a set range.

[0083] Furthermore, in step S3, when the iteration stops, the adversarial video is output. The iteration stops when the updated stylized video meets the requirement of misclassifying a targeted attack or a non-targeted attack. That is, confrontational video .

[0084] Example:

[0085] The embodiments of this invention are mainly divided into three parts. The first is that this embodiment proposes a video style selection method that is beneficial to adversarial attacks, so that the generated stylized video can maintain naturalness and move towards the classification decision boundary, or even cross the classification decision boundary.

[0086] The second aspect is that this embodiment extends image style transfer to video and introduces a time loss function to improve the consistency between frames before and after stylized video, thereby avoiding video flickering.

[0087] The third point is that this embodiment uses gradient estimation to iteratively optimize the stylized video, completing the video attack after a small number of iterations.

[0088] Therefore, this embodiment can achieve more efficient video attacks without changing the semantic information of the video, without limiting the scope of the perturbation.

[0089] The overall flowchart of this embodiment is as follows: Figure 2 As shown, firstly, the optimal style image is selected for the clean video from the style dataset. Then, video style transfer is performed, gradient estimation is performed on the obtained stylized video, and the video is iteratively optimized until the video is misclassified, and the adversarial video is output.

[0090] Style selection is the first part of this embodiment. Style selection serves two purposes. First, a suitable style image ensures that the stylized video is not easily distinguishable from the original video. Second, a correctly selected style image helps move the stylized video closer to the decision boundary, reducing the number of queries required during subsequent attacks. Specifically, this embodiment first randomly selects 70% of the videos in the video dataset and extracts video frames from each, forming a style dataset. The remaining videos form the attack dataset. Then, style images are selected for the clean videos in the attack dataset based on color theme similarity and target category confidence.

[0091] Since similar colors maintain better visual comfort after style transfer, this embodiment first quantizes the video pixel values ​​and uses median segmentation (MC) to obtain several color themes of the image. This embodiment calculates the color theme similarity between the clean video and the styled image in the HSV (Hue, Saturation, Value, hexagonal pyramid model) three-dimensional color space. Given a clean video... No. Style images randomly selected from a color theme and style dataset The Each color theme is located in the HSV three-dimensional color coordinate system. , The optimal style image is selected based on the color theme similarity between the clean video and the style image in the following formula. .

[0092]

[0093] in This is a style dataset.

[0094] In this embodiment, the average distance of three color themes is selected to measure the color similarity between the clean video and the styled image. Too many color themes will cause the generated video to carry too much information from the styled image, which is not conducive to maintaining the naturalness and visual comfort of the stylized video. For non-targeted attacks, this scheme uses color theme similarity to select the styled image.

[0095] To ensure that the stylized video is as close as possible to the decision boundary, this embodiment also proposes a target class confidence score to assist in selecting the style image during targeted attacks. Since style transfer can minimize the style loss between the initial video and the style image, the stylized video is similar to the style image in feature representation. When there is transferability between the feature extraction network and the target model, the video stylized from the style image in the target class is closer to the decision boundary of the target class. Therefore, adversarial videos can be created with fewer optimization steps after stylization. Specifically, firstly, the style image is processed according to the input frame count requirements of the video classifier. s By repeating this process to reach the required number of frames, a still video is obtained. Next, Input to classifier And obtain the label with the highest score and its corresponding confidence score. If The highest-scoring label is the target category. Then s The target category confidence score is denoted as Otherwise, s The target class confidence is set to 0. Therefore, the target class confidence can be expressed as:

[0096]

[0097] For targeted attacks, this embodiment considers both color theme similarity and target category confidence when selecting style images. In this case, the criteria for selecting style images are updated as follows:

[0098]

[0099] in It is a weighting coefficient used to balance the naturalness of stylized videos and the number of attack queries, and can be determined by the grid search method.

[0100] The color theme of all images and the confidence of all target categories can be calculated and stored before style transfer, which can save query volume and time costs when performing batch video attacks.

[0101] In some other embodiments, the following steps may be omitted:

[0102] When choosing a style, for targeted attacks, select based on two indicators: color theme similarity and target category confidence; for non-targeted attacks, select based only on color theme similarity.

[0103] Style transfer is the second part of this embodiment. Style transfer is a relatively mature technique in image processing, mainly composed of content loss, style loss, and total variance loss. Content loss minimizes the difference in high-order representation between the original image and the stylized image. Style loss aims to reduce the difference in style features between the stylized image and the stylized image. Total variance loss is used to improve the smoothness of the stylized image. This embodiment extends the above three losses to video and additionally designs a temporal loss to improve the consistency between consecutive frames in the video. The stylized video... frame With the frame The time loss between them is defined as:

[0104]

[0105] in , , These are the height, width, and number of channels of a single frame of video. This represents a warping function that outputs a warped frame from the input frame using pre-computed optical flow. Represents the optical flow middle The occlusion mask matrix of the frame.

[0106] Combining the four loss functions mentioned above, the total loss of video style transfer can be expressed as:

[0107]

[0108] in For video frame rate, , and These represent content loss, style loss, and total variance loss, respectively. , , and These are the weighting coefficients.

[0109] By minimizing the total loss, stylized videos can be made smooth and exhibit high temporal consistency. This temporal consistency in stylized videos can be preserved in generated adversarial examples. Therefore, the introduction of temporal loss provides a guarantee that adversarial examples can bypass temporal consistency-based adversarial defense mechanisms.

[0110] Adversarial example generation is the third part of this embodiment. The stylized video obtained after style transfer is close to the decision boundary, therefore only a small number of queries are needed for a successful attack. This embodiment uses a black-box gradient estimation method to further optimize the stylized video in the adversarial example generation part. Given a stylized video... Using dual sampling Surrounding by standard deviation Gaussian sampling There are several points. Using the logarithmic trick, the expected gradient can be transformed into the expectation of the gradient, yielding an approximate gradient estimate as:

[0111]

[0112] in Standard Gaussian noise, As a unit array, This is the misclassification loss function.

[0113] This embodiment uses the gradient estimation method described above to iteratively update the stylized video until it successfully crosses the decision boundary, at which point the attack stops and an adversarial example is obtained. Specifically, for non-targeted attacks, the stylized video has a certain probability of no longer being classified into its original category by the classifier; in this case, no additional query is needed for the attack to succeed.

[0114] A flowchart of a style transfer-based black-box video adversarial attack method in this embodiment is shown below. Figure 2 As shown, the specific steps include:

[0115] (1) Input clean videos and style datasets;

[0116] (2) Select an appropriate style image based on the style selection scheme;

[0117] (3) Perform style transfer on the clean video to obtain a stylized video;

[0118] (4) Perform gradient estimation on the stylized video and iteratively update it until the video is misclassified as the target category;

[0119] (5) Output the generated adversarial examples.

[0120] The beneficial effects of the embodiments of the present invention are as follows:

[0121] This invention addresses the problems of low query efficiency and weak resistance in video adversarial attacks by proposing a novel style transfer-based black-box video adversarial attack method. By considering semantically invariant, unrestricted adversarial perturbations, it improves the efficiency of black-box attacks and can bypass existing adversarial defense methods. Specifically, this embodiment proposes two style image selection metrics—color theme proximity and target category confidence—to address both video naturalness and query efficiency. These metrics help maintain the sensory comfort of the adversarial video and reduce the number of queries required. The time loss considered in this embodiment takes into account the consistency between consecutive frames of the video, bypassing defense detectors based on inter-frame pixel anomalies. Furthermore, existing adversarial defenses are designed for limited perturbations; since the unrestricted perturbations considered in this embodiment can extend the sample space to unknown regions beyond the defined range, it can easily bypass existing adversarial defense methods.

[0122] The embodiments of the present invention help to achieve low-cost black-box attacks on video classification systems in real-world scenarios. The generated adversarial samples are not easily identified by adversarial defense methods and cannot be detected by the human eye. It is an efficient and practical attack method.

[0123] The differences between the embodiments of the present invention and the prior art are as follows:

[0124] 1. Unrestricted perturbations with semantic invariance;

[0125] To ensure the stealth of adversarial perturbations, adversarial attacks typically limit the scope of the perturbation. However, this black-box approach results in two major drawbacks: a large number of queries and weak defense capabilities. The style transfer-based adversarial perturbation proposed in this embodiment does not limit the range of variation of the adversarial perturbation without altering the semantic information of the video. While such adversarial perturbations are observable to the human eye, the anomalies are imperceptible. Current attack research targeting video classification systems is relatively inefficient and does not consider unrestricted perturbations that maintain semantic integrity.

[0126] 2. Style selection strategy based on color theme proximity and target score confidence;

[0127] Current style transfer techniques in the image domain primarily target artistic styles, resulting in overly abstract stylized images that differ significantly from real-life images. Furthermore, abstract stylized images are easily detected as abnormal by the human eye, failing to meet the requirement of adversarial attacks not affecting visual comfort. Therefore, this solution uses real-life images as style images to avoid overly abstract stylized videos. When selecting style images, firstly, color theme similarity is used to obtain style images with colors close to the original clean video for style transfer, avoiding situations where excessive color deviation leads to unrealistic video tones. Secondly, to enable stylized videos to approach or even bypass decision boundaries, this solution proposes using target class confidence to select style images. Therefore, stylized videos require only a small number of queries to cross the decision boundary, significantly reducing the number of queries and improving attack efficiency.

[0128] 3. Introduce a temporal loss function for video style transfer;

[0129] Simply applying style transfer frame-by-frame to video can lead to inconsistencies in texture details between consecutive frames, and even cause flickering and unnatural changes during continuous playback. To address this issue, this solution leverages video optical flow information and introduces a temporal loss function to minimize the differences in pixel variations between adjacent frames. The content loss, style loss, and total variance loss from image style transfer are extended to video, and the temporal loss is then added to obtain the total loss for video style transfer. The resulting stylized video not only visually avoids the anomalies of continuous frame changes but also helps improve the resistance of adversarial examples to temporal consistency-based defense detectors.

[0130] Experimental Example

[0131] In the experimental example of this invention, the dataset is divided into a style dataset and an attack dataset in a ratio of 7 to 3. Images in the style dataset can be selected as style images for video style transfer, and images in the attack dataset are used for attack.

[0132] Randomly select a clean video from the dataset to be attacked. It also randomly selects a target category for the attack (different from the actual category of the clean video). ;

[0133] In targeted attacks, calculate the color theme proximity and target category confidence of all images in the clean video and style datasets;

[0134] Given a clean video No. Style images randomly selected from a color theme and style dataset The Each color theme is located in the HSV three-dimensional color coordinate system. , Color theme similarity Calculated;

[0135] Based on the input frame count requirements of the video classifier, the style images are... s By repeating this process to reach the required number of frames, a still video is obtained. Next, Input to classifier f And obtain the label with the highest score and its corresponding confidence score. Given a target category. y t The target category confidence score is calculated using the following formula:

[0136]

[0137] Compare the weighted sum of color theme similarity and target category confidence of images with different styles, and select the optimal style image. The optimal style image is obtained by the following formula:

[0138]

[0139] in For style datasets, μ The weighting coefficient is set to... ;

[0140] In non-targeted attacks, the color theme similarity between the clean video and all images in the style dataset is calculated, and the optimal style image is selected. The optimal style image is obtained by the following formula:

[0141]

[0142] Perform video style transfer on the clean video based on the optimal style image to obtain a stylized video. The loss function for video style transfer is shown in the following equation:

[0143]

[0144] in For video frame rate, , , and These represent content loss, style loss, total variance loss, and time loss, respectively. , , and As a weighting coefficient, in specific implementation, Set to 10, target under attack. The value is 75, in non-targeted attacks. It is 50. for , for ;

[0145] Using dual sampling in stylized videos Surrounding by standard deviation (Set as in target attack) In non-targeted attacks, it is set as Gaussian sampling (Set to 64) points, estimate The surrounding adversarial gradient. The gradient estimate can be calculated by the following formula:

[0146]

[0147] in Standard Gaussian noise, As a unit array, This is the misclassification loss function. In targeted attacks, the misclassification loss function is the difference between the model output and the target class. The cross-entropy, in non-targeted attacks, outputs the inverse of the cross-entropy of the model with respect to the true class;

[0148] Iterative updates to stylized videos The iterative update process can be represented by the following formula:

[0149]

[0150] in Step size, The disturbance threshold is set to 0.05. Represents a symbolic function. This means that input will be restricted to a set range.

[0151] When the iteration stops, output the adversarial video. The iteration stops when the updated stylized video meets the requirement of misclassifying a targeted attack or a non-targeted attack. That is, confrontational video .

[0152] Figures 3a-3c and Figures 4a-4c Examples of input clean video and adversarial video visualization results are provided. Figures 3a-3c The initial category for the attack is "sit-ups," and the target category is "handstand." Figures 4a-4c This is a non-targeted attack, and the initial category is "jump horse". Figure 3a and Figure 4a For an initial clean video, Figure 3a The actual category is "sit-ups". Figure 4a The true category is "upside down". Figure 3b and Figure 4b To select the optimal style image, Figure 3b It is an image selected from the "upside-down" category. Figure 4b It is the optimal style image selected for non-targeted attacks. Figure 3c and Figure 4c Adversarial videos that underwent style transfer and gradient iterative updates were all misclassified into other categories. Specifically... Figure 3c Misclassified as "upside down" Figure 4c It was misclassified as a "ping-pong ball hit". Although the post-attack confrontation video broke the perturbation limitations of traditional methods, it still retained the semantic information of the clean video.

[0153] The style dataset in this experiment includes, but is not limited to, the images in the dataset. Any image downloaded from the internet can be added to the style dataset, as long as the category of the image is within the category range of the dataset.

[0154] In this experimental example, the weight parameters can be changed according to the user's desired degree of style transfer. Reference values ​​are provided, but they are not fixed. The reference values ​​provided in this experimental example are as follows: Weight coefficients The value is 10, and the target is under attack. The value is 75, in non-targeted attacks. It is 50. for , for .

[0155] In this experimental example, a fixed style transfer model can be pre-trained offline for a fixed style image, and then an online attack can be implemented. Offline training of the fixed style transfer model specifically refers to: using a fixed style image and different input videos, optimizing the style transfer model parameters using the total loss mentioned in this scheme to obtain a transfer model with a fixed style. The online attack refers to: for a new input video, directly calling the pre-trained fixed style transfer model to directly obtain a stylized video, and then determining whether the stylized video is misclassified. If the stylized video can be misclassified, the attack is considered successful. Since the modifications to the input video do not require model training, a fast, online attack with zero latency can be achieved.

[0156] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, several equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or purpose, should be considered within the scope of protection of the present invention.

Claims

1. A black-box video adversarial attack method based on style transfer, characterized in that, Includes the following steps: S1. Design a style dataset and select the optimal style image for a clean video; S2. Perform video style transfer on the clean video based on the optimal style image to obtain a stylized video; S3. Estimate the gradient of the stylized video and update iteratively to output the adversarial video; Specifically, step S1 involves: randomly selecting a certain proportion of videos from the video dataset and extracting video frames to form a style dataset; the remaining videos form a dataset to be attacked; and then selecting the optimal style image based on color theme similarity and / or target category confidence. In targeted attacks, the color theme similarity and target category confidence of all images in the clean video and style dataset are calculated, and the optimal style image is selected, specifically as follows: Given a clean video No. Style images randomly selected from a color theme and style dataset The Each color theme is located in the HSV three-dimensional color coordinate system. , The color theme similarity is determined by Calculated; Based on the input frame count requirements of the video classifier, the style images are... s By repeating this process to reach the required number of frames, a still video is obtained. Next, Input to classifier f And obtain the label with the highest score and its corresponding confidence score; given the target category y t The target category confidence score is calculated using the following formula: ； Compare the weighted sum of color theme similarity and target category confidence of images with different styles, and select the optimal style image. The optimal style image mentioned above We obtain it from the following formula: ； in For style datasets, μ These are the weighting coefficients.

2. The style transfer-based adversarial attack method for black-box videos as described in claim 1, characterized in that, In step S1, the style dataset includes images in the dataset. Any image downloaded from the network can be added to the style dataset as long as the category of the image is within the category range of the dataset.

3. The style transfer-based adversarial attack method for black-box videos as described in claim 1, characterized in that, In non-targeted attacks, the color theme similarity between the clean video and all images in the style dataset is calculated, and the optimal style image is selected. The optimal style image mentioned above We obtain it from the following formula: ； in This is a style dataset.

4. The style transfer-based adversarial attack method for black-box videos as described in claim 1, characterized in that, In step S2, the loss function for video style transfer is shown in the following equation: ； in For video frame rate, , , and These represent content loss, style loss, total variance loss, and time loss, respectively. , , and These are the weighting coefficients.

5. The style transfer-based adversarial attack method for black-box videos as described in claim 4, characterized in that, In step S2, the stylized video's first... frame With the frame Time loss function between As shown in the following formula: ； in , , These are the height, width, and number of channels of a single frame of video. This represents a warping function that outputs a warped frame from the input frame using pre-computed optical flow. Represents the optical flow middle The occlusion mask matrix of the frame.

6. The style transfer-based black-box video adversarial attack method as described in claim 1, characterized in that, In step S3, the gradient estimation of the stylized video specifically involves using dual sampling on the stylized video. Surrounding by standard deviation Gaussian sampling One point, estimated The surrounding adversarial gradient; the gradient estimate is calculated by the following formula: ； in Standard Gaussian noise, As a unit array, The loss function is for misclassification. Wherein, the misclassification loss function In targeted attacks, the model output is correlated with the target category. The cross-entropy is the negative of the cross-entropy of the true class in non-targeted attacks.

7. The style transfer-based adversarial attack method for black-box videos as described in claim 1, characterized in that, In step S3, the stylized video is iteratively updated. The iterative update process can be represented by the following formula: ； in Step size, The perturbation threshold, Represents a symbolic function. This means that input will be restricted to a set range.

8. The style transfer-based black-box video adversarial attack method as described in claim 1 or 7, characterized in that, In step S3, when the iteration stops, the adversarial video is output. The iteration stopping condition is that the updated stylized video meets the requirement of misclassification as either a targeted attack or a non-targeted attack. That is, confrontational video .

Images