A method and device for generating black box adversarial samples

Abstract

The present invention relates to the field of intelligent algorithm adversarial technology, and provides a method and device for generating black-box adversarial samples. The present invention determines the shallow feature map, mid-level feature map, and deep feature map of each frame in the original sample as the original spatial features; divides the original sample according to feature similarity to obtain the original temporal features within each segment; determines the shallow feature map, mid-level feature map, and deep feature map of each frame in the initialized adversarial sample as the adversarial sample spatial features, and performs a pairwise fusion operation on multiple feature units in each segment to obtain the adversarial sample temporal features; measures the cosine similarity between the original spatial features and the adversarial sample spatial features, and measures the cosine similarity between the original temporal features and the adversarial sample temporal features to obtain an intermediate adversarial sample and iteratively optimizes the intermediate adversarial sample to ultimately obtain a target adversarial sample, thereby solving the problem of poor practicality of attack methods that directly use image models as video proxy models.

Description

Technical Field

[0001] The present invention relates to the field of intelligent algorithm adversarial technology, and in particular to a method and device for generating black-box adversarial samples. Background Art

[0002] Currently, video classification models based on deep neural networks have been widely used. These video classification models can automatically identify and analyze human movements in videos and are widely used in fields such as security monitoring and human-computer interaction. However, simply adding slight perturbations to the samples input to these video classification models can cause them to output incorrect judgment results. This poses a serious security threat risk in practical applications. For example, by attacking a video classification model, it can be made unable to identify adversarial video samples containing malicious information, which allows malicious information to spread through video channels, causing great harm. Therefore, in order to evaluate the security of video classification models and promote their optimization and upgrade, it is necessary to generate adversarial samples for testing video classification models.

[0003] Attacks using adversarial examples can be categorized as white-box or black-box attacks, depending on the attacker's understanding of the video classification model. Most existing black-box attack methods require training a proxy video model with the same architecture as the desired video classification model to generate adversarial examples. Due to the large volume of video data and the diverse structure of video classification models, this approach often fails to achieve the desired attack effectiveness in real-world applications, making it impractical.

[0004] Because videos can be viewed as multiple consecutive image frames, existing techniques use image models as proxy models to generate adversarial samples to attack video classification models. However, the inherent temporal correlation characteristics of video data cannot be easily captured using image models, resulting in poor adversarial sample attack effectiveness. Furthermore, both image models and video classification models are deep neural network models, and the features output by different layers in each contain different information. Directly using image models as proxy models ignores this characteristic, resulting in poor adversarial sample attack effectiveness.

[0005] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in this technical field. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method and device for generating black-box adversarial samples, the purpose of which is to use the information in the feature maps of different levels of each frame in the original sample to generate adversarial samples, thereby improving the attack success rate of adversarial samples in different video classification models; extracting original time features, generating and using adversarial sample time features, generating target adversarial samples that accurately destroy the temporal continuity of the video, and solving the problem of poor practicality of the attack method of directly using image models as video proxy models.

[0007] The present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a method for generating a black-box adversarial sample, comprising:

[0009] Determine the shallow feature map, the middle feature map, and the deep feature map of each frame in the original sample as the original spatial feature of the original sample; divide the original sample into multiple segments based on the feature similarity between different frames in the original sample; and perform a random shuffling operation on multiple feature units in each segment to obtain the original temporal feature of each segment;

[0010] Preprocess the original sample to obtain an initialized adversarial sample; determine the shallow feature map, mid-level feature map, and deep feature map of each frame in the initialized adversarial sample as the adversarial sample spatial features of the initialized adversarial sample; randomly obtain two to-be-fused video frames in each segment of the initialized adversarial sample, perform a pairwise fusion operation on the two to-be-fused video frames, and obtain the adversarial sample temporal features in each segment;

[0011] The cosine similarity between the original spatial features and the spatial features of the adversarial sample is measured to obtain a spatial loss function value; the cosine similarity between the original temporal features and the temporal features of the adversarial sample is measured to obtain a temporal loss function value; and the initialized adversarial sample is iteratively optimized according to the spatial loss function value and the temporal loss function value to obtain a target adversarial sample.

[0012] Furthermore, the shallow feature map, the middle feature map, and the deep feature map of each frame in the original sample are determined as the original spatial features of the original sample; the original sample is divided into multiple segments according to the feature similarity between different frames in the original sample; and the multiple feature units in each segment are randomly shuffled to obtain the original temporal features in each segment, including:

[0013] Inputting each frame of the original sample into the image model, obtaining edge texture information, abstract semantic information and high-order semantic information of the corresponding frame, using the edge texture information as a shallow feature map of each frame in the original sample, using the abstract semantic information as a middle feature map of each frame in the original sample, and using the high-order semantic information as a deep feature map of each frame in the original sample to obtain the original spatial features;

[0014] Calculating the cosine similarity between any two frames in the original sample; when the cosine similarity is greater than the similarity value, determining the corresponding two frames as belonging to the same segment, so as to divide the original sample into multiple segments;

[0015] The feature units in each segment are randomly replaced with feature units at corresponding positions in other frames in the same segment to obtain the original time features; wherein each segment includes multiple feature units.

[0016] Furthermore, the division parameters and the replacement number parameters are set;

[0017] The step of randomly replacing the feature units in each segment with feature units at corresponding positions in other frames in the same segment includes:

[0018] Dividing each frame in the segment into a plurality of feature units on an even basis according to the division parameters;

[0019] Determine the first b feature units in the corresponding frame in order from front to back; wherein b represents the replacement number parameter;

[0020] Randomly select another frame in the segment, and determine the first b feature units in the other frame;

[0021] The first b feature units in the corresponding frame are replaced by the first b feature units in the other frames to obtain the corresponding original time features.

[0022] Furthermore, the iteratively optimizing the initialized adversarial sample according to the spatial loss function value and the temporal loss function value to obtain a target adversarial sample includes:

[0023] Determine the sum of the spatial loss function value and the temporal loss function value as a comprehensive loss function value;

[0024] Iteratively optimizing the initialized adversarial sample according to the comprehensive loss function value until the number of iterations reaches a threshold;

[0025] The adversarial sample obtained in the last iteration is determined as the target adversarial sample.

[0026] Furthermore, the calculation formula of the spatial loss function value is:

[0027]

[0028] Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; The Lth extracted frame i j Layer feature map, j = {1, 2, 3}, the L1 layer feature map is the shallow feature map, the L2 layer feature map is the middle layer feature map, and the L3 layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; (V i +δ i ) represents the initialization of the adversarial sample; δ i represents the adversarial perturbation added to the i-th frame in the original sample.

[0029] Furthermore, the calculation formula of the time loss function value is:

[0030]

[0031] Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; The Lth extracted frame i j Layer feature map, j = {1, 2, 3}, the L1 layer feature map is the shallow feature map, the L2 layer feature map is the middle layer feature map, and the L3 layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; shuffle (V i ) represents the original time feature of the i-th frame after random shuffling operation; H(V i +δ i ) represents the fused video frame; (V i +δ i ) represents the initialization of the adversarial sample; δ i represents the adversarial perturbation added to the i-th frame in the original sample.

[0032] Furthermore, the iteratively optimizing the initialized adversarial sample according to the comprehensive loss function value until the number of iterations reaches a threshold includes:

[0033] Using the ADAM optimizer to adjust iterative optimization parameters based on the comprehensive loss function value to obtain optimized noise; wherein the iterative optimization parameters are control parameters for optimizing the intermediate adversarial samples obtained in the previous round;

[0034] The optimized noise is added to the original sample to obtain an intermediate adversarial sample.

[0035] Furthermore, the original sample is preprocessed to obtain an initialized adversarial sample; the shallow feature map, the middle feature map, and the deep feature map of each frame in the initialized adversarial sample are determined as the adversarial sample spatial features of the initialized adversarial sample; two video frames to be fused are randomly obtained in each segment of the initialized adversarial sample, and a pairwise fusion operation is performed on the two video frames to be fused to obtain the adversarial sample temporal features in each segment, including:

[0036] Adding a random Gaussian noise to the original sample to obtain a noisy video; initializing an adversarial perturbation with the same size as the original sample, and adding the adversarial perturbation to the noisy video to obtain an initialized adversarial sample;

[0037] Input each frame of the initialized adversarial sample into the image model to obtain adversarial sample spatial features;

[0038] According to the segment division result of the original sample, a plurality of segments in the initialization adversarial sample are obtained; in each segment of the initialization adversarial sample, a first video frame and a second video frame are randomly selected to obtain two corresponding video frames to be fused; and the first video frame and the second video frame are fused to obtain a fused video frame;

[0039] The fused video frame is input into an image model to obtain an adversarial sample temporal feature of the fused video frame.

[0040] In a second aspect, the present invention further provides a device for generating a black-box adversarial sample, comprising:

[0041] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to perform the method for generating black-box adversarial samples described in the first aspect.

[0042] In a third aspect, the present invention further provides a non-volatile computer storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors to complete the method for generating black-box adversarial samples described in the first aspect.

[0043] In a fourth aspect, a computer program product comprising instructions is provided, which, when executed on a computer or processor, causes the computer or processor to execute a method for generating black-box adversarial samples as described in the first to third aspects and any one of the above.

[0044] In the fifth aspect, the present invention also provides a black-box adversarial sample generation system, including a black-box adversarial sample generation device as in the second aspect, and using the black-box adversarial sample generation method as described in the first aspect to complete the interaction of the black-box adversarial sample generation device of the second aspect.

[0045] Different from the prior art, the present invention has at least the following beneficial effects:

[0046] The present invention uses the information in the shallow, middle and deep feature maps of each frame in the original sample to generate adversarial samples. It not only takes into account the instantaneous changes between video clips, but also obtains features from different levels of the image model, thereby improving the attack success rate of the target adversarial sample on the video feature information in the original sample; the content of the original sample often exists in multiple clips, and there is no certain time correlation between different clips. The present invention generates the target adversarial sample by extracting the original time features, generating and using the adversarial sample time features. While taking into account the time jumps of the video content in different clips, it realizes the capture of the time clues in the original sample and enables the target adversarial sample to accurately destroy the time continuity of the original sample, solving the problem of poor practicality of the attack method that directly uses the image model as the video proxy model. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0048] Figure 1 1 is a flow chart of a method for generating a black-box adversarial sample provided by an embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of a specific example of a method for generating black-box adversarial samples provided by an embodiment of the present invention;

[0050] Figure 3 1 is a schematic diagram of a process for generating a target adversarial sample according to an embodiment of the present invention;

[0051] Figure 4 is a flow chart of step 10 provided in an embodiment of the present invention;

[0052] Figure 5 is a schematic diagram of a specific example of using an intra-segment shuffling operation provided by an embodiment of the present invention;

[0053] Figure 6 is a flow chart of step 103 provided by an embodiment of the present invention;

[0054] Figure 7 is a flow chart of step 20 provided in an embodiment of the present invention;

[0055] Figure 8 is a flow chart of step 30 provided in an embodiment of the present invention;

[0056] Figure 9 is a flow chart of step 302 provided by an embodiment of the present invention;

[0057] Figure 10 2 is a schematic diagram of the architecture of a device for generating black-box adversarial samples provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] Unless the context requires otherwise, throughout the specification and claims, the term "including" is to be interpreted as meaning open inclusion, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific example" or "some examples" and the like are intended to indicate that the specific features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner, that is, although they may be carried in the embodiments or examples of the above terms due to reasons such as the order and position of appearance, it is not limited to that they can be carried in combination by one embodiment or example.

[0060] In the description of the present invention, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present disclosure.

[0061] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, for example, the description may also use the method of adding "A" and "B" at the end to describe the same type of nouns as two independent individuals. In this case, the corresponding features defined as "A" and "B" are only used to distinguish the description purposes of the same type of individuals, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0062] When describing some embodiments, the expressions “coupled”, “coupled” and “connected” and their derivatives may be used. For example, when describing some embodiments, the term “connected” may be used to indicate that two or more components are in direct physical or electrical contact with each other. For another example, when describing some embodiments, the term “coupled” may be used to indicate that two or more components are in direct physical or electrical contact. However, the term “connected” or “coupled” may also mean that two or more components are not in direct contact with each other, but still cooperate or interact with each other, such as “optical coupling”, “wireless connection”, etc. The embodiments disclosed herein are not necessarily limited to the contents of the present invention.

[0063] In the description of the present invention, the expression "A and / or B" (where A and B are used to formally represent specific characteristic contents) will be involved, and the corresponding expressions include the following three combinations: only A, only B, and a combination of A and B.

[0064] As used herein, "about," "substantially," or "approximately" includes the stated value and an average value that is within an acceptable range of deviation from the particular value as determined by one of ordinary skill in the art taking into account the measurements in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0065] Example 1:

[0066] In order to solve the above problems, Figure 1 As shown, an embodiment of the present invention provides a method for generating a black-box adversarial sample, including:

[0067] Step 10: Determine the shallow feature map, middle feature map and deep feature map of each frame in the original sample as the original spatial features of the original sample; divide the original sample into multiple segments according to the feature similarity between different frames in the original sample; and randomly shuffle the multiple feature units in each segment to obtain the original time features in each segment.

[0068] The original sample is video data. Since both the video classification model and the image model employ deep neural networks, and their intermediate layer output features share similarities, a cross-model attack method based on the image model is feasible. This cross-model attack method involves using an image model trained on a large-scale image dataset as a proxy model to attack the video classification model.

[0069] The feature maps output by different layers in the image model and the video classification model include different information. Using different layers for attacks will produce different effects. Therefore, the embodiment of the present invention extracts feature maps of different levels of each frame in the original sample, so as to use the information therein to generate adversarial samples for the corresponding level characteristics in the video classification model.

[0070] Video content often consists of multiple clips, with no specific temporal correlation between them. For example, an edited video often contains two unrelated scenes cut into the same segment. For example, one second of the video content shows someone speaking, and the next second shows an object or landscape. There is no temporal correlation between the two seconds. The present invention captures temporal features in video content by segmenting and randomly shuffling multiple feature units within each segment. Other methods for acquiring temporal features are also provided below.

[0071] The method for generating black-box adversarial samples in an embodiment of the present invention does not require additional training of a video proxy model, but instead uses an image model pre-trained on a large-scale dataset to generate target adversarial samples for a video classification model.

[0072] Among them, the method of obtaining shallow feature maps, middle feature maps and deep feature maps is selected by those skilled in the art according to the specific usage scenario; in an optional embodiment, an image model that implements the classification task can be used to process each frame of the original sample, and the corresponding shallow feature map, middle feature map and deep feature map are obtained from the image model; the shallow feature map can be edge texture information, the middle feature map can be abstract semantic information, and the deep feature map can be high-order semantic information. The specific method of calculating feature similarity and randomly shuffling multiple feature units in each segment will be described below. The specific type of image model in the embodiment of the present invention is selected by those skilled in the art according to the specific usage scenario and is not limited here; in the prior art, image models are used to classify image data, and the embodiment of the present invention uses image models to extract information of video frames.

[0073] Step 20: Preprocess the original sample to obtain an initialized adversarial sample; determine the shallow feature map, middle feature map and deep feature map of each frame in the initialized adversarial sample as the adversarial sample spatial features of the initialized adversarial sample; randomly obtain two video frames to be fused in each segment of the initialized adversarial sample, perform a pairwise fusion operation on the two video frames to be fused, and obtain the adversarial sample temporal features in each segment.

[0074] Among them, the initialized adversarial sample is video data; the initialized adversarial sample is obtained by adding adversarial perturbations to the original sample; the initialized adversarial sample can be divided into multiple segments according to the segment division of the original sample; each segment obtained in step 10 includes multiple feature units. The embodiment of the present invention performs pairwise fusion of the video frame image in the initialized adversarial sample (i.e., a video frame to be fused) with another random video frame image in the segment (i.e., another video frame to be fused) to enhance the aggressiveness of the target adversarial sample by randomly disrupting the feature units in each segment. The specific method of the pairwise fusion operation will be described below.

[0075] Step 30: Calculate the cosine similarity between the original spatial features and the spatial features of the adversarial sample to obtain a spatial loss function value; calculate the cosine similarity between the original temporal features and the temporal features of the adversarial sample to obtain a temporal loss function value; and iteratively optimize the initialized adversarial sample based on the spatial loss function value and the temporal loss function value to obtain a target adversarial sample.

[0076] The present invention uses the information in the shallow, middle and deep feature maps of each frame in the original sample to generate adversarial samples. It not only takes into account the instantaneous changes between video clips, but also obtains features from different levels of the image model, thereby improving the attack success rate of the target adversarial sample on the video feature information in the original sample; the content of the original sample often exists in multiple clips, and there is no certain time correlation between different clips. The present invention generates the target adversarial sample by extracting the original time features, generating and using the adversarial sample time features. While taking into account the time jumps of the video content in different clips, it realizes the capture of the time clues in the original sample and enables the target adversarial sample to accurately destroy the time continuity of the original sample, solving the problem of poor practicality of the attack method that directly uses the image model as the video proxy model.

[0077] The overall process of generating target adversarial samples in the embodiment of the present invention is as follows: Figure 2 and Figure 3 As shown below. The following first introduces the process of generating original spatial features and original time features in detail. Figure 4 As shown, the step 10 includes:

[0078] Step 101: Input each frame in the original sample into the image model to obtain edge texture information, abstract semantic information and high-order semantic information of the corresponding frame, use the edge texture information as the shallow feature map of each frame in the original sample, use the abstract semantic information as the middle feature map of each frame in the original sample, and use the high-order semantic information as the deep feature map of each frame in the original sample to obtain the original spatial features.

[0079] In the embodiment of the present invention, the original sample is first divided into multiple frames according to its original frame rate. T×C×H×W ,V i represents the i-th frame in the original sample, V i ∈R C×H×W , C represents the number of channels of the frame, H represents the height of the frame, W represents the width of the frame, and T represents the total number of frames of the original sample. Use the image model g(·) trained on a general image dataset ImageNet to generate target adversarial samples for the video classification model; each frame V of the original sample i It is sent to the image model g(·), which extracts feature maps of different layers. Represents the Lth frame of the i-th frame extracted from the image model jLayer features, where L = {l1, l2, l3} represents a list of selected layers. This embodiment of the present invention divides the total number of layers of the image model into three segments: the first segment is shallow features (edge ​​texture information of the frame), the second segment is mid-level features (abstract semantic information of the frame), and the third segment is deep features (high-level semantic information of the frame). A total of three layers are selected for feature attack, where l1 is used to obtain shallow feature maps, l2 is used to obtain mid-level feature maps, and l3 is used to obtain deep feature maps.

[0080] The embodiments of the present invention use image models trained on large-scale datasets to generate video adversarial samples, reducing the overhead of training video recognition proxy models. Without the need for additional video proxy model training, computing resource consumption is significantly reduced, and the stability of the attack effectiveness of the generated adversarial samples is improved. Furthermore, by utilizing the multi-layer feature space of the image model to extract information contained in the corresponding layers, the attack success rate of the generated target adversarial samples in different video classification models is improved.

[0081] Step 102: Calculate the cosine similarity between any two frames in the original sample; when the cosine similarity is greater than the similarity value, determine the corresponding two frames as belonging to the same segment, so as to divide the original sample into multiple segments.

[0082] Among them, the formula for cosine similarity is: A represents any frame in the original sample, and B represents another frame in the original sample. The similarity value is selected by those skilled in the art based on the specific usage scenario. In an optional embodiment, the similarity value can be 0.9, that is, when the cosine similarity between two frames is greater than 0.9, the two frames are determined to be in the same segment.

[0083] like Figure 2 As shown on the right side of the figure, it is determined whether any two frames are in the same clip. For example, the original sample V=Clip(V1,V2,…,V n ), n is the total number of segments divided, V i Represents the i-th segment divided; among them, a single segment represents the jth frame in the i-th segment, T i Represents the total number of frames in the i-th segment. Among them, the cosine similarity

[0084] Step 103: randomly replacing the feature units in each segment with feature units at corresponding positions in other frames in the same segment to obtain the original time features; wherein each segment includes multiple feature units.

[0085] The embodiment of the present invention uses cosine similarity to measure the feature similarity between the original spatial features of different frames in the original sample; then, the original spatial features in each segment are shuffled at the channel level using an intra-segment shuffling operation, that is, the feature units in each segment are randomly replaced with feature units at corresponding positions in other frames in the same segment. For example, Figure 5 The left side of the figure shows multiple segments divided from the original sample. A row represents a segment. The multiple blocks on each row are multiple feature units. The feature units in the same row are in the same segment. Figure 5 The middle right side shows the original time features obtained after the intra-fragment shuffling operation.

[0086] In an optional embodiment, the partition parameter and the replacement number parameter are set; Figure 6 As shown, step 103 includes:

[0087] Step 1031: Divide each frame in the segment into multiple feature units on an even basis according to the division parameters.

[0088] When the partition parameter is a and the replacement number parameter is b in this embodiment of the present invention, the original spatial features of each frame are first divided into a segments evenly according to the number of channels. For example, when a is set to 3, the features of each frame are evenly divided into 3 segments according to the number of channels, resulting in a corresponding plurality of feature units.

[0089] Step 1032: Determine the first b feature units in the corresponding frame in order from front to back; wherein b represents the replacement number parameter.

[0090] Then, segment b is selected from segment a and randomly replaced with the original spatial features of the corresponding segments in other frames in the current segment, b≤a. For example, when a is 3 and b is 2, as Figure 5 As shown, for each horizontal row of feature units, the first b feature units in the corresponding frame are determined from left to right.

[0091] Step 1033: Randomly select another frame in the segment and determine the first b feature units in the other frame.

[0092] Step 1034: Replace the first b feature units in the corresponding frame with the first b feature units in the other frames to obtain the corresponding original time features.

[0093] like Figure 5 As shown, the first and second segment features are randomly replaced with the first and second segment features of other frames in the current segment.

[0094] The original spatial features after being disrupted not only have their own features but also have the original spatial features of other frames in the same segment; therefore, the embodiment of the present invention uses the disrupted original spatial features in the segment as the original temporal features.

[0095] When using image models to generate target adversarial samples, the present invention considers the connections between multiple layers of features and the temporal continuity of video data. By calculating feature similarity and segmenting the samples, the temporal cues in the original sample are captured and the computational resources required are reduced. Furthermore, considering the temporal jumps between different segments, intra-segment shuffling is used to generate adversarial samples that precisely disrupt the temporal continuity of the video, further improving the success rate of video adversarial sample attacks.

[0096] In order to give a detailed introduction to the process of generating adversarial sample spatial features and adversarial sample temporal features, as shown in the following example: Figure 7 As shown, the step 20 includes:

[0097] Step 201: Add a random Gaussian noise to the original sample to obtain a noisy video; initialize an adversarial perturbation with the same size as the original sample, and add the adversarial perturbation to the noisy video to obtain an initialized adversarial sample.

[0098] First, we add random Gaussian noise to the original sample and define the video after adding the noise as the noisy video. This makes the target adversarial sample more robust. Then, we initialize an adversarial perturbation with the same size as the original sample and add this adversarial perturbation to the noisy video. This noisy video is defined as the initialized adversarial sample. Since we will subsequently iteratively optimize the added adversarial perturbation based on this initialized adversarial sample, we set the number of iterations to 0.

[0099] For example, the adversarial sample consists of the original sample and the adversarial perturbation δ, that is, V adv =V+δ. The size of the perturbation needs to be consistent with the original sample, δ∈R T×C×H×W , where δ i ∈R C×H×W represents the perturbation of the i-th frame, i={1,2,3,…,T}. In this embodiment, v is first set to (16 / 255) T×C×H×W Used to initialize the adversarial sample. If the number of iterations is not reached each time, a random Gaussian noise μ∈R is generated. T×C×H×W , add μ to V adv middle.

[0100] Step 202: Input each frame of the initialized adversarial sample into the image model to obtain adversarial sample spatial features.

[0101] First, the initialization adversarial sample is divided into multiple frames according to the original sample's original frame rate and fed into the image model. The image model processes each frame of the initialization adversarial sample in the same way as it processes each frame of the original sample. The image model extracts edge texture information from each frame of the initialization adversarial sample as the shallow feature map of that frame; extracts abstract semantic information from each frame of the initialization adversarial sample as the mid-level feature map of that frame; and extracts high-level semantic information from each frame of the initialization adversarial sample as the deep feature map of that frame.

[0102] Step 203: According to the segment division result of the original sample, multiple segments in the initialized adversarial sample are obtained; in each segment of the initialized adversarial sample, a first video frame and a second video frame are randomly selected to obtain two corresponding video frames to be fused; the first video frame and the second video frame are fused to obtain a fused video frame.

[0103] In an optional embodiment, the fusion ratio of the first video frame and the second video frame may be 1:1.

[0104] The following combination Figure 5 Explain the fusion operation:

[0105] First, the video V (i.e., the original sample) is divided into multiple clips according to the feature similarity between different video frames, and V = Clip (V1, V2, ..., V n ); where n represents the total number of segments that the video is divided into, V n Indicates fragment n, a single fragment Indicates the Tth in segment n n Video frame, T n Indicates the total number of video frames in this segment.

[0106] The embodiment of the present invention uses feature similarity to distinguish segments. The similarity within the same segment is greater than a threshold, i.e. in is the hyper-parameter threshold; through pairwise fusion operations, spatiotemporal information modeling is achieved only within the fragment; the embodiment of the present invention replaces the image translation operation in the prior art by randomly shuffling video frames to obtain long-distance time information.

[0107] When the image model g during training l (V)=M∈R T×C×H×W When, T represents the number of video frames of video V, C represents the number of feature channels of the middle layer of the model network, M represents the output of the middle layer of the model network, H represents the height of the feature map, and W represents the width of the feature map. i =[c i1 ,c i2 ,ci3 ,……,c iC ]=[c i1 …c ik ,c ik+1 …c i2k ,c i2k+1 …c iC ], where c ik represents the k-th channel of the i-th video frame, whose size is H×W, k is the base length of the random shuffling operation, k=floor(C÷p); where C represents the total length of the channel, p represents a preset hyperparameter, and the specific value of the preset hyperparameter is selected by those skilled in the art based on the specific usage scenario and experience. Floor(·) represents the rounding down operation after integer division.

[0108] After random shuffling, the following features are obtained:

[0109] IND1,IND2=shuffle(1,2,3,...,T n )

[0110] Get the fused video frame after modeling

[0111]

[0112] in, represents the i-th video frame in segment n.

[0113] In short, the embodiments of the present invention will Figure 5 The specific instance of the original sample on the left side is divided into P segments by channel, and then the first several (parameter) segments are randomly shuffled. The fused video features obtained in this way not only retain the feature information of the video frame in the original sample, but also integrate the feature information of other video frames in the segment. The pictures in each segment are basically consistent and have time information. According to the method of the embodiment of the present invention, the correlation information in the time dimension of the original sample can be quickly obtained. At the same time, it can be used to generate video adversarial samples that can also destroy the time information, greatly reducing the time overhead.

[0114] Step 204: Input the fused video frame into the image model to obtain the adversarial sample temporal features of the fused video frame.

[0115] The image model processes the fused video frames in the same way as it processes the original samples and each frame in the initialized adversarial samples, and will not be repeated here.

[0116] After obtaining the adversarial sample spatial features and adversarial sample temporal features, the embodiment of the present invention further calculates the cosine similarity between the original spatial features and the adversarial sample spatial features to obtain the spatial loss function value. The calculation formula of the spatial loss function value is:

[0117]

[0118] Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; The Lth extracted frame i j Layer feature map, j = {1, 2, 3}, the L1 layer feature map is the shallow feature map, the L2 layer feature map is the middle layer feature map, and the L3 layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; (V i +δ i ) represents the initialization of the adversarial sample; δ i represents the adversarial perturbation added to the i-th frame in the original sample.

[0119] In order to ensure that the human eye cannot observe the existence of the adversarial perturbation δ, it is necessary to make certain restrictions on δ; in an optional embodiment, the restriction ‖δ‖ ∞ ≤(16 / 255), where ‖·‖ ∞ Represents L p norm, st is the mathematical symbol representing the constraint.

[0120] Using the above calculation formula, the similarity between the feature space of the adversarial sample and the feature space of the original sample can be made smaller and smaller, that is, the adversarial sample (or intermediate adversarial sample) is initialized by optimizing the orthogonal part.

[0121] The embodiment of the present invention further calculates the cosine similarity between the original time feature and the adversarial sample time feature to obtain a time loss function value; the calculation formula of the time loss function value is:

[0122]

[0123] Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; The Lth extracted frame i jLayer feature map, j = {1, 2, 3}, the L1 layer feature map is the shallow feature map, the L2 layer feature map is the middle layer feature map, and the L3 layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; shuffle (V i ) represents the original time feature of the i-th frame after random shuffling operation; H(V i +δ i ) represents the fused video frame; (V i +δ i ) represents the initialization of the adversarial sample; δ i represents the adversarial perturbation added to the i-th frame in the original sample.

[0124] In an optional embodiment, similarly, the restriction on calculating the spatial loss function value is limited to ‖δ‖ ∞ ≤(16 / 255), where ‖·‖ ∞ Represents L p norm.

[0125] Using the above calculation formula, the similarity between the temporal features of the adversarial sample and the temporal features of the original sample can be made smaller and smaller, thereby completing the destruction of the temporal continuity of the video.

[0126] The embodiment of the present invention achieves the effect of attacking the features in the original sample in space and time by performing orthogonality in both space and time.

[0127] After obtaining the spatial loss function value and the temporal loss function value, as Figure 8 As shown, in step 30, the iterative optimization of the initialized adversarial sample according to the spatial loss function value and the temporal loss function value to obtain the target adversarial sample includes:

[0128] Step 301: Determine the sum of the spatial loss function value and the temporal loss function value as a comprehensive loss function value.

[0129] In order to generate comprehensive adversarial samples that attack both feature space and temporal features, the spatial loss function value and the temporal loss function value are jointly constrained to comprehensively optimize the adversarial perturbation. The final calculation formula for the comprehensive loss function value is as follows:

[0130] L=L c +L t

[0131] Through this comprehensive loss function value and the adaptive learning rate (ADAM) optimizer, the noise in the initialized adversarial sample can be optimized in a direction that is destructive to both the feature space and temporal continuity of the original sample.

[0132] Step 302: Iteratively optimize the initialized adversarial sample according to the comprehensive loss function value until the number of iterations reaches a threshold.

[0133] Add the optimized noise to the initial adversarial sample to generate a new intermediate adversarial sample for this iteration, and increase the number of iterations by 1.

[0134] Step 303: Determine the adversarial sample obtained in the last iteration as the target adversarial sample.

[0135] The threshold is a hyperparameter that controls the number of iterations, and the threshold is selected by those skilled in the art based on specific usage scenarios. In an optional embodiment, the threshold may be 60.

[0136] like Figure 9 As shown, step 302 includes:

[0137] Step 3021: Use the ADAM optimizer to adjust the iterative optimization parameters based on the comprehensive loss function value to obtain optimized noise; wherein, the iterative optimization parameters are control parameters for optimizing the intermediate adversarial samples obtained in the previous round.

[0138] Step 3022: Add the optimized noise to the original sample to obtain an intermediate adversarial sample.

[0139] The specific usage of the ADAM optimizer is determined by those skilled in the art based on specific usage scenarios and experience.

[0140] The black-box adversarial sample generation method of the present invention is applicable to different video classification models and has good generalization capabilities. Furthermore, the generated target adversarial samples can not only be used to attack video classification models, but also help researchers discover vulnerabilities in image models, thereby enhancing the defense capabilities of video classification models and improving their reliability and security in practical applications.

[0141] The present invention adds adversarial perturbations to attack different feature levels and temporal continuity of the original samples. While ensuring a high success rate of black-box attacks using target adversarial samples, it reduces the visual difference between the target adversarial samples and the original samples, and at the same time improves the generation rate of target adversarial samples.

[0142] Example 2:

[0143] like Figure 10 FIG2 is a schematic diagram of the architecture of a device for generating black-box adversarial samples according to an embodiment of the present invention. The device for generating black-box adversarial samples according to this embodiment includes one or more processors 21 and a memory 22. Figure 10 A processor 21 is taken as an example.

[0144] The processor 21 and the memory 22 may be connected via a bus or other means. Figure 10 The bus connection is taken as an example.

[0145] Memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the method for generating black-box adversarial examples in this embodiment. Processor 21 executes the method for generating black-box adversarial examples by running the non-volatile software programs and instructions stored in memory 22.

[0146] The memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory 22 may optionally include a memory remotely located relative to the processor 21, and such remote memory may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0147] The program instructions / modules are stored in the memory 22. When executed by the one or more processors 21, the method for generating black-box adversarial samples in the above-mentioned embodiment is executed, for example, each step of the method for generating black-box adversarial samples in the embodiment of the present invention described above is executed.

[0148] An embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, for example Figure 10 A processor 21 can enable the above one or more processors to execute the method for generating black-box adversarial samples in the specific embodiment of the present invention, for example, to execute the various steps of the method for generating black-box adversarial samples in the embodiment of the present invention described above; it can also realize Figure 10 The various modules and units described above; or executing the method for generating a black box adversarial sample in a specific embodiment of the present invention, for example, executing the various steps of the method for generating a black box adversarial sample in the embodiment of the present invention described above; or Figure 10 The various modules and units described.

[0149] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific content can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0150] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk or an optical disk, etc.

[0151] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating black-box adversarial samples, characterized in that: include: Determine the shallow feature map, the middle feature map, and the deep feature map of each frame in the original sample as the original spatial feature of the original sample; divide the original sample into multiple segments based on the feature similarity between different frames in the original sample; and perform a random shuffling operation on multiple feature units in each segment to obtain the original temporal feature of each segment; Preprocessing the original sample to obtain an initialized adversarial sample; determining the shallow feature map, the middle feature map, and the deep feature map of each frame in the initialized adversarial sample as the adversarial sample spatial features of the initialized adversarial sample; Randomly obtaining two to-be-fused video frames in each segment of the initialized adversarial sample, performing a pairwise fusion operation on the two to-be-fused video frames, and obtaining a temporal feature of the adversarial sample in each segment; The cosine similarity between the original spatial features and the spatial features of the adversarial sample is measured to obtain a spatial loss function value; the cosine similarity between the original temporal features and the temporal features of the adversarial sample is measured to obtain a temporal loss function value; and the initialized adversarial sample is iteratively optimized according to the spatial loss function value and the temporal loss function value to obtain a target adversarial sample.

2. The method for generating black-box adversarial samples according to claim 1, characterized in that: The shallow feature map, the middle feature map and the deep feature map of each frame in the original sample are determined as the original spatial features of the original sample; and the original sample is divided into multiple segments according to the feature similarity between different frames in the original sample; A random shuffling operation is performed on multiple feature units in each segment to obtain the original time features in each segment, including: Inputting each frame of the original sample into the image model, obtaining edge texture information, abstract semantic information and high-order semantic information of the corresponding frame, using the edge texture information as a shallow feature map of each frame in the original sample, using the abstract semantic information as a middle feature map of each frame in the original sample, and using the high-order semantic information as a deep feature map of each frame in the original sample to obtain the original spatial features; Calculating the cosine similarity between any two frames in the original sample; when the cosine similarity is greater than the similarity value, determining the corresponding two frames as belonging to the same segment, so as to divide the original sample into multiple segments; The feature units in each segment are randomly replaced with feature units at corresponding positions in other frames in the same segment to obtain the original time features; wherein each segment includes multiple feature units.

3. The method for generating black-box adversarial samples according to claim 2, characterized in that: Set the division parameters and replacement number parameters; The step of randomly replacing the feature units in each segment with feature units at corresponding positions in other frames in the same segment includes: Dividing each frame in the segment into a plurality of feature units on an even basis according to the division parameters; Determine the first b feature units in the corresponding frame in order from front to back; wherein b represents the replacement number parameter; Randomly select another frame in the segment, and determine the first b feature units in the other frame; The first b feature units in the corresponding frame are replaced by the first b feature units in the other frames to obtain the corresponding original time features.

4. The method for generating black-box adversarial samples according to claim 1, wherein: The iteratively optimizing the initialized adversarial sample according to the spatial loss function value and the temporal loss function value to obtain a target adversarial sample includes: Determine the sum of the spatial loss function value and the temporal loss function value as a comprehensive loss function value; Iteratively optimizing the initialized adversarial sample according to the comprehensive loss function value until the number of iterations reaches a threshold; The adversarial sample obtained in the last iteration is determined as the target adversarial sample.

5. The method for generating black-box adversarial samples according to claim 4, characterized in that: The calculation formula of the spatial loss function value is: Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; The Lth extracted frame i j Layer feature map, j = {1, 2, 3}, the L1 layer feature map is the shallow feature map, the L2 layer feature map is the middle layer feature map, and the L3 layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; (V i +δ i ) represents the initialization of the adversarial sample; δ i represents the adversarial perturbation added to the i-th frame in the original sample.

6. The method for generating black-box adversarial samples according to claim 4, characterized in that: The calculation formula of the time loss function value is: Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; The Lth extracted frame i j Layer feature map, j = {1, 2, 3}, the L1 layer feature map is the shallow feature map, the L2 layer feature map is the middle layer feature map, and the L3 layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; shuffle (V i ) represents the original time feature of the i-th frame after random shuffling operation; H(V i +δ i ) represents the fused video frame; (V i +δ i ) represents the initialization of the adversarial sample; δ i represents the adversarial perturbation added to the i-th frame in the original sample.

7. The method for generating black-box adversarial samples according to claim 4, characterized in that: The iteratively optimizing the initialized adversarial sample according to the comprehensive loss function value until the number of iterations reaches a threshold comprises: Using the ADAM optimizer to adjust iterative optimization parameters based on the comprehensive loss function value to obtain optimized noise; wherein the iterative optimization parameters are control parameters for optimizing the intermediate adversarial samples obtained in the previous round; The optimized noise is added to the original sample to obtain an intermediate adversarial sample.

8. The method for generating black-box adversarial samples according to claim 1, characterized in that: The original sample is preprocessed to obtain an initialized adversarial sample; the shallow feature map, the middle feature map and the deep feature map of each frame in the initialized adversarial sample are determined as the adversarial sample spatial features of the initialized adversarial sample; Randomly obtain two video frames to be fused in each segment of the initialized adversarial sample, perform a pairwise fusion operation on the two video frames to be fused, and obtain the adversarial sample temporal features in each segment, including: Adding a random Gaussian noise to the original sample to obtain a noisy video; initializing an adversarial perturbation with the same size as the original sample, and adding the adversarial perturbation to the noisy video to obtain an initialized adversarial sample; Input each frame of the initialized adversarial sample into the image model to obtain adversarial sample spatial features; According to the segment division result of the original sample, a plurality of segments in the initialization adversarial sample are obtained; in each segment of the initialization adversarial sample, a first video frame and a second video frame are randomly selected to obtain two corresponding video frames to be fused; and the first video frame and the second video frame are fused to obtain a fused video frame; The fused video frame is input into an image model to obtain an adversarial sample temporal feature of the fused video frame.

9. A device for generating black-box adversarial samples, characterized in that: The device for generating black-box adversarial samples includes at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor. After being executed by the processor, the instructions are used to implement the method for generating black-box adversarial samples described in any one of claims 1 to 8.

10. A non-volatile computer storage medium, characterized in that The computer storage medium stores computer-executable instructions, which are executed by one or more processors to complete the method for generating black-box adversarial samples according to any one of claims 1 to 8.

Images