Black box confrontation sample generation method and device

Abstract

The invention relates to the technical field of intelligent algorithm confrontation, and provides a black box confrontation sample generation method and device. The method comprises the following steps: determining a shallow-layer feature map, a middle-layer feature map and a deep-layer feature map of each frame in an original sample as original spatial features; dividing the original sample according to the feature similarity to obtain an original time feature in each fragment; determining a shallow-layer feature map, a middle-layer feature map and a deep-layer feature map of each frame in the initialized adversarial sample as adversarial sample space features, and performing pairwise fusion operation on a plurality of feature units in each fragment to obtain adversarial sample time features; calculating the cosine similarity between the original spatial features and the confrontation sample spatial features, calculating the cosine similarity between the original time features and the confrontation sample time features, obtaining an intermediate confrontation sample, carrying out the iterative optimization of the intermediate confrontation sample, and finally obtaining a target confrontation sample. The problem that an attack method directly using an image model as a video proxy model is poor in practicability is solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent algorithm adversarial technology, and in particular to a method and device for generating black-box adversarial samples. Background Art

[0002] Currently, video classification models based on deep neural networks have been widely used. These video classification models can automatically identify and analyze human actions in videos and are widely used in security monitoring, human-computer interaction and other fields. However, by simply adding slight disturbances to the samples input into these video classification models, these video classification models can output incorrect judgment results. As a result, there will be serious security threat risks in the actual application of video classification models; for example, by attacking the video classification model, it can be made unable to identify adversarial video samples with malicious information, which causes malicious information to be spread through the video channel, which is extremely harmful. Therefore, in order to evaluate the security of video classification models and promote the optimization and upgrading of video classification models, it is necessary to generate adversarial samples for testing video classification models.

[0003] Attacks using adversarial samples can be divided into white-box attacks and black-box attacks, depending on the attacker's understanding of the video classification model. Most of the existing black-box attack methods require training an additional video proxy model with the same architecture as the video classification model to generate adversarial samples. Due to the larger volume of video data and the more diverse structure of video classification models, this method often fails to achieve the desired attack effect in real applications and has poor practicality.

[0004] Since a video can be viewed as multiple consecutive image frames, there are existing methods in the prior art that use image models as video proxy models to generate adversarial samples to attack video classification models. However, the temporal correlation characteristics inherent in video data cannot be simply obtained through image models, resulting in poor attack effects of adversarial samples generated using this method. In addition, both image models and video classification models are deep neural network models, and the features output by different layers in the two have different information. The attack method that directly uses image models as video proxy models ignores this characteristic, and the generated adversarial samples have poor attack effects.

[0005] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in the field of this technology. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a method and device for generating black-box adversarial samples, the purpose of which is to use the information in the feature maps of different levels of each frame in the original sample to generate adversarial samples, thereby improving the attack success rate of adversarial samples in different video classification models; extracting original time features, generating and using adversarial sample time features, generating target adversarial samples that accurately destroy the temporal continuity of the video, and solving the problem of poor practicality of the attack method that directly uses the image model as the video proxy model.

[0007] The present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a method for generating a black-box adversarial sample, comprising:

[0009] Determine the shallow feature map, the middle feature map and the deep feature map of each frame in the original sample as the original spatial feature of the original sample; divide the original sample into multiple segments according to the feature similarity between different frames in the original sample; perform random shuffling operation on multiple feature units in each segment to obtain the original time feature in each segment;

[0010] Preprocess the original sample to obtain an initialized adversarial sample; determine the shallow feature map, the middle feature map and the deep feature map of each frame in the initialized adversarial sample as the adversarial sample spatial features of the initialized adversarial sample; randomly obtain two video frames to be fused in each segment of the initialized adversarial sample, perform a pairwise fusion operation on the two video frames to be fused, and obtain the adversarial sample temporal features in each segment;

[0011] The cosine similarity between the original spatial feature and the spatial feature of the adversarial sample is calculated to obtain a spatial loss function value; the cosine similarity between the original temporal feature and the temporal feature of the adversarial sample is calculated to obtain a temporal loss function value; and according to the spatial loss function value and the temporal loss function value, the initialized adversarial sample is iteratively optimized to obtain a target adversarial sample.

[0012] Furthermore, the shallow feature map, the middle feature map and the deep feature map of each frame in the original sample are determined as the original spatial features of the original sample; the original sample is divided into multiple segments according to the feature similarity between different frames in the original sample; and the multiple feature units in each segment are randomly shuffled to obtain the original time features in each segment, including:

[0013] Input each frame in the original sample into the image model to obtain edge texture information, abstract semantic information and high-order semantic information of the corresponding frame, use the edge texture information as a shallow feature map of each frame in the original sample, use the abstract semantic information as a middle feature map of each frame in the original sample, and use the high-order semantic information as a deep feature map of each frame in the original sample to obtain the original spatial feature;

[0014] Calculating the cosine similarity between any two frames in the original sample; when the cosine similarity is greater than the similarity value, determining the corresponding two frames as belonging to the same segment, so as to divide the original sample into a plurality of segments;

[0015] The feature units in each segment are randomly replaced with feature units at corresponding positions in other frames in the same segment to obtain the original time features; wherein each segment includes multiple feature units.

[0016] Further, the partitioning parameters and the replacement number parameters are set;

[0017] The step of randomly replacing the feature units in each segment with feature units at corresponding positions in other frames in the same segment comprises:

[0018] According to the division parameters, each frame in the segment is evenly divided into a plurality of feature units;

[0019] Determine the first b feature units in the corresponding frame in order from front to back, wherein b represents the replacement number parameter;

[0020] Randomly select another frame in the segment, and determine the first b feature units in the another frame;

[0021] The first b feature units in the corresponding frame are replaced by the first b feature units in the other frames to obtain the corresponding original time features.

[0022] Further, the iteratively optimizing the initialized adversarial sample according to the spatial loss function value and the temporal loss function value to obtain a target adversarial sample includes:

[0023] Determine the sum of the spatial loss function value and the temporal loss function value as a comprehensive loss function value;

[0024] Iteratively optimizing the initialized adversarial sample according to the comprehensive loss function value until the number of iterations reaches a threshold;

[0025] The adversarial sample obtained in the last iteration is determined as the target adversarial sample.

[0026] Furthermore, the calculation formula of the spatial loss function value is:

[0027]

[0028] Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; represents the Lth extracted frame i j Layer feature map, j = {1, 2, 3}, the L1th layer feature map is the shallow feature map, the L2th layer feature map is the middle feature map, and the L3th layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; (V i +δ i ) represents the initialization of adversarial samples; δ i Represents the adversarial perturbation added to the i-th frame in the original sample.

[0029] Furthermore, the calculation formula of the time loss function value is:

[0030]

[0031] Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; represents the Lth extracted i-th frame j Layer feature map, j = {1, 2, 3}, the L1th layer feature map is the shallow feature map, the L2th layer feature map is the middle feature map, and the L3th layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; shuffle (V i ) represents the original time feature of the i-th frame after random shuffling; H(V i +δ i ) represents the fused video frame; (V i +δ i ) represents the initialization of adversarial samples; δ i Represents the adversarial perturbation added to the i-th frame in the original sample.

[0032] Further, the iterative optimization of the initialized adversarial sample according to the comprehensive loss function value until the number of iterations reaches a threshold comprises:

[0033] Using the ADAM optimizer to adjust the iterative optimization parameters based on the comprehensive loss function value to obtain optimized noise; wherein the iterative optimization parameters are control parameters for optimizing the intermediate adversarial samples obtained in the previous round;

[0034] The optimized noise is added to the original sample to obtain an intermediate adversarial sample.

[0035] Furthermore, the original sample is preprocessed to obtain an initialized adversarial sample; the shallow feature map, the middle feature map and the deep feature map of each frame in the initialized adversarial sample are determined as the adversarial sample spatial features of the initialized adversarial sample; two video frames to be fused are randomly obtained in each segment of the initialized adversarial sample, and the two video frames to be fused are fused in pairs to obtain the adversarial sample temporal features in each segment, including:

[0036] Add a random Gaussian noise to the original sample to obtain a noisy video; initialize an adversarial perturbation with the same size as the original sample, and add the adversarial perturbation to the noisy video to obtain an initialized adversarial sample;

[0037] Input each frame of the initialized adversarial sample into the image model to obtain adversarial sample spatial features;

[0038] According to the segment division result of the original sample, a plurality of segments in the initialization adversarial sample are obtained; in each segment of the initialization adversarial sample, a first video frame and a second video frame are randomly selected to obtain two corresponding video frames to be fused; and the first video frame and the second video frame are fused to obtain a fused video frame;

[0039] The fused video frame is input into an image model to obtain adversarial sample temporal features of the fused video frame.

[0040] In a second aspect, the present invention further provides a device for generating a black-box adversarial sample, comprising:

[0041] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the method for generating black-box adversarial samples described in the first aspect.

[0042] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the method for generating black-box adversarial samples described in the first aspect.

[0043] In a fourth aspect, a computer program product comprising instructions is provided. When the instructions are executed on a computer or a processor, the computer or the processor executes a method for generating black-box adversarial samples as described in the first to third aspects and any one of the above.

[0044] In a fifth aspect, the present invention also provides a black-box adversarial sample generation system, comprising a black-box adversarial sample generation device as in the second aspect, and using the black-box adversarial sample generation method as described in the first aspect to complete the interaction of the black-box adversarial sample generation device of the second aspect.

[0045] Different from the prior art, the present invention has at least the following beneficial effects:

[0046] The present invention uses the information in the shallow, middle and deep feature maps of each frame in the original sample to generate adversarial samples. It not only takes into account the instantaneous changes between video clips, but also obtains features from different levels of the image model, thereby improving the attack success rate of the target adversarial sample on the video feature information in the original sample. The content of the original sample often exists in multiple clips, and there is no certain time correlation between different clips. The present invention generates the target adversarial sample by extracting the original time features, generating and using the adversarial sample time features. While taking into account the time jumps in the video content in different clips, it realizes the capture of the time clues in the original sample, and enables the target adversarial sample to accurately destroy the time continuity of the original sample, thereby solving the problem of poor practicality of the attack method that directly uses the image model as the video proxy model. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 It is a flowchart of a method for generating a black-box adversarial sample provided by an embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of a specific example of a method for generating a black-box adversarial sample provided by an embodiment of the present invention;

[0050] Figure 3 It is a schematic diagram of a process of generating a target adversarial sample provided by an embodiment of the present invention;

[0051] Figure 4 is a flow chart of step 10 provided in an embodiment of the present invention;

[0052] Figure 5 is a schematic diagram of a specific example of using an intra-segment shuffle operation provided by an embodiment of the present invention;

[0053] Figure 6 is a flowchart of step 103 provided by an embodiment of the present invention;

[0054] Figure 7 is a flow chart of step 20 provided in an embodiment of the present invention;

[0055] Figure 8 is a flow chart of step 30 provided in an embodiment of the present invention;

[0056] Fig. 9 is a flowchart of step 302 provided by an embodiment of the present invention;

[0057] Fig.10 It is a schematic diagram of the architecture of a device for generating black-box adversarial samples provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] Unless the context requires otherwise, throughout the specification and claims, the term "including" is to be interpreted as open inclusion, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "examples", "specific examples" or "some examples" and the like are intended to indicate that specific features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner, that is, although they may be carried in the embodiments or examples of the above terms due to reasons such as the order and position of appearance, it is not limited to that they can be carried in combination by one embodiment or example.

[0060] In the description of the present invention, it is necessary to understand that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present disclosure.

[0061] In the description of the present invention, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more. In addition, for example, the same type of nouns may be described as two independent individuals by adding "A" and "B" at the end. In this case, the corresponding features defined as "A" and "B" are only used to distinguish the same type of individuals for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features.

[0062] When describing some embodiments, the expressions "coupling", "coupling" and "connection" and their derivatives may be used. For example, when describing some embodiments, the term "connection" may be used to indicate that two or more components are in direct physical or electrical contact with each other. For another example, when describing some embodiments, the term "coupling" may be used to indicate that two or more components are in direct physical or electrical contact. However, the terms "connection" or "coupling" may also refer to two or more components that are not in direct contact with each other, but still cooperate or interact with each other, such as "optical path coupling", "wireless connection", etc. The embodiments disclosed herein are not necessarily limited to the contents of the present invention.

[0063] In the description of the present invention, the expression "A and / or B" (where A and B are used to formally represent specific characteristic contents) will be involved, and the corresponding expressions include the following three combinations: only A, only B, and a combination of A and B.

[0064] As used herein, "about," "substantially," or "approximately" includes the stated value and an average value that is within an acceptable range of deviation from the particular value as determined by one of ordinary skill in the art taking into account the measurements in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0065] Embodiment 1:

[0066] In order to solve the above problems, Figure 1 As shown, an embodiment of the present invention provides a method for generating a black box adversarial sample, comprising:

[0067] Step 10: Determine the shallow feature map, the middle feature map and the deep feature map of each frame in the original sample as the original spatial features of the original sample; divide the original sample into multiple segments according to the feature similarity between different frames in the original sample; and randomly shuffle the multiple feature units in each segment to obtain the original time features in each segment.

[0068] The original sample is video data. Since the network model structures of the video classification model and the image model are both deep neural networks, and the output features of the intermediate layers of the two models have certain similarities, the cross-model attack method based on the image model has certain feasibility; the cross-model attack method refers to: using the image model trained on a large-scale image dataset as a proxy model to attack the video classification model.

[0069] The feature maps output by different layers in the image model and the video classification model include different information. Using different layers for attacks will produce different effects. Therefore, the embodiment of the present invention extracts feature maps of different levels of each frame in the original sample, so as to utilize the information therein to generate adversarial samples for the corresponding level characteristics in the video classification model.

[0070] Video content often contains multiple clips, and there is no certain time association between different clips; for example, in an edited video, two unrelated pictures are often edited into the same video. For example, the previous second of the video content is someone talking, and the next second is a picture of an object or scenery. There is no time association between these two seconds. The embodiment of the present invention captures the time features in the video content by dividing the clips and randomly shuffling the multiple feature units in each clip. Other ways of obtaining the time features will be provided below.

[0071] The method for generating black-box adversarial samples in an embodiment of the present invention does not require additional training of a video proxy model, but instead uses an image model pre-trained on a large-scale dataset to generate target adversarial samples for a video classification model.

[0072] Among them, the method of obtaining the shallow feature map, the middle feature map and the deep feature map is selected by the technicians in this field according to the specific usage scenario; in an optional embodiment, each frame of the original sample can be processed using an image model that implements the classification task, and the corresponding shallow feature map, middle feature map and deep feature map are obtained from the image model; the shallow feature map can be edge texture information, the middle feature map can be abstract semantic information, and the deep feature map can be high-order semantic information. The specific method of calculating feature similarity and randomly shuffling multiple feature units in each segment will be described below. The specific type of the image model of the embodiment of the present invention is selected by the technicians in this field according to the specific usage scenario and is not limited here; the image model in the prior art is used to classify image data, and the embodiment of the present invention uses the image model to extract information of the video frame.

[0073] Step 20: Preprocess the original sample to obtain an initialized adversarial sample; determine the shallow feature map, the middle feature map and the deep feature map of each frame in the initialized adversarial sample as the adversarial sample spatial features of the initialized adversarial sample; randomly obtain two video frames to be fused in each segment of the initialized adversarial sample, perform pairwise fusion operations on the two video frames to be fused, and obtain the adversarial sample temporal features in each segment.

[0074] Among them, the initialized adversarial sample is video data; the initialized adversarial sample is obtained by adding adversarial perturbations to the original sample; the initialized adversarial sample can be divided into multiple segments according to the segment division of the original sample; each segment obtained in step 10 includes multiple feature units. The embodiment of the present invention performs pairwise fusion of the video frame image in the initialized adversarial sample (i.e., a video frame to be fused) with another random video frame image in the segment (i.e., another video frame to be fused) to randomly shuffle the feature units in each segment, so as to improve the aggressiveness of the target adversarial sample. The specific method of the pairwise fusion operation will be described below.

[0075] Step 30: Calculate the cosine similarity between the original spatial feature and the spatial feature of the adversarial sample to obtain the spatial loss function value; calculate the cosine similarity between the original temporal feature and the temporal feature of the adversarial sample to obtain the temporal loss function value; iteratively optimize the initialized adversarial sample according to the spatial loss function value and the temporal loss function value to obtain the target adversarial sample.

[0076] The present invention uses the information in the shallow, middle and deep feature maps of each frame in the original sample to generate adversarial samples. It not only takes into account the instantaneous changes between video clips, but also obtains features from different levels of the image model, thereby improving the attack success rate of the target adversarial sample on the video feature information in the original sample. The content of the original sample often exists in multiple clips, and there is no certain time correlation between different clips. The present invention generates the target adversarial sample by extracting the original time features, generating and using the adversarial sample time features. While taking into account the time jumps in the video content in different clips, it realizes the capture of the time clues in the original sample, and enables the target adversarial sample to accurately destroy the time continuity of the original sample, thereby solving the problem of poor practicality of the attack method that directly uses the image model as the video proxy model.

[0077] The overall process of generating a target adversarial sample in the embodiment of the present invention is as follows: Figure 2 and Figure 3 As shown below. The following first introduces the process of generating original spatial features and original temporal features in detail. Figure 4 As shown, the step 10 includes:

[0078] Step 101: Input each frame in the original sample into the image model to obtain edge texture information, abstract semantic information and high-order semantic information of the corresponding frame, use the edge texture information as the shallow feature map of each frame in the original sample, use the abstract semantic information as the middle feature map of each frame in the original sample, and use the high-order semantic information as the deep feature map of each frame in the original sample to obtain the original spatial features.

[0079] In the embodiment of the present invention, the original sample is first divided into multiple frames according to its original frame rate. T×C×H×W ,V i represents the i-th frame in the original sample, V i ∈R C×H×W , C represents the number of channels of the frame, H represents the height of the frame, W represents the width of the frame, and T represents the total number of frames of the original sample. Use the image model g(·) trained on a general image dataset ImageNet to generate target adversarial samples for the video classification model; i It is sent to the image model g(·), which extracts feature maps of different layers. Represents the Lth frame of the i-th frame extracted from the image model jLayer features, where L = {l1, l2, l3} represents a list of selected layers. The embodiment of the present invention divides the total number of layers of the image model into three sections, the first section is the shallow feature (edge ​​texture information of the frame), the second section is the middle feature (abstract semantic information of the frame), and the third section is the deep feature (high-order semantic information of the frame). A total of 3 layers are selected for feature attack, where l1 is used to obtain the shallow feature map, l2 is used to obtain the middle feature map, and l3 is used to obtain the deep feature map.

[0080] The embodiment of the present invention uses an image model trained on a large-scale data set to generate video adversarial samples, reducing the additional overhead of training a video recognition proxy model. Since there is no need to train an additional video proxy model, the consumption of computing resources is significantly reduced, and the stability of the attack effect of the generated adversarial samples is improved. At the same time, since the multi-layer feature space of the image model is used to extract the information contained in the corresponding layer, the attack success rate of the target adversarial samples finally generated in different video classification models is improved.

[0081] Step 102: Calculate the cosine similarity between any two frames in the original sample; when the cosine similarity is greater than the similarity value, determine the corresponding two frames as belonging to the same segment, so as to divide the original sample into multiple segments.

[0082] Among them, the formula for cosine similarity is: A represents any frame in the original sample, and B represents another frame in the original sample. The similarity value is selected by a person skilled in the art according to a specific usage scenario; in an optional embodiment, the similarity value may be 0.9, that is, when the cosine similarity between two frames is greater than 0.9, the two frames are determined to be in the same segment.

[0083] like Figure 2 As shown on the right side of the figure, it is determined whether any two frames are in the same clip. For example, the original sample V = Clip (V1, V2, ..., V n ), n is the total number of segments divided, V i Represents the i-th segment divided; among them, a single segment represents the jth frame in the ith segment, T i represents the total number of frames in the i-th segment. Among them, cosine similarity

[0084] Step 103: randomly replace the feature units in each segment with feature units at corresponding positions in other frames in the same segment to obtain the original time features; wherein each segment includes multiple feature units.

[0085] The embodiment of the present invention uses cosine similarity to measure the feature similarity between the original spatial features of different frames in the original sample; then, the original spatial features in each segment are shuffled at the channel level using an intra-segment shuffling operation, that is, the feature units in each segment are randomly replaced with feature units at corresponding positions in other frames in the same segment. Figure 5 The left side of the figure shows multiple segments divided from the original sample. A row represents a segment. The multiple blocks on each row are multiple feature units. The feature units in the same row are in the same segment. Figure 5 The middle right side shows the original temporal features obtained after the intra-fragment shuffle operation.

[0086] In an optional embodiment, the partitioning parameter and the replacement number parameter are set; Figure 6 As shown, step 103 includes:

[0087] Step 1031: Divide each frame in the segment into multiple feature units on an even basis according to the division parameters.

[0088] When the division parameter of the embodiment of the present invention is a and the replacement number parameter is b, the original spatial features of each frame are first divided into a segments evenly according to the number of channels. For example, when a is set to 3, the features of each frame are evenly divided into 3 segments according to the number of channels to obtain a corresponding plurality of feature units.

[0089] Step 1032: Determine the first b feature units in the corresponding frame in order from front to back; wherein b represents the replacement number parameter.

[0090] Then, segment b is selected from segment a and randomly replaced with the original spatial features of the corresponding segments in other frames in the current segment, b≤a. For example, when a is 3 and b is 2, Figure 5 As shown, for each horizontal row of feature units, the first b feature units in the corresponding frame are determined from left to right.

[0091] Step 1033: Randomly select another frame in the segment, and determine the first b feature units in the other frame.

[0092] Step 1034: Replace the first b feature units in the corresponding frame with the first b feature units in the other frames to obtain the corresponding original time features.

[0093] like Figure 5 As shown, the first and second segment features are randomly replaced with the first and second segment features of other frames in the current segment.

[0094] The original spatial features after being disrupted not only have their own original features but also have the original spatial features of other frames in the same segment; therefore, the embodiment of the present invention uses the disrupted original spatial features in the segment as the original temporal features.

[0095] When using the image model to generate the target adversarial sample, the embodiment of the present invention takes into account the connection between multiple layers of features and the temporal continuity of the video data. By calculating the feature similarity and dividing the segments, the temporal clues in the original sample are captured and the required computing resources are reduced; considering the time jumps in different segments, the intra-segment shuffling operation is used to generate adversarial samples that accurately destroy the temporal continuity of the video, further improving the attack success rate of the video adversarial sample.

[0096] In order to give a detailed introduction to the process of generating adversarial sample spatial features and adversarial sample temporal features, Figure 7 As shown, the step 20 includes:

[0097] Step 201: Add a random Gaussian noise to the original sample to obtain a noisy video; initialize an adversarial perturbation with the same size as the original sample, and add the adversarial perturbation to the noisy video to obtain an initialized adversarial sample.

[0098] First, add a random Gaussian noise to the original sample, and define the video after adding the noise as the noisy video. By adding random Gaussian noise, the target adversarial sample has higher robustness. Then, initialize an adversarial perturbation with the same size as the original sample, and add the adversarial perturbation to the noisy video, and define the added noisy video as the initialized adversarial sample; since the added adversarial perturbation needs to be iteratively optimized based on the initialized adversarial sample in the future, the number of iterations is set to 0 at this time.

[0099] For example, the adversarial sample consists of the original sample and the adversarial perturbation δ, that is, V adv =V+δ. The size of the perturbation needs to be consistent with the original sample, δ∈R T×C×H×W , where δ i ∈R C×H×W represents the disturbance of the i-th frame, i = {1, 2, 3, ..., T}. In this embodiment, v is first set to (16 / 255) T×C×H×W Used to initialize adversarial samples. If the number of iterations is not reached each time, a random Gaussian noise μ∈R is generated. T×C×H×W , add μ to V adv middle.

[0100] Step 202: Input each frame of the initialized adversarial sample into the image model to obtain adversarial sample spatial features.

[0101] First, the initialized adversarial sample is divided into multiple frames according to the original frame rate of the original sample and input into the image model. The image model processes each frame in the initialized adversarial sample in the same way as each frame in the original sample. The image model extracts the edge texture information of each frame in the initialized adversarial sample as the shallow feature map of the frame; extracts the abstract semantic information of each frame in the initialized adversarial sample as the middle feature map of the frame; extracts the high-order semantic information of each frame in the initialized adversarial sample as the deep feature map of the frame.

[0102] Step 203: According to the segment division result of the original sample, multiple segments in the initialized adversarial sample are obtained; in each segment of the initialized adversarial sample, a first video frame and a second video frame are randomly selected to obtain two corresponding video frames to be fused; the first video frame and the second video frame are fused to obtain a fused video frame.

[0103] In an optional embodiment, the fusion ratio of the first video frame and the second video frame may be 1:1.

[0104] Combine the following Figure 5 The following describes the fusion operation:

[0105] First, the video V (i.e., the original sample) is divided into multiple clips according to the feature similarity between different video frames, and V = Clip (V1, V2, ..., V n ), where n represents the total number of segments that the video is divided into, V n Indicates fragment n, a single fragment represents the Tth n Video frame, T n Indicates the total number of video frames in this segment.

[0106] The embodiment of the present invention uses feature similarity to distinguish segments. The similarity within the same segment is greater than a threshold, that is, in is the hyper-parameter threshold; through pairwise fusion operations, spatiotemporal information modeling is achieved only within the fragment; the embodiment of the present invention replaces the image translation operation in the prior art by randomly disrupting the video frames to obtain long-distance time information.

[0107] When the image model g in the training process l (V)=M∈R T×C×H×W , where T represents the number of video frames of video V, C represents the number of feature channels of the middle layer of the model network, M represents the output of the middle layer of the model network, H represents the height of the feature map, and W represents the width of the feature map. i =[c i1 ,c i2 ,ci3 ,……,c iC ]=[c i1 …c ik ,c ik+1 …c i2k ,c i2k+1 …c iC ], where c ik represents the kth channel of the i-th video frame, whose size is H×W, k is the reference length of the random shuffle operation, k=floor(C÷p); wherein C represents the total length of the channel, p represents a preset hyperparameter, and the specific value of the preset hyperparameter is selected by technical personnel in this field according to the specific usage scenario and experience, and floor(·) represents the rounding down operation after integer division.

[0108] After random shuffling, the following features are obtained:

[0109] IND1,IND2=shuffle(1,2,3,...,T n )

[0110] Get the modeled fused video frame

[0111]

[0112] in, represents the i-th video frame in segment n.

[0113] In short, the embodiments of the present invention will Figure 5 The specific instance of the original sample on the left is divided into P segments by channel, and then the first several (parameter) segments are randomly shuffled. The fused video features obtained in this way not only retain the feature information of the video frame in the original sample, but also integrate the feature information of other video frames in the segment. The pictures in each segment are basically consistent and have time information. According to the method of the embodiment of the present invention, the correlation information on the time dimension in the original sample can be quickly obtained. At the same time, a video adversarial sample that can also destroy the time information is generated, which greatly reduces the time cost.

[0114] Step 204: Input the fused video frame into an image model to obtain adversarial sample temporal features of the fused video frame.

[0115] The image model's processing of the fused video frames is the same as that of the original samples and each frame in the initialized adversarial samples, which will not be repeated here.

[0116] After obtaining the adversarial sample spatial features and adversarial sample temporal features, the embodiment of the present invention further calculates the cosine similarity between the original spatial features and the adversarial sample spatial features to obtain the spatial loss function value. The calculation formula of the spatial loss function value is:

[0117]

[0118] Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; represents the Lth extracted i-th frame j Layer feature map, j = {1, 2, 3}, the L1th layer feature map is the shallow feature map, the L2th layer feature map is the middle feature map, and the L3th layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; (V i +δ i ) represents the initialization of adversarial samples; δ i Represents the adversarial perturbation added to the i-th frame in the original sample.

[0119] In order to ensure that the human eye cannot observe the existence of the adversarial perturbation δ, it is necessary to make certain restrictions on δ; in an optional embodiment, the restriction ‖δ‖ ∞ ≤(16 / 255), where ‖·‖ ∞ Represents L p norm, and st is a mathematical symbol representing the constraint.

[0120] By using the above calculation formula, the similarity between the feature space of the adversarial sample and the feature space of the original sample can be made smaller and smaller, that is, the adversarial sample (or intermediate adversarial sample) is initialized by optimizing the orthogonal part.

[0121] The embodiment of the present invention further calculates the cosine similarity between the original time feature and the adversarial sample time feature to obtain the time loss function value; the calculation formula of the time loss function value is:

[0122]

[0123] Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; represents the Lth extracted i-th frame jLayer feature map, j = {1, 2, 3}, the L1th layer feature map is the shallow feature map, the L2th layer feature map is the middle feature map, and the L3th layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; shuffle (V i ) represents the original time feature of the i-th frame after random shuffling; H(V i +δ i ) represents the fused video frame; (V i +δ i ) represents the initialization of adversarial samples; δ i Represents the adversarial perturbation added to the i-th frame in the original sample.

[0124] In an optional embodiment, similarly, the restriction when calculating the spatial loss function value is limited to ‖δ‖ ∞ ≤(16 / 255), where ‖·‖ ∞ Represents L p Norm.

[0125] By using the above calculation formula, the similarity between the temporal features of the adversarial sample and the temporal features of the original sample can be made smaller and smaller, thereby completing the destruction of the temporal continuity of the video.

[0126] The embodiment of the present invention achieves the effect of attacking the features in the original sample in space and time by performing orthogonality in both space and time.

[0127] After obtaining the spatial loss function value and the temporal loss function value, as Figure 8 As shown, in step 30, the iterative optimization of the initialized adversarial sample according to the spatial loss function value and the temporal loss function value to obtain the target adversarial sample includes:

[0128] Step 301: Determine the sum of the spatial loss function value and the temporal loss function value as a comprehensive loss function value.

[0129] In order to generate comprehensive adversarial samples that attack both feature space and time features, the spatial loss function value and the temporal loss function value are used for joint constraints to comprehensively optimize the adversarial perturbation. The final calculation formula for the comprehensive loss function value is as follows:

[0130] L=L c +L t

[0131] Through the comprehensive loss function value and the adaptive learning rate (ADAM) optimizer, the noise in the initialized adversarial sample can be optimized in a direction that is destructive to the feature space and time continuity of the original sample.

[0132] Step 302: Iteratively optimize the initialized adversarial sample according to the comprehensive loss function value until the number of iterations reaches a threshold.

[0133] Add the optimized noise to the initialized adversarial sample to generate a new intermediate adversarial sample for this iteration, and increase the number of iterations by 1.

[0134] Step 303: Determine the adversarial sample obtained in the last iteration as the target adversarial sample.

[0135] The threshold is a hyperparameter that controls the number of iterations, and the threshold is selected by a person skilled in the art according to a specific usage scenario; in an optional embodiment, the threshold may be 60.

[0136] like Fig. 9 As shown, step 302 includes:

[0137] Step 3021: Use the ADAM optimizer to adjust the iterative optimization parameters based on the comprehensive loss function value to obtain optimized noise; wherein the iterative optimization parameters are control parameters for optimizing the intermediate adversarial samples obtained in the previous round.

[0138] Step 3022: Add the optimized noise to the original sample to obtain an intermediate adversarial sample.

[0139] The specific usage of the ADAM optimizer is determined by those skilled in the art based on specific usage scenarios and experience.

[0140] The method for generating black-box adversarial samples in the embodiment of the present invention is applicable to different video classification models and has good generalization ability. In addition, the generated target adversarial samples can not only be used to attack video classification models, but also help researchers discover the vulnerabilities of image models, thereby enhancing the defense capability of video classification models and improving their reliability and security in practical applications.

[0141] The present invention adds adversarial perturbations to attack different feature levels and time continuity of original samples. While ensuring a high success rate of black-box attacks using target adversarial samples, it reduces the visual difference between target adversarial samples and original samples and improves the generation rate of target adversarial samples.

[0142] Embodiment 2:

[0143] like Fig.10 , which is a schematic diagram of the architecture of a device for generating a black-box adversarial sample according to an embodiment of the present invention. The device for generating a black-box adversarial sample according to this embodiment includes one or more processors 21 and a memory 22. Fig.10 A processor 21 is taken as an example.

[0144] The processor 21 and the memory 22 may be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0145] The memory 22 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs and non-volatile computer executable programs, such as the method for generating black-box adversarial samples in this embodiment. The processor 21 executes the method for generating black-box adversarial samples by running the non-volatile software programs and instructions stored in the memory 22.

[0146] The memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 22 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0147] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, the method for generating black-box adversarial samples in the above-mentioned embodiment is executed, for example, each step of the method for generating black-box adversarial samples in the embodiment of the present invention described above is executed.

[0148] An embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, such as Fig.10 A processor 21 may enable the one or more processors to execute the method for generating a black-box adversarial sample in a specific embodiment of the present invention, for example, to execute each step of the method for generating a black-box adversarial sample in the embodiment of the present invention described above; and may also implement Fig.10 The various modules and units described above; or executing the method for generating a black-box adversarial sample in a specific embodiment of the present invention, for example, executing the various steps of the method for generating a black-box adversarial sample in the embodiment of the present invention described above; it can also be implemented Fig.10 The various modules and units described.

[0149] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0150] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0151] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating black-box adversarial samples, characterized in that: include: Determine the shallow feature map, the middle feature map and the deep feature map of each frame in the original sample as the original spatial feature of the original sample; divide the original sample into multiple segments according to the feature similarity between different frames in the original sample; perform random shuffling operation on multiple feature units in each segment to obtain the original time feature in each segment; Preprocessing the original sample to obtain an initialized adversarial sample; determining a shallow feature map, a middle feature map, and a deep feature map of each frame in the initialized adversarial sample as adversarial sample spatial features of the initialized adversarial sample; Randomly obtain two video frames to be fused in each segment of the initialized adversarial sample, perform a pairwise fusion operation on the two video frames to be fused, and obtain a temporal feature of the adversarial sample in each segment; The cosine similarity between the original spatial feature and the spatial feature of the adversarial sample is calculated to obtain a spatial loss function value; the cosine similarity between the original temporal feature and the temporal feature of the adversarial sample is calculated to obtain a temporal loss function value; and according to the spatial loss function value and the temporal loss function value, the initialized adversarial sample is iteratively optimized to obtain a target adversarial sample.

2. The method for generating black-box adversarial samples according to claim 1, characterized in that: The shallow feature map, the middle feature map and the deep feature map of each frame in the original sample are determined as the original spatial features of the original sample; and the original sample is divided into a plurality of segments according to the feature similarity between different frames in the original sample; The multiple feature units in each segment are randomly shuffled to obtain the original time features in each segment, including: Input each frame in the original sample into the image model to obtain edge texture information, abstract semantic information and high-order semantic information of the corresponding frame, use the edge texture information as a shallow feature map of each frame in the original sample, use the abstract semantic information as a middle feature map of each frame in the original sample, and use the high-order semantic information as a deep feature map of each frame in the original sample to obtain the original spatial feature; Calculating the cosine similarity between any two frames in the original sample; when the cosine similarity is greater than the similarity value, determining the corresponding two frames as belonging to the same segment, so as to divide the original sample into a plurality of segments; The feature units in each segment are randomly replaced with feature units at corresponding positions in other frames in the same segment to obtain the original time features; wherein each segment includes multiple feature units.

3. The method for generating black-box adversarial samples according to claim 2, characterized in that: Set the division parameters and replacement number parameters; The step of randomly replacing the feature units in each segment with feature units at corresponding positions in other frames in the same segment comprises: According to the division parameters, each frame in the segment is evenly divided into a plurality of feature units; Determine the first b feature units in the corresponding frame in order from front to back, wherein b represents the replacement number parameter; Randomly select another frame in the segment, and determine the first b feature units in the another frame; The first b feature units in the corresponding frame are replaced by the first b feature units in the other frames to obtain the corresponding original time features.

4. The method for generating a black-box adversarial sample according to claim 1, characterized in that: The iteratively optimizing the initialized adversarial sample according to the spatial loss function value and the temporal loss function value to obtain a target adversarial sample includes: Determine the sum of the spatial loss function value and the temporal loss function value as a comprehensive loss function value; Iteratively optimizing the initialized adversarial sample according to the comprehensive loss function value until the number of iterations reaches a threshold; The adversarial sample obtained in the last iteration is determined as the target adversarial sample.

5. The method for generating a black-box adversarial sample according to claim 4, characterized in that: The calculation formula of the spatial loss function value is: Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; represents the Lth extracted i-th frame j Layer feature map, j = {1, 2, 3}, the L1th layer feature map is the shallow feature map, the L2th layer feature map is the middle feature map, and the L3th layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; (V i +δ i ) represents the initialization of adversarial samples; δ i Represents the adversarial perturbation added to the i-th frame in the original sample.

6. The method for generating black-box adversarial samples according to claim 4, characterized in that: The calculation formula of the time loss function value is: Among them, V i represents the i-th frame in the original sample, T i represents the total number of frames of segment i; represents the Lth extracted i-th frame j Layer feature map, j = {1, 2, 3}, the L1th layer feature map is the shallow feature map, the L2th layer feature map is the middle feature map, and the L3th layer feature map is the deep feature map; CosSim (x, y) represents the cosine similarity between the vector x and the vector y; shuffle (V i ) represents the original time feature of the i-th frame after random shuffling; H(V i +δ i ) represents the fused video frame; (V i +δ i ) represents the initialization of adversarial samples; δ i Represents the adversarial perturbation added to the i-th frame in the original sample.

7. The method for generating a black-box adversarial sample according to claim 4, characterized in that: The iterative optimization of the initialized adversarial sample according to the comprehensive loss function value until the number of iterations reaches a threshold comprises: Using the ADAM optimizer to adjust the iterative optimization parameters based on the comprehensive loss function value to obtain optimized noise; wherein the iterative optimization parameters are control parameters for optimizing the intermediate adversarial samples obtained in the previous round; The optimized noise is added to the original sample to obtain an intermediate adversarial sample.

8. The method for generating a black-box adversarial sample according to claim 1, characterized in that: The original sample is preprocessed to obtain an initialized adversarial sample; a shallow feature map, a middle feature map, and a deep feature map of each frame in the initialized adversarial sample are determined as adversarial sample spatial features of the initialized adversarial sample; Randomly obtain two video frames to be fused in each segment of the initialized adversarial sample, perform a pairwise fusion operation on the two video frames to be fused, and obtain the adversarial sample time features in each segment, including: Add a random Gaussian noise to the original sample to obtain a noisy video; initialize an adversarial perturbation with the same size as the original sample, and add the adversarial perturbation to the noisy video to obtain an initialized adversarial sample; Input each frame of the initialized adversarial sample into the image model to obtain adversarial sample spatial features; According to the segment division result of the original sample, a plurality of segments in the initialization adversarial sample are obtained; in each segment of the initialization adversarial sample, a first video frame and a second video frame are randomly selected to obtain two corresponding video frames to be fused; and the first video frame and the second video frame are fused to obtain a fused video frame; The fused video frame is input into an image model to obtain adversarial sample temporal features of the fused video frame.

9. A device for generating black-box adversarial samples, characterized in that: The device for generating black-box adversarial samples includes at least one processor and a memory, and the at least one processor and the memory are connected via a data bus. The memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to implement the method for generating black-box adversarial samples described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method for generating black-box adversarial samples described in any one of claims 1-8.

Images