Gradient estimation method and device during adversarial sample generation in video model boundary black box

Abstract

The invention discloses a gradient estimation method and device during adversarial sample generation in a video model boundary black box. The method comprises the following steps: determining the number of gradient estimation samples based on the current access times to a video model and the preset maximum access times allowed by the video model during adversarial sample generation in the boundary black box; performing Gaussian distribution processing on the gradient estimation sample, sampling to obtain noise, generating a test sample based on the noise and the sample, and determining a boundary feedback result of the video model according to the test sample; adding one to the current access times of the video model to obtain corresponding test samples and boundary feedback results, and repeating a preset number of times to obtain multiple groups of test samples and corresponding boundary feedback results; and determining a gradient based on noise and boundary feedback results corresponding to all the test samples. The method can reduce the deviation of gradient estimation, thereby reducing the noise of an adversarial video, and improving the quality of the adversarial video.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a method and device for estimating gradients when generating adversarial samples in a boundary black box of a video model. Background Art

[0002] In recent years, neural networks have been widely used in video models, but they are easily affected or interfered by adversarial samples, which causes the video model to be greatly disturbed when performing classification predictions and the prediction results to be seriously biased. Therefore, research in related fields is of great significance. Adversarial samples in the boundary black box are generated by continuously querying the final classification label of the video model to obtain information. Among them, in generating adversarial samples, gradient estimation is a relatively important step, which can determine the update direction of the generated adversarial video, that is, the adversarial sample. In the existing technology, when performing gradient estimation, the gradient estimation strategy cannot be dynamically adjusted according to the generation process of the adversarial sample, thereby generating more noise when generating adversarial videos, which ultimately leads to low quality of the adversarial video.

[0003] Therefore, how to reduce the deviation of gradient estimation, thereby reducing the noise of adversarial videos and improving the quality of adversarial videos, is a technical problem to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of the present invention is to solve the technical problem in the prior art that the deviation in gradient estimation is large, which leads to more noise in the adversarial video.

[0005] To achieve the above technical objectives, on the one hand, the present invention provides a method for estimating gradients when generating adversarial samples in a black box boundary of a video model, the method comprising: Determine the number of gradient estimation samples based on the current number of visits to the video model when generating the boundary black box adversarial sample and the maximum number of visits allowed by the preset video model; Performing Gaussian distribution processing on the gradient estimation sample and sampling to obtain noise, generating a test sample based on the noise and the sample, and then determining a boundary feedback result of the video model based on the test sample; After adding one to the current number of visits to the video model, the corresponding test samples and boundary feedback results are obtained, and after repeating the process for a preset number of times, multiple groups of the test samples and their corresponding boundary feedback results are obtained; The gradient is determined based on the noise and boundary feedback results corresponding to all test samples.

[0006] Furthermore, the gradient is determined based on the noise and boundary feedback results corresponding to all test samples, specifically including: Determining the impact factor of the corresponding noise according to the noise and boundary feedback results corresponding to each group of test samples; All influencing factors are averaged, and the average is used as the gradient.

[0007] Furthermore, the influence factor corresponding to the noise is determined specifically by the following formula: , Where, z is the impact factor, y Feedback result for the boundary, For noise.

[0008] Furthermore, the test samples are generated by the following formula: , Where, For the test sample, x is the gradient estimation sample, is the noise amplification factor, For noise.

[0009] Furthermore, the boundary feedback result is specifically determined by the following formula: , Where, y Feedback result for the boundary, For the video model, is a test sample, where the boundary feedback result is the boundary value of the original probability value, that is, 1 or -1.

[0010] Furthermore, the number of gradient estimation samples is determined based on the current number of visits to the video model and the maximum number of visits allowed by the preset video model when generating the boundary black box adversarial sample, and is specifically determined by the following formula: , Where, N is the number of samples for gradient estimation, is the lower limit of the mean parameter, q is the number of visits to the video model during adversarial sample generation, m is the mean parameter, r is the scale parameter, Q The maximum number of accesses allowed for a video model.

[0011] Furthermore, the lower limit of the mean parameter is determined by the following formula: , The ratio parameter is specifically used to calculate the lower limit of the mean parameter.

[0012] On the other hand, the present invention also provides a device for estimating gradients when generating adversarial samples in a black box boundary of a video model, the device comprising: A first determination module is used to determine the number of gradient estimation samples based on the current number of visits to the video model when generating the boundary black box adversarial sample and the maximum number of visits allowed by the preset video model; A second determination module is configured to perform Gaussian distribution processing on the gradient estimation sample and sample it to obtain noise, generate a test sample based on the noise and the sample, and then determine a boundary feedback result of the video model based on the test sample; A repetition module is used to obtain corresponding test samples and boundary feedback results after adding one to the current number of visits to the video model, and to obtain multiple groups of the test samples and their corresponding boundary feedback results after repeating a preset number of times; The third determination module is used to determine the gradient based on the noise and boundary feedback results corresponding to all test samples.

[0013] The present invention provides a method and device for estimating gradients when generating adversarial samples in a boundary black box of a video model. Compared to existing technologies, this method determines the number of gradient estimation samples based on the current number of video model accesses and the preset maximum number of accesses allowed by the video model when generating boundary black box adversarial samples. The gradient estimation samples are subjected to Gaussian distribution processing and sampled to obtain noise. Test samples are generated based on the noise and samples, and then boundary feedback results of the video model are determined based on the test samples. The current number of video model accesses is incremented by one to obtain corresponding test samples and boundary feedback results. This is repeated a preset number of times to obtain multiple groups of test samples and their corresponding boundary feedback results. Finally, the gradient is determined based on the noise and boundary feedback results corresponding to all test samples. This method can reduce the deviation of gradient estimation, thereby reducing the noise of the adversarial video and improving the quality of the adversarial video. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 1 is a flow chart of a method for estimating gradients when generating adversarial samples in a black box within a video model boundary, as provided in an embodiment of this specification; Figure 2 The figure shows a schematic diagram of the structure of the gradient estimation device for generating adversarial samples in the boundary black box of the video model provided by an embodiment of this specification. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0017] like Figure 1 The flowchart of the gradient estimation method for generating adversarial samples in the black box of the boundary of the video model provided by the embodiment of this specification is shown. Although this specification provides the method operation steps or device structure shown in the following embodiments or drawings, the method or device may include more or fewer operation steps or module units after partial merger based on routine or no creative labor. In the steps or structures that do not have a necessary causal relationship logically, the execution order of these steps or the module structure of the device is not limited to the execution order or module structure shown in the embodiments or drawings of this specification. When the method or module structure is applied in actual devices, servers or terminal products, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or drawings (for example, in a parallel processor or multi-threaded processing environment, or even in a distributed processing or server cluster implementation environment).

[0018] Bounded black-box adversarial sample generation is an algorithm that generates samples by testing a model. It mainly consists of four steps: Step 1: Find boundary adversarial samples: In short, the algorithm finds a sample that makes the model more sensitive. For example, if you want to find a boundary adversarial sample for a cat picture, you can take a dog picture and add some pixels from the dog picture to the cat picture. In this way, the modified picture has both cat and dog features. The model is more sensitive to this sample and produces an "ambiguous" feeling. Step 2: Gradient estimation: The algorithm randomly adds some noise to the boundary adversarial examples. At this time, the model's classification of the noisy samples will randomly change. Sometimes it is classified as a cat, and sometimes it is classified as a dog. After repeated testing, the algorithm knows in which direction to add noise to make the model classify the image as a dog. This direction is the gradient. Step 3: Sample update: Based on the gradient calculated in the second step, the boundary adversarial sample generated in the first step can be updated. The updated sample is called an adversarial sample. After the update, the adversarial sample will make the model more inclined to classify it as a dog.

[0019] Step 4: Repeat steps 1 to 3 until the generated adversarial example looks very much like a cat, but the model classifies it as a dog. Then the algorithm achieves its goal.

[0020] The present application solution is an improvement to the gradient estimation in step 2.

[0021] The gradient estimation method for generating adversarial samples in the video model boundary black box provided in the embodiment of this description can be applied to terminal devices such as clients and servers, such as Figure 1 As shown, the method specifically includes the following steps: Step S101: Determine the number of gradient estimation samples based on the current number of visits to the video model when generating the boundary black box adversarial sample and the maximum number of visits allowed by the preset video model.

[0022] Specifically, check Is it true? If so, exit; otherwise, execute the subsequent steps.

[0023] in, q and Q They represent the current number of visits to the video model and the maximum number of model visits specified by the environment. This step is mainly used to check whether the number of times the algorithm visits the model exceeds the limit. If it exceeds the limit, it means that boundary black box testing can no longer be performed, and gradient estimation cannot be performed. Otherwise, you can continue.

[0024] Select a mean parameter m and a scale parameter r, where m is used to calculate the lower bound value and r is the scale used to calculate the lower bound value. The lower bound of the mean parameter is determined by the following formula: , The proportion parameter is specifically used to calculate the lower limit of the mean parameter to ensure the minimum value of the number of samples. By setting the minimum value, performance is guaranteed.

[0025] according to Calculate the number of samples used for this gradient estimate: , Where, N is the number of samples, is the lower limit of the mean parameter, q is the number of visits to the video model during adversarial sample generation, m is the mean parameter, r is the scale parameter, Q The maximum number of accesses allowed for a video model. N The formula can be obtained, the number of samples N along with qThe value of the algorithm gradually increases with the increase of the value of the boundary, and this change is gradual, not fixed or exponential. This reasonable change, combined with the minimum value idea in the previous step, can not only ensure the lower limit of the algorithm performance, but also continuously optimize the algorithm performance by gradually increasing the number of samples as the boundary black box test process proceeds.

[0026] Step S102: Perform Gaussian distribution processing on the gradient estimation sample and sample it to obtain noise, generate a test sample based on the noise and the sample, and then determine the boundary feedback result of the video model according to the test sample.

[0027] Specifically, the test samples are generated by the following formula: , Where, For the test sample, x is the gradient estimation sample, is the noise amplification factor, For noise; It is a fixed parameter that can be adjusted and added to the test sample The intensity of the noise, The larger it is, the stronger the noise added is, and the greater the impact on the sample. The smaller it is, the weaker the added noise is and the smaller the impact on the sample is.

[0028] The boundary feedback result is determined by the following formula: , Where, y Feedback result for the boundary, For the video model, is a test sample, where the boundary feedback result is the boundary value of the original probability value, that is, 1 or -1; y If it is 1, it means that the added noise affects the model’s x If the value is -1, it means that the added noise has no effect on the model x classification.

[0029] The impact factor corresponding to the noise is determined by the following formula: , Where, z is the impact factor, y Feedback result for the boundary, is noise; impact factor z for y and The product of , which represents the impact of this sampling noise on the model, z The larger it is, the more effective the noise of this sampling is; otherwise, it means the noise effect of this sampling is poor.

[0030] Step S103: adding one to the current number of accesses to the video model to obtain corresponding test samples and boundary feedback results, and repeating the process a preset number of times to obtain multiple groups of test samples and their corresponding boundary feedback results.

[0031] Step S104: Determine the gradient based on the noise and boundary feedback results corresponding to all test samples.

[0032] Specifically, this scheme explores the effects of different noises on x The impact factor was calculated for each sampling noise. z , and ultimately multiple impact factors z The mean is the final gradient, which means that this algorithm takes into account the influence of all noises and calculates the optimal noise through the idea of ​​​​mean, that is, the final gradient.

[0033] Finally, after the gradient is determined, the current number of visits to the video model needs to be updated, that is, the number of visits is increased by a preset number.

[0034] Based on the above-mentioned method for estimating gradients when generating adversarial samples in a black box at the boundary of a video model, one or more embodiments of this specification also provide a platform and terminal for estimating gradients when generating adversarial samples in a black box at the boundary of a video model. The platform or terminal may include devices, software, modules, plug-ins, servers, clients, etc. that use the methods described in the embodiments of this specification and are combined with necessary hardware implementation devices. Based on the same innovative concept, the systems in one or more embodiments provided in the embodiments of this specification are as described in the following embodiments. Since the implementation scheme and method for solving the problem of the system are similar, the implementation of the specific system in the embodiments of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. The terms "unit" or "module" used below can be a combination of software and / or hardware that implements predetermined functions. Although the system described in the following embodiments is preferably implemented in software, hardware and a combination of software and hardware are also possible and conceived.

[0035] Specifically, Figure 2 This is a schematic diagram of the module structure of an embodiment of the gradient estimation device for generating adversarial samples in the video model boundary black box provided in this specification, such as Figure 2 As shown, the gradient estimation device for generating adversarial samples in the video model boundary black box provided in this specification includes: A first determination module 201 is configured to determine the number of gradient estimation samples based on the current number of accesses to the video model and the maximum number of accesses allowed by the preset video model when generating the boundary black box adversarial sample; A second determination module 202 is configured to perform Gaussian distribution processing on the gradient estimation sample and sample it to obtain noise, generate a test sample based on the noise and the sample, and then determine a boundary feedback result of the video model based on the test sample; A repetition module 203 is configured to obtain corresponding test samples and boundary feedback results by adding one to the current number of accesses to the video model, and to obtain multiple groups of the test samples and their corresponding boundary feedback results after repeating the process for a preset number of times; The third determination module 204 is configured to determine a gradient based on the noise and boundary feedback results corresponding to all test samples.

[0036] It should be noted that the above-mentioned system may also include other implementation methods according to the description of the corresponding method embodiment. The specific implementation methods can refer to the description of the above-mentioned corresponding method embodiment, and will not be described one by one here.

[0037] An embodiment of the present application further provides an electronic device, including: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the method provided in the above embodiment.

[0038] The electronic device provided by the embodiment of the present application stores executable instructions of a processor in a memory. When the processor executes the executable instructions, it can determine the number of gradient estimation samples based on the current number of accesses to the video model when generating a boundary black box adversarial sample and the maximum number of accesses allowed by the preset video model; Gaussian distribution processing is performed on the gradient estimation samples and sampled to obtain noise, and a test sample is generated based on the noise and sample, and then the boundary feedback result of the video model is determined based on the test sample; the current number of accesses to the video model is increased by one to obtain the corresponding test sample and boundary feedback result, and after repeating the process a preset number of times, multiple groups of the test samples and their corresponding boundary feedback results are obtained; and the gradient is determined based on the noise and boundary feedback results corresponding to all the test samples. This can reduce the deviation of the gradient estimation, thereby reducing the noise of the adversarial video and improving the quality of the adversarial video.

[0039] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0040] The methods or devices described in the above embodiments of this specification can implement business logic through computer programs and record them on storage media. The storage media can be read and executed by a computer to achieve the effects of the solutions described in the embodiments of this specification, such as: Determine the number of gradient estimation samples based on the current number of visits to the video model when generating the boundary black box adversarial sample and the maximum number of visits allowed by the preset video model; Performing Gaussian distribution processing on the gradient estimation sample and sampling to obtain noise, generating a test sample based on the noise and the sample, and then determining a boundary feedback result of the video model based on the test sample; After adding one to the current number of visits to the video model, the corresponding test samples and boundary feedback results are obtained, and after repeating the process for a preset number of times, multiple groups of the test samples and their corresponding boundary feedback results are obtained; The gradient is determined based on the noise and boundary feedback results corresponding to all test samples.

[0041] The storage medium may include a physical device for storing information, typically digitizing the information and then storing it in a medium utilizing electrical, magnetic, or optical means. Examples of such storage media include: devices that store information electrically, such as various types of memory devices like RAM and ROM; devices that store information magnetically, such as hard disks, floppy disks, magnetic tapes, magnetic core memories, bubble memories, and USB flash drives; and devices that store information optically, such as CDs and DVDs. Of course, other types of readable storage media exist, such as quantum memories and graphene memories.

[0042] The embodiments of this specification are not limited to those that must comply with industry communication standards, standard computer resource data update and data storage rules, or the situations described in one or more embodiments of this specification. Certain industry standards or slightly modified implementation plans based on the implementation described in the embodiments using custom methods or embodiments can also achieve the same, equivalent, or similar implementation effects as the above embodiments, or the expected implementation effects after deformation. The embodiments obtained by applying these modified or deformed data acquisition, storage, judgment, processing methods, etc. can still fall within the scope of the optional implementation plans of the embodiments of this specification.

[0043] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel ATMEL AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in pure computer-readable program code, the controller can also be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the means for implementing various functions included therein can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0044] The device embodiments described above are merely illustrative. For example, the division of units described is merely a logical functional division. Actual implementations may employ alternative divisions, such as combining or integrating multiple units or plug-ins into another system, or omitting or disabling certain features. Furthermore, the coupling or direct coupling or communication connection shown or discussed between devices or units may be through interfaces, or indirect coupling or communication connection between devices or units may be electrical, mechanical, or otherwise.

[0045] These computer program instructions can also be loaded onto a computer or other programmable resource data updating device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0046] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referenced to the partial description of the method embodiment. In the description of this specification, reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in an appropriate manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of different embodiments or examples, unless they are mutually inconsistent.

[0047] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A gradient estimation method for generating adversarial samples in a black box video model boundary, characterized by: The method comprises: Determine the number of gradient estimation samples based on the current number of visits to the video model when generating the boundary black box adversarial sample and the maximum number of visits allowed by the preset video model; Performing Gaussian distribution processing on the gradient estimation sample and sampling to obtain noise, generating a test sample based on the noise and the sample, and then determining a boundary feedback result of the video model based on the test sample; After adding one to the current number of visits to the video model, the corresponding test samples and boundary feedback results are obtained, and after repeating the process for a preset number of times, multiple groups of the test samples and their corresponding boundary feedback results are obtained; The gradient is determined based on the noise and boundary feedback results corresponding to all test samples.

2. The method for estimating gradients when generating adversarial samples in a black box video model boundary as claimed in claim 1, wherein: The gradient is determined based on the noise and boundary feedback results corresponding to all test samples, specifically including: Determining the impact factor of the corresponding noise according to the noise and boundary feedback results corresponding to each group of test samples; All influencing factors are averaged, and the average is used as the gradient.

3. The method for estimating gradients when generating adversarial samples in a black box video model boundary as claimed in claim 2, wherein: Specifically, the impact factor corresponding to the noise is determined by the following formula: ， Where, z is the impact factor, y Feedback result for the boundary, For noise.

4. The method for estimating gradients when generating adversarial samples in a black box video model boundary as claimed in claim 1, wherein: Specifically, the test samples are generated by the following formula: ， Where, For the test sample, x is the gradient estimation sample, is the noise amplification factor, For noise.

5. The method for estimating gradients when generating adversarial samples in a black box video model boundary as claimed in claim 1, wherein: The boundary feedback result is specifically determined by the following formula: ， Where, y Feedback result for the boundary, For the video model, is a test sample, where the boundary feedback result is the boundary value of the original probability value, that is, 1 or -1.

6. The method for estimating gradients when generating adversarial samples in a black box video model boundary as claimed in claim 1, wherein: The number of gradient estimation samples is determined based on the number of visits to the video model currently and the maximum number of visits allowed by the preset video model when generating the boundary black box adversarial sample. The number is specifically determined by the following formula: ， Where, N is the number of samples for gradient estimation, is the lower limit of the mean parameter, q is the number of visits to the video model during adversarial sample generation, m is the mean parameter, r is the scale parameter, Q The maximum number of accesses allowed for a video model.

7. The method for estimating gradients when generating adversarial samples in a video model boundary black box according to claim 6, wherein: The lower limit of the mean parameter is determined by the following formula: ， The ratio parameter is specifically used to calculate the lower limit of the mean parameter.

8. A device for estimating gradients when generating adversarial samples in a black box video model boundary, characterized in that: The device comprises: A first determination module is used to determine the number of gradient estimation samples based on the current number of visits to the video model when generating the boundary black box adversarial sample and the maximum number of visits allowed by the preset video model; A second determination module is configured to perform Gaussian distribution processing on the gradient estimation sample and sample it to obtain noise, generate a test sample based on the noise and the sample, and then determine a boundary feedback result of the video model based on the test sample; A repetition module is used to obtain corresponding test samples and boundary feedback results after adding one to the current number of visits to the video model, and to obtain multiple groups of the test samples and their corresponding boundary feedback results after repeating a preset number of times; The third determination module is used to determine the gradient based on the noise and boundary feedback results corresponding to all test samples.

Images