A black box adversarial sample generation method, device, equipment and medium

Abstract

The application discloses a black box adversarial sample generation method and device, equipment and medium, and relates to the field of deep neural networks, which comprises the following steps: acquiring a reference deformation image, processing the reference deformation image to obtain a misalignment shadow, and generating a black box adversarial sample based on the misalignment shadow; detecting the black box adversarial sample, if the detection fails, performing text image semantic segmentation on the black box adversarial sample to obtain a background image and text position information; and performing background color filling and text pixel filling operations on the misalignment shadow based on the background image and the text position information to obtain a target black box adversarial sample. Through the above technical scheme of the application, the stability of black box adversarial sample generation can be improved, and the universality of black box adversarial sample generation can be improved.

Description

Technical Field

[0001] This invention relates to the field of deep neural networks, and in particular to a method, apparatus, device, and medium for generating black-box adversarial examples. Background Technology

[0002] Deep neural network-based scene text recognition overcomes the shortcomings of traditional scene text recognition, such as difficulty in detecting irregular text and heavy reliance on manually constructed features. It is widely used in various text recognition tasks, including license plate recognition, document automation, road sign recognition, and optical character recognition. However, it also inherits the vulnerability of deep neural networks to adversarial attacks; even subtle perturbations added to benign samples can mislead deep neural network-based scene text recognition models into giving incorrect results. Although there is a wealth of research on adversarial attacks, it mainly focuses on image classification, object detection, and instance segmentation. Due to the irregular shape of scene text images, attack schemes targeting victim models such as image classifiers and object detectors cannot be directly applied to deep neural network-based scene text recognizers. Furthermore, scene text images contain numerous tiny objects, making attacks on neural network-based scene text recognizers far more challenging than classic image classification and object detection. Song et al. first conducted research on adversarial attacks on text detectors, and subsequently, some attack schemes targeting scene text recognizers and detectors emerged. However, these schemes remain at the stage of white-box attack research based on constructing adversarial examples using the gradients of victim models. In reality, adversaries cannot obtain confidential information such as the victim's model network structure and gradients. Furthermore, existing attack schemes still employ a construction pattern of adding adversarial perturbations to benign images when generating adversarial examples. This results in unnatural adversarial examples that are easily perceived by the human eye and difficult to evade defense measures.

[0003] As can be seen from the above, how to increase the stability and universality of black-box adversarial example generation is a problem to be solved in this field. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for generating black-box adversarial examples, which can increase the stability and versatility of black-box adversarial example generation. The specific solution is as follows:

[0005] Firstly, this application discloses a black-box adversarial example generation method, including:

[0006] A baseline deformed image is acquired, and the baseline deformed image is processed to obtain misaligned shadows. Black-box adversarial examples are generated based on the misaligned shadows.

[0007] The black-box adversarial sample is detected. If the detection fails, the black-box adversarial sample is subjected to text image semantic segmentation to obtain the background image and text location information.

[0008] Based on the background image and the text position information, background color filling and text pixel filling operations are performed on the misaligned shadow to obtain the target black-box adversarial sample.

[0009] Optionally, the step of acquiring a reference deformed image and processing the reference deformed image to obtain a misaligned shadow includes:

[0010] Obtain the baseline deformation image;

[0011] The reference deformed image is processed by channel translation transformation using single-channel misalignment to obtain misaligned shadows.

[0012] Optionally, the detection of the black-box adversarial sample includes:

[0013] Generate sample detection conditions based on visual perception business requirements.

[0014] Determine whether the black-box adversarial sample meets the sample detection conditions.

[0015] Optionally, performing text image semantic segmentation on the black-box adversarial sample to obtain background image and text location information includes:

[0016] The black-box adversarial sample is subjected to image binarization to obtain the processed black-box adversarial sample;

[0017] The processed black-box adversarial sample is subjected to text image semantic segmentation to obtain background image and text location information.

[0018] Optionally, the step of performing background color filling and text pixel filling operations on the misaligned shadow based on the background image and the text position information includes:

[0019] The number of pixels in the misaligned shadow is determined, and the background color is filled into the misaligned shadow based on the background image and the number of pixels.

[0020] Based on the text location information and the misaligned shadow, the misaligned text is determined, and a text pixel filling operation is performed on the misaligned text.

[0021] Optionally, generating black-box adversarial examples based on the misaligned shadows includes:

[0022] Obtain the preset counter-disturbance value;

[0023] Samples are generated based on the misaligned shadows, and the adversarial perturbation value is added to the samples to obtain black-box adversarial samples.

[0024] Optionally, the step of generating samples based on the misaligned shadows and adding the adversarial perturbation value to the samples to obtain black-box adversarial samples includes:

[0025] The reference deformed image is subjected to distortion transformation, image noise processing and geometric transformation to obtain the reference deformed image after image deformation processing. The reference deformed image after image deformation processing is then subjected to channel misalignment transformation to obtain the image after channel misalignment transformation.

[0026] Samples are generated based on the misaligned shadows, and the adversarial perturbation value is added to the samples;

[0027] A preset function is invoked, and the black-box adversarial sample is generated based on the baseline deformed image after the image deformation processing, the image after the channel misalignment transformation processing, and the sample.

[0028] Secondly, this application discloses a black-box adversarial sample generation device, comprising:

[0029] The image acquisition module is used to acquire a reference deformed image, process the reference deformed image to obtain misaligned shadows, and generate black-box adversarial samples based on the misaligned shadows.

[0030] The detection module is used to detect the black-box adversarial sample. If the detection fails, the black-box adversarial sample is subjected to text image semantic segmentation to obtain the background image and text location information.

[0031] The target black-box adversarial sample generation module is used to perform background color filling and text pixel filling operations on the misaligned shadow based on the background image and the text position information to obtain the target black-box adversarial sample.

[0032] Thirdly, this application discloses an electronic device, including:

[0033] Memory, used to store computer programs;

[0034] A processor is used to execute the computer program to implement the aforementioned black-box adversarial sample generation method.

[0035] Fourthly, this application discloses a computer storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed black-box adversarial sample generation method.

[0036] As can be seen, this application provides a black-box adversarial example generation method, including acquiring a benchmark deformed image, processing the benchmark deformed image to obtain a misaligned shadow, generating a black-box adversarial example based on the misaligned shadow; detecting the black-box adversarial example, and if the detection fails, performing text image semantic segmentation on the black-box adversarial example to obtain a background image and text position information; and performing background color filling and text pixel filling operations on the misaligned shadow based on the background image and the text position information to obtain a target black-box adversarial example. This application generates adversarial examples by using single-channel misalignment on the image, breaking the inherent pattern of existing schemes that generate adversarial perturbations by relying on the internal details of the victim model and then adding them to benign images to construct adversarial examples. By not relying on the internal details of the target model when constructing adversarial examples, the generated adversarial examples have good transferability. At the same time, the adversarial perturbation exists in the form of a shadow, and the shadow is generated by relying on the essential properties of the benign image, so the adversarial examples constructed by this model are more natural. In addition, the channel translation pixel displacement used in generating adversarial examples is not constrained by the dataset or the victim model, so the generated adversarial examples have universality. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0038] Figure 1 This is a flowchart of a black-box adversarial sample generation method disclosed in this application;

[0039] Figure 2 This is a flowchart of a black-box adversarial sample generation method disclosed in this application;

[0040] Figure 3 This is an example diagram of a black-box adversarial sample generation method disclosed in this application;

[0041] Figure 4 This is a schematic diagram of the structure of a black-box adversarial sample generation device disclosed in this application;

[0042] Figure 5 This application provides a structural diagram of an electronic device. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Deep neural network-based scene text recognition overcomes the shortcomings of traditional scene text recognition, such as difficulty in detecting irregular text and heavy reliance on manually constructed features. It is widely used in various text recognition tasks, including license plate recognition, document automation, road sign recognition, and optical character recognition. However, it also inherits the vulnerability of deep neural networks to adversarial attacks; even subtle perturbations added to benign samples can mislead deep neural network-based scene text recognition models into giving incorrect results. Although there is a wealth of research on adversarial attacks, it mainly focuses on image classification, object detection, and instance segmentation. Due to the irregular shape of scene text images, attack schemes targeting victim models such as image classifiers and object detectors cannot be directly applied to deep neural network-based scene text recognizers. Furthermore, scene text images contain numerous tiny objects, making attacks on neural network-based scene text recognizers far more challenging than classic image classification and object detection. Song et al. first conducted research on adversarial attacks on text detectors, and subsequently, some attack schemes targeting scene text recognizers and detectors emerged. However, these schemes remain at the stage of white-box attack research based on constructing adversarial examples using the gradients of victim models. In reality, adversaries cannot obtain confidential information such as the victim's model network structure and gradients. Furthermore, existing attack schemes still employ a construction method of adding adversarial perturbations to benign images when generating adversarial examples. This results in unnatural-looking adversarial examples that are easily detected by the human eye and difficult to evade defense measures. Therefore, increasing the stability and versatility of black-box adversarial example generation is a problem that needs to be solved in this field.

[0045] See Figure 1 As shown in the figure, this invention discloses a black-box adversarial sample generation method, which may specifically include:

[0046] Step S11: Obtain a reference deformed image, process the reference deformed image to obtain misaligned shadows, and generate black-box adversarial samples based on the misaligned shadows.

[0047] In this embodiment, a baseline deformed image is acquired. A single-channel misalignment transformation is performed on the baseline deformed image to obtain misaligned shadows. Then, black-box adversarial examples are generated based on these misaligned shadows.

[0048] Specifically, generating black-box adversarial samples based on the misaligned shadows includes: obtaining a preset adversarial perturbation value; then performing distortion transformation, image noise processing, and geometric transformation on a reference deformed image to obtain a reference deformed image after image deformation processing; performing channel misalignment transformation processing on the reference deformed image after image deformation processing to obtain an image after channel misalignment transformation processing; generating samples based on the misaligned shadows and adding the adversarial perturbation value to the samples; calling a preset function; and generating the black-box adversarial samples based on the reference deformed image after image deformation processing, the image after channel misalignment transformation processing, and the samples.

[0049] Step S12: Detect the black-box adversarial sample. If the detection fails, perform text image semantic segmentation on the black-box adversarial sample to obtain the background image and text location information.

[0050] In this embodiment, sample detection conditions are generated based on visual perception business requirements. Then, it is determined whether the black-box adversarial sample meets the sample detection conditions. If the sample detection conditions are not met, it means that the detection fails. Then, text image semantic segmentation is performed on the black-box adversarial sample to obtain background image and text location information.

[0051] Step S13: Based on the background image and the text position information, perform background color filling and text pixel filling operations on the misaligned shadow to obtain the target black box adversarial sample.

[0052] In this embodiment, the true label of the reference deformed image X is l = {l1, l2, ..., l...} n}, where n represents the length of the true label sequence. For the text recognizer R (victim model), we can find an adversarial perturbation δ such that R(X′)=l′, where X′ represents the adversarial sample generated after adding a perturbation to the benign text recognition image, i.e., X′=X+δ, and l′ represents the adversarial label, l′={l1′,l2′...,l n n′ represents the length of the adversarial label. It's important to note that n′ is not necessarily equal to n. Based on the above description, the generated adversarial sample X′ for X is represented as follows:

[0053] minδ;

[0054] s, t, X′=X+δ, R(X′)=l′, l′≠l, δ≤ε;

[0055] Here, ε represents the perturbation constraint to control the difference between benign and adversarial examples. Unlike previous studies, we no longer employ the construction method of adding adversarial perturbations to benign images to generate adversarial examples. Instead, we directly process benign samples through channels to construct adversarial examples. The target black-box adversarial example is represented as follows:

[0056] minD(X′,X);

[0057] s, t, X′=F(S(X)), R(X′)=l′, l′≠l, D(X′,X)≤ε;

[0058] Where D(X′, X) is used to measure the distance between X′ and X, F(X) represents channel misalignment transformation, and S(X) represents image deformation technique.

[0059] Image channel transformation is essentially a type of image translation transformation, but unlike other transformations, we only perform translation transformations on a single channel of the image. If X is a three-channel color image, we define X... (0) Represents the R-channel image, X (1) Represents the G-channel image, X (2) Representing the B-channel image, the single-channel misalignment transform F(X) can be expressed as:

[0060]

[0061] Where i = 0, 1, 2, Δx, Δy represent the pixel displacement of a single channel. S(X) represents the baseline image deformation technique; this paper only introduces warp transformation, image noise, and geometric transformation. For warp transformation, S(X) can be expressed as: S(X) = B(H, H+C), where B() represents the bilinear interpolation function, H represents the coordinate matrix of the image, and C represents the displacement field.

[0062] C = (C0 * G)λ;

[0063]

[0064] * indicates a convolution operation, C0 represents a random matrix of the same size as H with values ​​between (-1, 1), whose elements can be represented as (a, b), (a, b) ∈ (-1, 1), ν represents the deformation coefficient, and λ is a scaling factor for the deformation intensity, used to control the deformation intensity. For image noise transformation, S(X) can be expressed as:

[0065]

[0066]

[0067]

[0068] Where c and d represent the range of uniform noise values, μ represents the mean of Gaussian noise, σ represents the standard deviation of noise, and e and f represent the gray values ​​of salt-and-pepper noise. For image geometric transformation, S(X) can be expressed as: S(X) = B(H, M);

[0069] in, The image transformation matrix can be solved as follows:

[0070]

[0071] in, Indicates the source and target points. It can be represented as a scaling transformation matrix Rotation transformation matrix θ represents the rotation angle, and the shear transformation matrix is... Affine transformation matrix Translation transformation matrix Indicates the target point.

[0072] In this embodiment, a baseline deformed image is acquired and processed to obtain a misaligned shadow. A black-box adversarial example is generated based on the misaligned shadow. The black-box adversarial example is then detected. If the detection fails, text image semantic segmentation is performed on the black-box adversarial example to obtain a background image and text position information. Based on the background image and the text position information, background color filling and text pixel filling operations are performed on the misaligned shadow to obtain the target black-box adversarial example. This application generates adversarial examples by using single-channel misalignment on the image, breaking the inherent pattern of existing schemes that rely on the internal details of the victim model to generate adversarial perturbations and then add them to benign images to construct adversarial examples. By not relying on the internal details of the target model when constructing adversarial examples, the generated adversarial examples have good transferability. Furthermore, the adversarial perturbation exists in the form of a shadow, and the shadow is generated using the essential properties of the benign image. Therefore, the adversarial examples constructed by this model are more natural. In addition, the channel translation pixel displacement used in generating adversarial examples is not constrained by the dataset or the victim model, thus the generated adversarial examples have universality.

[0073] See Figure 2 As shown in the figure, this invention discloses a black-box adversarial sample generation method, which may specifically include:

[0074] Step S21: Obtain a reference deformed image, process the reference deformed image to obtain misaligned shadows, and generate black-box adversarial samples based on the misaligned shadows.

[0075] Step S22: Detect the black-box adversarial sample. If the detection fails, perform image binarization on the black-box adversarial sample to obtain the processed black-box adversarial sample. Then, perform text image semantic segmentation on the processed black-box adversarial sample to obtain the background image and text location information.

[0076] Step S23: Determine the pixel quantity of the misaligned shadow, fill the misaligned shadow with background color based on the background image and the pixel quantity, then determine the misaligned text based on the text position information and the misaligned shadow, and perform text pixel filling operation on the misaligned text to obtain the target black box adversarial sample.

[0077] In this embodiment, it should be noted that, due to the different image sizes, as the displacement of single-channel misaligned pixels increases, some adversarial examples generated in the images may affect visual perception (text discoloration or blurring), such as... Figure 3 As shown. To solve this problem, we fill the misaligned shadow with a background color similar to the background. The specific operation can be described as follows: First, we use text segmentation techniques to semantically segment the text image to determine the background and text positions. Since the pixels in the text image are relatively uniform, we use image binarization to segment the text image. Second, based on the number of misaligned pixels, we fill the misaligned shadow (the position of the text in the benign image) with a similar background color, and the misaligned text is filled with the text pixels from the benign image, such as... Figure 3 As shown in “G_8_S”, Figure 3 Adversarial examples (multi-pixel displacement) "G_X" indicates that the G channel is shifted X pixels to the right and downward, "Rotate" indicates image rotation, and "Segment" indicates the segmentation result of image binarization. "G_8_S" indicates that the G channel is shifted 8 pixels and filled with other pixels. The adversarial examples generated in this application have good transferability, naturalness, and versatility. Specifically, this scheme generates adversarial examples by using single-channel displacement on slightly deformed images, breaking the inherent pattern of existing schemes that generate adversarial perturbations by relying on the internal details of the victim model and then adding them to benign images to construct adversarial examples. This scheme does not rely on the internal details of the target model when constructing adversarial examples, thus the generated adversarial examples have good transferability. At the same time, the adversarial perturbations of this scheme exist in the form of shadows, and the shadows are generated by relying on the essential properties of benign images, so the adversarial examples constructed by this model are more natural. In addition, the amount of channel translation pixel displacement used by this scheme to generate adversarial examples is not constrained by the dataset or the victim model, thus the generated adversarial examples have versatility.

[0078] In this embodiment, a baseline deformed image is acquired and processed to obtain a misaligned shadow. A black-box adversarial example is generated based on the misaligned shadow. The black-box adversarial example is then detected. If the detection fails, text image semantic segmentation is performed on the black-box adversarial example to obtain a background image and text position information. Based on the background image and the text position information, background color filling and text pixel filling operations are performed on the misaligned shadow to obtain the target black-box adversarial example. This application generates adversarial examples by using single-channel misalignment on the image, breaking the inherent pattern of existing schemes that rely on the internal details of the victim model to generate adversarial perturbations and then add them to benign images to construct adversarial examples. By not relying on the internal details of the target model when constructing adversarial examples, the generated adversarial examples have good transferability. Furthermore, the adversarial perturbation exists in the form of a shadow, and the shadow is generated using the essential properties of the benign image. Therefore, the adversarial examples constructed by this model are more natural. In addition, the channel translation pixel displacement used in generating adversarial examples is not constrained by the dataset or the victim model, thus the generated adversarial examples have universality.

[0079] See Figure 4 As shown, this embodiment of the invention discloses a black-box adversarial sample generation device, which may specifically include:

[0080] Image acquisition module 11 is used to acquire a reference deformed image, process the reference deformed image to obtain misaligned shadows, and generate black-box adversarial samples based on the misaligned shadows;

[0081] The detection module 12 is used to detect the black-box adversarial sample. If the detection fails, the black-box adversarial sample is subjected to text image semantic segmentation to obtain the background image and text location information.

[0082] The target black-box adversarial sample generation module 13 is used to perform background color filling and text pixel filling operations on the misaligned shadow based on the background image and the text position information to obtain the target black-box adversarial sample.

[0083] In this embodiment, a baseline deformed image is acquired and processed to obtain a misaligned shadow. A black-box adversarial example is generated based on the misaligned shadow. The black-box adversarial example is then detected. If the detection fails, text image semantic segmentation is performed on the black-box adversarial example to obtain a background image and text position information. Based on the background image and the text position information, background color filling and text pixel filling operations are performed on the misaligned shadow to obtain the target black-box adversarial example. This application generates adversarial examples by using single-channel misalignment on the image, breaking the inherent pattern of existing schemes that rely on the internal details of the victim model to generate adversarial perturbations and then add them to benign images to construct adversarial examples. By not relying on the internal details of the target model when constructing adversarial examples, the generated adversarial examples have good transferability. Furthermore, the adversarial perturbation exists in the form of a shadow, and the shadow is generated using the essential properties of the benign image. Therefore, the adversarial examples constructed by this model are more natural. In addition, the channel translation pixel displacement used in generating adversarial examples is not constrained by the dataset or the victim model, thus the generated adversarial examples have universality.

[0084] In some specific embodiments, the image acquisition module 11 may specifically include:

[0085] The image acquisition module is used to acquire the reference deformation image;

[0086] The misaligned shadow determination module is used to perform channel translation transformation on the reference deformed image using single-channel misalignment to obtain misaligned shadows.

[0087] In some specific embodiments, the detection module 12 may specifically include:

[0088] The sample detection condition generation module is used to generate sample detection conditions based on visual perception business requirements.

[0089] The judgment module is used to determine whether the black-box adversarial sample meets the sample detection conditions.

[0090] In some specific embodiments, the detection module 12 may specifically include:

[0091] An image binarization processing module is used to perform image binarization processing on the black-box adversarial sample to obtain the processed black-box adversarial sample.

[0092] The text image semantic segmentation module is used to perform text image semantic segmentation on the processed black-box adversarial sample to obtain background image and text location information.

[0093] In some specific embodiments, the target black-box adversarial example generation module 13 may specifically include:

[0094] A pixel quantity determination module is used to determine the pixel quantity of the misaligned shadow, and fill the misaligned shadow with background color based on the background image and the pixel quantity;

[0095] The misaligned text determination module is used to determine the misaligned text based on the text position information and the misaligned shadow, and to perform text pixel filling operation on the misaligned text.

[0096] In some specific embodiments, the target black-box adversarial example generation module 13 may specifically include:

[0097] The counter-disturbance value acquisition module is used to acquire preset counter-disturbance values;

[0098] The black-box adversarial sample determination module is used to generate samples based on the misaligned shadows and add the adversarial perturbation value to the samples to obtain black-box adversarial samples.

[0099] In some specific embodiments, the target black-box adversarial example generation module 13 may specifically include:

[0100] The channel misalignment transformation module is used to perform distortion transformation, image noise processing, and geometric transformation on the reference deformed image to obtain the reference deformed image after image deformation processing. The channel misalignment transformation is then performed on the reference deformed image after image deformation processing to obtain the image after channel misalignment transformation.

[0101] An adversarial perturbation value addition module is used to generate samples based on the misaligned shadows and add the adversarial perturbation values ​​to the samples;

[0102] The black-box adversarial sample generation module is used to call a preset function and generate the black-box adversarial sample based on the baseline deformed image after the image deformation processing, the image after the channel misalignment transformation processing, and the sample.

[0103] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the black-box adversarial sample generation method performed by the electronic device disclosed in any of the foregoing embodiments.

[0104] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0105] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0106] The operating system 221 manages and controls the various hardware devices on the electronic device 20 and the computer program 222 to enable the processor 21 to perform operations and processing on the data 223 in the memory 22. The operating system 221 can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the black-box adversarial example generation method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the black-box adversarial example generation device from external devices, as well as data collected by its own input / output interface 25.

[0107] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0108] Furthermore, embodiments of this application also disclose a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements the steps of the black-box adversarial sample generation method disclosed in any of the foregoing embodiments.

[0109] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0110] The foregoing has provided a detailed description of a black-box adversarial sample generation method, apparatus, device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A black-box adversarial example generation method, characterized in that, include: A baseline deformed image is acquired, and the baseline deformed image is processed to obtain misaligned shadows. Black-box adversarial examples are generated based on the misaligned shadows. The black-box adversarial sample is detected. If the detection fails, the black-box adversarial sample is subjected to text image semantic segmentation to obtain the background image and text location information. Based on the background image and the text position information, background color filling and text pixel filling operations are performed on the misaligned shadow to obtain the target black box adversarial sample; The step of acquiring a reference deformed image and processing the reference deformed image to obtain a misaligned shadow includes: acquiring the reference deformed image; and performing a channel translation transformation on the reference deformed image using a single-channel misalignment to obtain a misaligned shadow. The step of generating black-box adversarial samples based on the misaligned shadows includes: obtaining a preset adversarial perturbation value; generating samples based on the misaligned shadows, and adding the adversarial perturbation value to the samples to obtain black-box adversarial samples.

2. The black-box adversarial example generation method according to claim 1, characterized in that, The detection of the black-box adversarial sample includes: Generate sample detection conditions based on visual perception business requirements. Determine whether the black-box adversarial sample meets the sample detection conditions.

3. The black-box adversarial example generation method according to claim 1, characterized in that, The step of performing text-image semantic segmentation on the black-box adversarial sample to obtain background image and text location information includes: The black-box adversarial sample is subjected to image binarization to obtain the processed black-box adversarial sample; The processed black-box adversarial sample is subjected to text image semantic segmentation to obtain background image and text location information.

4. The black-box adversarial example generation method according to claim 1, characterized in that, The step of filling the misaligned shadow with background color and text pixels based on the background image and the text position information includes: The number of pixels in the misaligned shadow is determined, and the background color is filled into the misaligned shadow based on the background image and the number of pixels. Based on the text location information and the misaligned shadow, the misaligned text is determined, and a text pixel filling operation is performed on the misaligned text.

5. The black-box adversarial example generation method according to claim 1, characterized in that, The step of generating samples based on the misaligned shadows and adding the adversarial perturbation value to the samples to obtain black-box adversarial samples includes: The reference deformed image is subjected to distortion transformation, image noise processing and geometric transformation to obtain the reference deformed image after image deformation processing. The reference deformed image after image deformation processing is then subjected to channel misalignment transformation to obtain the image after channel misalignment transformation. Samples are generated based on the misaligned shadows, and the adversarial perturbation value is added to the samples; A preset function is invoked, and the black-box adversarial sample is generated based on the baseline deformed image after the image deformation processing, the image after the channel misalignment transformation processing, and the sample.

6. A black-box adversarial sample generation device, characterized in that, include: The image acquisition module is used to acquire a reference deformed image, process the reference deformed image to obtain misaligned shadows, and generate black-box adversarial samples based on the misaligned shadows. The detection module is used to detect the black-box adversarial sample. If the detection fails, the black-box adversarial sample is subjected to text image semantic segmentation to obtain the background image and text location information. The target black-box adversarial sample generation module is used to perform background color filling and text pixel filling operations on the misaligned shadow based on the background image and the text position information to obtain the target black-box adversarial sample. Specifically, the image acquisition module is used to acquire a reference deformed image; and to perform channel translation transformation on the reference deformed image using single-channel misalignment to obtain misaligned shadows. Specifically, the image acquisition module is used to acquire a preset adversarial perturbation value; generate samples based on the misaligned shadows; and add the adversarial perturbation value to the samples to obtain black-box adversarial samples.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the black-box adversarial sample generation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the black-box adversarial sample generation method as described in any one of claims 1 to 5.

Images