Digital domain visible light adversarial sample performance test method

Abstract

The invention provides a digital domain visible light adversarial sample performance testing method, and relates to the technical field of adversarial sample performance testing. The method comprises the following steps: training a target detector; generating a digital domain visible light confrontation sample; giving a resistance index before a resistance sample performance test; fourier transform is carried out on the adversarial sample; superimposing a transfer function on the adversarial sample after Fourier transform; inverse Fourier transform is carried out on the adversarial sample after superposition of the transfer function; providing countermeasure indexes after the countermeasure sample performance test; giving a resistance relative change condition before and after a resistance sample performance test; according to the method, the influence of an imaging environment and imaging equipment on the performance of the visible light adversarial sample is considered, the robustness and stability testing means of the digital domain adversarial sample are enriched, and meanwhile, the method has important guiding significance for design and generation of the physical domain adversarial sample.

Description

Technical Field

[0001] The present invention belongs to the technical field of adversarial sample performance testing, and in particular, relates to a digital domain visible light adversarial sample performance testing method. Background Art

[0002] Digital domain visible light adversarial samples refer to tiny perturbations added to digital images or data that are almost visually imperceptible but can cause machine learning models (such as image classifiers, object detectors, etc.) to make incorrect predictions or decisions.

[0003] Adversarial samples generated in the digital domain are generally difficult to adapt to the real physical world, that is, their adversarial properties are difficult to maintain under different imaging environments and imaging devices. However, the prior art lacks a robustness and stability testing method for adversarial samples in the visible light digital domain. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a digital domain visible light adversarial sample performance testing method.

[0005] A first aspect of the present invention discloses a method for testing performance of a digital domain visible light adversarial sample; the method comprises:

[0006] Step 1: construct a normal sample based on the collected digital image or data, and use the normal sample to train the target detector D, and record the average target detection accuracy M1 corresponding to the normal sample;

[0007] Step 2: Add preset disturbances to the collected digital image or data to generate a digital domain visible light adversarial sample A1;

[0008] Step 3: Send the digital domain visible light adversarial sample A1 to the trained target detector D, and record the average target detection accuracy M2 corresponding to A1;

[0009] Step 4: Based on the difference between M1 and M2, obtain the adversarial index EOCM1 before the adversarial sample performance test of the trained target detector D;

[0010] Step 5, perform Fourier transform on A1 to obtain a Fourier transformed digital domain visible light adversarial sample A2;

[0011] Step 6: Based on A2, superimpose the transfer function to obtain the superimposed result A3;

[0012] Step 7, perform inverse Fourier transform on A3 to obtain a superposition result A4 after inverse Fourier transform;

[0013] Step 8: Send A4 to the trained target detector D, and record the average target detection accuracy M3 corresponding to A4;

[0014] Step 9: Based on the difference between M1 and M3, the adversarial index EOCM2 after the adversarial sample performance test of the trained target detector D is obtained;

[0015] Step 10: Based on EOCM1 and EOCM2, obtain the relative change index EOCM of the adversarial sample performance test before and after the training of the target detector D, and then determine the adversarial nature of the adversarial sample.

[0016] In the step 1, the target detector D includes an SSD network, a Yolo v9 network or a Faster RCNN network.

[0017] The step 6 specifically includes:

[0018] Based on A2, the atmospheric turbulence transfer function, aerosol modulation transfer function, diffraction limit transfer function, aberration transfer function, image sensor transfer transfer function, image sensor spatial filter transfer function and image sensor temporal filter transfer function are superimposed to obtain the superimposed result A3.

[0019] The specific process of superposition transfer function is as follows:

[0020] A3=A2×MTF turb ×MTF aero ×MTF diff ×MTF aber ×MTF CCD ×MTF ds ×MTF dt

[0021] Among them, MTF turb is the atmospheric turbulence transfer function; MTF aero is the aerosol modulation transfer function; MTF diff is the diffraction-limited transfer function; MTF aber is the aberration transfer function; MTF CCD MTF is the image sensor transfer function. ds is the spatial filter transfer function of the image sensor; MTF dt is the image sensor time filter transfer function; where,

[0022]

[0023] Where ν is the angular spatial frequency; is the refractive index structure constant; λ is the wavelength of visible light; L is the path length;

[0024]

[0025] in, is the angular space cutoff frequency, α is the aerosol particle diameter; k a is the absorption coefficient; k s is the scattering coefficient;

[0026]

[0027] in, is the spatial cutoff frequency of the incoherent optical system, D0 is the effective aperture of the optical system, λ s is the central wavelength of incoherent light; f x 、f y are the spatial frequencies in the horizontal and vertical directions, respectively;

[0028]

[0029] Where σ is the standard deviation measured in angles;

[0030]

[0031] Where N is the number of bits of the CCD; η is the transfer efficiency; f t is the temporal frequency; f tc is the clock frequency;

[0032]

[0033] Among them, α and β are the spatial angles of the rectangular detector;

[0034]

[0035] Among them, f is the spatial frequency; f0 is the frequency corresponding to 3dB.

[0036] In step 10, the EOCM calculation method is as follows:

[0037]

[0038] In step 10, if EOCM>0, the adversarial nature of the visible light adversarial sample in the digital domain is relatively weakened; if EOCM=0, the adversarial nature of the visible light adversarial sample in the digital domain is relatively unchanged; if EOCM<0, the adversarial nature of the visible light adversarial sample in the digital domain is relatively enhanced.

[0039] A second aspect of the present invention discloses a digital domain visible light adversarial sample performance testing system, the system comprising:

[0040] The first processing module is configured to construct a normal sample according to the collected digital image or data, train the target detector D using the normal sample, and record the target average detection accuracy M1 corresponding to the normal sample;

[0041] A second processing module is configured to add a preset disturbance to the collected digital image or data to generate a digital domain visible light adversarial sample A1;

[0042] The third processing module is configured to send the digital domain visible light adversarial sample A1 to the trained target detector D, and record the target average detection accuracy M2 corresponding to A1;

[0043] The fourth processing module is configured to obtain an adversarial index EOCM1 before the adversarial sample performance test for the trained target detector D based on the difference between M1 and M2;

[0044] A fifth processing module is configured to perform Fourier transform on A1 to obtain a Fourier transformed digital domain visible light adversarial sample A2;

[0045] A sixth processing module is configured to, based on A2, superimpose the transfer function to obtain a superimposed result A3;

[0046] A seventh processing module is configured to perform an inverse Fourier transform on A3 to obtain a superposition result A4 after the inverse Fourier transform;

[0047] The eighth processing module is configured to send A4 to the trained target detector D, and record the target average detection accuracy M3 corresponding to A4;

[0048] A ninth processing module is configured to obtain, based on the difference between M1 and M3, an adversarial index EOCM2 after the adversarial sample performance test for the trained target detector D;

[0049] The tenth processing module is configured to obtain, based on EOCM1 and EOCM2, an adversarial relative change index EOCM before and after the adversarial sample performance test of the trained target detector D, and then determine the adversarial nature of the adversarial sample.

[0050] The third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in any one of the digital domain visible light adversarial sample performance testing methods in the first aspect of the present disclosure are implemented.

[0051] The fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the digital domain visible light adversarial sample performance testing methods in the first aspect of the present disclosure.

[0052] In summary, the solution proposed in the present invention has the following technical effects: the technical solution of the present invention takes into account the impact of the imaging environment and imaging equipment on the performance of visible light adversarial samples, enriches the robustness and stability testing methods of adversarial samples in the digital domain, and at the same time has important guiding significance for the design and generation of adversarial samples in the physical domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 Flow chart of a method for testing performance of digital domain visible light adversarial samples according to an embodiment of the present invention;

[0055] Figure 2 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] According to an embodiment of the present invention, in a first aspect, a method for testing performance of a digital domain visible light adversarial sample is provided; see Figure 1 , the method comprising:

[0058] Step 1: construct a normal sample based on the collected digital image or data, and use the normal sample to train the target detector D, and record the average target detection accuracy M1 corresponding to the normal sample;

[0059] In the step 1, the target detector D includes but is not limited to an SSD network, a Yolo v9 network or a Faster RCNN network.

[0060] Step 2: Add preset disturbances to the collected digital image or data to generate a digital domain visible light adversarial sample A1; in step 2, the adversarial sample generation method includes but is not limited to Fast Gradient Sign Method (FGSM), Carlini and Wagner's Attack (C&W), Full-coverage Camouflage Attack (FCA), etc.

[0061] Step 3: Send the digital domain visible light adversarial sample A1 to the trained target detector D, and record the average target detection accuracy M2 corresponding to A1;

[0062] Step 4: Based on the difference between M1 and M2, the adversarial index EOCM1 before the adversarial sample performance test of the trained target detector D is obtained; the calculation method of EOCM1 is as follows:

[0063] EOCM1=M1-M2

[0064] Step 5, perform Fourier transform on A1 to obtain a Fourier transformed digital domain visible light adversarial sample A2;

[0065] Step 6: Based on A2, superimpose the transfer function to obtain the superimposed result A3;

[0066] In step S6, the specific process of superimposing the transfer function is as follows:

[0067] A3=A2×MTF turb ×MTF aero ×MTF diff ×MTF aber ×MTF CCD ×MTF ds ×MTF dt

[0068] In step 6, the atmospheric turbulence transfer function is as follows:

[0069]

[0070] Wherein, ν is the angular spatial frequency, which indicates the number of cycles per radian, and its relationship with the spatial frequency is ν = fυ (f is the focal length, υ is the spatial frequency); is the refractive index structure constant; λ is the wavelength; L is the path length.

[0071] In step 6, the aerosol transfer function is as follows:

[0072]

[0073] in, is the angular space cutoff frequency, α is the aerosol particle diameter; k a is the absorption coefficient, k s is the scattering coefficient, and the sum of the two is the extinction coefficient k e , that is, k e (h) = (3.912 / R m )exp(-h / 1200); R m is the meteorological visibility distance; h is the altitude.

[0074] In step 6, the diffraction-limited (optical) transfer function is as follows:

[0075]

[0076] in, is the spatial cutoff frequency of the incoherent optical system (in c / mrad), D0 is the effective aperture of the optical system (in mm); λ s is the central wavelength of the incoherent light (in μm), the average working wavelength can be taken as (λ1+λ2) / 2, [λ1,λ2] is the working wavelength range; f x 、f y are the spatial frequencies in the horizontal and vertical directions respectively.

[0077] In step 6, the aberration transfer function is as follows:

[0078]

[0079] Where σ is the standard deviation measured in angles, σ = σ c / f, in c / mrad, is directly related to the percentage of energy in the dispersion circle. Assuming that all energy is concentrated in the dispersion circle, the standard deviation σ c It is equal to 1 / 4 of the diameter of the diffusion circle. This assumption can approximate the shape of the diffusion spot well and is easy to control. The diameter of the diffusion spot can be determined by the following formula:

[0080]

[0081] In step 6, the image sensor transfer function is as follows:

[0082]

[0083] Where N is the number of bits of the CCD; η is the transfer efficiency; f t is the temporal frequency (in Hz); f tc is the clock frequency in Hz.

[0084] In step 6, the image sensor spatial filter transfer function is as follows:

[0085]

[0086] Among them, α and β are the spatial angles of the rectangular detector; f x 、f y are the spatial frequencies in the horizontal and vertical directions respectively.

[0087] In step 6, the image sensor time filter transfer function is as follows:

[0088]

[0089] Where, f is the spatial frequency (in c / mrad); f0 is the frequency corresponding to 3dB (in c / mrad), (τ is the detector carrier lifetime (in μs)).

[0090] In step 6, the atmospheric turbulence transfer function and the aerosol modulation transfer function are used to characterize the imaging environment; the diffraction limit (optical) transfer function, the aberration transfer function, the image sensor transfer transfer function, the image sensor spatial filtering transfer function and the image sensor temporal filtering transfer function are used to characterize the imaging device.

[0091] Step 7, perform inverse Fourier transform on A3 to obtain a superposition result A4 after inverse Fourier transform;

[0092] Step 8: Send A4 to the trained target detector D, and record the average target detection accuracy M3 corresponding to A4;

[0093] Step 9: Based on the difference between M1 and M3, the adversarial index EOCM2 after the adversarial sample performance test of the trained target detector D is obtained; the calculation method of EOCM2 is as follows:

[0094] EOCM2=M1-M3

[0095] Step 10: Based on EOCM1 and EOCM2, obtain the relative change index EOCM of the adversarial sample performance test before and after the training of the target detector D, and then determine the adversarial nature of the adversarial sample.

[0096] The EOCM calculation method is as follows:

[0097]

[0098] In step 10, if EOCM>0, the adversarial nature of the visible light adversarial sample in the digital domain is relatively weakened; if EOCM=0, the adversarial nature of the visible light adversarial sample in the digital domain is relatively unchanged; if EOCM<0, the adversarial nature of the visible light adversarial sample in the digital domain is relatively enhanced.

[0099] The second aspect of the present invention discloses a digital domain visible light adversarial sample performance testing system, the system is used to implement any of the methods in the above embodiments. The system includes:

[0100] The first processing module is configured to construct a normal sample according to the collected digital image or data, train the target detector D using the normal sample, and record the target average detection accuracy M1 corresponding to the normal sample;

[0101] A second processing module is configured to add a preset disturbance to the collected digital image or data to generate a digital domain visible light adversarial sample A1;

[0102] The third processing module is configured to send the digital domain visible light adversarial sample A1 to the trained target detector D, and record the target average detection accuracy M2 corresponding to A1;

[0103] The fourth processing module is configured to obtain an adversarial index EOCM1 before the adversarial sample performance test for the trained target detector D based on the difference between M1 and M2;

[0104] A fifth processing module is configured to perform Fourier transform on A1 to obtain a Fourier transformed digital domain visible light adversarial sample A2;

[0105] A sixth processing module is configured to, based on A2, superimpose the transfer function to obtain a superimposed result A3;

[0106] A seventh processing module is configured to perform an inverse Fourier transform on A3 to obtain a superposition result A4 after the inverse Fourier transform;

[0107] The eighth processing module is configured to send A4 to the trained target detector D, and record the target average detection accuracy M3 corresponding to A4;

[0108] A ninth processing module is configured to obtain, based on the difference between M1 and M3, an adversarial index EOCM2 after the adversarial sample performance test for the trained target detector D;

[0109] The tenth processing module is configured to obtain, based on EOCM1 and EOCM2, an adversarial relative change index EOCM before and after the adversarial sample performance test of the trained target detector D, and then determine the adversarial nature of the adversarial sample.

[0110] The third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in any one of the digital domain visible light adversarial sample performance testing methods in the first aspect of the present disclosure are implemented.

[0111] Figure 2 is a structural diagram of an electronic device according to an embodiment of the present invention, such as Figure 2 As shown, the electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the housing of the electronic device, or an external keyboard, touch pad or mouse, etc.

[0112] Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a structural diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the technical solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0113] The fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in any one of the steps in the method for testing performance of digital domain visible light adversarial samples in the first aspect of the present disclosure.

[0114] In summary, the technical solution proposed in the present invention has the following technical effects: the technical solution of the present invention takes into account the influence of the imaging environment and the imaging device on the performance of visible light adversarial samples, enriches the robustness and stability testing methods of adversarial samples in the digital domain, and at the same time has important guiding significance for the design and generation of adversarial samples in the physical domain.

[0115] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solutions described in the above embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A digital domain visible light adversarial sample performance testing method, characterized in that: The method comprises: Step 1: construct a normal sample based on the collected digital image or data, and use the normal sample to train the target detector, and record the average target detection accuracy M1 corresponding to the normal sample; Step 2: Add a preset disturbance to the collected digital image or data to generate a digital domain visible light adversarial sample A1; Step 3: Send the digital domain visible light adversarial sample A1 to the trained target detector, and record the target average detection accuracy M2 corresponding to the digital domain visible light adversarial sample A1; Step 4: Calculate the difference between the target average detection accuracy M1 and the target average detection accuracy M2 to obtain the adversarial index EOCM1 before the adversarial sample performance test of the trained target detector; Step 5, perform Fourier transform on the digital domain visible light adversarial sample A1 to obtain the digital domain visible light adversarial sample A2 after Fourier transform; Step 6, superimposing the transfer function on the digital domain visible light adversarial sample A2 to obtain a first superposition result A3; Step 7, performing inverse Fourier transform on the first superposition result A3 to obtain a second superposition result A4; Step 8, sending the second superposition result A4 to the trained target detector D, and recording the target average detection accuracy M3 corresponding to the second superposition result A4; Step 9, calculate the difference between the target average detection accuracy M1 and the target average detection accuracy M3, and obtain the adversarial index EOCM2 after the adversarial sample performance test of the trained target detector; Step 10: Based on the adversarial index EOCM1 and the adversarial index EOCM2, the adversarial relative change index EOCM before and after the adversarial sample performance test of the trained target detector is obtained, so as to determine the adversarial nature of the digital domain visible light adversarial sample A1 according to the adversarial relative change index EOCM.

2. The method according to claim 1, characterized in that In the step 1, the target detector includes an SSD network, a Yolo v9 network, or a Faster RCNN network.

3. The method according to claim 1, characterized in that The step 6 specifically includes: For the adversarial sample A2, the atmospheric turbulence transfer function, the aerosol modulation transfer function, the diffraction limit transfer function, the aberration transfer function, the image sensor transfer transfer function, the image sensor spatial filtering transfer function and the image sensor temporal filtering transfer function are superimposed to obtain a first superposition result A3.

4. The method according to claim 3, characterized in that The specific process of superposition transfer function is as follows: <h2 style=";text-align:left;direction:ltr">A3 = A2 × MTF<h2 style=";text-align:left;direction:ltr"> turb <h2 style=";text-align:left;direction:ltr"> ×MTF<h2 style=";text-align:left;direction:ltr"> aero <h2 style=";text-align:left;direction:ltr"> ×MTF<h2 style=";text-align:left;direction:ltr"> diff <h2 style=";text-align:left;direction:ltr"> ×MTF<h2 style=";text-align:left;direction:ltr"> aber <h2 style=";text-align:left;direction:ltr"> ×MTF<h2 style=";text-align:left;direction:ltr"> CCD <h2 style=";text-align:left;direction:ltr"> ×MTF<h2 style=";text-align:left;direction:ltr"> ds <h2 style=";text-align:left;direction:ltr"> ×MTF<h2 style=";text-align:left;direction:ltr"> dt Among them, MTF turb is the atmospheric turbulence transfer function; MTF aero is the aerosol modulation transfer function; MTF diff is the diffraction-limited transfer function; MTF aber is the aberration transfer function; MTF CCD MTF is the image sensor transfer function. ds is the spatial filter transfer function of the image sensor; MTF dt is the image sensor temporal filter transfer function.

5. The method according to claim 1, characterized in that In step 10, the EOCM calculation method is as follows:

6. The method according to claim 5, characterized in that In step 10, if EOCM>0, it is determined that the adversarial nature of the visible light adversarial sample in the digital domain is relatively weakened; if EOCM=0, it is determined that the adversarial nature of the visible light adversarial sample in the digital domain is relatively unchanged; if EOCM<0, it is determined that the adversarial nature of the visible light adversarial sample in the digital domain is relatively enhanced.

7. A digital domain visible light adversarial sample performance testing system, characterized in that: The system comprises: The first processing module is configured to construct a normal sample according to the collected digital image or data, train the target detector using the normal sample, and record the target average detection accuracy M1 corresponding to the normal sample; A second processing module is configured to add a preset disturbance to the collected digital image or data to generate a digital domain visible light adversarial sample A1; The third processing module is configured to send the digital domain visible light adversarial sample A1 to the trained target detector, and record the target average detection accuracy M2 corresponding to the digital domain visible light adversarial sample A1; The fourth processing module is configured to calculate the difference between the target average detection accuracy M1 and the target average detection accuracy M2, and obtain the adversarial index EOCM1 before the adversarial sample performance test for the trained target detector D; A fifth processing module is configured to perform Fourier transform on the digital domain visible light adversarial sample A1 to obtain a digital domain visible light adversarial sample A2 after Fourier transform; A sixth processing module is configured to superimpose a transfer function on the digital domain visible light adversarial sample A2 to obtain a first superposition result A3; a seventh processing module, configured to perform inverse Fourier transform on the first superposition result A3 to obtain a second superposition result A4; An eighth processing module is configured to send the second superposition result A4 to the trained target detector D, and record the target average detection accuracy M3 corresponding to the second superposition result A4; A ninth processing module is configured to calculate the difference between the target average detection accuracy M1 and the target average detection accuracy M3, and obtain an adversarial index EOCM2 after the adversarial sample performance test for the trained target detector D; The tenth processing module is configured to obtain the adversarial relative change index EOCM before and after the adversarial sample performance test of the trained target detector based on the adversarial index EOCM1 and the adversarial index EOCM2, so as to determine the adversarial nature of the digital domain visible light adversarial sample A1 according to the adversarial relative change index EOCM.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps in the digital domain visible light adversarial sample performance testing method described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the digital domain visible light adversarial sample performance testing method described in any one of claims 1 to 6 are implemented.

Images