Image physical adversarial attack system and method based on LED local illumination modulation and rolling shutter effect

Abstract

The application discloses an image physical adversarial attack system and method based on LED local illumination modulation and rolling shutter effect, and the system comprises a local adversarial attack disturbance signal generation module, an LED modulation driving module, an LED illumination source, a CMOS image sensor and an image recognition model; the application generates a high-frequency brightness change signal by modulating the LED illumination source, utilizes the rolling shutter effect of the CMOS image sensor to implant invisible brightness information disturbance in the image acquisition process of a target object, and performs strong hidden physical adversarial attack on the image recognition model; the local light illumination micromodel of the application simulates the interference image by modeling through a Gaussian function, setting a specific modulation signal and superimposed output; the design batch sampling binary signal gradient optimization algorithm is used to optimize the modulation signal through reverse propagation, so that the accuracy and success rate of the adversarial attack are improved; and the adversarial attack disturbance form of the application can be generalized and migrated to different image samples and image recognition models.

Description

Technical Field

[0001] This invention belongs to the field of image recognition and artificial intelligence security technology, specifically relating to an image physics adversarial attack system and method based on LED local illumination modulation and rolling shutter effect. Background Technology

[0002] Adversarial attacks refer to attacks that add designed perturbations to input samples, causing deep learning models to output incorrect results. Adversarial attacks targeting image recognition models can be categorized into two types based on their attack methods: digital adversarial attacks and physical adversarial attacks. Early research on adversarial attack examples focused on digital adversarial attacks, where attackers could modify input samples at the pixel level. However, in the real physical world, targets are typically captured by image sensors and then directly transmitted to the target model. Therefore, it is difficult for attackers to directly access the image after it has been captured internally and add pixel-level perturbations. Compared to digital adversarial attacks, physical adversarial attacks directly implant perturbations in the real physical world, posing a serious threat to the target system and improving the feasibility of adversarial attacks.

[0003] The current new direction of physical adversarial attacks against image recognition models is optical-based attacks. These attacks mainly use optical means such as natural light, modulated light, or lasers to project onto objects in the real physical world to perform perturbation attacks. The perturbation patterns are set and optimized in combination with the target characteristics. This requires significant computational costs, which reduces generalization and has the disadvantage of being susceptible to ambient light, resulting in low executability. Summary of the Invention

[0004] To address the problems of high computational cost, low generalization, and low executability of existing optical-based image recognition adversarial attacks, this invention proposes an image physics adversarial attack system and method based on LED local illumination modulation and rolling shutter effect. A local illumination differentiable model is established, and Gaussian function modeling is used to set specific modulation signals and superimpose images simulating illumination interference. Simultaneously, a batch sampling binarized signal gradient optimization algorithm is designed to optimize the modulation signal through backpropagation, improving the accuracy and success rate of the adversarial attack. This invention effectively solves the problems of high computational cost, low generalization, and poor executability in existing optical adversarial attacks.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides an image physics adversarial attack system based on LED local illumination modulation and rolling shutter effect, including a local adversarial attack perturbation signal generation module, an LED signal modulation driving module, an LED illumination source, a CMOS image sensor and an image recognition model;

[0007] The local adversarial attack perturbation signal generation module is connected to the LED signal modulation driving module and is used to generate image adversarial samples and adversarial attack perturbation signals. Based on a local illumination differentiable model under the rolling shutter effect and a gradient-based batch sampling binarized signal gradient optimization algorithm, the local illumination differentiable model generates image adversarial samples and adversarial attack perturbation signals with local illumination perturbation information, and optimizes the parameters of the adversarial attack perturbation signals. The local illumination differentiable model is constructed by simulating the LED illumination intensity distribution using a Gaussian function, mathematically modeling brightness flicker simulation signal modulation, and channel conversion superimposed modulation illumination. The Gaussian function simulating the LED illumination intensity distribution describes the light intensity of the LED illumination area, the mathematically modeling brightness flicker simulation signal modulation generates a specific modulation pulse signal to cause the LED illumination source to produce high-speed brightness flicker, and the channel conversion superimposed modulation illumination determines the relationship between pixel values ​​before and after superimposed illumination in a real physical world scene.

[0008] The LED signal modulation driving module is connected to the local adversarial attack disturbance signal generation module and the LED lighting source. It is used to convert the generated adversarial attack disturbance signal into an LED lighting modulation signal and drive the LED lighting source to achieve lighting in different switching states.

[0009] The LED lighting source is placed in front of the subject to be identified to provide ambient lighting and to carry out physical countermeasures attacks. It implants brightness information disturbances invisible to the human eye into the target object area through high-frequency brightness modulation.

[0010] The CMOS image sensor is placed in front of the subject to be identified for image acquisition, and uses a rolling shutter exposure mechanism to expose and image line by line.

[0011] The image recognition model is connected to a CMOS image sensor and is a publicly available pre-trained model used for subject detection and classification of acquired images.

[0012] As a preferred technical solution, the Gaussian function simulating the light intensity distribution of LED lighting is specifically as follows:

[0013] The illuminated area is divided into a spot area and a diffusion area that do not interfere with each other. Both the spot area and the diffusion area follow a multidimensional normal distribution. The light intensity decay rate of the spot area is significantly faster than that of the diffusion area, and the light intensity peak is located at the center of the spot area.

[0014] The total light intensity at any point in the set space is composed of the linear superposition of the light intensity of the spot area and the light intensity of the diffuse area, and the maximum light intensity value is reached at the center coordinate point.

[0015] The attenuation rate of light intensity in the spot region from the center to the periphery is quantized by a first hyperparameter, which is positively correlated with the spot diameter; the attenuation rate of light intensity in the diffusion region from the center to the periphery is quantized by a second hyperparameter.

[0016] Define spatial location influence factors: The first factor represents the location of the set point. x The squared normalized distance from the center position; the second factor represents the setpoint at... y Standardized squared distance from the center position; third factor representation x direction and y Statistical correlation of directional changes;

[0017] Gaussian mode light intensity coefficients are set for the light spot region and the diffusion region, respectively, and the Gaussian mode light intensity coefficients are positively correlated with the maximum light intensity value.

[0018] As a preferred technical solution, the mathematical modeling of brightness flicker simulation signal modulation specifically includes:

[0019] Anti-illuminance is generated by the combined effect of spatial light intensity distribution and temporal modulation perturbation, where the final pixel value at any location point depends on the product of the original light intensity value and the integral of the modulation signal within a single line of exposure time of the CMOS sensor;

[0020] The modulation signal is a periodic rectangular pulse sequence with a constant height equal to the brightness value in the on state and a width equal to the modulation period duration. The on or off state of the LED in each period is controlled by a binary symbol sequence.

[0021] The sequence number of the binary symbol is in one-to-one correspondence with the row number of the CMOS image sensor, so that the independent LED brightness state is captured during the exposure of each row of pixels.

[0022] An adversarial perturbation pattern is generated as a normalized result of the modulation signal integral during exposure, and the adversarial perturbation pattern is embedded in the image spatial domain as a brightness perturbation carrier.

[0023] As a preferred technical solution, the channel conversion and superposition modulation illumination specifically includes:

[0024] The brightness channel of the original image is extracted as the basis for illumination interference;

[0025] By using a target object mask to define the effective area of ​​the anti-lighting effect, it is ensured that the disturbance only illuminates the surface of the target object;

[0026] Linearly superimpose the counter-lighting within the defined area with the original brightness channel;

[0027] Set the overflow value for the brightness channel. When the superimposed value exceeds the overflow value, the image will be truncated to prevent overexposure.

[0028] The final generated adversarial image contains brightness perturbations invisible to the human eye, and the adversarial image exhibits alternating bright and dark stripe features.

[0029] As a preferred technical solution, the execution flow of the gradient-based batch sampling binarized signal gradient optimization algorithm is as follows:

[0030] By using reparameterization technology, a differentiable function for the binary signal of LED lighting status is designed, converting the discrete binary signal into a continuous probability form;

[0031] Introducing Gamba noise into the differentiable function of a binarized signal enhances sampling randomness and increases the randomness of the sampling operation.

[0032] Gradient iteration is used to back-optimize the probability of the original signal in the digital domain, simultaneously performing optimization based on the current probability in each iteration. N Sub-sampling, superimposed with illumination to obtain N One adversarial example, N Each sample is input into the neural network as a batch for gradient calculation. The process is repeated iteratively until the loss function is maximized. The loss function consists of the model loss and the modulation signal transition loss. The model loss is used to increase the difference between the predicted label value and the true label value of the attack target. The modulation signal transition loss is used to minimize the state switching frequency of adjacent rows of modulation signals to reduce the switching between bright and dark stripes.

[0033] As a preferred technical solution, the formula for calculating the loss function is: αJ modle－ βJ jump ;

[0034] ;

[0035] ;

[0036] in, J modle For the model's loss, J jump This is the transition loss of the modulated signal. α and β These represent the influencing factors of model loss and jump loss, respectively. These influencing factors are set based on learning experience. N This represents the maximum number of samples output in each iteration. n The serial number of the adversarial sample. f ( X () represents the model output for clean samples. f ( X adv ) represents the model output for adversarial examples with localized illumination perturbations. For the first k The digital domain simulates the original signal. Representative and The adjacent next row of digital fields simulates the original signal. l This is the signal length.

[0037] As a preferred technical solution, the method of using a rolling shutter exposure mechanism for line-by-line exposure imaging specifically includes:

[0038] The photodiodes are turned on and off row by row, and each adjacent row of pixels is exposed at a fixed time difference. By controlling the periodic changes of the LED illumination source, the LED illumination source is in two different states, on and off, during the short exposure time of the CMOS image sensor and when exposing adjacent rows. This produces alternating bright and dark stripes in the image, forming brightness changes that are imperceptible to the human eye.

[0039] Secondly, this invention provides an adversarial attack method for an image physics adversarial attack system based on LED local illumination modulation and rolling shutter effect, comprising the following steps:

[0040] (1) Given an initial signal for the LED lighting source, the initial signal is derived from a switching probability signal. π 0 and π A two-dimensional vector composed of 1;

[0041] (2) Use reparameterization technology to binarize and differentiable process the initial signal of the LED lighting source, use Gamper noise to increase the randomness of the sampling operation, and output the true value of the modulated signal;

[0042] (3) Use the local illumination differentiable model as the generation model for adversarial examples to generate image adversarial examples and adversarial attack perturbation signals;

[0043] (4) Input the adversarial examples of the image into the image recognition model, calculate the result of the loss function, backpropagate to update the signal probability, until the result of the loss function is maximized;

[0044] (5) Convert the adversarial attack disturbance signal into an LED lighting source modulation signal and use a CMOS image sensor to perform physical adversarial attacks on the real shooting scene.

[0045] As a preferred technical solution, in step (4), the loss function consists of the model loss and the modulation signal transition loss, which is used to increase the difference between the predicted label value and the true label value of the attack target and reduce the switching of bright and dark stripes.

[0046] As a preferred technical solution, in step (4), the loss function optimization adopts a batch sampling strategy: each iteration generates N Composed of adversarial examples batchSimultaneously calculate the gradient update signal probability.

[0047] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0048] (1) This invention proposes a local illumination differentiable model under the rolling shutter effect, which describes the influence of light source intensity changes on image brightness. Through this model, the parameters of the modulation signal can be optimized, thereby improving the success rate of the attack.

[0049] (2) This invention proposes a gradient-based batch sampling binarized signal gradient optimization algorithm, which performs differentiable design on the binarized signal and generates the optimal adversarial disturbance by optimizing the parameters of the modulation signal, thereby improving the success rate of adversarial attacks.

[0050] (3) This invention proposes a generalized physical adversarial attack method based on local illumination modulation: This method utilizes the rolling shutter effect of CMOS image sensors to implant brightness perturbations that are imperceptible to the human eye during image acquisition, thereby inducing the deep learning system to make incorrect decisions. This method focuses on the modulation of "local" illumination, that is, using external light sources to generate perturbations only in the target object area, rather than adding perturbations to the entire image. This localized processing not only improves the concealment of the attack, but also reduces the dependence on the entire scene. While enhancing the generalization ability of adversarial perturbations, it further improves the concealment and feasibility of optical physical adversarial attacks. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of an image physics-based adversarial attack system based on LED local illumination modulation and rolling shutter effect, according to an embodiment of the present invention.

[0053] Figure 2 This is a flowchart illustrating the image physics adversarial attack method based on LED local illumination modulation and rolling shutter effect according to an embodiment of the present invention. Detailed Implementation

[0054] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0055] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0056] The rolling shutter effect refers to the distortion, tilting, stretching, or jelly-like jitter that occurs in images when shooting fast-moving objects or when the camera itself moves rapidly, due to the specific working principle of digital cameras or mobile phone cameras using CMOS image sensors.

[0057] like Figure 1 As shown, this embodiment provides an image physics adversarial attack system based on LED local illumination modulation and rolling shutter effect. The system includes a local adversarial attack perturbation signal generation module, an LED signal modulation driving module, an LED illumination source, a CMOS image sensor, and an image recognition model. The present invention generates a high-frequency brightness change signal by modulating the LED illumination source, and utilizes the rolling shutter effect of the CMOS image sensor to implant brightness information perturbation invisible to the human eye during the target object image acquisition process, thereby carrying out a highly covert physical adversarial attack on the image recognition model.

[0058] Furthermore, the local adversarial attack perturbation signal generation module is used to generate image adversarial samples and adversarial attack perturbation signals, specifically as follows:

[0059] Based on the local illumination differentiable model under the rolling shutter effect and the gradient-based batch sampling binarized signal gradient optimization algorithm, adversarial examples of images with local illumination perturbation information, adversarial attack perturbation signals, and parameters for optimizing adversarial attack perturbation signals are generated, respectively. The construction steps of the local illumination differentiable model are to simulate the LED lighting intensity distribution through Gaussian functions, simulate signal modulation by mathematical modeling brightness flicker, and superimpose modulation of the lighting through channel conversion. The gradient-based batch sampling binarized signal gradient optimization algorithm uses Gumbel-SoftMax reparameterization technology to establish a differentiable model of the binary signal of the LED lighting state, introduces Gumbel noise to increase the randomness of the sampling operation, optimizes the binarized signal using gradients, and uses batch sampling to solve the problem of signal probability medianization.

[0060] Furthermore, the Gaussian function simulates the light intensity distribution of LED lighting, specifically as follows:

[0061] The illuminated area is divided into a spot area and a diffusion area that do not interfere with each other. Both of them follow a multidimensional normal distribution. The light intensity decay rate in the spot area is significantly faster than that in the diffusion area, and the light intensity peak is located at the center of the spot area.

[0062] The total light intensity at any point in the set space is composed of the linear superposition of the light intensity of the spot area and the light intensity of the diffuse area, and the maximum light intensity value is reached at the center coordinate point.

[0063] The attenuation rate of light intensity in the spot region from the center to the periphery is quantized by a first hyperparameter, which is positively correlated with the spot diameter; the attenuation rate of light intensity in the diffusion region from the center to the periphery is quantized by a second hyperparameter.

[0064] Define the spatial location influence factors: the first factor represents the squared standardized distance of the point from the center in the x-direction; the second factor represents the squared standardized distance of the point from the center in the y-direction; the third factor characterizes the statistical correlation between the changes in the x-direction and the y-direction.

[0065] Gaussian mode light intensity coefficients are set for the light spot region and the diffusion region, respectively, and the Gaussian mode light intensity coefficients are positively correlated with the maximum light intensity value.

[0066] Furthermore, the expression for the luminous intensity of an LED lighting source at a certain point in the illuminated area is:

[0067] ;

[0068] ;

[0069] ;

[0070] ;

[0071] in, I ( x,y ) is a point in the illuminated area ( x,y ) light intensity, It is a Gaussian function model. I max The light intensity at the peak of the light intensity, with the center of the circle as the center. o The coordinates are ( x 0 ,y 0), I 1 represents the light intensity of the light spot area. I 2 represents the light intensity in the diffusion region. s z For hyperparameters, z =1 indicates the rate at which the light intensity in the spot area decreases from the center to the periphery, which is related to the spot diameter. d related, z =2 indicates the rate at which the light intensity in the diffusion region decreases from the center to the periphery. r The correlation is in two directions. l 1, l 2 represents the Gaussian mode intensity coefficients of the spot region and the diffusion region, respectively, and the maximum intensity. I max related, a Represented as the light point at x The degree to which the direction deviates from the center of the circle. b Represented as the light point at y The degree to which the direction deviates from the center of the circle. c express x , y Statistical correlation of directional changes.

[0072] Furthermore, the mathematical modeling of brightness flicker analog signal modulation specifically includes:

[0073] Anti-illuminance is generated by the combined effect of spatial light intensity distribution and temporal modulation perturbation, where the final pixel value at any location point depends on the product of the original light intensity value and the integral of the modulation signal within a single line of exposure time of the CMOS sensor;

[0074] The modulation signal is a periodic rectangular pulse sequence with a constant height equal to the brightness value in the on state and a width equal to the modulation period duration. The on or off state of the LED in each period is controlled by a binary symbol sequence.

[0075] The sequence number of the binary symbol is in one-to-one correspondence with the row number of the CMOS image sensor, so that the independent LED brightness state is captured during the exposure of each row of pixels.

[0076] An adversarial perturbation pattern is generated as a normalized result of the modulation signal integral during exposure, and this pattern is embedded in the image spatial domain as a brightness perturbation carrier.

[0077] It is understood that the mathematical modeling brightness flicker simulation signal modulation is used to generate a specific modulation pulse signal to cause the LED flashlight light to produce high-speed brightness flicker. The expression for the anti-lighting effect produced by the modulation of the LED flashlight is:

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] in, To modulate the anti-glare generated by the LED lighting source, For an LED lighting source at a certain point ( x,y The pixel value at () t e The exposure time per line for a CMOS image sensor. R To introduce counter-perturbation patterns, R ( k ) indicates a signal r ( t ) for the k The impact of the line r ( t (for time) t The changing LED brightness modulation signal t For the conversion gain of the CMOS image sensor, k For the row number of the exposure on the CMOS image sensor. t k This refers to the time when each pixel row of a CMOS image sensor begins exposure. a k Represented as the first k The row digital field simulates the original signal and is a sequence of binary symbols with values ​​of 1 or 0. m ( t - kT s ) represents a shape with a height of 1 and a width of 1. T s A rectangular pulse.

[0083] Furthermore, the channel conversion and superposition modulation of the illumination specifically refers to:

[0084] The brightness channel of the original image is extracted as the basis for illumination interference;

[0085] By using a target object mask to define the effective area of ​​the anti-lighting effect, it is ensured that the disturbance only illuminates the surface of the target object;

[0086] Linearly superimpose the counter-lighting within the defined area with the original brightness channel;

[0087] Set the overflow value for the brightness channel. When the superimposed value exceeds the overflow value, the image will be truncated to prevent overexposure.

[0088] The final generated adversarial image contains brightness perturbations invisible to the human eye, and the perturbation pattern presents a feature of alternating bright and dark stripes.

[0089] It is understandable that the channel conversion and superimposed modulation illumination are used to determine the relationship between pixel value changes before and after superimposed illumination in a real physical world scene; the image is converted from RGB space to HSI space, and only the influence of LED illumination on the I channel in HSI space is considered, reducing the complexity of the problem of pixel value changes before and after superimposed illumination; the expression for the adversarial image captured under local illumination attack is:

[0090] ;

[0091] ;

[0092] ;

[0093] in, X adv These are adversarial images captured under localized illumination attacks. I X To modulate the interference effect of LED light, the brightness channel of the original image is used. or The mask for the target object. This describes the effect of an LED light source illuminating a target object. c max Indicates the brightness channel I Overflow value.

[0094] Furthermore, the execution flow of the gradient-based batch sampling binarized signal gradient optimization algorithm is as follows:

[0095] By using reparameterization technology, a differentiable function for the binary signal of LED lighting status is designed, converting the discrete binary signal into a continuous probability form;

[0096] Introducing Gamba noise into the differentiable function of a binarized signal enhances sampling randomness and increases the randomness of the sampling operation.

[0097] Gradient iteration is used to back-optimize the probability of the original signal in the digital domain, simultaneously performing optimization based on the current probability in each iteration. N Sub-sampling, superimposed with illumination to obtain N One adversarial example, NEach sample is input into the neural network as a batch for gradient calculation. The process is repeated iteratively until the loss function is maximized. The loss function consists of the model loss and the modulation signal transition loss. The model loss is used to increase the difference between the predicted label value and the true label value of the attack target. The modulation signal transition loss is used to minimize the state switching frequency of adjacent rows of modulation signals to reduce the switching between bright and dark stripes.

[0098] In one specific embodiment, the reparameterization technique employs Gumbel-SoftMax, an important method in the field of deep learning for solving the problem of non-differentiability of discrete random variables. Its core purpose is to "approximately" sample discrete categories in scenarios where gradients need to be calculated (such as training neural networks using backpropagation), while keeping the parameters of the sampling process differentiable with respect to the probability distribution.

[0099] Furthermore, the formula for calculating the differentiable function of a binarized signal is:

[0100] ;

[0101] ;

[0102] ;

[0103] ;

[0104] in, p The real value of the modulated signal. This is an approximation of the modulated signal. The sampled vector, It is an approximate representation of the one-hot vector. m It's a hyperparameter. π 0 represents the probability that the signal is 0, i.e., the signal is off; π 1 represents the probability that the signal is 1. This represents the Gamper noise of the signal.

[0105] Furthermore, the formula for calculating the loss function is: αJ modle－ βJ jump ;

[0106] ;

[0107] ;

[0108] in, J modle For the model's loss, J jump This is the transition loss of the modulated signal.α and β These represent the influencing factors of model loss and jump loss, respectively. These influencing factors are set based on learning experience. N This represents the maximum number of samples output in each iteration. n The serial number of the adversarial sample. f ( X () represents the model output for clean samples. f ( X adv ) represents the model output for adversarial examples with localized illumination perturbations. For the first k The digital domain simulates the original signal. Representative and The adjacent next row of digital fields simulates the original signal. l This is the signal length.

[0109] Furthermore, the LED signal modulation driving module is used to convert the adversarial attack disturbance signal generated by the local adversarial attack disturbance signal generation module into an LED lighting modulation signal, load the modulation signal and drive the LED to achieve lighting in different switching states.

[0110] Furthermore, the LED lighting source is used to provide ambient lighting and to implement physical adversarial attacks. By modulating the LED lighting source to generate high-frequency brightness variation signals, and utilizing the rolling shutter effect of the CMOS image sensor, invisible brightness information perturbations are implanted during the target object image acquisition process. These brightness information perturbations affect the features extracted by the image recognition model, reducing the detection success rate and classification accuracy of the image recognition model, thus achieving a highly covert physical adversarial attack on the image recognition model.

[0111] Furthermore, the CMOS image sensor is the image acquisition module of the image recognition system, which is placed in front of the subject to be recognized to perform exposure and image imaging.

[0112] Furthermore, the image recognition model is a publicly available image processing model with pre-trained weights, used to detect and classify the main subjects contained in the image input by the image acquisition module.

[0113] like Figure 2 As shown, this embodiment also provides an image physics adversarial attack method based on LED local illumination modulation and rolling shutter effect, including the following steps:

[0114] (1) Given the initial signal of the LED lighting source v initial signal v It is determined by the switching probability signal π 0 and π A two-dimensional vector composed of 1;

[0115] (2) Use Gumbel-SoftMax technology to perform binarization and differentiability processing of the initial signal of the LED lighting source, use Gumbel noise to increase the randomness of the sampling operation, and output the true value of the modulated signal. p ;

[0116] (3) Use the local illumination differentiable model as the generation model for adversarial examples to generate image adversarial examples and adversarial attack perturbation signals;

[0117] (4) Input the image adversarial example into the image recognition model, calculate the result of the loss function, and backpropagate to update the signal probability. v Repeat (1) to (3) until the loss function result is maximized;

[0118] Furthermore, the loss function consists of the model loss and the modulation signal transition loss. The optimization objective of the loss function is to increase the difference between the predicted label value and the true label value of the attack target and to reduce the switching between bright and dark stripes.

[0119] Furthermore, the loss function optimization employs a batch sampling strategy: each iteration generates... n Composed of adversarial examples batch Simultaneously calculate the gradient update signal probability.

[0120] (5) Convert the adversarial attack disturbance signal into an LED lighting source modulation signal and use a CMOS image sensor to perform physical adversarial attacks on the real shooting scene.

[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0122] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. An image physical adversarial attack system based on LED local illumination modulation and rolling shutter effect, characterized in that, The local adversarial attack disturbance signal generation module, the LED signal modulation driving module, the LED illumination source, the CMOS image sensor and the image recognition model are included. The local adversarial attack disturbance signal generation module is connected with the LED signal modulation driving module, and is used for generating image adversarial samples and adversarial attack disturbance signals; the local adversarial attack disturbance signal generation module generates image adversarial samples and adversarial attack disturbance signals with local light disturbance information based on a local light illumination differentiable model under a rolling shutter effect and a gradient-based batch sampling binary signal gradient optimization algorithm, and optimizes parameters of the adversarial attack disturbance signals; the local light illumination differentiable model is constructed by simulating LED illumination intensity distribution by a Gaussian function, modeling brightness flicker simulation signal modulation, and channel conversion superposition modulation illumination; the Gaussian function simulating LED illumination intensity distribution is used for describing the light intensity of the LED illumination area, the mathematical modeling brightness flicker simulation signal modulation is used for generating a specific modulation pulse signal to make the LED illumination source produce high-speed brightness flicker, and the channel conversion superposition modulation illumination is used for determining the change relationship of pixel values before and after superimposed illumination in a real physical world scene. The LED signal modulation driving module is connected with the local adversarial attack disturbance signal generation module and the LED illumination source, and is used for converting the generated adversarial attack disturbance signals into LED illumination modulation signals, and driving the LED illumination source to realize illumination in different switching states. The LED illumination source is placed in front of the to-be-identified subject, and is used for providing environmental illumination and implementing physical adversarial attacks, and implants brightness information disturbance invisible to human eyes in the target object area through high-frequency brightness modulation. The CMOS image sensor is placed in front of the to-be-identified subject, and is used for image acquisition and row-by-row exposure imaging by using a rolling shutter exposure mechanism. The image recognition model is connected with the CMOS image sensor, is a public pre-training model, and is used for subject detection and classification on the collected images.

2. The system of claim 1, wherein the LED-based local illumination modulation and rolling shutter effect are used to physically attack the image. The Gaussian function simulates the LED illumination intensity distribution, and specifically: The illumination area is divided into non-interfering spot areas and diffusion areas, and the spot areas and the diffusion areas both obey a multi-dimensional normal distribution, wherein the light intensity decay rate of the spot areas is significantly faster than that of the diffusion areas, and the light intensity peak value is located at the center position of the spot areas; The total light intensity at any point in space is composed of the linear superposition of the light intensity of the spot areas and the light intensity of the diffusion areas, and the maximum light intensity value is reached at the center coordinate point; The first hyperparameter is used to quantify the decay rate of the light intensity of the spot areas from the center to the periphery, and the first hyperparameter is positively correlated with the spot diameter; the second hyperparameter is used to quantify the decay rate of the light intensity of the diffusion areas from the center to the periphery; Defining the spatial position impact factor: a first factor represents the setpoint in x the normalized square distance of the direction from the center position; The second factor represents the normalized square distance of the setpoint from y the standard deviation of the direction from the center position; the third factor represents x the statistical correlation of the direction from y the standard deviation of the direction The Gaussian model light intensity coefficients of the spot areas and the diffusion areas are set respectively, and the Gaussian model light intensity coefficients are positively correlated with the maximum light intensity value.

3. The system of claim 1, wherein the LED-based local illumination modulation and rolling shutter effect are used to physically attack the image. The mathematical modeling brightness flicker simulation signal modulation is specifically: The adversarial illumination is generated by the joint action of the spatial light intensity distribution and the time modulation disturbance, wherein the final pixel value at any position point depends on the product of the original light intensity value and the integral of the modulation signal within the single-row exposure time of the CMOS sensor. The modulation signal is a periodic rectangular pulse sequence, the height is constant as the on-state brightness value, the width is equal to the modulation period length, and the on or off state of the LED in each period is controlled by a binary symbol sequence; The sequence number of the binary symbol sequence corresponds to the CMOS image sensor row number one by one, so that an independent LED brightness state is captured during the exposure of each row of pixels; The adversarial disturbance pattern is generated as the normalized result of the integral of the modulation signal during exposure, which is embedded in the image spatial domain as a brightness disturbance carrier.

4. The system of claim 1, wherein the LED-based local illumination modulation and rolling shutter effect are used to physically attack the image. The channel conversion superimposes the modulation light, specifically: Extract the brightness channel of the original image as the basis for light interference; Limit the effective action area of the adversarial light through the target object mask to ensure that the disturbance only illuminates the target object surface; Linearly superimpose the adversarial light in the limited area with the original brightness channel; Set the overflow value of the brightness channel, and perform truncation processing when the superimposed value exceeds the overflow value to prevent overexposure during imaging; Finally, generate an adversarial image containing invisible brightness disturbances to the human eye, which presents the characteristics of alternating light and dark stripes.

5. The system of claim 1, wherein the LED-based local illumination modulation and rolling shutter effect are used to physically attack the image. The execution process of the gradient-based batch sampling binary signal gradient optimization algorithm is: Use the reparameterization technique to design a differentiable function of the LED lighting state binary signal, converting the discrete binary signal to a continuous probability form; Introduce Gumble noise into the differentiable function of the binary signal to enhance the randomness of the sampling operation; Simultaneously based on the current probability in one iteration N Sub-sampling, superimposing illumination N An adversarial sample, N A batch of samples is input into a neural network for gradient calculation, and repeated iterations are performed until the loss function is maximized. The loss function is composed of a model loss and a modulation signal jump loss. The model loss is used to increase the difference between the predicted label value and the true label value of the attack target. The modulation signal jump loss is used to minimize the state switching frequency of adjacent rows of modulation signals to reduce the switching of bright and dark stripes.

6. The system of claim 5, wherein the LED-based local illumination modulation and rolling shutter effect are physically counteracted by the image. The loss function is calculated as αJ modle－ βJ jump ; ； ； wherein, J modle is the loss of the model, J jump is the jump loss of the modulated signal, α and β respectively represent the influence factors of the model loss and the jump loss, the influence factors are set according to the experience of learning, N is the maximum value of the number of samples output per iteration, n is the serial number of the adversarial sample, f ( X ) is the model output of the clean sample, f ( X adv ) is the model output of the adversarial sample with local light disturbance, is the first k row of the digital domain analog original signal, represents the next row of the digital domain analog original signal adjacent to , and l is the signal length.

7. The system of claim 1, wherein the LED-based local illumination modulation and rolling shutter effect are used to physically attack the image. The use of a rolling shutter exposure mechanism for row-by-row exposure imaging is as follows: Open and close the photodiode row by row, and each adjacent row of pixels is exposed at a fixed time difference; by controlling the periodic change of the LED lighting source, the LED lighting source is in two different states of on and off during the short time of CMOS image sensor exposure imaging and when adjacent rows are exposed, alternating light and dark stripes are produced in the imaging image, forming invisible brightness changes to the human eye.

8. The adversarial attack method of claim 1-7 based on LED local illumination modulation and rolling shutter effect of image physical adversarial attack system, characterized in that, The steps include: (1) a given LED lighting source initial signal, said initial signal is a two-dimensional vector consisting of a switching probability signal π 0 and π 1 ; π 0 represents the probability of the signal being 0; π 1 represents the probability of the signal being 1; (2) Use the reparameterization technique to perform binary differentiable processing of the initial signal of the LED lighting source, use Gumble noise to increase the randomness of the sampling operation, and output the true value of the modulation signal; (3) Use the local light differentiable model as the generation model of the adversarial sample to generate the image adversarial sample and adversarial attack disturbance signal; (4) Input the image adversarial sample into the image recognition model to calculate the result of the loss function, and update the signal probability through back propagation until the loss function result is maximized; (5) Convert the adversarial attack disturbance signal to the LED lighting source modulation signal, and use the CMOS image sensor to physically attack the real shooting scene.

9. The method of claim 8, wherein, In step (4), the loss function is composed of the loss of the model and the jump loss of the modulation signal, which is used to increase the difference between the predicted label value and the true label value of the attack target and reduce the switching of the light and dark stripes.

10. The method of claim 8, wherein, In step (4), the loss function optimization adopts a batch sampling strategy: each iteration generates N a group of adversarial samples batch , and the gradient update signal probability is calculated synchronously.

Images