Image physical confrontation attack system and method based on LED local illumination modulation and rolling shutter effect

Abstract

The invention discloses an image physical confrontation attack system and method based on LED local illumination modulation and a rolling shutter effect. The system comprises a local confrontation attack disturbance signal generation module, an LED modulation driving module, an LED illumination source, a CMOS image sensor and an image recognition model. A high-frequency brightness change signal is generated by modulating an LED illumination source, brightness information disturbance invisible to human eyes is implanted in the target object image acquisition process by utilizing the rolling shutter effect of a CMOS image sensor, and strong hidden physical countermeasure attack is performed on an image recognition model; according to the local illumination micromodel, modeling is carried out through a Gaussian function, a specific modulation signal is set, and a simulation interference image is superposed and output; a batch sampling binary signal gradient optimization algorithm is designed to carry out back propagation to optimize modulation signals, so that the accuracy and success rate of attack resistance are improved; the attack disturbance resisting form can be generalized and migrated to different image samples and image recognition models.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition and artificial intelligence security technology, and specifically relates to an image physical counterattack system and method based on LED local lighting modulation and rolling shutter effect. Background Art

[0002] Adversarial attacks involve adding designed perturbations to input samples to cause deep learning models to output erroneous results. Adversarial attacks against image recognition models can be categorized into two types: digital and physical. Early research on adversarial attack samples focused on digital attacks, in which attackers can modify input samples at the pixel level. However, in the real physical world, targets are typically captured by image sensors and then directly transmitted to the target model. This makes it difficult for attackers to directly access the image captured within the system and add pixel-level perturbations. Compared to digital adversarial attacks, physical adversarial attacks directly implant perturbations in the real physical world, posing a serious threat to the target system and making adversarial attacks more feasible.

[0003] The new direction of physical adversarial attacks against image recognition models is optical-based attacks, which mainly use optical means such as natural light, modulated light or lasers to project onto objects in the real physical world to perform perturbation attacks. The perturbation pattern is set and optimized based on the target characteristics. This requires significant computational costs, which reduces generalization and has the disadvantage of being easily affected by ambient light, resulting in low executability. Summary of the Invention

[0004] In response to the problems of high computational cost, low generalization and low feasibility of existing optical-based image recognition counterattacks, the present invention proposes an image physical counterattack system and method based on LED local illumination modulation and rolling shutter effect. A local illumination differentiable model is established, and an image simulating illumination interference is output by Gaussian function modeling, setting a specific modulation signal and superimposing the output. At the same time, a batch sampling binarization signal gradient optimization algorithm is designed to back-propagate and optimize the modulation signal to improve the accuracy and success rate of the counterattack. The present invention can effectively solve the problems of high computational cost, low generalization and poor feasibility of existing optical counterattacks.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides an image physics adversarial attack system based on LED local illumination modulation and rolling shutter effect, comprising a local adversarial attack disturbance signal generation module, an LED signal modulation drive module, an LED illumination source, a CMOS image sensor, and an image recognition model; The local adversarial attack disturbance signal generation module is connected to the LED signal modulation driving module, and is used to generate image adversarial samples and adversarial attack disturbance signals; the local adversarial attack disturbance signal generation module generates image adversarial samples and adversarial attack disturbance signals with local illumination disturbance information based on the local illumination differentiable model under the rolling shutter effect and the gradient-based batch sampling binary signal gradient optimization algorithm, and optimizes the parameters of the adversarial attack disturbance signal; the local illumination differentiable model is constructed by simulating the LED illumination intensity distribution through Gaussian function, mathematically modeling the brightness flicker simulation signal modulation, and channel conversion superposition modulated illumination; the Gaussian function simulated LED illumination intensity distribution is used to describe the light intensity of the LED illumination area, the mathematically modeled brightness flicker simulation signal modulation is used to generate a specific modulated pulse signal to make the LED illumination source produce high-speed brightness flicker, and the channel conversion superposition modulated illumination is used to determine the change relationship between the pixel values ​​before and after the superimposed illumination in the real physical world scene; The LED signal modulation driving module is connected to the local counter-attack disturbance signal generating module and the LED lighting source, and is used to convert the generated counter-attack disturbance signal into an LED lighting modulation signal, and drive the LED lighting source to realize lighting in different switching states; The LED lighting source is placed in front of the subject to be identified to provide ambient lighting and implement physical countermeasure attacks, implanting brightness information disturbances invisible to the human eye in the target object area through high-frequency brightness modulation; The CMOS image sensor is placed in front of the subject to be identified, used for image acquisition, and uses a rolling shutter exposure mechanism to expose the image line by line; The image recognition model is connected to the CMOS image sensor and is a public pre-trained model used to detect and classify subjects in captured images.

[0006] As a preferred technical solution, the Gaussian function simulates the LED lighting intensity distribution, specifically: The illumination area is divided into a spot area and a diffusion area that do not interfere with each other. Both the spot area and the diffusion area obey a multidimensional normal distribution. The light intensity decay rate of the spot area is significantly faster than that of the diffusion area, and the light intensity peak is located at the center of the spot area. The total light intensity at any point in the set space is composed of the linear superposition of the light intensity in the spot area and the light intensity in the diffusion area, and reaches the maximum light intensity value at the central coordinate point; The attenuation rate of the light intensity in the spot area from the center to the periphery is quantified by a first hyperparameter, wherein the first hyperparameter is positively correlated with the diameter of the spot; the attenuation rate of the light intensity in the diffusion area from the center to the periphery is quantified by a second hyperparameter; Define the spatial position influence factors: The first factor represents the set point xThe normalized square of the distance from the center position; the second factor represents the set point y The square of the normalized distance of the direction from the center position; the third factor representation x Direction and y statistical relevance of directional changes; The Gaussian mode light intensity coefficients of the light spot area and the diffusion area are set respectively, and the Gaussian mode light intensity coefficients are positively correlated with the maximum light intensity value.

[0007] As a preferred technical solution, the mathematical modeling brightness flicker analog signal modulation is specifically as follows: The adversarial illumination is generated by the combined effect of spatial light intensity distribution and temporal modulation perturbation, where the final pixel value at any position depends on the product of the original light intensity value and the integral of the modulation signal within the exposure time of a single line of the CMOS sensor; The modulation signal is a periodic rectangular pulse sequence, the height of which is constant at the brightness value of the on state, the width of which is equal to the modulation period, and the on or off state of the LED in each period is controlled by a binary symbol sequence; The serial numbers of the binary symbol sequence correspond one-to-one with the row numbers of the CMOS image sensor, so that each row of pixels captures an independent LED brightness state during exposure; An adversarial perturbation pattern is generated as the normalized result of the integration of the modulation signal during exposure, and the adversarial perturbation pattern is embedded in the image space domain as a brightness perturbation carrier.

[0008] As a preferred technical solution, the channel conversion superposition modulated illumination is specifically as follows: Extract the brightness channel of the original image as the basis of light interference; The effective area of ​​the anti-illumination is limited by the target object mask to ensure that the perturbation only illuminates the surface of the target object; Linearly superimpose the adversarial illumination in the limited area with the original brightness channel; Set the overflow value of the brightness channel. When the superposition value exceeds the overflow value, it will be truncated to prevent overexposure of the image. Finally, an adversarial image containing brightness perturbations that are invisible to the human eye is generated, and the adversarial image presents the characteristics of alternating light and dark stripes.

[0009] As a preferred technical solution, the execution process of the gradient-based batch sampling binarization signal gradient optimization algorithm is as follows: The reparameterization technique is used to design a differentiable function of the binary signal of LED lighting status, and the discrete binary signal is converted into a continuous probability form. Introducing Gambel noise into the differentiable function of the binary signal enhances the randomness of sampling and increases the randomness of the sampling operation; The gradient iteration is used to reversely optimize the probability of the original signal simulated in the digital domain, and the current probability is simultaneously used in one iteration. N Subsampling, superimposing lighting to obtain N adversarial examples, N The samples are input into the neural network as a batch for gradient calculation, and the iteration is repeated until the loss function is maximized. The loss function consists of the model loss and the jump loss of the modulation signal. The loss of the model is used to increase the difference between the predicted label value and the true label value of the attack target, and the jump loss of the modulation signal is used to minimize the state switching frequency of the modulation signal in adjacent rows to reduce the switching between bright and dark stripes.

[0010] As a preferred technical solution, the loss function is calculated as follows: αJ modle－ βJ jump ; ; ; in, J modle is the loss of the model, J jump is the transition loss of the modulation signal, α and β They are respectively expressed as the impact factors of model loss and jump loss, and the impact factors are set according to learning experience. N is the maximum number of samples output in each iteration, n is the serial number of the adversarial sample, f ( X ) is the model output of the clean sample, f ( X adv ) is the model output of the adversarial sample with local illumination perturbations, For the k Line digital domain analog original signal, Representatives and The next row of digital domain simulates the original signal. l is the signal length.

[0011] As a preferred technical solution, the rolling shutter exposure mechanism is used to expose the image line by line, specifically: The photodiodes are turned on and off row by row, and each adjacent row of pixels is exposed with a fixed time difference. By controlling the periodic changes of the LED illumination source, the LED illumination source is in two different states, on and off, during the brief time when the CMOS image sensor is exposing the image and when exposing adjacent rows. This produces alternating light and dark stripes in the image, forming brightness changes that are imperceptible to the human eye.

[0012] In a second aspect, the present invention provides a method for countering an attack on an image physics counterattack system based on LED local illumination modulation and rolling shutter effect, comprising the following steps: (1) Given an initial signal of an LED lighting source, the initial signal is a switching probability signal π 0 and π 1 two-dimensional vector; (2) Use reparameterization technology to perform binary differentiable processing of the initial signal of the LED lighting source, use Gambel noise to increase the randomness of the sampling operation, and output the true value of the modulated signal; (3) Using the local illumination differentiable model as the generative model for adversarial samples, we generate image adversarial samples and adversarial attack perturbation signals; (4) Input the image adversarial sample into the image recognition model, calculate the result of the loss function, and backpropagate to update the signal probability until the loss function result is maximized; (5) The adversarial attack disturbance signal is converted into an LED lighting source modulation signal, and a CMOS image sensor is used to perform a physical adversarial attack on the real shooting scene.

[0013] As a preferred technical solution, in step (4), the loss function is composed of the loss of the model and the jump loss of the modulation signal, which is used to increase the difference between the predicted label value and the true label value of the attack target and reduce the switching of light and dark stripes.

[0014] As a preferred technical solution, in step (4), the loss function optimization adopts a batch sampling strategy: each iteration generates N adversarial examples batch , synchronously calculate the gradient update signal probability.

[0015] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) This paper proposes a differentiable model of local illumination under the rolling shutter effect, which describes the impact of light source intensity changes on image brightness. Through this model, the parameters of the modulation signal can be optimized, thereby improving the success rate of the attack.

[0016] (2) This paper proposes a gradient optimization algorithm for batch sampling binarized signals based on gradient, performs a differentiable design on the binarized signal, generates the optimal adversarial perturbation by optimizing the parameters of the modulated signal, and improves the success rate of the adversarial attack.

[0017] (3) This paper proposes a generalized physical adversarial attack method based on local illumination modulation: This method utilizes the rolling shutter effect of CMOS image sensors to implant brightness perturbations that are imperceptible to the human eye during image acquisition, thereby inducing deep learning systems to make incorrect decisions. This method focuses on "local" illumination modulation, that is, using external light sources to generate perturbations only in the target object area, rather than adding perturbations to the entire image. This localized processing not only improves the stealthiness of the attack, but also reduces dependence on the entire scene. While enhancing the generalization ability of the adversarial perturbation, it further improves the stealth and feasibility of optical physical adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 Schematic diagram of an image physics countermeasure attack system based on LED local illumination modulation and rolling shutter effect according to an embodiment of the present invention.

[0020] Figure 2 Schematic diagram of the flow of an image physics countermeasure attack method based on LED local illumination modulation and rolling shutter effect according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0022] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0023] The rolling shutter effect refers to the distortion, tilt, stretching or jelly-like jitter that occurs in images when shooting fast-moving objects or the camera itself moves quickly in digital cameras or mobile phone cameras using CMOS image sensors due to their specific working mode.

[0024] like Figure 1 As shown, this embodiment provides an image physical adversarial attack system based on LED local illumination modulation and rolling shutter effect, which includes a local adversarial attack disturbance signal generation module, an LED signal modulation drive module, an LED illumination source, a CMOS image sensor, and an image recognition model. The present invention modulates the LED illumination source to generate a high-frequency brightness change signal, utilizes the rolling shutter effect of the CMOS image sensor, and implants brightness information disturbances that are invisible to the human eye during the target object image acquisition process, thereby performing a strong and covert physical adversarial attack on the image recognition model.

[0025] Furthermore, the local adversarial attack disturbance signal generation module is used to generate image adversarial samples and adversarial attack disturbance signals, specifically: A local illumination differentiable model based on the rolling shutter effect and a gradient-based batch sampling binary signal gradient optimization algorithm are used to generate image adversarial samples with local illumination disturbance information, adversarial attack disturbance signals, and optimize the parameters of the adversarial attack disturbance signals, respectively. The construction steps of the local illumination differentiable model are to simulate the LED lighting intensity distribution through Gaussian function, mathematically model the brightness flicker simulation signal modulation, and channel conversion and superposition modulated illumination. The gradient-based batch sampling binary signal gradient optimization algorithm uses the Gumbel-SoftMax reparameterization technology to establish a differentiable model of the LED lighting state binary signal, introduces Gumbel noise to increase the randomness of the sampling operation, uses gradient to optimize the binary signal, and adopts batch sampling measures to solve the signal probability median problem.

[0026] Furthermore, the Gaussian function simulates the LED lighting intensity distribution, specifically: The illumination area is divided into a spot area and a diffusion area that do not interfere with each other. Both areas obey a multidimensional normal distribution. The light intensity decay rate in the spot area is significantly faster than that in the diffusion area, and the light intensity peak is located at the center of the spot area. The total light intensity at any point in the set space is composed of the linear superposition of the light intensity in the spot area and the light intensity in the diffusion area, and reaches the maximum light intensity value at the central coordinate point; The attenuation rate of the light intensity in the spot area from the center to the periphery is quantified by a first hyperparameter, wherein the first hyperparameter is positively correlated with the diameter of the spot; the attenuation rate of the light intensity in the diffusion area from the center to the periphery is quantified by a second hyperparameter; Define the spatial position influencing factors: the first factor represents the square of the normalized distance of the point from the center position in the x direction; the second factor represents the square of the normalized distance of the point from the center position in the y direction; the third factor represents the statistical correlation of the changes in the x and y directions; The Gaussian mode light intensity coefficients of the light spot area and the diffusion area are set respectively, and the Gaussian mode light intensity coefficients are positively correlated with the maximum light intensity value.

[0027] Furthermore, the light intensity of the LED lighting source at a certain point in the illumination area is expressed as: ; ; ; ; in, I ( x,y ) is a point in the illumination area ( x,y ) light intensity, is the Gaussian function model, I max is the light intensity at the peak value, and the center of the circle is o The coordinates of x 0 ,y 0), I 1 is the light intensity in the spot area, I 2 is the light intensity in the diffusion area, s z is a hyperparameter, z =1 indicates the attenuation rate of the light intensity from the center to the periphery of the spot area, which is related to the spot diameter d related, z =2 indicates the attenuation rate of the light intensity in the diffusion area from the center to the periphery. r is the correlation in two directions, l 1, l 2 are Gaussian mode light intensity coefficients of the spot area and the diffusion area, respectively, and the maximum light intensity I max related, a Indicated as the illumination point at x The degree of deviation from the center of the circle in direction, b Indicated as the illumination point at y The degree of deviation from the center of the circle in direction, c express x 、 y Statistical correlations of directional changes.

[0028] Furthermore, the mathematical modeling brightness flicker analog signal modulation is specifically: The adversarial illumination is generated by the combined effect of spatial light intensity distribution and temporal modulation perturbation, where the final pixel value at any position depends on the product of the original light intensity value and the integral of the modulation signal within the exposure time of a single line of the CMOS sensor; The modulation signal is a periodic rectangular pulse sequence, the height of which is constant at the brightness value of the on state, the width of which is equal to the modulation period, and the on or off state of the LED in each period is controlled by a binary symbol sequence; The serial numbers of the binary symbol sequence correspond one-to-one with the row numbers of the CMOS image sensor, so that each row of pixels captures an independent LED brightness state during exposure; An adversarial perturbation pattern is generated as the normalized result of the integration of the modulation signal during exposure, which is embedded into the image space domain as a brightness perturbation carrier.

[0029] It is understandable that the mathematical modeling brightness flicker analog signal modulation is used to generate a specific modulation pulse signal to make the LED flashlight light produce high-speed brightness flicker. The expression for the anti-illumination generated by the modulated LED flashlight is: ; ; ; ; in, To modulate the opposing light generated by the LED lighting source, For LED lighting source at a certain point ( x, y ), t e is the exposure time of each line of the CMOS image sensor, R is the adversarial perturbation pattern introduced, R ( k ) indicates a signal r ( t ) k The impact of the line, r ( t ) is the time t Changing LED brightness modulation signal, t is the conversion gain of the CMOS image sensor, k is the row number of the CMOS image sensor exposure, t k is the time when each pixel row of the CMOS image sensor starts to be exposed, a k Expressed as k The digital domain simulates the original signal and is a sequence of binary symbols, whose values ​​are 1 or 0. m ( t - kT s ) is represented by a height of 1 and a width of T s rectangular pulse.

[0030] Furthermore, the channel conversion superposition modulated illumination is specifically as follows: Extract the brightness channel of the original image as the basis of light interference; The effective area of ​​the anti-illumination is limited by the target object mask to ensure that the perturbation only illuminates the surface of the target object; Linearly superimpose the adversarial illumination in the limited area with the original brightness channel; Set the overflow value of the brightness channel. When the superposition value exceeds the overflow value, it will be truncated to prevent overexposure of the image. Finally, an adversarial image containing brightness perturbations that are invisible to the human eye is generated, and the perturbation pattern presents the characteristics of alternating light and dark stripes.

[0031] It can be understood that the channel conversion superimposed modulated illumination is used to determine the change relationship between pixel values ​​before and after superimposed illumination in real physical world scenes; the image is converted from RGB space to HSI space, and only the impact of LED illumination on the I channel in HSI space is considered, which reduces the complexity of the problem of pixel value changes before and after superimposed illumination; the expression of the adversarial image captured under local illumination attack is: ; ; ; in, X adv is an adversarial image taken under local illumination attack, I X To modulate the LED light interference effect, the brightness channel of the original image is used. or is the mask of the target object, The effect of LED lighting source on the target object. c max Represents the brightness channel I Overflow value.

[0032] Furthermore, the execution process of the gradient-based batch sampling binarization signal gradient optimization algorithm is as follows: The reparameterization technique is used to design a differentiable function of the binary signal of LED lighting status, and the discrete binary signal is converted into a continuous probability form. Introducing Gambel noise into the differentiable function of the binary signal enhances the randomness of sampling and increases the randomness of the sampling operation; The gradient iteration is used to reversely optimize the probability of the original signal simulated in the digital domain, and the current probability is simultaneously used in one iteration. N Subsampling, superimposing lighting to obtain N adversarial examples, N The samples are input into the neural network as a batch for gradient calculation, and the iteration is repeated until the loss function is maximized. The loss function consists of the model loss and the jump loss of the modulation signal. The loss of the model is used to increase the difference between the predicted label value and the true label value of the attack target, and the jump loss of the modulation signal is used to minimize the state switching frequency of the modulation signal in adjacent rows to reduce the switching between bright and dark stripes.

[0033] In a specific embodiment, the reparameterization technology uses Gumbel-SoftMax, which is an important method for solving the problem of non-differentiability of discrete random variables in the field of deep learning. Its core purpose is to "approximately" sample discrete categories in scenarios where gradient calculation is required (such as using backpropagation to train neural networks), while keeping the sampling process differentiable with respect to the parameters of the probability distribution.

[0034] Furthermore, the calculation formula of the differentiable function of the binary signal is: ; ; ; ; in, p is the real value of the modulating signal, is the approximate value of the modulation signal, is the sampling vector, is an approximate representation of a one-hot vector, m is a hyperparameter, π 0 represents the probability that the signal is 0, that is, the signal is off; π 1 represents the probability that the signal is 1, represents the Gambel noise of the signal.

[0035] Furthermore, the loss function is calculated as αJ modle－ βJ jump ; ; ; in, J modle is the loss of the model, J jump is the transition loss of the modulation signal, α andβ They are respectively expressed as the impact factors of model loss and jump loss, and the impact factors are set according to learning experience. N is the maximum number of samples output for each iteration, n is the serial number of the adversarial sample, f ( X ) is the model output of the clean sample, f ( X adv ) is the model output of the adversarial sample with local illumination perturbations, For the k Line digital domain analog original signal, Representatives and The next row of digital domain simulates the original signal. l is the signal length.

[0036] Furthermore, the LED signal modulation driving module is used to convert the anti-attack disturbance signal generated by the local anti-attack disturbance signal generation module into an LED lighting modulation signal, load the modulation signal and drive the LED to achieve lighting in different switching states.

[0037] Furthermore, the LED illumination source is used to provide ambient lighting and implement physical adversarial attacks. By modulating the LED illumination source to generate a high-frequency brightness variation signal, and leveraging the rolling shutter effect of the CMOS image sensor, a brightness perturbation invisible to the human eye is implanted during the target object image acquisition process. This brightness perturbation affects the features extracted by the image recognition model, reducing the detection success rate and classification accuracy of the image recognition model, thereby achieving a highly concealed physical adversarial attack on the image recognition model.

[0038] Furthermore, the CMOS image sensor is an image acquisition module of the image recognition system, which is placed in front of the subject to be recognized for exposure and image formation.

[0039] Furthermore, the image recognition model is a public image processing model with pre-trained weights, which is used to detect and classify the subjects contained in the image input by the image acquisition module.

[0040] like Figure 2 As shown, this embodiment also provides an image physics countermeasure attack method based on LED local illumination modulation and rolling shutter effect, including the following steps: (1) Given the initial signal of LED lighting source v , the initial signal v The switching probability signal π 0 and π 1 two-dimensional vector; (2) Use Gumbel-SoftMax technology to perform binary differentiable processing of the initial signal of the LED lighting source, use Gumbel noise to increase the randomness of the sampling operation, and output the true value of the modulated signal p ; (3) Using the local illumination differentiable model as the generative model for adversarial samples, we generate image adversarial samples and adversarial attack perturbation signals; (4) Input the image adversarial sample into the image recognition model, calculate the result of the loss function, and backpropagate to update the signal probability v , repeat (1) to (3) until the loss function result is maximized; Furthermore, the loss function is composed of the loss of the model and the jump loss of the modulation signal. The optimization goal of the loss function is to increase the difference between the predicted label value and the true label value of the attack target and reduce the switching between bright and dark stripes.

[0041] Furthermore, the loss function optimization adopts a batch sampling strategy: each iteration generates n adversarial examples batch , synchronously calculate the gradient update signal probability.

[0042] (5) The adversarial attack disturbance signal is converted into an LED lighting source modulation signal, and a CMOS image sensor is used to perform a physical adversarial attack on the real shooting scene.

[0043] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0044] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0045] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. Image physics counterattack system based on LED local illumination modulation and rolling shutter effect, characterized by: It includes a local adversarial attack disturbance signal generation module, an LED signal modulation drive module, an LED lighting source, a CMOS image sensor, and an image recognition model; The local adversarial attack disturbance signal generation module is connected to the LED signal modulation driving module, and is used to generate image adversarial samples and adversarial attack disturbance signals; the local adversarial attack disturbance signal generation module generates image adversarial samples and adversarial attack disturbance signals with local illumination disturbance information based on the local illumination differentiable model under the rolling shutter effect and the gradient-based batch sampling binary signal gradient optimization algorithm, and optimizes the parameters of the adversarial attack disturbance signal; the local illumination differentiable model is constructed by simulating the LED illumination intensity distribution through Gaussian function, mathematically modeling the brightness flicker simulation signal modulation, and channel conversion superposition modulated illumination; the Gaussian function simulated LED illumination intensity distribution is used to describe the light intensity of the LED illumination area, the mathematically modeled brightness flicker simulation signal modulation is used to generate a specific modulated pulse signal to make the LED illumination source produce high-speed brightness flicker, and the channel conversion superposition modulated illumination is used to determine the change relationship between the pixel values ​​before and after the superimposed illumination in the real physical world scene; The LED signal modulation driving module is connected to the local counter-attack disturbance signal generating module and the LED lighting source, and is used to convert the generated counter-attack disturbance signal into an LED lighting modulation signal, and drive the LED lighting source to realize lighting in different switching states; The LED lighting source is placed in front of the subject to be identified, and is used to provide ambient lighting and implement physical countermeasure attacks, by implanting brightness information disturbances invisible to the human eye in the target object area through high-frequency brightness modulation; The CMOS image sensor is placed in front of the subject to be identified, used for image acquisition, and uses a rolling shutter exposure mechanism to expose the image line by line; The image recognition model is connected to the CMOS image sensor and is a public pre-trained model used to detect and classify subjects in captured images.

2. The image physics counterattack system based on LED local illumination modulation and rolling shutter effect according to claim 1 is characterized in that: The Gaussian function simulates the LED lighting intensity distribution, specifically: The illumination area is divided into a spot area and a diffusion area that do not interfere with each other. Both the spot area and the diffusion area obey a multidimensional normal distribution. The light intensity decay rate of the spot area is significantly faster than that of the diffusion area, and the light intensity peak is located at the center of the spot area. The total light intensity at any point in the set space is composed of the linear superposition of the light intensity in the spot area and the light intensity in the diffusion area, and reaches the maximum light intensity value at the central coordinate point; The attenuation rate of the light intensity in the spot area from the center to the periphery is quantified by a first hyperparameter, wherein the first hyperparameter is positively correlated with the diameter of the spot; the attenuation rate of the light intensity in the diffusion area from the center to the periphery is quantified by a second hyperparameter; Define the spatial position influence factors: The first factor represents the set point x The square of the normalized distance of the direction from the central position; The second factor represents the set point y The square of the normalized distance of the direction from the center position; the third factor representation x Direction and y statistical relevance of directional changes; The Gaussian mode light intensity coefficients of the light spot area and the diffusion area are set respectively, and the Gaussian mode light intensity coefficients are positively correlated with the maximum light intensity value.

3. The image physics counterattack system based on LED local illumination modulation and rolling shutter effect according to claim 1 is characterized in that: The mathematical modeling brightness flicker analog signal modulation is specifically: The adversarial illumination is generated by the combined effect of spatial light intensity distribution and temporal modulation perturbation, where the final pixel value at any position depends on the product of the original light intensity value and the integral of the modulation signal within the exposure time of a single line of the CMOS sensor; The modulation signal is a periodic rectangular pulse sequence, the height of which is constant at the brightness value of the on state, the width of which is equal to the modulation period, and the on or off state of the LED in each period is controlled by a binary symbol sequence; The serial numbers of the binary symbol sequence correspond one-to-one with the row numbers of the CMOS image sensor, so that each row of pixels captures an independent LED brightness state during exposure; An adversarial perturbation pattern is generated as the normalized result of the integration of the modulation signal during exposure, and the adversarial perturbation pattern is embedded in the image space domain as a brightness perturbation carrier.

4. The image physics counterattack system based on LED local illumination modulation and rolling shutter effect according to claim 1, characterized in that: The channel conversion superposition modulated illumination is specifically as follows: Extract the brightness channel of the original image as the basis of light interference; The effective area of ​​the anti-illumination is limited by the target object mask to ensure that the perturbation only illuminates the surface of the target object; Linearly superimpose the adversarial illumination in the limited area with the original brightness channel; Set the overflow value of the brightness channel. When the superposition value exceeds the overflow value, it will be truncated to prevent overexposure of the image. Finally, an adversarial image containing brightness perturbations that are invisible to the human eye is generated, and the adversarial image presents the characteristics of alternating light and dark stripes.

5. The image physics counterattack system based on LED local illumination modulation and rolling shutter effect according to claim 1 is characterized in that: The execution process of the gradient-based batch sampling binary signal gradient optimization algorithm is as follows: The reparameterization technique is used to design a differentiable function of the binary signal of LED lighting status, and the discrete binary signal is converted into a continuous probability form. Introducing Gambel noise into the differentiable function of the binary signal enhances the randomness of sampling and increases the randomness of the sampling operation; The gradient iteration is used to reversely optimize the probability of the original signal simulated in the digital domain, and the current probability is simultaneously used in one iteration. N Subsampling, superimposing lighting to obtain N adversarial examples, N The samples are input into the neural network as a batch for gradient calculation, and the iteration is repeated until the loss function is maximized. The loss function consists of the model loss and the jump loss of the modulation signal. The loss of the model is used to increase the difference between the predicted label value and the true label value of the attack target, and the jump loss of the modulation signal is used to minimize the state switching frequency of the modulation signal in adjacent rows to reduce the switching between bright and dark stripes.

6. The image physics counterattack system based on LED local illumination modulation and rolling shutter effect according to claim 5, characterized in that: The loss function is calculated as αJ modle－ βJ jump ; ； ； in, J modle is the loss of the model, J jump is the transition loss of the modulation signal, α and β They are respectively expressed as the impact factors of model loss and jump loss, and the impact factors are set according to learning experience. N is the maximum number of samples output for each iteration, n is the serial number of the adversarial sample, f ( X ) is the model output of the clean sample, f ( X adv ) is the model output of the adversarial sample with local illumination perturbations, For the k Line digital domain analog original signal, Representatives and The next row of digital domain simulates the original signal. l is the signal length.

7. The image physics counterattack system based on LED local illumination modulation and rolling shutter effect according to claim 1, characterized in that: The rolling shutter exposure mechanism is used to expose the image line by line, specifically: The photodiodes are turned on and off row by row, and each adjacent row of pixels is exposed with a fixed time difference. By controlling the periodic changes of the LED illumination source, the LED illumination source is in two different states, on and off, during the brief time when the CMOS image sensor is exposing the image and when exposing adjacent rows. This produces alternating light and dark stripes in the image, forming brightness changes that are imperceptible to the human eye.

8. The method for countering an attack on an image physics counterattack system based on LED local illumination modulation and rolling shutter effect according to any one of claims 1 to 7, characterized in that: The steps include: (1) Given an initial signal of an LED lighting source, the initial signal is a switching probability signal π 0 and π 1 two-dimensional vector; π 0 means the probability that the signal is 0; π 1 represents the probability that the signal is 1; (2) Use reparameterization technology to perform binary differentiable processing of the initial signal of the LED lighting source, use Gambel noise to increase the randomness of the sampling operation, and output the true value of the modulated signal; (3) Using the local illumination differentiable model as the generative model for adversarial samples, we generate image adversarial samples and adversarial attack perturbation signals; (4) Input the image adversarial sample into the image recognition model, calculate the result of the loss function, and backpropagate to update the signal probability until the loss function result is maximized; (5) The adversarial attack disturbance signal is converted into an LED lighting source modulation signal, and a CMOS image sensor is used to perform a physical adversarial attack on the real shooting scene.

9. The anti-attack method according to claim 8, characterized in that: In step (4), the loss function is composed of the model loss and the jump loss of the modulation signal, which is used to increase the difference between the predicted label value and the true label value of the attack target and reduce the switching of light and dark stripes.

10. The anti-attack method according to claim 8, characterized in that: In step (4), the loss function optimization adopts a batch sampling strategy: each iteration generates N adversarial examples batch , synchronously calculate the gradient update signal probability.

Images