Visible light visual perception physical countermeasure attack method and device based on reflected light
Through the optimization algorithm based on reflected light, the problem of difficult modification of patches or textures and light source dependence in the existing technology is solved, and the effective anti-attack effect is achieved.
Patent Information
- Application Number
- CN202510082123.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-20
- Publication Date
- 2025-06-03
AI Technical Summary
When prior art engages in antagonistic attacks in the physical world, contact methods are difficult to modify adversarial patches or textures, and contactless methods rely on light sources. The availability and effectiveness of light sources are limited by environmental conditions, resulting in poor attack effects.
Through the visible light visual perception physical adversarial attack method based on reflected light, the shape and color of reflected light is optimized using a preset reflected light optimization algorithm, so that it can project reflected light that misleads the neural network on the target object, thereby achieving adversarial attack.
This method can make the target object misidentified and has high concealment, solving the problems of difficulty in modifying patches or textures and light source dependence in the prior art.
Smart Images
Figure CN120088613A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular, to a visible light vision perception physical adversarial attack method and device based on reflected light. Background Art
[0002] In recent years, with the continuous progress and development of machine learning theory and technology, especially the breakthrough progress in the fields of computer vision and multimedia, machine learning has been widely applied in technical fields such as autonomous driving, target detection, medical image processing, biological image recognition, and face recognition. However, the rapid development of the machine learning field has also brought many security problems. Some studies have shown that the output of deep neural networks is highly sensitive to small changes in the input. For example, for an image classification task, by adding some imperceptible perturbations to the original image, incorrect output results can be caused by the deep neural network. Among them, the image with added perturbations is called an adversarial sample.
[0003] However, the method of adding perturbations to all pixels of the original sample cannot be extended to the physical world because in the physical world, only the target object can be modified and the environment cannot be modified. To construct adversarial samples that also have attacks in the physical world, some current studies have pointed out that the constraint of the imperceptibility of adversarial samples can be relaxed, and instead, the proportion of adversarial perturbations in the picture can be constrained.
[0004] Currently, adversarial attack technologies for the physical world can be divided into contact type and non-contact type. Contact type physical adversarial attack technology refers to pasting printed adversarial perturbations on the outer surface of the target object to generate adversarial samples for adversarial attacks. Common contact type physical adversarial attack technologies are mainly implemented based on adversarial patches or adversarial textures. The adversarial attack implemented based on adversarial patches optimizes an adversarial patch image without considering the situation that the actual target object may not be flat. The adversarial attack implemented based on adversarial textures optimizes the texture of the flattened view of the target object plane, which is more conforming to the shape of the object and more in line with the actual situation during the optimization process. However, in practical applications, both contact type physical adversarial attack technologies implemented based on adversarial patches and those based on adversarial textures have the following two problems: First, after deploying the adversarial patch or adversarial texture, it cannot be modified; second, the generated adversarial patch or adversarial texture is very easy to attract attention.
[0005] The non-contact adversarial attack technology refers to adding adversarial perturbations to a target object in a remote non-contact manner to generate adversarial samples for adversarial attacks. Existing non-contact adversarial attack technologies mainly rely on light sources, such as sunlight, laser, and light projected by a projector, to add adversarial perturbations to the target object. However, in practical applications, the existing non-contact adversarial attack technologies that rely on light sources still have the following problems: the projection direction of sunlight cannot be autonomously planned, and in some cases, it is impossible to construct a shadow on the target object; light sources such as lasers and projectors can only be used in a relatively dark environment, and when sunlight is strong, it will suppress artificial light, resulting in the failure of the attack; the adversarial perturbations randomly generated by the light source usually make it difficult for the deep neural network to misidentify. Summary of the Invention
[0006] To solve some or all of the above technical problems existing in the prior art, the present invention provides a visible light vision perception physical adversarial attack method and device based on reflected light.
[0007] The technical solution of the present invention is as follows:
[0008] In a first aspect, a visible light vision perception physical adversarial attack method based on reflected light is provided. The method includes:
[0009] Obtain the neural network model to be attacked and the target object image;
[0010] Based on the obtained neural network model and target object image, use a preset reflected light optimization algorithm to optimize the reflected light, and obtain the shape and color of the reflected light that can make the neural network model identify the target object in the target object image as other categories;
[0011] Irradiate the target object to be hidden with a light source, place a light-transmitting plate with a set-shaped light-transmitting hole in front of the light source, and place a colored transparent glass sheet in front of the light-transmitting plate to apply reflected light with the same shape and color as the shape and color of the optimized reflected light on the target object to be hidden.
[0012] In some optional embodiments, the reflected light optimization algorithm includes the following steps:
[0013] Step 21, based on the obtained target object image, use a preset population initialization algorithm to obtain a population and the optimization direction corresponding to each individual in the population. The population includes multiple sub-populations, the sub-population includes multiple individuals, and an individual represents a set of optimization variables. A set of optimization variables includes: the x coordinate of the center of the circle, the y coordinate of the center of the circle, the radius of the circle, transparency, RGB channel values, and multiple angle values. The coordinates of the center of the circle are the coordinates in the coordinate system of the target object image;
[0014] Step 22: Based on the obtained population, use a preset adversarial sample generation algorithm to generate an adversarial sample corresponding to each individual in the population;
[0015] Step 23: Input each adversarial sample into the neural network model to obtain the prediction result output by the neural network model;
[0016] Step 24: Determine whether there is an adversarial sample whose probability of being predicted by the neural network model as the same category as the target object in the target object image is less than a set threshold. If so, output the individual corresponding to the current adversarial sample as the optimal individual and end the optimization. If not, continue to the next step;
[0017] Step 25: Update the optimization direction corresponding to each individual using a preset optimization direction update formula;
[0018] Step 26: Based on the updated optimization direction, update each individual in the population using a preset optimization variable update formula to obtain an updated population, and return to Step 22.
[0019] In some alternative embodiments, the population initialization algorithm includes the following steps:
[0020] Randomly sample a value as the radius r of the circle from a preset radius value range, randomly sample a coordinate as the center coordinate (x, y) of the circle from a preset center coordinate value range, and randomly sample the optimization direction v corresponding to the radius from a preset optimization direction value range r , the optimization direction v corresponding to the x coordinate of the center x and the optimization direction v corresponding to the y coordinate of the center y ;
[0021] Step 212: Based on the sampled radius r of the circle and the center coordinate (x, y), randomly sample a value as the transparency α from a preset transparency value range, randomly sample an R channel value R, a G channel value G, and a B channel value B from a preset RGB channel value range, and randomly sample multiple angle values a 1 , a 2 , …, a n , to obtain a set of optimization variables (x, y, r, α, R, G, B, a 1 , a 2 , …, a n ) as an individual of the sub-population, and randomly sample the optimization direction v corresponding to the transparency from a preset optimization direction value range α , the optimization direction v corresponding to the R channel value R , the optimization direction v corresponding to the G channel value G , the optimization direction v corresponding to the B channel value B, and the optimization directions v corresponding to multiple angular values 1 , v 2 , …, v n , the optimization direction corresponding to the individual is obtained (v x , v y , v r , v α , v R , v G , v B , v 1 , v 2 , …, v n );
[0022] Step 213, loop step 212 until the first preset number of individuals and their corresponding optimization directions are obtained, and then proceed to the next step;
[0023] Step 214, loop steps 211 - 213 until the second preset number of subpopulations is obtained.
[0024] In some alternative embodiments, the adversarial sample generation algorithm includes the following steps:
[0025] Step 221, sequentially select a subpopulation from the population;
[0026] Step 222, after each selection of a subpopulation, based on the selected subpopulation, sequentially select an individual from the subpopulation, and calculate the points corresponding to the angular value on the target object image according to the radius, center coordinates, and angular value in the optimization variables corresponding to the selected individual. Sort the obtained multiple points to minimize the distance between adjacent points. Based on the sorting result of the multiple points, connect the adjacent two points with a straight line on the target object image to obtain the reflected light of the corresponding shape. Adjust the transparency and color of the reflected light according to the transparency and RGB channel values in the optimization variables corresponding to the selected individual to obtain the adversarial sample corresponding to the individual.
[0027] In some alternative embodiments, the optimization direction update formula is expressed as:
[0028] v i,j (t) = Wv i,j (t - 1) + C 1 κ 1 (q pbest - q i,j (t - 1)) +
[0029] C 2 κ 2 (q gbest - q i,j (t - 1)) + C 3 κ 3(q sgbest -q i,j (t - 1))
[0030] wherein, v i,j (t) represents the optimization direction corresponding to the j-th individual in the i-th sub-population at the t-th iteration cycle, v i,j (t - 1) represents the optimization direction corresponding to the j-th individual in the i-th sub-population at the (t - 1)-th iteration cycle, q i,j (t - 1) represents the j-th individual in the i-th sub-population at the (t - 1)-th iteration cycle, q pbest represents the historical optimal variable corresponding to the current individual, q gbest represents the historical optimal individual in the entire population, q sgbest represents the historical optimal individual in the sub-population where the current individual is located, W represents the inertia weight, C 1 represents the first learning factor, κ 1 represents the first scaling factor, C 2 represents the second learning factor, κ 2 represents the second scaling factor, C 3 represents the third learning factor, κ 3 represents the third scaling factor.
[0031] In some alternative embodiments, the optimization variable update formula is expressed as:
[0032] q i,j (t) = q i,j (t - 1) + v I,j (t);
[0033] wherein, q i,j (t) represents the j-th individual in the i-th sub-population at the t-th iteration cycle, q i,j (t - 1) represents the j-th individual in the i-th sub-population at the (t - 1)-th iteration cycle, v i,j (t) represents the optimization direction corresponding to the j-th individual in the i-th sub-population at the t-th iteration cycle.
[0034] In a second aspect, there is also provided a visible light vision perception physical adversarial attack device based on reflected light, and the device includes:
[0035] A target acquisition unit, configured to acquire a neural network model to be attacked and a target object image;
[0036] An optimization unit, configured to optimize the reflected light based on the acquired neural network model and the target object image by using a preset reflected light optimization algorithm, so as to obtain the shape and color of the reflected light that can enable the neural network model to recognize the target object in the target object image as other categories;
[0037] A light source capable of emitting visible light;
[0038] A reflector disposed in front of the light source for reflecting the visible light emitted by the light source;
[0039] A light-transmitting plate having a light-transmitting hole of a set shape, disposed in front of the reflector for adjusting the shape of the reflected light of the reflector to a specified shape;
[0040] A colored transparent glass sheet disposed in front of the light-transmitting plate for adjusting the color of the reflected light to a specified color.
[0041] The main advantages of the technical solution of the present invention are as follows:
[0042] The visible light vision perception physical adversarial attack method based on reflected light of the present invention optimizes the shape and color of the reflected light according to the neural network model to be attacked, and remotely projects the reflected light of the corresponding shape and color on the target object by using the light source, so that the target object can be misrecognized with high concealment. Description of the Drawings
[0043] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the present invention. The illustrative embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0044] Figure 1 is a flowchart of the visible light vision perception physical adversarial attack method based on reflected light provided by an embodiment of the present invention;
[0045] Figure 2 is a structural schematic diagram of the visible light vision perception physical adversarial attack device provided by an embodiment of the present invention, wherein the target acquisition unit and the optimization unit are not shown. Detailed Embodiments
[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0047] The technical solutions provided by the embodiments of the present invention will be described in detail below in conjunction with the drawings.
[0048] Refer to Figure 1 , an embodiment of the present invention provides a visible light vision perception physical adversarial attack method based on reflected light, and the method includes the following steps:
[0049] Step 1: Obtain the neural network model to be attacked and the target object image;
[0050] In the embodiments of the present invention, according to the neural network model to be actually attacked, the target object image is determined and obtained.
[0051] In the embodiments of the present invention, the target object image is the image for detection by the neural network model. For example, if the neural network model to be attacked is used for object classification detection, the target object image is an image containing an object of the corresponding category.
[0052] In an embodiment of the present invention, if the neural network model to be attacked can be directly obtained, directly obtain the neural network model to be attacked; if the neural network model to be attacked cannot be directly obtained, select one from the existing trained neural network models with the same use.
[0053] Step 2: Based on the obtained neural network model and target object image, use a preset reflected light optimization algorithm to optimize the reflected light, and obtain the shape and color of the reflected light that can make the neural network model recognize the target object in the target object image as other categories;
[0054] Step 3: Use a light source to irradiate the target object to be hidden, place a light-transmitting plate with a light-transmitting hole of a set shape in front of the light source, and place a colored transparent glass sheet in front of the light-transmitting plate, so as to apply reflected light with the same shape and color as the shape and color of the optimized reflected light on the target object to be hidden.
[0055] In the embodiments of the present invention, the light source can be a natural light source, such as sunlight, or an artificial light source, such as laser, projector, flashlight light, etc.
[0056] In the embodiments of the present invention, the shape of the light-transmitting hole on the light-transmitting plate is the same as the shape of the reflected light obtained in Step 2.
[0057] In the embodiments of the present invention, the parameters of the transparent glass sheet are adjusted according to the color of the reflected light obtained in Step 2, so that the color of the reflected light projected onto the target object through the colored transparent glass sheet is the same as the color of the reflected light obtained in Step 2.
[0058] The visible light visual perception physical adversarial attack method based on reflected light provided by the embodiments of the present invention can optimize the shape and color of the reflected light according to the neural network model to be attacked, and remotely project the reflected light of the corresponding shape and color on the target object by using a light source, so that the target object can be misrecognized and has high concealment.
[0059] In the embodiments of the present invention, the optimization problem of the shape and color of the reflected light can be modeled as the following mathematical problem:
[0060] Set the target object image as X, and the position and shape of the reflected light are represented by N points P i (x i , y i ). It is composed of (x i , y i ) representing the coordinates of the i-th point in the image. The color of the reflected light is determined by the RGB channel values (R, G, B) and the transparency variable α. The function A represents the function of drawing the reflected light into the image X in the digital space;
[0061] Based on the above settings, during the optimization process, the shape and color of the reflected light are composed of the variables (x 1 , y 1 ,..., x N , y N , R, G, B, α). (x 1 , y 1 ) represents the coordinates of the 1st point in the image, and (x N , y N ) represents the coordinates of the Nth point in the image. The adversarial sample generated according to the image X and the reflected light can be expressed as:
[0062] X adv = A(X, P, C, α);
[0063] The optimization problem can be expressed as:
[0064] Find(P, C, α);
[0065] s.t. f(X adv ) ≠ f(X);
[0066] Among them, X adv represents the adversarial sample generated according to the image X and the point coordinate parameter P, the RGB channel value parameter C, and the transparency variable α. f(X) represents the output result of the neural network model for the input X, and f(X adv ) represents the output result of the neural network model for the input X adv .
[0067] It can be seen from the above optimization problem that the number of optimization variables is positively correlated with the shape of the reflected light. For example, when the reflected light is triangular, the reflected light can be composed of 3 points, and at this time, the dimension of the optimization variable is 10 dimensions. When the reflected light is quadrilateral, the reflected light can be composed of 4 points, and at this time, the dimension of the optimization variable is 12 dimensions. In order to further reduce the dimension of the optimization variables, in the embodiments of the present invention, the following modeling method is adopted for the shape of the reflected light:
[0068] Initialize a value r as the radius of the circle, initialize a point (x, y) as the center of the circle, and initialize an angular value a1 , calculate the point P on the circle corresponding to the angle value according to the radius, the center of the circle, and the angle value 1 (x 1 , y 1 ), and the symmetric point P' 1 (x 1 , y 1 ) of the point P 1 (x 1 , y 1 ). According to the number of sides N of the shape of the required reflected light, initialize N - 2 angle values, find the points on the circle corresponding to each angle value, and connect the points in sequence in the clockwise or counterclockwise direction around the circle to obtain the reflected light of the corresponding shape.
[0069] For example, when constructing a triangular reflected light, based on the initialized radius, center of the circle, and angle value a 1 , add another angle value a 2 , find the point P 2 (x, y) on the circle corresponding to the angle value, and connect the three points to obtain the triangular reflected light; when constructing a quadrilateral reflected light, based on the initialized radius, center of the circle, and angle value a 1 , add two angle values a 2 and a 3 , find the points P 2 (x, y) and the point P 3 (x, y) on the circle corresponding to the two angle values, and connect the four points to obtain the quadrilateral reflected light.
[0070] In the embodiment of the present invention, by adopting the above-mentioned modeling method for the shape of the reflected light, the optimization of the two variables of coordinates x and y can be converted into the optimization of a single variable of the angle value a, which can reduce the dimension of the optimization variables corresponding to the shape of the reflected light, improve the optimization speed, reduce the optimization cost, and avoid the need to calculate the constraint that the connections between points do not cross when determining the shape of the reflected light according to the point coordinates.
[0071] Furthermore, based on the above-mentioned modeling and analysis results for the optimization problems of the shape and color of the reflected light, in the embodiment of the present invention, the reflected light optimization algorithm specifically includes the following steps:
[0072] Step 21, based on the acquired target object image, use the preset population initialization algorithm to obtain the population and the optimization direction corresponding to each individual in the population;
[0073] In an embodiment of the present invention, the population includes multiple sub - populations, each sub - population includes multiple individuals, and an individual represents a set of optimization variables. The set of optimization variables includes: the x - coordinate of the center of the circle, the y - coordinate of the center of the circle, the radius of the circle, transparency, RGB channel values, and multiple angle values. Among them, the coordinates of the center of the circle are the coordinates in the target object image coordinate system.
[0074] In an embodiment of the present invention, the number of angle values in a set of optimization variables is determined according to the shape of the reflected light required in practice. When the required shape of the reflected light is linear, the number of angle values is one; when the required shape of the reflected light is triangular, the number of angle values is two; when the required shape of the reflected light is quadrilateral, the number of angle values is three; when the required shape of the reflected light is an N - sided polygon, the number of angle values is N - 1.
[0075] In an embodiment of the present invention, the x - coordinate of the center of the circle, the y - coordinate of the center of the circle, and the radius of the circle in different individuals under the same sub - population are the same.
[0076] Step 22: Based on the obtained population, use a preset adversarial sample generation algorithm to generate an adversarial sample corresponding to each individual in the population;
[0077] In an embodiment of the present invention, each individual in the population can generate a corresponding adversarial sample.
[0078] Step 23: Input each adversarial sample into the neural network model to obtain the prediction result output by the neural network model;
[0079] Step 24: Determine whether there is an adversarial sample whose probability of being predicted by the neural network model as the same category as the target object in the target object image is less than the set threshold. If so, output the individual corresponding to the current adversarial sample as the optimal individual and end the optimization. If not, continue to the next step;
[0080] In an embodiment of the present invention, the set threshold is specifically set according to actual requirements.
[0081] In an embodiment of the present invention, the smaller the probability that the adversarial sample is predicted by the neural network model as the same category as the target object in the target object image, the better the attack performance of the adversarial sample.
[0082] Step 25: Update the optimization direction corresponding to each individual using a preset optimization direction update formula;
[0083] Step 26: Based on the updated optimization direction, update each individual in the population using a preset optimization variable update formula to obtain the updated population, and return to Step 22.
[0084] In the embodiments of the present invention, by using the above-mentioned reflected light optimization algorithm to optimize and solve the shape and color of the reflected light, the shape and color of the reflected light with optimal attack performance can be obtained. And by setting sub-populations in the population to perform dual-population optimization and solution, the optimization search efficiency can be further improved.
[0085] Furthermore, in the embodiments of the present invention, the population initialization algorithm specifically includes the following steps:
[0086] Step 211: Randomly sample a value as the radius r of the circle from the preset radius value range, randomly sample a coordinate as the center coordinate (x, y) of the circle from the preset center coordinate value range, and randomly sample the optimization direction v corresponding to the radius from the preset optimization direction value range r , the optimization direction v corresponding to the x coordinate of the center x and the optimization direction v corresponding to the y coordinate of the center y ;
[0087] In the embodiments of the present invention, the radius value range, the center coordinate value range, and the optimization direction value range are specifically set according to actual requirements.
[0088] Specifically, in the embodiments of the present invention, the radius value range is set to [0, min(H, W) / 2], and the center coordinate value range is set to [r, H - r] to [r, W - r], where H and W respectively represent the height and width of the target object image, and r represents the sampled radius value, so that the generated circle is within the target object image range.
[0089] Step 212: Based on the sampled radius r of the circle and the center coordinate (x, y), randomly sample a value as the transparency α from the preset transparency value range, randomly sample an R channel value R, a G channel value G, and a B channel value B from the preset RGB channel value range, and randomly sample multiple angle values a 1 , a 2 , …, a n , to obtain a set of optimization variables (x, y, r, α, R, G, B, a 1 , a 2 , …, a n ) as an individual of the sub-population, and randomly sample the optimization direction v corresponding to the transparency from the preset optimization direction value range α , the optimization direction v corresponding to the R channel value R , the optimization direction v corresponding to the G channel value G , the optimization direction v corresponding to the B channel value B , and the optimization direction v corresponding to multiple angle values 1 , v 2 , …, vn , the optimization direction corresponding to the individual (v x , v y , v r , v α , v R , v G , v B , v 1 , v 2 , …, v n )
[0090] In the embodiments of the present invention, the transparency value range, the RGB channel value range, the angle value range, and the optimization direction value range are specifically set according to actual requirements.
[0091] Step 213: Repeat Step 212 until the first preset number of individuals and their corresponding optimization directions are obtained, and then proceed to the next step;
[0092] In the embodiments of the present invention, the first preset number is set according to the number of individuals included in the sub-population required in actuality.
[0093] Step 214: Repeat Steps 211 - 213 until the second preset number of sub-populations is obtained.
[0094] In the embodiments of the present invention, the second preset number is set according to the number of sub-populations included in the population required in actuality.
[0095] Furthermore, in the embodiments of the present invention, the adversarial sample generation algorithm specifically includes the following steps:
[0096] Step 221: Select one sub-population from the population in sequence;
[0097] Step 222: After each selection of a sub-population, based on the selected sub-population, select one individual from the sub-population in sequence, and calculate the points corresponding to the angle value on the target object image according to the radius, center coordinates, and angle value in the optimization variables corresponding to the selected individual. Sort the obtained multiple points so that the distance between adjacent points is minimized. Based on the sorting result of the multiple points, connect two adjacent points with a straight line on the target object image to obtain the reflected light of the corresponding shape. Adjust the transparency and color of the reflected light according to the transparency and RGB channel values in the optimization variables corresponding to the selected individual to obtain the adversarial sample corresponding to the individual.
[0098] In the embodiments of the present invention, taking the first angle value as an example, the coordinates of the point corresponding to the angle value are calculated using the following formula:
[0099]
[0100] where, (x1 , y 1 ) represents the angular value a 1 The coordinates of the corresponding point, (x, y) represents the coordinates of the center of the circle, and r represents the radius of the circle.
[0101] Furthermore, in the embodiments of the present invention, for the first angular value among the optimization variables, two points are calculated and generated. Among them, the first point corresponding to the first angular value is calculated using the above formula, and the second point corresponding to the first angular value is calculated using the following formula:
[0102] x′ 1 = 2×x - x 1 ;
[0103] y′ 1 = 2×y - y 1 ;
[0104] Among them, (x′ 1 , y′ 1 ) represents the coordinates of the second point corresponding to the angular value a 1 corresponding thereto.
[0105] In this way, it is possible to further reduce the dimension of the optimization variables, reduce the optimization cost, and improve the optimization efficiency.
[0106] In the embodiments of the present invention, the obtained multiple points are sorted using the minimum point distance algorithm so that the distance between adjacent points is minimized.
[0107] Furthermore, in the embodiments of the present invention, the optimization direction update formula is expressed as:
[0108] v i,j (t) = Wv i,j (t - 1) + C 1 κ 1 (q pbest - q i,j (t - 1)) +
[0109] C 2 κ 2 (q gbest - q i,j (t - 1)) + C 3 κ 3 (q sgbest - q i,j (t - 1))
[0110] Among them, v i,j (t) represents the optimization direction corresponding to the jth individual in the ith subpopulation during the tth iteration loop, v i,j (t - 1) represents the optimization direction corresponding to the jth individual in the ith subpopulation during the (t - 1)th iteration loop, qi,j (t - 1) represents the j-th individual in the i-th sub-population during the (t - 1)-th iteration loop, and q pbest represents the historical optimal variable corresponding to the current individual, and q gbest represents the historical optimal individual in the entire population, and q sgbest represents the historical optimal individual in the sub-population where the current individual is located. W represents the inertia weight, and C 1 represents the first learning factor, and κ 1 represents the first scaling factor, and C 2 represents the second learning factor, and κ 2 represents the second scaling factor, and C 3 represents the third learning factor, and κ 3 represents the third scaling factor.
[0111] In the embodiments of the present invention, the inertia weight, learning factors, and scaling factors are specifically set according to actual requirements.
[0112] Furthermore, in the embodiments of the present invention, the optimization variable update formula is expressed as:
[0113] q i,j (t) = q i,j (t - 1) + v I,j (t);
[0114] Among them, q i,j (t) represents the j-th individual in the i-th sub-population during the t-th iteration loop, and q i,j (t - 1) represents the j-th individual in the i-th sub-population during the (t - 1)-th iteration loop, and v i,j (t) represents the optimization direction corresponding to the j-th individual in the i-th sub-population during the t-th iteration loop.
[0115] Referring to Figure 2 , in the second aspect, the embodiments of the present invention also provide a visible light vision perception physical adversarial attack device based on reflected light, and the device includes:
[0116] A target acquisition unit for acquiring the neural network model to be attacked and the target object image;
[0117] An optimization unit for optimizing the reflected light based on the acquired neural network model and target object image by using a preset reflected light optimization algorithm to obtain the shape and color of the reflected light that can make the neural network model recognize the target object in the target object image as other categories;
[0118] A light source capable of emitting visible light;
[0119] A reflecting mirror arranged in front of the light source for reflecting the visible light emitted by the light source;
[0120] A light-transmitting plate, provided with light-transmitting holes of a set shape, is arranged in front of a reflector and is used to adjust the shape of the reflected light of the reflector into a specified shape;
[0121] A colored transparent glass sheet is arranged in front of the light-transmitting plate and is used to adjust the color of the reflected light into a specified color.
[0122] The functions and achieved technical effects of each unit and component in the visible light visual perception physical countermeasure attack device based on reflected light provided by the embodiments of the present invention respectively correspond to those of the method described in the above embodiments and will not be elaborated here.
[0123] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. In addition, "front", "rear", "left", "right", "up" and "down" in this article are referred to the placement state shown in the drawings.
[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for physical countermeasure attack of visible light visual perception based on reflected light, characterized in that: include: Obtain the neural network model to be attacked and the target object image; Based on the acquired neural network model and the target object image, the reflected light is optimized using a preset reflected light optimization algorithm to obtain a shape and color of the reflected light that enables the neural network model to recognize the target object in the target object image as other categories; A light source is used to illuminate the target object to be hidden, and a light-transmitting plate with a light-transmitting hole of a set shape is placed in front of the light source, and a colored transparent glass slide is placed in front of the light-transmitting plate to apply reflected light with the same shape and color as the optimized reflected light shape and color to the target object to be hidden.
2. The method for physical countermeasure against attack of visible light visual perception based on reflected light according to claim 1, characterized in that: The reflected light optimization algorithm comprises the following steps: Step 21, based on the acquired target object image, using a preset population initialization algorithm to obtain the optimization direction corresponding to the population and each individual in the population, the population includes multiple sub-populations, the sub-population includes multiple individuals, and one individual represents a set of optimization variables, and the set of optimization variables includes: the x-coordinate of the center of the circle, the y-coordinate of the center of the circle, the radius of the circle, the transparency, the RGB channel value and multiple angle values, and the coordinates of the center of the circle are the coordinates in the target object image coordinate system; Step 22, based on the obtained population, generate an adversarial sample corresponding to each individual in the population using a preset adversarial sample generation algorithm; Step 23, input each adversarial sample into the neural network model to obtain the prediction result output by the neural network model; Step 24, determine whether there is a certain adversarial sample predicted by the neural network model as the same category as the target object in the target object image, and the probability is less than the set threshold. If so, the individual corresponding to the current adversarial sample is output as the optimal individual and the optimization ends. If not, proceed to the next step; Step 25, using a preset optimization direction update formula to update the optimization direction corresponding to each individual; Step 26, based on the updated optimization direction, each individual in the population is updated using a preset optimization variable update formula to obtain an updated population, and then returns to step 22.
3. The method for physical countermeasure against attack of visible light visual perception based on reflected light according to claim 2, characterized in that: The population initialization algorithm comprises the following steps: Randomly sample a value from the preset radius range as the radius r of the circle, randomly sample a coordinate from the preset center coordinate range as the coordinate (x, y) of the center of the circle, and randomly sample the optimization direction v corresponding to the radius from the preset optimization direction range r , the optimization direction v corresponding to the x-coordinate of the center of the circle x The optimized direction v corresponding to the y coordinate of the center of the circle y ; Step 212, based on the radius r and the center coordinates (x, y) of the circle obtained by sampling, randomly sample a value from a preset transparency value range as the transparency α, randomly sample an R channel value R, a G channel value G and a B channel value B from a preset RGB channel value range, and randomly sample multiple angle values a1, a2, ..., a from a preset angle value range. n , and obtain a set of optimization variables (x, y, r, α, R, G, B, a1, a2, …, a n ) as an individual of the subpopulation, and randomly sample the optimization direction v corresponding to the transparency from the preset optimization direction value range α , R channel value corresponding to the optimization direction v R , the optimization direction v corresponding to the G channel value G , the optimization direction v corresponding to the B channel value B , and the optimization directions v1,v2,…,v corresponding to multiple angle values n , and obtain the optimization direction corresponding to the individual (v x ,v y ,v r ,v α ,v R ,v G ,v B ,v1,v2,…,v n ); Step 213, looping through step 212 until a first preset number of individuals and their corresponding optimization directions are obtained, and then proceeding to the next step; Step 214, looping through steps 211 to 213 until a second preset number of sub-populations is obtained.
4. The method for physical countermeasure against attacks on visible light visual perception based on reflected light according to claim 2, characterized in that: The adversarial sample generation algorithm comprises the following steps: Step 221, selecting a sub-population in the population in turn; Step 222, after each subpopulation is selected, one individual in the subpopulation is selected in turn based on the selected subpopulation, and the point corresponding to the angle value on the target object image is calculated according to the radius, center coordinates and angle value in the optimization variables corresponding to the selected individual, and the obtained multiple points are sorted so that the distance between adjacent points is minimized, and based on the sorting results of the multiple points, two adjacent points on the target object image are connected by a straight line to obtain reflected light of the corresponding shape, and the transparency and color of the reflected light are adjusted according to the transparency and RGB channel values in the optimization variables corresponding to the selected individual to obtain the adversarial sample corresponding to the individual.
5. The method for physical countermeasure against attack of visible light visual perception based on reflected light according to claim 2, characterized in that: The optimization direction update formula is expressed as: v i,j (t)=Wv i,j (t-1)+C1κ1(q pbest -q i,j (t-1))+C2κ2(q gbest -q i,j (t-1))+C3κ3(q sgbest -q i,j (t-1)) Among them, v i,j (t) represents the optimization direction corresponding to the jth individual in the ith subpopulation at the tth iteration cycle, v i,j (t-1) represents the optimization direction corresponding to the jth individual in the ith subpopulation at the t-1th iteration cycle, q i,j (t-1) represents the jth individual in the ith subpopulation at the t-1th iteration cycle, q pbest represents the historical optimal variable corresponding to the current individual, q gbest represents the historical best individual in the entire population, q sgbest represents the historical optimal individual in the subpopulation where the current individual is located, W represents the inertia weight, C1 represents the first learning factor, κ1 represents the first scaling factor, C2 represents the second learning factor, κ2 represents the second scaling factor, C3 represents the third learning factor, and κ3 represents the third scaling factor.
6. The method for physical countermeasure against attack of visible light visual perception based on reflected light according to claim 2, characterized in that: The optimization variable update formula is expressed as: q i,j (t)=q i,j (t-1)+v i,j (t); Among them, q i,j (t) represents the jth individual in the ith subpopulation at the tth iteration cycle, q i,j (t-1) represents the jth individual in the ith subpopulation at the t-1th iteration cycle, v i,j (t) represents the optimization direction corresponding to the jth individual in the i-th subpopulation in the t-th iteration cycle.
7. A visible light visual perception physical counterattack device based on reflected light, characterized in that: The device comprises: A target acquisition unit, used to acquire a neural network model to be attacked and an image of a target object; An optimization unit, configured to optimize the reflected light using a preset reflected light optimization algorithm based on the acquired neural network model and the target object image, to obtain a shape and color of the reflected light that enables the neural network model to recognize the target object in the target object image as other categories; A light source capable of emitting visible light; A reflector, arranged in front of the light source, for reflecting the visible light emitted by the light source; A light-transmitting plate, which is provided with a light-transmitting hole of a set shape and is arranged in front of the reflector, and is used to adjust the shape of the reflected light of the reflector to a specified shape; The colored transparent glass sheet is arranged in front of the light-transmitting plate and is used for adjusting the color of the reflected light to a specified color.