An adversarial patch generation optimization method combined with timing feature information, a system and a storage medium

Abstract

The application provides a kind of combined timing feature information's countermeasure patch generation optimization method, system and storage medium, method includes: step S1, data collection and loading: open source data set, simulation data set, custom real scene data set are loaded, obtain total loading frame number;Step S2, calculate countermeasure patch position: initialize countermeasure patch parameter, read the data of target frame, obtain the label parameter information provided by open source data set, simulation data set, custom real scene data set, generate countermeasure image;Step S3, combine historical BEV information batch optimization: adopt multi-frame timing inference setting, calculate the total loss of continuous multi-frame output result, based on total loss back propagation and gradient optimization, iteration generates final countermeasure patch.The beneficial effects of the present application are: the method of the present application combines the timing BEV feature information provided by the countermeasure patch with the optimization method of the traditional countermeasure patch, and better optimization effect is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and in particular to an adversarial patch generation optimization method and system combining time sequence feature information and a storage medium. BACKGROUND

[0002] Deep neural networks (DNNs) are representative technologies of artificial intelligence and have been widely used in many fields of life and production. Although DNNs perform excellently in terms of performance, they are vulnerable to adversarial attacks and can output incorrect results. Currently, various studies have confirmed the feasibility of using adversarial patches to implement adversarial attacks on target detection tasks in the field of autonomous driving, and have used this method to expose the defects of DNNs or autonomous driving models that involve perception and decision-making.

[0003] In practical applications, adversarial patches can be used to strengthen the robustness of existing DNNs through adversarial training, or they can be used as a safety evaluation indicator to measure whether autonomous driving can accurately perceive the surrounding environment and output correct decision-making information. Due to their physical realizability, adversarial patches have great significance in reducing abnormal influences and promoting technology landing.

[0004] BEVFormer is a target detection network that uses time sequence self-attention mechanisms and historical BEV features to perceive the surrounding environment. Since participants (targets) in real-world scenarios are constantly moving, historical BEV information can greatly improve the accuracy of target detection and trajectory prediction.

[0005] Currently, there are adversarial attacks based on adversarial patches, which cover a wide range of achievements in hiding real targets and creating false targets in the digital and physical domains.

[0006] Cheng et al. placed adversarial patches on the road surface or targets and optimized based on the results of each frame, resulting in a significant decline in the performance of the target detection network. Wang et al. placed adversarial patches on the road surface and optimized based on the BEV features of each frame, obtaining adversarial patches that can effectively create false targets in the real world.

[0007] Traditional adversarial patch optimization methods, although using target detection networks such as BEVFormer as attack targets, do not consider the influence of time sequence self-attention mechanisms and historical BEV features used by target detection networks on attack results. These optimization methods are based on the detection results of a single frame (time step) to optimize adversarial patches, ignoring the feature transmission effect of adversarial patches in dynamic scenarios. In both the digital and physical domains, the BEV information provided by adversarial patches is similar to that of other participants, and it will have an impact on subsequent prediction results.

[0008] Cheng et al. Although effective on multi-frame data, it is only based on the optimization of the results of the adversarial patch on a certain frame, ignoring the influence of the adversarial patch on the historical BEV and the effective information it may provide in the gradient transmission process. Similarly, although the method uses BEV features for optimization, it does not consider the information contained in the historical BEV by the adversarial patch. SUMMARY

[0009] In order to solve the defects in the prior art, the present application provides an adversarial patch generation optimization method combined with time sequence feature information, which combines the time sequence BEV feature information provided by the adversarial patch with the optimization method of the traditional adversarial patch to obtain better optimization effect.

[0010] The present application discloses an adversarial patch generation optimization method combined with time sequence feature information, comprising:

[0011] Step S1, data collection and loading: collect and load open source data sets, simulation data sets and custom real scene data sets to obtain the total number of loaded frames;

[0012] Step S2, calculate the adversarial patch position: initialize the adversarial patch parameters, read the target frame data, obtain the annotation parameter information provided by the open source data set, the simulation data set and the custom real scene data set, such information is uniformly provided by the data collector after completing the annotation, based on the obtained parameters, calculate the position of the adversarial patch corner point in the global coordinate system, and generate an adversarial image accordingly;

[0013] Step S3, combine historical BEV information batch optimization: use multi-frame time sequence reasoning setting to calculate the total loss of continuous multi-frame output results, perform back propagation and gradient optimization based on the total loss, and iteratively generate the final adversarial patch.

[0014] As a further improvement of the present application, the step S1 specifically comprises:

[0015] Step S101, open source data set: use the open source data set to obtain parameter information: self-vehicle coordinate system to global coordinate system conversion matrix T e2g And global coordinate system to camera coordinate system conversion matrix T g2c ;

[0016] Step S102, simulation data set: use the simulation software to render the obtained data set in real time, and obtain parameter information similar to step S101 according to the setting of the simulation software;

[0017] Step S103, custom real scene data set: obtain data by deploying sensor equipment by itself, customize the annotation of the data, and obtain parameter information similar to step S101 according to the sensor parameters;

[0018] Step S104: Select steps S101, S102, and S103 to construct continuous frame data of any scene in the dataset, and set the number of preheating frames N for a certain scene. pre The number of frames N used in training etu By default, the last frame used in training is the target frame. t Add the two frame counts together to get the total number of loaded frames N. load .

[0019] As a further improvement of the present invention, in step S101, parameter information is obtained using open-source datasets commonly used in the field of autonomous driving, such as NuScenes and KITTI; the global coordinate system to camera coordinate system transformation matrix T g2c Transformation matrix T from global coordinate system to local coordinate system g2e The transformation matrix T from the local coordinate system to the radar coordinate system e2l The transformation matrix T from radar coordinate system to camera coordinate system l2c Obtained by performing matrix multiplication.

[0020] As a further improvement of the present invention, step S2 specifically includes:

[0021] Step S201, Initialize adversarial patch parameters: Set the shape and size of the adversarial patch, randomly generate it based on the target mean and variance, or read it based on an existing pattern, obtain the adversarial patch pattern P, and determine the corner positions of the adversarial patch Corners. init :

[0022]

[0023] Corner tl Corner represents the row vector of coordinates of the top left corner point. tr Corner br Corner bl These represent the coordinate row vectors for the top right, bottom right, and bottom left sides, respectively. W and H are the width and height of the adversarial patch, respectively, with the default unit being meters. The ideal displacement of the adversarial patch coordinates within the target frame is also determined. ego_patch = [Δx, Δy, Δz], where Δx, Δy, and Δz are the displacements along the x-axis, y-axis, and z-axis in the vehicle coordinate system, respectively, and the rotations around the positive directions of the local coordinate system's x-axis, y-axis, and z-axis are the forward, leftward, and upward angles, respectively;

[0024] Step S202: Obtain relevant parameters of the target frame; read the data of the target frame to obtain the parameter information mentioned in step S101;

[0025] Step S203: Calculate the global coordinates of the corner points of the adversarial patch; first, construct the rotation matrix R and the transformation matrix T. p2e :

[0026] R = R z (γ)·R y (β)·R x (α)

[0027]

[0028] Among them, R x (α), R y (β), R z (γ) are the rotation matrices for rotating the corner point about the x-axis by α degrees, the y-axis by β degrees, and the z-axis by γ degrees in the local coordinate system, respectively, and are defined as follows:

[0029]

[0030]

[0031] Then, the global coordinates of the corner points of the adversarial patch are calculated (Corners). global And save it for later use:

[0032] Corners global =Corners init ·T p2e ·T e2g

[0033] Step S204: Generate adversarial images; traverse all loaded frames and obtain the k-th frame ({0, ..., N}). load -1} frames k The parameter information mentioned in step S101 is then used to transform the coordinates of the adversarial patch corner points in the global coordinate system to the camera coordinate system in each subsequent frame.

[0034]

[0035] in It is the transformation matrix from the global coordinate system to the camera coordinate system in the k-th frame, x k y k z k These are the x-axis, y-axis, and z-axis column vectors of the adversarial patch corner point in the k-th frame in the camera coordinate system. Then, the coordinates of the adversarial patch corner point in the camera coordinate system are normalized to obtain the coordinates of the adversarial patch corner point in the image coordinate system.

[0036]

[0037] The adversarial patch pattern was then scaled to the frame. k The original image k The size is then determined, and perspective transformation is performed based on the positions of the original corner points and the target corner points to obtain the perspective-transformed image in the original image. k Scenario-based adversarial patch

[0038]

[0039] Where Resize(P,image) k ) Scale the adversarial patch P to the image. k size, These are the coordinates of the four corner points of the scaled adversarial patch, starting from the top left and moving clockwise, in the image coordinate system. images k The width and height of the image, and then based on the adversarial patch... The mask is calculated at the position in the middle as follows.

[0040]

[0041] in for The pixel value at (i,j); finally, the original image. k Counter-patch images and mask Fusion Generative Adversarial Images

[0042]

[0043] After generating the adversarial image, the corresponding original image is replaced with the adversarial image to obtain adversarial data.

[0044] As a further improvement of the present invention, step S3 specifically includes:

[0045] Step S301, Initial Frame Inference: Output the inference result of the initial frame, and at the same time generate the BEV information of the initial frame and use it as the historical BEV information of the next frame for inference.

[0046] Step S302, Non-initial frame inference: Combine the BEV information of the previous frame to participate in inference, and combine the BEV information of the previous frame to output the BEV information of the current frame, output the inference result of the frame, and at the same time generate the BEV information of the current frame combined with the BEV information of the previous frame as the historical BEV information of the next frame to participate in inference.

[0047] Step S303, Batch optimization: N generated from the frames participating in training etuThe loss is selectively calculated for each result, the selected loss is accumulated and backpropagated, the gradient of the adversarial patch is then obtained and optimized, and the final adversarial patch is obtained after several iterations.

[0048] As a further improvement of the present invention, in step S303, the adversarial patch will participate in subsequent inferences after participating in the BEV of a certain frame inference, and the impact of the BEV feature extraction module and historical BEV on the result loss will be considered during backpropagation.

[0049] As a further improvement of the present invention, in step S303, the result of the target frame is necessary, while the results of other frames are optional.

[0050] The present invention also discloses an adversarial patch generation optimization system with joint temporal feature information, comprising: a memory, a processor, and a computer program stored in the memory, wherein the computer program is configured to implement the steps of the method described in the present invention when invoked by the processor.

[0051] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the method described in the present invention when invoked by a processor.

[0052] The beneficial effects of this invention are: 1. The method of this invention combines the temporal BEV feature information provided by the adversarial patch with the optimization method of the traditional adversarial patch, and obtains better optimization results; 2. The method of this invention takes into account the influence of the BEV feature extraction module and historical BEV on the result loss during backpropagation, so the final loss will cover multiple data flow branches, contain more information, and guide more accurate optimization results. Attached Figure Description

[0053] Figure 1 This is the overall optimized process of the method of the present invention. Detailed Implementation

[0054] This invention discloses an optimized method for generating adversarial patches using joint temporal feature information, comprising:

[0055] Step S1, Data Collection and Loading: Collect and load open-source datasets, simulation datasets, and custom real-world scenario datasets to obtain the total number of loaded frames;

[0056] Step S2, Calculate the adversarial patch position: Initialize the adversarial patch parameters, read the data of the target frame, obtain the parameter information mentioned in the open source dataset, calculate the global coordinates of the corner points of the adversarial patch, and generate the adversarial image;

[0057] Step S3, batch optimization based on historical BEV (bird's-eye view) information: adopt multi-frame temporal inference setting, calculate the total loss of output results of multiple consecutive frames, perform backpropagation and gradient optimization based on the total loss, and iteratively generate the final adversarial patch.

[0058] Step S1 specifically includes:

[0059] Step S101, Open source dataset: Use commonly used open source datasets in the field of autonomous driving, such as NuScenes and KITTI, to obtain the parameter information provided by the dataset (1) Transformation matrix T from vehicle coordinate system to global coordinate system e2g and (2) the transformation matrix T from the global coordinate system to the camera coordinate system g2c (This can be achieved by using the global coordinate system to local coordinate system transformation matrix T) g2e The transformation matrix T from the local coordinate system to the radar coordinate system e2l The transformation matrix T from radar coordinate system to camera coordinate system l2c Obtained by performing matrix multiplication.

[0060] Note: The original transformation matrices mentioned above are used for column coordinate operations. For ease of explanation in the following text, these transformation matrices are transposes of the original matrices and are all applicable to row coordinates. Furthermore, the transformation matrix from coordinate system a to coordinate system b is T. a2b And (T) a2b ) -1 =T b2a This represents the transformation matrix from coordinate system b to coordinate system a, and will not be further explained later.

[0061] Step S102, Simulation Dataset; The dataset is obtained by real-time rendering using simulation software such as CARLA, and parameter information similar to that in step S101 is obtained according to the settings of the simulation software.

[0062] Step S103, Customize Real-Scene Dataset: Acquire data by deploying your own sensor devices, perform custom annotations on the data, and obtain parameter information similar to that in S101 based on the sensor parameters.

[0063] Step S104: Select steps S101, S102, and S103 to construct continuous frame data of any scene in the dataset, and set the number of preheating frames N for a certain scene. pre The number of frames N used in training etu (By default, the last frame used in training is the target frame) t Add the two frame counts together to get the total number of loaded frames N. load .

[0064] Step S2 specifically includes:

[0065] Step S201, Initialize adversarial patch parameters: Set the shape and size of the adversarial patch, randomly generate it based on the target mean and variance, or read it based on an existing pattern, obtain the adversarial patch pattern P, and determine the corner positions of the adversarial patch Corners. init :

[0066]

[0067] Corner tl The row vector represents the coordinates of the top left corner point. Similarly, in clockwise order, there are row vectors representing the top right, bottom right, and bottom left corner points. W and H represent the width and height of the adversarial patch, respectively, with the default unit being meters. This also determines the coordinates of the adversarial patch within the Frame. t Translation of ideal displacement ego_patch = [Δx, Δy, Δz], where Δx, Δy, and Δz are the displacements along the x, y, and z axes in the vehicle coordinate system, respectively, and the angles α, β, and γ of the rotation about the local coordinate system (the positive directions of the x, y, and z axes are forward, left, and up, respectively).

[0068] Step S202, Obtain target frame related parameters: Read target frame Frame t The data is used to obtain the parameter information mentioned in step S101.

[0069] Step S203, calculate the global coordinates of the corner points of the adversarial patch: First, construct the rotation matrix R and the transformation matrix T. p2e :

[0070] R = R z (γ)·R y (β)·R x (α)

[0071]

[0072] T x2y A is the transformation matrix from the x-coordinate system to the (to is the same as 2)y-coordinate system; T For the transpose of matrix A, due to Translation ego_patch Since the matrix is ​​a row vector, the last column of the transformation matrix is ​​the offset, so transpose is used to construct the correct transformation matrix.

[0073] Among them, R x (α), R y (β), R z (γ) are the rotation matrices for rotating the corner point about the x-axis by α degrees, the y-axis by β degrees, and the z-axis by γ degrees in the local coordinate system, respectively, and are defined as follows:

[0074]

[0075] Then, the global coordinates of the corner points of the adversarial patch are calculated (Corners). global And save it for later use:

[0076] Corners global =Corners init ·T p2e ·T e2g Step S204, Generate adversarial image: Traverse all loaded frames and obtain the k-th frame ({0,…,N) load -1} frames k The parameter information mentioned in step S101 is then used to transform the coordinates of the adversarial patch corner points in the global coordinate system to the camera coordinate system in each subsequent frame.

[0077]

[0078] in It is the transformation matrix from the global coordinate system to the camera coordinate system in the k-th frame, x k y k z k These are the x-axis, y-axis, and z-axis column vectors of the adversarial patch corner point in the k-th frame, respectively, in the camera coordinate system. Then, the coordinates of the adversarial patch corner point in the camera coordinate system are normalized to obtain the coordinates of the adversarial patch corner point in the image coordinate system.

[0079]

[0080] The adversarial patch pattern was then scaled to the frame. k The original image k Size, then perform perspective transformation based on the positions of the original corner points and the target corner points, and obtain the perspective transformed image. k Scenario-based adversarial patch

[0081]

[0082] Where Resize(P,image) k ) Scale the adversarial patch P to the image. k size, These are the coordinates of the four corner points of the scaled adversarial patch, starting from the top left and moving clockwise, in the image coordinate system. images k The width and height of the image. Then, based on the adversarial patch... The mask is calculated at the position in the middle as follows.

[0083]

[0084] in for The pixel value at (i,j). Finally, the original image... k Counter-patch images and mask Fusion Generative Adversarial Images

[0085]

[0086] After generating the adversarial image, the corresponding original image is replaced with the adversarial image to obtain adversarial data.

[0087] Step S3 includes:

[0088] Experimental setup: Overall data flow direction as follows Figure 1 As shown in the diagram. Assume a target frame in a certain batch is frame t, and the two preceding frames involved in the inference of frame t are frames t-1 and t-2. The red line in the diagram represents the data flow of frame t-1, affecting the output result R of frame t-1. t-1 And the output result R of frame t t The experiment only considers N. etu The inference results of N frames used in training will be used to... etu The inference results are collected into a list, and the loss for each inference result is calculated and summed to form the total loss. After the total loss is calculated, backpropagation is performed, and then the gradient of the adversarial patch is obtained and optimized. After several iterations, the final adversarial patch is obtained.

[0089] Step S301, Initial Frame Inference. Since the initial frame is the first frame of the batch training, no historical BEV information is involved in the inference. This part of the inference will output the inference result of this frame, and at the same time generate the BEV information of this frame and use it as the historical BEV information for the next frame in the inference.

[0090] Step S302, Non-initial frame inference. During inference, the non-initial frame incorporates the BEV information from the previous frame and outputs the BEV information for the current frame. This part of the inference will output the inference result for that frame, while simultaneously generating the BEV information for the current frame, combined with the BEV information from the previous frame, as historical BEV information for the next frame to participate in the inference.

[0091] Step S304, Batch optimization: N generated from the frames participating in training etu The loss is calculated selectively for each result (target frame). tThe result of the first frame is necessary, while the results of other frames are optional. The selected loss is accumulated and backpropagated. The BEV of the adversarial patch after participating in the inference of a certain frame will also participate in subsequent inferences. Therefore, the backpropagation takes into account the impact of the BEV feature extraction module and the historical BEV on the result loss. Thus, the final loss will cover multiple data stream branches, containing more information and guiding more accurate optimization results.

[0092] The present invention also discloses an adversarial patch generation optimization system with joint temporal feature information, comprising: a memory, a processor, and a computer program stored in the memory, wherein the computer program is configured to implement the steps of the method described in the present invention when invoked by the processor.

[0093] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the method described in the present invention when invoked by a processor.

[0094] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. An optimized method for generating adversarial patches using joint temporal feature information, characterized in that, include: Step S1, Data Collection and Loading: Collect and load open-source datasets, simulation datasets, and custom real-world scenario datasets to obtain the total number of loaded frames; Step S2, calculate the position of the adversarial patch: initialize the adversarial patch parameters, read the target frame data, and obtain the annotation parameter information provided by the open source dataset, simulation dataset, and custom real scene dataset. This information is uniformly provided by the data collector. Based on the obtained parameters, calculate the position of the corner point of the adversarial patch in the global coordinate system and generate the adversarial image accordingly. Step S3, batch optimization based on historical BEV information: adopt multi-frame temporal inference setting, calculate the total loss of output results of multiple consecutive frames, perform backpropagation and gradient optimization based on the total loss, and iteratively generate the final adversarial patch; Step S3 specifically includes: Step S301, Initial Frame Inference: Output the inference result of the initial frame, and at the same time generate the BEV information of the initial frame and use it as the historical BEV information of the next frame for inference. Step S302, Non-initial frame inference: Combine the BEV information of the previous frame to participate in inference, and combine the BEV information of the previous frame to output the BEV information of the current frame, output the inference result of the frame, and at the same time generate the BEV information of the current frame combined with the BEV information of the previous frame as the historical BEV information of the next frame to participate in inference. Step S303, Batch optimization: N generated from the frames participating in training etu The loss is selectively calculated for each result, the selected loss is accumulated and backpropagated, the gradient of the adversarial patch is then obtained and optimized, and the final adversarial patch is obtained after several iterations.

2. The anti-patch generation optimization method according to claim 1, characterized in that, Step S1 specifically includes: Step S101, Open Source Dataset: Use the open source dataset to obtain parameter information: Transformation matrix T from vehicle coordinate system to global coordinate system. e2g and the transformation matrix T from the global coordinate system to the camera coordinate system g2c ; Step S102, Simulation Dataset: The dataset obtained by real-time rendering using simulation software, and parameter information similar to that in step S101 obtained according to the settings of the simulation software; Step S103, Customize real-world dataset: Acquire data through self-deployed sensor devices, perform custom annotations on the data, and obtain parameter information similar to that in step S101 based on sensor parameters; Step S104: Select steps S101, S102, and S103 to construct continuous frame data of any scene in the dataset, and set the number of preheating frames N for a certain scene. pre The number of frames N used in training etu By default, the last frame used in training is the target frame. t Add the two frame counts together to get the total number of loaded frames N. load .

3. The method for optimizing anti-patch generation according to claim 2, characterized in that, In step S101, parameter information is obtained using NuScenes and KITTI, open-source datasets commonly used in the field of autonomous driving; the global coordinate system to camera coordinate system transformation matrix T g2c Transformation matrix T from global coordinate system to local coordinate system g2e The transformation matrix T from the local coordinate system to the radar coordinate system e2l The transformation matrix T from radar coordinate system to camera coordinate system l2c Obtained by performing matrix multiplication.

4. The method for optimizing anti-patch generation according to claim 2, characterized in that, Step S2 specifically includes: Step S201, Initialize adversarial patch parameters: Set the shape and size of the adversarial patch, randomly generate it based on the target mean and variance, or read it based on an existing pattern, obtain the adversarial patch pattern P, and determine the corner positions of the adversarial patch Corners. init : Corner tl Corner represents the row vector of coordinates of the top left corner point. tr Corner br Corner bl These represent the coordinate row vectors for the top right, bottom right, and bottom left sides, respectively. W and H are the width and height of the adversarial patch, respectively, with the default unit being meters. The ideal displacement of the adversarial patch coordinates within the target frame is also determined. ego_patch = [Δx, Δy, Δz], where Δx, Δy, and Δz are the displacements along the x-axis, y-axis, and z-axis in the vehicle coordinate system, respectively, and the rotations around the positive directions of the local coordinate system's x-axis, y-axis, and z-axis are the forward, leftward, and upward angles, respectively; Step S202: Obtain relevant parameters of the target frame; read the data of the target frame to obtain the parameter information mentioned in step S101; Step S203: Calculate the global coordinates of the corner points of the adversarial patch; first, construct the rotation matrix R and the transformation matrix T. p2e : R=R z (c)·R y (b)·R x (a) Among them, A T Let R be the transpose of matrix A. x (α), R y (β), R z (γ) are the rotation matrices for rotating the corner point about the x-axis by α degrees, the y-axis by β degrees, and the z-axis by γ degrees in the local coordinate system, respectively, and are defined as follows: Then, the global coordinates of the corner points of the adversarial patch are calculated (Corners). global And save it for later use: Corners global =Corners init ·T p2e ·T e2g Step S204: Generate adversarial images; traverse all loaded frames and obtain the k-th frame ({0, ..., N}). load -1} frames k The parameter information mentioned in step S101 is then used to transform the coordinates of the adversarial patch corner points in the global coordinate system to the camera coordinate system in each subsequent frame. in It is the transformation matrix from the global coordinate system to the camera coordinate system in the k-th frame, x k y k z k These are the x-axis, y-axis, and z-axis column vectors of the adversarial patch corner point in the k-th frame in the camera coordinate system. Then, the coordinates of the adversarial patch corner point in the camera coordinate system are normalized to obtain the coordinates of the adversarial patch corner point in the image coordinate system. The adversarial patch pattern was then scaled to the frame. k The original image k The size is then determined, and perspective transformation is performed based on the positions of the original corner points and the target corner points to obtain the perspective-transformed image in the original image. k Scenario-based adversarial patch Where Resize(P,image) k ) Scale the adversarial patch P to the image. k size, These are the coordinates of the four corner points of the scaled adversarial patch, starting from the top left and moving clockwise, in the image coordinate system. images k The width and height of the image, and then based on the adversarial patch... The mask is calculated at the position in the middle as follows. in for The pixel value at (i,j); finally, the original image. k Counter-patch images and mask Fusion Generative Adversarial Images After generating the adversarial image, the corresponding original image is replaced with the adversarial image to obtain adversarial data.

5. The method for optimizing anti-patch generation according to claim 1, characterized in that, In step S303, the adversarial patch will participate in subsequent inferences after participating in the BEV of a certain frame inference. During backpropagation, the impact of the BEV feature extraction module and historical BEV on the result loss is considered.

6. The method for optimizing anti-patch generation according to claim 1, characterized in that, In step S303, the result of the target frame is necessary, while the results of other frames are optional.

7. An adversarial patch generation optimization system that combines temporal feature information, characterized in that, include: A memory, a processor, and a computer program stored on the memory, the computer program being configured to implement the steps of the method of any one of claims 1-6 when invoked by the processor.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to implement the steps of the method according to any one of claims 1-6 when invoked by a processor.

Images