Reversible anti-interference method, device, equipment and medium for camera images

Abstract

The invention discloses a reversible adversarial interference method, device, equipment and medium for camera images, and relates to the technical field of data adversarial interference. The method comprises: obtaining an original image and dividing it into superpixel blocks. According to the superpixel blocks, the initial gradient is calculated and then an initial perturbation contribution matrix is ​​constructed. According to the initial perturbation contribution matrix, an initial perturbation is constructed. The initial perturbation is superimposed on the original image to obtain an initial adversarial sample. Subsequent steps are iterated until the iteration is completed to obtain a final adversarial sample and a corresponding final perturbation. Based on the adversarial sample generated in the previous iteration, the iterative gradient is calculated and channel enhancement is performed through a channel attention network. The iterative gradient and the gradient of the previous iteration are dynamically fused through an attenuation factor to obtain a new gradient. According to the new gradient, a new perturbation contribution matrix is ​​constructed. According to the new perturbation contribution matrix, a perturbation increment is constructed to accumulate perturbations, and the accumulated perturbations are amplitude clipped to obtain new perturbations. The new perturbation is superimposed on the original image to obtain a new adversarial sample.

Description

Technical Field

[0001] The present invention relates to the technical field of data anti-interference, and in particular to a reversible anti-interference method, device, equipment and medium for camera images. Background Art

[0002] With the rapid development of autonomous driving technology, the number of cameras installed in cars is increasing, and these cameras can capture a large amount of image data during driving. This data not only contains road environment and traffic conditions, but may also involve sensitive information, such as infrastructure in a specific area, private places or other confidential objects.

[0003] Due to the imperfect data collection and storage mechanism, these image data can easily be used to train related models without the permission of the data owner. This phenomenon not only leads to a great risk of leakage of scene-level sensitive information, but also the images of confidential infrastructure, special places or sensitive objects involved in the data collection process may be improperly obtained and used, and infringes on the rights and interests of the data owner. Summary of the invention

[0004] The present invention provides a reversible anti-interference method, device, equipment and medium for camera images to improve at least one of the above technical problems.

[0005] In a first aspect, the present invention provides a reversible anti-interference method for a camera image, comprising:

[0006] Get the original image.

[0007] The original image is divided into a plurality of super-pixel blocks by a super-pixel segmentation algorithm.

[0008] According to the superpixel block, an initial gradient is calculated through a proxy model and a loss function.

[0009] According to the initial gradient, an initial disturbance contribution matrix is ​​constructed.

[0010] An initial disturbance is constructed according to the initial disturbance contribution matrix.

[0011] The initial perturbation is superimposed on the original image to obtain an initial adversarial sample.

[0012] Iterate the subsequent steps until the iteration is completed to obtain the final adversarial sample and the corresponding final perturbation.

[0013] Based on the adversarial sample generated in the previous iteration, the iterative gradient is calculated through the proxy model and loss function.

[0014] Recalibrate the iterative gradient through a channel attention network to generate a channel attention weight matrix, and perform a Hadamard product operation on the weight matrix and the iterative gradient to obtain the iterative gradient after channel enhancement;

[0015] The iterative gradient and the gradient of the previous iteration are dynamically fused through the attenuation factor to obtain a new gradient.

[0016] According to the new gradient, a new disturbance contribution matrix is ​​constructed.

[0017] According to the new disturbance contribution matrix, a disturbance increment is constructed to accumulate disturbance, and the accumulated disturbance is amplitude-cut to obtain a new disturbance.

[0018] The new perturbation is superimposed on the original image to obtain a new adversarial sample.

[0019] Furthermore, the calculation model of the initial gradient is: In the formula, is the initial gradient, represents the gradient, For the original image, for functions used for dimension expansion and padding, For the proxy model, is the true category label.

[0020] Furthermore, based on the adversarial sample generated in the previous iteration, the iterative gradient is calculated through the proxy model and the loss function, including:

[0021] Input the adversarial sample generated in the previous iteration into the proxy model , get the prediction results . Wherein, the proxy model is a 2D classification prediction model.

[0022] The loss function is calculated based on the prediction results. The adversarial sample generated in the previous iteration is input and the true category label The loss value is . for functions used for dimension expansion and padding, is the adversarial sample obtained in the previous iteration, is the true category label.

[0023] The iterative gradient is calculated according to the loss function. The calculation model of the iterative gradient is: .in, For the The gradient of the iteration, represents the gradient, The adversarial sample generated in the previous iteration, for functions used for dimension expansion and padding, is the true category label.

[0024] Furthermore, the iterative gradient is feature recalibrated through a channel attention network to generate a channel attention weight matrix, and the weight matrix is ​​subjected to a Hadamard product operation with the iterative gradient to obtain the iterative gradient after channel enhancement, specifically including:

[0025] Performing a global average pooling operation on the iterative gradient, compressing the features of each channel, and obtaining a feature vector;

[0026] Use a fully connected layer to transform the feature vector, and then apply the ReLU activation function to the transformed feature vector to introduce nonlinearity;

[0027] A fully connected layer is then used to restore the feature vector to its shape before the transformation, and then the Sigmoid activation function is applied to limit the weight of each channel to the range of [0,1] to obtain the channel attention weight matrix;

[0028] Finally, the generated channel attention weight matrix is ​​subjected to a Hadamard product operation with the iterative gradient to obtain the iterative gradient after channel enhancement;

[0029] Furthermore, the fusion model of the new gradient is: In the formula, For the new gradient, is the attenuation factor, is the gradient of the previous iteration, For the Iterative gradient of iteration .

[0030] Furthermore, according to the new gradient, a new disturbance contribution matrix is ​​constructed, which specifically includes:

[0031] According to the new gradient, an absolute value matrix is ​​obtained. In the formula, is the absolute value matrix, For the new gradient.

[0032] According to the absolute value matrix, a new disturbance contribution matrix is ​​constructed. The construction model of the new disturbance contribution matrix is: In the formula, For the The new perturbation contribution matrix of the iteration, is the natural exponential function, Indicates Line The absolute value of the gradient of the superpixel in the column, Indicates The absolute value of the gradient of the superpixel, Represents the constraints of the model, Indicates the image height, Indicates the image width, represents the height of the superpixel, Represents the width of the superpixel.

[0033] Furthermore, the calculation model of the initial disturbance is: In the formula, is the initial disturbance, is the unit disturbance, is the symbolic function, is the initial perturbation contribution matrix.

[0034] Furthermore, according to the new disturbance contribution matrix, a disturbance increment is constructed to accumulate disturbances, and the accumulated disturbances are amplitude-cut to obtain new disturbances, which specifically includes:

[0035] According to the new disturbance contribution matrix, a disturbance increment is generated. The calculation model of the disturbance increment is: In the formula, For the The perturbation increment of the iteration, is the unit disturbance, is the symbolic function, For the The new perturbation contribution matrix for iteration .

[0036] According to the disturbance increment, the disturbance is accumulated to obtain a new disturbance. The accumulation model of the new disturbance is: In the formula, For new disturbances, is the perturbation obtained in the previous iteration, For the The perturbation increment for iterations.

[0037] For new disturbances Amplitude clipping is performed to ensure that the invisibility constraint is met. That is: In the formula, for functions used for dimension expansion and padding, for Norm, is the unit disturbance, is the maximum disturbance multiple.

[0038] Further, according to the new disturbance contribution matrix, a disturbance increment is constructed to accumulate disturbance, and the accumulated disturbance is amplitude-cut to obtain a new disturbance, which also includes:

[0039] Based on the new perturbation , using implicit scene protection (ISP) to perform black-box attacks on the proxy model of the unknown model. Taking the perturbation and the original image as input, the direction Randomly selected based on the chosen perturbation. Adding perturbations to the model changes the confidence , if the direction Failure to reduce , then the direction Instead, for each perturbation point, record its contribution to the decrease in confidence. After that, the three points that produced the most significant decrease in model confidence are identified and selected from the recordings. These points are excluded and, in subsequent iterations, only the remaining points to which additional perturbations can be applied are enhanced.

[0040] Furthermore, a reversible adversarial interference method for a camera image further includes: performing perturbation decoding according to the final perturbation and the final adversarial sample to obtain the original image. The perturbation is superimposed on the original image, and the final adversarial sample is perturbation decoded, which is achieved by grayscale invariant reversible data embedding.

[0041] In the second aspect, the present invention provides a reversible anti-interference device for camera images, which includes an image acquisition module, a superpixel module, an initial gradient module, an initial contribution module, an initial perturbation module, an initial superposition module, an iteration module, an iterative gradient module, a new map module, a new contribution module, a new perturbation module and an iterative superposition module.

[0042] The image acquisition module is used to acquire the original image.

[0043] The superpixel module is used to divide the original image into multiple superpixel blocks through a superpixel segmentation algorithm.

[0044] The initial gradient module is used to calculate the initial gradient according to the super pixel block through the proxy model and the loss function.

[0045] The initial contribution module is used to construct an initial disturbance contribution matrix according to the initial gradient.

[0046] The initial disturbance module is used to construct an initial disturbance according to the initial disturbance contribution matrix.

[0047] The initial superposition module is used to superimpose the initial perturbation onto the original image to obtain an initial adversarial sample.

[0048] The iteration module is used to iterate the subsequent steps until the iteration is completed to obtain the final adversarial sample and the corresponding final perturbation.

[0049] The iterative gradient module is used to calculate the iterative gradient based on the adversarial samples generated in the previous iteration through the proxy model and loss function.

[0050] An attention module is used to perform feature recalibration on the iterative gradient through a channel attention network, generate a channel attention weight matrix, and perform a Hadamard product operation on the weight matrix and the iterative gradient to obtain the iterative gradient after channel enhancement;

[0051] The new map module is used to dynamically fuse the iterative gradient and the gradient of the previous iteration through a decay factor to obtain a new gradient.

[0052] The new contribution module is used to construct a new disturbance contribution matrix according to the new gradient.

[0053] The new disturbance module is used to construct a disturbance increment according to the new disturbance contribution matrix to accumulate disturbances, and to perform amplitude clipping on the accumulated disturbances to obtain new disturbances.

[0054] The iterative superposition module is used to superimpose the new perturbation onto the original image to obtain a new adversarial sample.

[0055] In a third aspect, the present invention provides a reversible anti-interference device for camera images, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a reversible anti-interference method for camera images as described in any paragraph of the first aspect.

[0056] In a fourth aspect, the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a reversible anti-interference method for a camera image as described in any paragraph of the first aspect.

[0057] By adopting the above technical solution, the present invention can achieve the following technical effects:

[0058] A reversible adversarial interference method for camera images in an embodiment of the present invention reduces the data quality of unauthorized models by embedding adversarial perturbations, thereby being used to protect camera-based autonomous driving data from unauthorized use. At the same time, it enables legitimate users to recover the original data through advanced steganography. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the specific implementation methods of the present invention. It should be understood that the following drawings only show certain specific implementation methods of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 The present invention is a logic block diagram of a reversible anti-interference method for camera images.

[0061] Figure 2 This is a schematic diagram showing that when the image is not disturbed, the model can normally recognize the information in the image.

[0062] Figure 3 Schematic diagram showing that after the image is disturbed, the image cannot recognize the information in the image.

[0063] Figure 4 Schematic diagram of the effect before and after disturbance.

[0064] Figure 5 The present invention is an algorithm framework diagram of a reversible anti-interference method for camera images.

[0065] Figure 6 This is the overall framework diagram of steganography in image processing.

[0066] Figure 7 This is a visualization of the reversible adversarial interference method on the Grad-CAM heat map.

[0067] Figure 8 It is a visualization diagram of the image and its quality under different noise concentrations of the reversible anti-interference method.

[0068] Fig. 9 This is a visualization of the 3D detection results of the reversible adversarial interference method in BEVDet to achieve scene-locked encoding and decoding.

[0069] Fig.10 It is a visualization of the 3D occupancy prediction results of the reversible adversarial interference method in DHD scene-locked encoding and decoding.

[0070] Fig.11 This is a visualization of the Grad-CAM heat map of clean images, adversarial images, and restored images. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Example 1, please refer to Figures 1 to 11 The first embodiment of the present invention provides a reversible anti-interference method for a camera image, which can be performed by a reversible anti-interference device for a camera image (hereinafter referred to as: interference device). In particular, it is performed by one or more processors in the interference device to implement steps S01 to S13.

[0073] First, the symbols used in the embodiment are defined. The original image of the input clean scene is , represents a real number, represents the image dimension, Indicates the image height, represents the image width. The superpixel size is , represents the height of the superpixel, represents the width of the superpixel. The true category label is , the true bounding box is , the classification category is , proxy model The output result is . The disturbance is , the perturbation is generated by gradient binary encoding (GBE, Gradient Binary Encoding). is the unit disturbance, is the maximum disturbance multiple, is a symbolic function. represents the gradient, Indicates increment, is a function for dimension expansion and padding.

[0074] The following is a brief description of the steps of the reversible anti-interference method for camera images.

[0075] S01. Input a clean original image .

[0076] Preferably, the original image is an image taken by a camera on a car. It is understandable that the interference device can be an electronic device with computing performance such as a portable notebook computer, a desktop computer, a server, a smart phone or a tablet computer.

[0077] After the original image is input, the algorithm iteration begins, and the first iteration uses the original image as the processing object. Subsequent iterations all use the initial adversarial sample generated in the previous iteration as the processing object.

[0078] The data processing flow of the first iteration includes steps S02 to S05.

[0079] S02, dividing the original image into a plurality of super pixel blocks by a super pixel segmentation algorithm. Specifically, the super pixel segmentation algorithm can effectively reduce the amount of calculation of the algorithm. The super pixel segmentation algorithm is a prior art, and the present invention will not be described in detail here.

[0080] S03. Calculate the initial gradient according to the superpixel block through the proxy model and loss function .

[0081] The calculation model of the initial gradient is: In the formula, represents the gradient, For the original image, for functions used for dimension expansion and padding, For the proxy model, is the true category label.

[0082] Specifically, the calculation model of the initial gradient in step S03 is the same as the calculation model of the new gradient in step S08. The only difference is that step S03 uses the original image for calculation, while step S08 uses the adversarial sample generated in the previous iteration for calculation. Therefore, please refer to step S08 for the detailed calculation process of the gradient.

[0083] S04. Construct an initial disturbance contribution matrix based on the initial gradient .

[0084] Specifically, the initial disturbance contribution matrix of step S04 is The calculation model of is the same as the calculation model of the new disturbance contribution matrix in step S10. It is just that step S04 uses the initial gradient for calculation, while step S10 uses the new gradient for calculation. Therefore, the detailed calculation process of the disturbance contribution matrix is ​​referred to step S10.

[0085] S05. Construct an initial disturbance according to the initial disturbance contribution matrix .

[0086] The calculation model of the initial disturbance is: In the formula, is the unit disturbance, is the symbolic function, is the initial perturbation contribution matrix.

[0087] Based on the initial perturbation contribution matrix and gradient direction, The initial perturbation of the norm. Specifically, the initial perturbation of step S05 The calculation model of is the same as the calculation model of the new disturbance in step S12. It is just that step S04 uses the initial gradient for calculation, while step S10 uses the new gradient for calculation. Therefore, for the detailed calculation process of the disturbance, refer to step S10.

[0088] S06: superimpose the initial disturbance onto the original image to obtain an initial adversarial sample Among them, the calculation model of the initial adversarial sample is: .

[0089] The data processing flow of subsequent iterations after the first iteration includes steps S07 to S13.

[0090] S08, based on the adversarial sample generated in the previous iteration, the iterative gradient is calculated through the proxy model and the loss function. Preferably, step S08 specifically includes steps S081 to S083.

[0091] S081. Input the adversarial sample generated in the previous iteration into the proxy model , get the prediction results . Wherein, the proxy model is a 2D classification prediction model.

[0092] S082, calculating the loss function according to the prediction result. The adversarial sample generated in the previous iteration is input and the true category label The loss value is . for functions used for dimension expansion and padding, is the adversarial sample obtained in the previous iteration, is the true category label.

[0093] S083. Calculate the iterative gradient according to the loss function. The calculation model of the iterative gradient is: .in, For the The gradient of the iteration, represents the gradient, The adversarial sample generated in the previous iteration, for functions used for dimension expansion and padding, is the true category label.

[0094] Specifically, the purpose of adding perturbations is to make it impossible for the model to detect the content in the image.

[0095] .

[0096] In the formula, For the proxy model, The adversarial sample generated in the previous iteration, is the output of the proxy model, is the true category label, Represents the constraints of the model, for functions used for dimension expansion and padding, For disturbance, express Norm, is the unit disturbance, is the maximum disturbance multiple.

[0097] Therefore, in this embodiment, a 2D classification prediction model is used as a proxy model to predict the adversarial sample generated in the previous iteration, and the loss function is calculated based on the prediction result. The effect of the disturbance is judged by the gradient of the loss function.

[0098] The operation of adding disturbances to the original image using the reversible adversarial interference method of a camera image of the present invention is defined as Noise Serialization Encoding (NSE). NSE is a coding module specially designed for scene detection and scene perception tasks. Therefore, it is different from the single-class weakly supervised classification attack task, and its main purpose is to achieve semantic interference and detection failure.

[0099] In order to destroy the confidence of the target in the scene, the invention designs the following loss function for the proxy model:

[0100] .

[0101] in, Represents the adversarial sample generated by the previous iteration and the true category label The loss value at that time. yes One-hot encoding of is a logarithmic function, is the natural exponential function, For the proxy model, input and the correct category of the image The output of Classification category, is the number of classification categories, is the serial number of the classification category, For the proxy model, input and The output of the classification categories, is the number of bounding boxes, is the ordinal number of the bounding box, represents the predicted bounding box, represents the true bounding box, represents the bounding box predicted by the surrogate model, Indicates The ground-truth bounding box, is the L1 norm.

[0102] To generate adversarial samples, this embodiment calculates pixel gradients from the loss function and adds perturbations to increase the loss for non-targeted attacks. Given the variability of gradient values ​​at different locations, applying uniform perturbations will have different effects on the loss function. Therefore, this embodiment prioritizes more significant perturbations at points that have a greater impact on the loss function, as these areas are more sensitive to input changes that may significantly affect the final classification decision, thereby increasing the effectiveness of the attack. Therefore, the gradient is obtained by smoothing the gradient of each superpixel.

[0103] The calculation model of iterative gradient is:

[0104] .

[0105] .

[0106] In the formula, For the The iterative gradient of the iteration, represents the gradient, The adversarial sample generated in the previous iteration, for functions used for dimension expansion and padding, is the true category label. Indicates the number of rows, Indicates the number of columns, Indicates Line The gradient of the superpixel of the column, represents the image dimension, Indicates the image height, Indicates the image width.

[0107] S09, recalibrating the iterative gradient through a channel attention network to generate a channel attention weight matrix, and performing a Hadamard product operation on the weight matrix and the iterative gradient to obtain the iterative gradient after channel enhancement;

[0108] Specifically, the iterative gradient is a multidimensional tensor whose shape is related to the number of channels and spatial size of the image. The channel attention network first performs a global average pooling operation on the iterative gradient in the spatial dimension, compresses the features of each channel into a scalar value, and obtains a feature vector with a shape of 1×C, where C is the image dimension (i.e., the number of channels). Then, a fully connected layer is used to transform the feature vector and output a feature vector with a shape of 1×C / r, where r is the reduction ratio (i.e., reduction ratio) to reduce the number of parameters and computational complexity. Then, the ReLU activation function is applied to the output of the fully connected layer to introduce nonlinearity. Then, another fully connected layer is used to restore the feature vector to a shape of 1×C. Then, the Sigmoid activation function is applied to the feature vector restored by the fully connected layer, and the weight of each channel is limited to the range of [0, 1] to obtain the channel attention weight matrix. Finally, the generated channel attention weight matrix is ​​Hadamard producted with the iterative gradient and output. That is, the gradient features of each channel are multiplied by the corresponding weights to enhance important channel features and suppress unimportant channel features. The iterative gradient after channel enhancement will be used in the subsequent gradient fusion step to generate more discriminative new gradients and guide more effective perturbation generation.

[0109] S10, dynamically fuse the iterative gradient and the gradient of the previous iteration through the attenuation factor to obtain a new gradient. Specifically, the fusion model of the new gradient is:

[0110] .

[0111] In the formula, For the new gradient, is the attenuation factor, is the gradient of the previous iteration, For the Iterative gradient of iteration .

[0112] S11. Construct a new disturbance contribution matrix according to the new gradient. Preferably, step S11 includes steps S111 to S112.

[0113] S111, obtaining an absolute value matrix according to the new gradient. In the formula, is the absolute value matrix, is the new gradient. Preferably, in the first iteration, .

[0114] S112, constructing a new disturbance contribution matrix according to the absolute value matrix to determine the sensitivity of superpixels in different blocks.

[0115] The construction model of the new disturbance contribution matrix is:

[0116] .

[0117] In the formula, For the The new perturbation contribution matrix of the iteration, is the natural exponential function, Indicates Line The absolute value of the gradient of the superpixel in the column, Indicates The absolute value of the gradient of the superpixel, Represents the constraints of the model, Indicates the image height, Indicates the image width, represents the height of the superpixel, Represents the width of the superpixel.

[0118] It represents the impact of perturbations at different locations on the loss function, that is, the contribution fraction of the gradient to the deviation in the loss function. Taking into account the changes in contributions from different blocks, the generated perturbation values ​​are quantized into multiple levels.

[0119] S12. According to the new disturbance contribution matrix, a disturbance increment is constructed to accumulate disturbances, and the accumulated disturbances are amplitude-cut to obtain new disturbances.

[0120] S121. Generate a disturbance increment according to the new disturbance contribution matrix.

[0121] The calculation model of disturbance increment is:

[0122] .

[0123] Where: For the The perturbation increment of the iteration, is the unit disturbance, is the symbolic function, For the The new perturbation contribution matrix for iteration .

[0124] S122. Accumulate disturbances according to the disturbance increment to obtain new disturbances.

[0125] The accumulation model of new disturbances is:

[0126] .

[0127] Where: For new disturbances, is the perturbation obtained in the previous iteration, For the The perturbation increment for iterations.

[0128] S123, New disturbance Amplitude clipping is performed to ensure that the invisibility constraint is met. That is: Specifically, according to the new disturbance, use The norm limiting method limits the maximum perturbation amplitude of each pixel to In which, is the unit disturbance, is the maximum disturbance multiple. Norm restriction on new perturbations Amplitude clipping can ensure that the perturbation satisfies the invisibility constraint.

[0129] After constructing the new perturbation contribution matrix, the current perturbation and adversarial samples need to be updated. First, the perturbation is updated based on the perturbation obtained in the previous iteration and the new perturbation contribution matrix. Specifically, adding the current contribution matrix to the existing perturbation can further amplify the perturbation effect in the target area while retaining the previously accumulated perturbation information. This gradual accumulation method can effectively focus the perturbation on the area that has the greatest impact on the model prediction results.

[0130] However, in order to control the amplitude of the perturbation and maintain the similarity between the adversarial sample and the original image, it is necessary to perform a cropping operation on the updated perturbation. The norm restriction method limits the maximum perturbation amplitude of each pixel to This means that even after multiple rounds of iterations, the total intensity of the perturbation will not exceed the preset maximum perturbation multiple, thus ensuring that the generated adversarial samples have minimal perceptual changes. In addition, this clipping operation can also avoid outlier interference that may occur during gradient calculation, making the perturbation more stable.

[0131] Based on the above embodiment, in an optional embodiment of the present invention, step S12 also includes step S124.

[0132] S124, based on new disturbance , using implicit scene protection (ISP) to perform black-box attacks on the proxy model of the unknown model. Taking the perturbation and the original image as input, the direction Randomly selected based on the chosen perturbation. Adding perturbations to the model changes the confidence , if the direction Failure to reduce , then the direction Instead, for each perturbation point, record its contribution to the decrease in confidence. After that, the three points that produce the most significant decrease in model confidence are identified and selected from the records. These points are excluded, and in subsequent iterations, only the remaining points where additional perturbations can be applied are enhanced. Specifically, since the perturbations of the superpixel blocks may have reached their maximum threshold, further increasing these perturbations may have limited direct impact on the confidence. Therefore, these points are excluded.

[0133] S13, superimposing the new disturbance onto the original image to obtain a new adversarial sample. The calculation model of the new adversarial sample is: In the formula, For new adversarial samples, For the original image, For new disturbance.

[0134] Once the new perturbation is calculated, it is applied to the current adversarial sample to generate the next iterative adversarial sample. In this process, the pixel values ​​are bounded to ensure that the generated adversarial samples remain numerically legal. Specifically, each pixel value needs to be in the range of 0 to 1 (assuming that the pixel values ​​are normalized. If not normalized, it is in the range of 0 to 255). In this way, the generation of abnormal adversarial samples, such as negative pixels or pixels exceeding the maximum value, can be avoided. At the same time, the mapping function It is used to apply the pruned perturbations to adversarial samples to further optimize their distribution. This iterative update process is the key to adversarial sample generation, which continuously optimizes the perturbation direction and strength in each iteration to improve the effectiveness of adversarial attacks.

[0135] like Figure 5 The algorithm framework diagram of the reversible anti-interference method is shown. The RED in the figure represents reversible data embedding.

[0136] S07. After performing multiple iterations from step S07 to step S12 until the iterations are completed, the adversarial sample and perturbation outputted from the last iteration are taken as the final adversarial sample and the corresponding final perturbation.

[0137] After all iterations are completed, the final adversarial sample and the corresponding perturbation are returned as output. At this point, the adversarial sample has gone through multiple rounds of gradient optimization and perturbation accumulation, and can effectively interfere with the proxy model. Because each iteration strictly controls the range of perturbations while maximizing the adversarial effect, the final adversarial sample is not only highly aggressive but also visually highly similar to the original sample.

[0138] Specifically, the final adversarial sample It is expressed as:

[0139]

[0140] is a function for dimension expansion and padding. Due to the use of superpixels, is effectively used as a simplified perturbation block with a two-dimensional shape, specifically, Indicates location The perturbation of the superpixel at , and the function Extend the disturbance to the coverage area ,in( and , Indicates a channel. Definition is a unit disturbance, and a three-digit code is used to represent The amplitude of the disturbance at , used to indicate the count of unit disturbance.

[0141] The perturbation generation step of this embodiment is an efficient perturbation compression technique that uses superpixels to replace individual pixels. The storage requirements are reduced by applying gradient smoothing to superpixels, thereby meeting the limitations of reversible data embedding RDE-GI in terms of encodable byte length while maintaining adversarial efficacy. Therefore, even if the data space is reduced, the perturbation still effectively challenges the model.

[0142] It should be noted that the traditional reversible data embedding (RDE) method often introduces distortion in the grayscale representation of the image, which is crucial for feature analysis. Therefore, the embodiment of the present invention adopts the grayscale invariance reversible data embedding (RDE-GI) method to superimpose the disturbance on the original image. RDE-GI uses the R and B channels of the color image to embed information, and adjusts the pixel value in the G channel to ensure grayscale invariance.

[0143] From step S01 to step S13, perturbations of different sensitivities are constructed through superpixel blocks and cleverly embedded into the adversarial image using RDE-GI technology. In the process of generating adversarial samples, the perturbations are encoded into binary information streams and then embedded into the adversarial image along with related auxiliary data, which preserves the adversarial features of the image while ensuring the integrity of the embedded information.

[0144] like Figure 2 As shown in , clean scene data can be directly used by the model to identify the content in the picture. Figure 3As shown in , after adding disturbances, the scene data loses the perception of vehicles on the road and cannot recognize the content in the picture. Figure 7 As shown in the figure, the upper part of the picture shows the privacy protection effect for the target, and the lower part of the picture shows the privacy protection effect for the scene.

[0145] A reversible adversarial interference method for camera images in an embodiment of the present invention reduces the data quality of unauthorized models by embedding adversarial perturbations, thereby being used to protect camera-based autonomous driving data from unauthorized use. At the same time, it enables legitimate users to recover the original data through advanced steganography.

[0146] RDE-GI is a reversible embedding technology, so as long as the disturbance is obtained through authorization, the RDE-GI technology can be used to remove the hidden information. Therefore, a reversible anti-interference method for camera images also includes step S14.

[0147] S14. Perform perturbation decoding according to the final perturbation and the final adversarial sample to obtain the original image.

[0148] Specifically, in the entire process from step S01 to step S13, all disturbances When the original image needs to be restored, the RDE-GI technology is used to extract the hidden information, and the system can perform reverse operations. Accurately restore the original image.

[0149] This enables accurate reconstruction of the perturbation after it has been removed, thus helping to restore the original image with minimal loss. With this innovative approach, not only is the effectiveness of adversarial attacks maintained, but the reversibility of the process is also ensured, allowing for seamless restoration of an approximate original image. The weighted update mechanism of the perturbation contribution matrix during encoding avoids gradient oscillations, combined with The constraint and benchmark superposition strategy achieves strict reversibility while ensuring the effectiveness of adversarial attacks, providing the autonomous driving system with the ability to recover safely under adversarial interference.

[0150] Embodiment 2: The present invention provides a reversible anti-interference device for camera images, which includes an image acquisition module, a superpixel module, an initial gradient module, an initial contribution module, an initial perturbation module, an initial superposition module, an iteration module, an iterative gradient module, a new map module, a new contribution module, a new perturbation module and an iterative superposition module.

[0151] The image acquisition module is used to acquire the original image.

[0152] The superpixel module is used to divide the original image into multiple superpixel blocks through a superpixel segmentation algorithm.

[0153] The initial gradient module is used to calculate the initial gradient according to the super pixel block through the proxy model and the loss function.

[0154] The initial contribution module is used to construct an initial disturbance contribution matrix according to the initial gradient.

[0155] The initial disturbance module is used to construct an initial disturbance according to the initial disturbance contribution matrix.

[0156] The initial superposition module is used to superimpose the initial perturbation onto the original image to obtain an initial adversarial sample.

[0157] The iteration module is used to iterate the subsequent steps until the iteration is completed to obtain the final adversarial sample and the corresponding final perturbation.

[0158] The iterative gradient module is used to calculate the iterative gradient based on the adversarial samples generated in the previous iteration through the proxy model and loss function.

[0159] The new map module is used to dynamically fuse the iterative gradient and the gradient of the previous iteration through a decay factor to obtain a new gradient.

[0160] The new contribution module is used to construct a new disturbance contribution matrix according to the new gradient.

[0161] The new disturbance module is used to construct a disturbance increment according to the new disturbance contribution matrix to accumulate disturbances, and to perform amplitude clipping on the accumulated disturbances to obtain new disturbances.

[0162] The iterative superposition module is used to superimpose the new perturbation onto the original image to obtain a new adversarial sample.

[0163] Embodiment 3: The present invention provides a reversible anti-interference device for camera images, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a reversible anti-interference method for camera images as described in any paragraph of Embodiment 1.

[0164] Embodiment 4: The present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a reversible anti-interference method for a camera image as described in any paragraph of Embodiment 1.

[0165] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0166] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0167] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0168] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0169] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0170] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0171] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0172] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A reversible anti-interference method for camera images, characterized in that: Include: Get the original image; Dividing the original image into multiple super-pixel blocks by a super-pixel segmentation algorithm; According to the superpixel block, an initial gradient is calculated through a proxy model and a loss function; According to the initial gradient, construct an initial disturbance contribution matrix; Constructing an initial disturbance according to the initial disturbance contribution matrix; Superimposing the initial perturbation onto the original image to obtain an initial adversarial sample; Iterate the subsequent steps until the iteration is completed to obtain the final adversarial sample and the corresponding final perturbation; Based on the adversarial sample generated in the previous iteration, the iterative gradient is calculated through the proxy model and the loss function; Recalibrate the iterative gradient through a channel attention network to generate a channel attention weight matrix, and perform a Hadamard product operation on the weight matrix and the iterative gradient to obtain the iterative gradient after channel enhancement; The iterative gradient and the gradient of the previous iteration are dynamically fused through the attenuation factor to obtain a new gradient; According to the new gradient, construct a new disturbance contribution matrix; According to the new disturbance contribution matrix, a disturbance increment is constructed to accumulate disturbance, and the accumulated disturbance is amplitude-cut to obtain a new disturbance; Superimposing the new perturbation onto the original image to obtain a new adversarial sample; The calculation model of the initial disturbance is: ; In the formula, is the initial disturbance, is the unit disturbance, is the symbolic function, is the initial perturbation contribution matrix; According to the new disturbance contribution matrix, a disturbance increment is constructed to accumulate disturbance, and the accumulated disturbance is amplitude-cut to obtain a new disturbance, which specifically includes: According to the new disturbance contribution matrix, a disturbance increment is generated; wherein the calculation model of the disturbance increment is: ; In the formula, For the The perturbation increment of the iteration, is the unit disturbance, is the symbolic function, For the The new perturbation contribution matrix for the iteration; According to the disturbance increment, disturbance accumulation is performed to obtain a new disturbance; the accumulation model of the new disturbance is: ; In the formula, For new disturbances, is the perturbation obtained in the previous iteration, For the The perturbation increment for the iteration; For new disturbances Amplitude clipping is performed to ensure that the invisibility constraints are met; that is: ; In the formula, for functions used for dimension expansion and padding, for Norm, is the unit disturbance, is the maximum disturbance multiple; According to the new disturbance contribution matrix, a disturbance increment is constructed to accumulate disturbance, and the accumulated disturbance is amplitude-cut to obtain a new disturbance, which also includes: Based on the new perturbation , perform black-box attacks on proxy models of unknown models using implicit scene protection; take perturbations and original images as input, direction Based on the chosen perturbation, random selection; in the direction Adding perturbations to the model changes the confidence , if the direction Failure to reduce , then the direction Instead, for each perturbation point, record its contribution to the decrease in confidence; at a predefined perturbation iteration After that, the three points that produced the most significant decrease in model confidence are identified and selected from the recordings. These points are excluded and, in subsequent iterations, only the remaining points to which additional perturbations can be applied are enhanced.

2. The reversible anti-interference method for camera images according to claim 1, characterized in that: The iterative gradient is feature recalibrated through a channel attention network to generate a channel attention weight matrix, and the weight matrix is ​​subjected to a Hadamard product operation with the iterative gradient to obtain the iterative gradient after channel enhancement, specifically including: Performing a global average pooling operation on the iterative gradient, compressing the features of each channel, and obtaining a feature vector; Use a fully connected layer to transform the feature vector, and then apply the ReLU activation function to the transformed feature vector to introduce nonlinearity; A fully connected layer is then used to restore the feature vector to its shape before the transformation, and then the Sigmoid activation function is applied to limit the weight of each channel to the range of [0,1] to obtain the channel attention weight matrix; Finally, the generated channel attention weight matrix is ​​subjected to a Hadamard product operation with the iterative gradient to obtain the iterative gradient after channel enhancement; Based on the adversarial sample generated in the previous iteration, the iterative gradient is calculated through the proxy model and loss function, including: Input the adversarial sample generated in the previous iteration into the proxy model , get the prediction results ; Wherein, the proxy model is a 2D classification prediction model; The loss function is calculated based on the prediction results; wherein the adversarial sample generated in the previous iteration is input and the true category label The loss value is ; for functions used for dimension expansion and padding, is the adversarial sample obtained in the previous iteration, is the true category label; The iterative gradient is calculated according to the loss function; wherein the calculation model of the iterative gradient is: ;in, For the The gradient of the iteration, represents the gradient, The adversarial sample generated in the previous iteration, for functions used for dimension expansion and padding, is the true category label; The calculation model of the initial gradient is: ; In the formula, is the initial gradient, represents the gradient, For the original image, for functions used for dimension expansion and padding, For the proxy model, is the true category label.

3. The reversible anti-interference method for camera images according to claim 1, characterized in that: The fusion model of the new gradient is: ; In the formula, For the new gradient, is the attenuation factor, is the gradient of the previous iteration, For the Iterative gradient of iteration .

4. The reversible anti-interference method for camera images according to claim 1, characterized in that: According to the new gradient, a new disturbance contribution matrix is ​​constructed, which specifically includes: According to the new gradient, an absolute value matrix is ​​obtained; wherein, ; In the formula, is the absolute value matrix, For the new gradient; According to the absolute value matrix, a new disturbance contribution matrix is ​​constructed; the construction model of the new disturbance contribution matrix is: ; In the formula, For the The new perturbation contribution matrix of the iteration, is the natural exponential function, Indicates Line The absolute value of the gradient of the superpixel in the column, Indicates The absolute value of the gradient of the superpixel, Represents the constraints of the model, Indicates the image height, Indicates the image width, represents the height of the superpixel, Represents the width of the superpixel.

5. The reversible anti-interference method for camera images according to claim 1, characterized in that: Also includes: According to the final perturbation and the final adversarial sample, perturbation decoding is performed to obtain the original image; wherein, superimposing the perturbation on the original image and perturbation decoding on the final adversarial sample are achieved through grayscale invariant reversible data embedding.

6. A reversible anti-interference device for camera images, characterized in that: A reversible anti-interference method for a camera image as claimed in any one of claims 1 to 5; the reversible anti-interference device comprises: An image acquisition module, used for acquiring original images; A superpixel module, used for dividing the original image into multiple superpixel blocks by a superpixel segmentation algorithm; An initial gradient module, used to calculate an initial gradient according to the superpixel block through a proxy model and a loss function; An initial contribution module, used for constructing an initial perturbation contribution matrix according to the initial gradient; An initial disturbance module, used for constructing an initial disturbance according to the initial disturbance contribution matrix; An initial superposition module, used for superimposing the initial perturbation onto the original image to obtain an initial adversarial sample; Iteration module, used to iterate the subsequent steps until the iteration is completed, and obtain the final adversarial sample and the corresponding final perturbation; The iterative gradient module is used to calculate the iterative gradient based on the adversarial samples generated in the previous iteration through the proxy model and loss function; An attention module is used to perform feature recalibration on the iterative gradient through a channel attention network, generate a channel attention weight matrix, and perform a Hadamard product operation on the weight matrix and the iterative gradient to obtain the iterative gradient after channel enhancement; A new map module, used for dynamically fusing the iterative gradient and the gradient of the previous iteration through a decay factor to obtain a new gradient; A new contribution module, used for constructing a new perturbation contribution matrix according to the new gradient; A new disturbance module is used to construct a disturbance increment according to the new disturbance contribution matrix to accumulate disturbances, and to clip the amplitude of the accumulated disturbance to obtain a new disturbance; The iterative superposition module is used to superimpose the new perturbation onto the original image to obtain a new adversarial sample.

7. A reversible anti-interference device for camera images, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory; the computer program can be executed by the processor to implement a reversible anti-interference method for a camera image as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute a reversible anti-interference method for a camera image as described in any one of claims 1 to 5.

Images