Cascade attack resisting method for visual perception of unmanned aerial vehicle in rainy days

Through the cascading adversarial attack method, the adversarial disturbances of the rain removal network and the downstream task network are integrated into the rain diagram, solving the problems of poor attack performance and non-concealment in the existing technology, and realizing the visual perception system of the drone Effective attack.

CN119006309APending Publication Date: 2024-11-22UNIV OF ELECTRONICS SCI & TECH OF CHINA +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202410988931.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Existing adversarial attack methods are difficult to effectively attack integrated systems of rain-draining networks and downstream mission networks in drones, especially in rainy scenarios. Due to the huge differences in the structure, learning objectives and training strategies of the two, the attack performance is poor and Not hidden.

Method used

A cascaded adversarial attack method is proposed, which divides the rain map into rain area and non-rain areas through binary masks, generates adversarial perturbations of rain-retaining attacks and task attacks, and integrates them into the input rain map to generate the final The confrontational sample is used to attack both the rain and downstream mission networks.

Benefits of technology

It is realized that without exceeding the perturbation change limit, it can generate confrontation samples that can effectively affect the entire integrated system, ensure attack performance and follow the principle of concealment, and can have a fatal impact in the visual perception of drones on rainy days.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006309A_ABST
    Figure CN119006309A_ABST
Patent Text Reader

Abstract

The invention discloses a cascade confrontation attack method for visual perception of an unmanned aerial vehicle in rainy days, innovatively provides cascade confrontation attack, and utilizes a binary mask to divide a rain map into a rain area and a non-rain area. One area is used for attacking the rain removal network and aims to retain rainwater in the output image; and the other area is used for attacking the downstream task network and aims to induce misclassification of the image content. The two groups of obviously different and non-overlapping disturbances are concentrated on one input rain map to generate a final adversarial sample, and the downstream task can be seriously influenced even though the adversarial sample is subjected to preprocessing operation of the rain removal network, namely, the adversarial sample capable of attacking the rain removal network and the downstream task network at the same time is generated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of counterattack, and more specifically, relates to a cascade counterattack method for visual perception of unmanned aerial vehicles in rainy days. Background Art

[0002] Drones often perform visual perception tasks in rainy scenes. Due to the destructive effect of rain on the downstream task network, a rain removal network is usually used to pre-process the rain image, and then the visual perception model is used to achieve the expected task objectives. This integrated system consisting of a rain removal network and a downstream task network is widely deployed in drone systems.

[0003] However, this setting, i.e., ensemble system, has never been explored by adversarial attack methods. Existing adversarial attack methods are currently limited to attacking only one type of network mentioned above. The reason is that the two networks from different task levels (low-level and high-level) have huge differences in network structure, learning objectives, data annotations, and training strategies. Therefore, common methods in the adversarial attack field to attack multiple models (such as universal perturbations and ensemble attacks) are ineffective in this case.

[0004] When attacking a single model in the above integrated system in isolation, if the rain removal network is attacked alone, the optimization process of the generated adversarial perturbation lacks key information such as parameters and labels of the downstream task network, making it difficult to generate perturbations that have a fatal impact on the downstream task. On the other hand, adversarial perturbations are noise, and the rain removal network can mitigate their impact through its own recovery ability, that is, attacking only the downstream task network without considering the rain removal network will change the optimized adversarial perturbation distribution, thereby affecting the attack performance. If we attack both models at the same time, the challenge is how to combine these two sets of perturbations on the input rain map, because the input of the integrated system is only a rain map. An intuitive method is to directly add these two sets of perturbations to the input rain map, but this will interfere with their respective optimized perturbation distributions, thereby destroying the performance of each attack. More importantly, this will exceed the limit value of the perturbation change, making the perturbation visible to the naked eye, violating the hidden principle of adversarial attacks, that is, the human eye cannot perceive the existence of perturbations.

[0005] Due to the above difficulties, existing methods have not explored attacks on such integrated systems, and only focus on attacks on specific downstream task networks and rain removal networks. There are already mature and powerful attack algorithms for attacks on downstream task networks such as classification, segmentation, and detection. As for the research between rain removal networks and adversarial attacks, the work of Yu et al. has demonstrated the vulnerability of rain removal networks to adversarial attacks. In addition, they also studied various types of adversarial attacks on rain removal problems and analyzed the impact of these attack methods on human and machine vision tasks. However, these works all focus on a single type and a single network, ignoring the coexistence of rain removal networks and downstream task networks in actual systems, which makes their methods perform poorly in real integrated systems. Therefore, the integrated system setting of drones in real scenarios ensures their normal operation in rainy scenes and avoids malicious attacks from third parties to a certain extent.

[0006] As mentioned above, existing attack methods have the following disadvantages: (1) There is a huge gap between the two networks from different fields and different task levels (high-level and low-level) in terms of architecture, learning objectives, data annotation, and training strategies, which makes common methods for attacking multiple models such as general perturbations and integrated attacks ineffective. (2) When attacking only a single model, the generated adversarial samples usually show very poor attack performance because the differences between models designed for different tasks in the integrated system are not considered. (3) If two models are attacked at the same time, forcing two sets of perturbations to be added to the input image at the same time will change their respective distributions, thereby destroying the effectiveness of their respective attacks, and will exceed the maximum threshold of perturbation change set during the optimization process, violating the principle of concealment. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a cascade adversarial attack method for the visual perception of drones in rainy days, so as to generate adversarial samples that can simultaneously attack the deraining network and the downstream task network.

[0008] To achieve the above-mentioned purpose, the present invention provides a cascade counterattack method for visual perception of drones in rainy days, which is characterized by comprising the following steps:

[0009] (1) Initialization

[0010] Sample the initial rain-retention attack adversarial perturbation from the uniform distribution U(-∈,∈) Among them, ∈ is the perturbation limit;

[0011] (2) If the adversarial attack is a cascaded adversarial attack - from rain to task, then:

[0012] 2.1) First, iterate the number of times t from 1 to T RRA The rain retention attack against the perturbation iteration:

[0013]

[0014]

[0015] in, is the rain-retaining attack adversarial disturbance after t iterations, is the rain attack adversarial disturbance before t iterations, Proj(·) represents Project it into the norm limit (-∈,∈) and put X rain + Project into the valid space (0,1);

[0016]

[0017] is the rain attack disturbance loss About the rain retention attack against disturbances The gradient of , sgn represents the sign of the extracted gradient, α controls the step size of each iteration, and Mask is a binary rain mask, where a value of 1 represents an area with rain and a value of 0 represents an area without rain;

[0018] Among them, the rain attack disturbance loss for:

[0019] The rain retention attack generated by the rain retention attack is used to counter the disturbance Add to Rain Chart X rain In the deraining network f rain Get the output rain map

[0020]

[0021] Then, measure the rain map X rain arrive The distance between them is taken as the perturbation loss

[0022]

[0023] Among them, 0 is a zero matrix, which is related to the rain map have the same size, ||·||2 means finding the bi-norm of the matrix;

[0024] The T RRA The result of the iteration is the rain attack against the disturbance Denoted as the rain-retaining attack against the disturbance δ RRA ;

[0025] 2.2) Then calculate the task attack adversarial disturbance δ TA :

[0026]

[0027] δ TA =clip(δ TA , -∈,∈);

[0028] in, is the cross entropy loss L TA (f task (X clean +δ RRA ),y) about the non-rain image X clean The gradient of task (X clean +δ RRA ) is the rain map X rain Non-rain image x clean Add rain retention attack to counter the disturbance δ RRA Afterwards, through the semantic segmentation network f task The result of the semantic segmentation task after y is used as the label of the semantic segmentation task, L TA (.,.) means calculating the cross entropy loss between the two, and clip is the clipping function;

[0029] (3) If the adversarial attack is a cascaded adversarial attack - from task to rain, then:

[0030] 3.1) First, calculate the task attack adversarial disturbance δ TA :

[0031]

[0032] δ TA =clip(δ TA -∈,∈);

[0033] 3.2) Then the number of iterations t increases from 1 to T RRA Perform rain retention attack against disturbance iteration:

[0034]

[0035]

[0036] Among them, the rain attack disturbance loss for:

[0037] Task attack adversarial perturbation δ TA , the rain-retaining attack adversarial disturbance generated by the rain-retaining attack Add to Rain Chart X rain In the deraining network f rain Get the output rain map

[0038]

[0039] Then, measure the rain map X rain arrive The distance between them is taken as the perturbation loss

[0040]

[0041] Among them, 0 is a zero matrix, which is related to the rain map have the same size, ||·||2 means finding the bi-norm of the matrix;

[0042] The T RRA The result of the iteration is the rain attack against the disturbance Denoted as the rain-retaining attack against the disturbance δ RRA ;

[0043] (4) Output adversarial sample image

[0044]

[0045]

[0046] The object of the present invention is achieved in this way.

[0047] The present invention innovatively proposes a cascade adversarial attack method for the visual perception of drones in rainy days, which divides the rain image into rainy areas and non-rainy areas using a binary mask. One area is used to attack the deraining network, aiming to retain the rain in the output image; the other area is used to attack the downstream task network, aiming to induce misclassification of the image content. By concentrating these two sets of obviously different and non-overlapping perturbations on an input rain image, the final adversarial sample is generated. Even after the preprocessing operation of the deraining network, the downstream task will be seriously affected, that is, an adversarial sample that can simultaneously attack the deraining network and the downstream task network is generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of a specific implementation method of the cascade counterattack method for the visual perception of unmanned aerial vehicles in rainy days of the present invention;

[0049] Figure 2 It is a schematic diagram of the principle of a specific implementation method of the cascade counterattack method for the visual perception of unmanned aerial vehicles in rainy days in the present invention. DETAILED DESCRIPTION

[0050] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0051] The rain image captured by the drone through the visual sensor is first sent to the preprocessing model to obtain the image after rain removal, and then sent to the downstream task network to obtain the final result output. Due to the wide application of semantic segmentation tasks in the fields of computer vision and drones, the present invention selects the semantic segmentation task to represent many downstream tasks, and the present invention also has other downstream tasks. The present invention attacks the integrated system composed of the rain removal network and the semantic segmentation network in the drone. The attack strategy is to add a small amount of visually imperceptible adversarial perturbations to the input rain image to deteriorate the output of the rain removal network and the semantic segmentation network. The ultimate goal is to hinder the correct execution of the semantic segmentation network.

[0052] Specifically, the present invention innovatively proposes a cascade adversarial attack, which uses a binary mask to divide the image into rainy areas and non-rainy areas. One area is used to attack the deraining network, aiming to retain the rain in the output image; the other area is used to attack the semantic segmentation network, aiming to induce misclassification of the image content. By concentrating these two sets of obviously different and non-overlapping perturbations on an input rain map, the final adversarial sample is generated, and even after the preprocessing operation of the deraining network, the downstream tasks will be seriously affected. The following is a detailed description of the present invention.

[0053] 1. Rain Attack

[0054] Use X rain represents the input rain map of the integrated system, δ represents the adversarial perturbation that is imperceptible to human vision, which is limited to l p norm bound ∈, so the adversarial sample It can be expressed as:

[0055]

[0056] ||δ|| p ≤∈ (1)

[0057] In actual scenarios, rain has a significant interference effect on downstream tasks, and its presence will greatly interfere with the correct execution of the downstream task network. Based on this observation, the present invention attacks the rain removal network to destroy its rain removal ability. Specifically, the present invention proposes a method called "Remain Rain Attack" (RRA), which aims to make the rain in the input rain map difficult to detect and remove, so as to retain a large amount of rain to disrupt the normal execution of the semantic segmentation task. The present invention represents the adversarial perturbation generated by the Remain Rain Attack as δ RRA, and add the disturbance to the rain map X rain In the deraining network f rain Get the output This operation can be expressed as:

[0058]

[0059] Then, define the objective function L RRA To measure X rain arrive Distance between:

[0060]

[0061] Here 0 is a zero matrix, and The invention uses l2 Euler distance to measure the distance between input and output, which indicates the degree of rain retention. Here, Mask is a binary rain mask, where the value 1 indicates the area with rain, and the value 0 indicates the area without rain. In the rain retention attack, only the area with rain is concerned, while the area without rain is ignored. Therefore, in L RRA Adding Mask can improve L RRA The goal of this invention is to minimize L RRA , to ensure that rainwater can be retained to the greatest extent, the optimization of the objective function can be formally expressed as the following process:

[0062]

[0063] st||δ RRA || p ≤∈ (4)

[0064] The present invention uses PGD (Projected Gradient Descent) to solve the above optimization problem. In order to obtain the optimal δ RRA , do the following:

[0065]

[0066] here is the perturbation loss about gradient.

[0067] sgn represents the sign of the extracted gradient, α controls the step size of each iteration, and Proj(·) represents the Project it into the norm limit (-∈,∈) and put X rain +δ RRA Projected into the effective space (0,1). The initial is sampled from the uniform distribution U(-∈,∈). RRAAfter iterations, the final adversarial perturbation is obtained And as the final rain retention attack against the disturbance δ RRA .

[0068] 2. Cascade adversarial attacks

[0069] Although the adversarial perturbation generated by the rain retention attack can retain rain and affect the execution of downstream tasks, as mentioned in the previous section, since the downstream task network does not participate in the rain retention attack adversarial perturbation δ RRA The generation of , the rain retention attack can only partially affect the integrated system and cannot pose a fatal threat to the entire UAV mission. In fact, we observe that powerful adversarial attack algorithms tend to generate adversarial perturbations in the main object area of ​​the image. This is because the target object is usually the most important factor affecting the model's decision, and adversarial perturbations in the main area of ​​the object can more effectively destroy the model's prediction. Inspired by this, the present invention uses Mask to constrain the perturbations generated by the rain retention attack, that is, δ RRA This perturbation covering the entire image is multiplied by the Mask, leaving only the perturbation in the rainy area. In each iteration of , the present invention uses the following formula based on formula (5) to obtain

[0070]

[0071] The rain in the rain image only occupies a small part of the rain image, and the rain is evenly distributed in the image. However, most of the main objects in the image are located in the non-rain area. Therefore, for most of the remaining non-rain areas in the input rain image, we use them to attack the downstream task network.

[0072] For the attack on the downstream task network, the existing attack algorithms have shown strong performance. This paper takes the simple and efficient FGSM as an example, and calls the entire attack process on the semantic segmentation network task attack (TA). For the input of the task attack, we represent it as X clean , which is the rain map X rain We assume that y is the label of the semantic segmentation task, and the entire task attack process can be expressed as:

[0073]

[0074] Here L TA represents the cross entropy loss, f task represents the semantic segmentation network, and our goal is to maximize L TA , to maximally interfere with the correct prediction of the downstream task network. Task attack adversarial perturbation δ TA Obtained by the following formula:

[0075]

[0076] ||δ TA || p ≤∈ (8)

[0077] By cutting the rain map, we get two non-overlapping perturbations: δ RRA and δ TA By changing δ RRA and δ TA Added to the input rain image, we seamlessly integrate the adversarial perturbations generated by two models with significant differences into a single image, which we call Cascaded Adversarial Attack (CAA), which enables us to attack two very different models simultaneously without exceeding the constraint limit ∈. Therefore, the final adversarial sample It can be expressed as:

[0078]

[0079] Due to the significant differences in the structures and parameters of the two models, as well as the different optimization goals of the rain retention attack and the task attack, the generated adversarial perturbation δ RRA and δ TA The difference is huge. The core reason is that the two use different optimization formulas to optimize the perturbation on different networks without any interaction. Therefore, in order to get closer to δ RRA and δ TA To improve the perturbation distribution distance so that it can be integrated more smoothly on the input rain map, we design two variants of cascaded adversarial attacks, called Cascaded Adversarial Attack from Rain to Task (CAA-RT) and Cascaded Adversarial Attack from Task to Rain (CAA-TR). They improve the attack performance by introducing a carefully designed perturbation interaction mechanism to minimize the difference between the two generated adversarial perturbations. Specifically, CAA-RT first performs a rain retention attack, which transforms the perturbation δ RRA Added to the mission attack input X clean Then execute the task attack to get δ TA Finally, we use formula (9) to get the adversarial sample In contrast, in CAA-TR, the task attack is first applied and the generated perturbation δ TA Add to X rain Then, we perform a rain attack to obtain δ RRASimilar to CAA-RT, the final adversarial sample is obtained by formula (9). RRA and δ in CAA-RT TA The generation formula is as follows:

[0080]

[0081]

[0082] Figure 1 , 2 They are respectively a flow chart and a principle schematic diagram of a specific implementation method of the cascade counterattack method for visual perception of unmanned aerial vehicles in rainy days of the present invention.

[0083] In this embodiment, if Figure 1 , 2 As shown, the cascade counterattack method for the visual perception of unmanned aerial vehicles in rainy days of the present invention comprises the following steps:

[0084] Step S1: Initialization

[0085] Sample the initial rain-retention attack adversarial perturbation from the uniform distribution U(-∈,∈) Among them, ∈ is the perturbation limit.

[0086] Step S2: If the adversarial attack is Cascaded Adversarial Attack - From Rain to Task (CAA-RT), then:

[0087] Step S2.1: Perform rain-retaining attack adversarial perturbation iteration to obtain rain-retaining attack adversarial perturbation δ RRA :

[0088] First, the number of iterations t is from 1 to T RRA The rain retention attack against the perturbation iteration:

[0089]

[0090]

[0091] in, is the rain-retaining attack adversarial disturbance after t iterations, is the rain attack adversarial disturbance before t iterations, Proj(·) represents Project it into the norm limit (-∈,∈) and put X rain + Project into the valid space (0,1);

[0092]

[0093] is the rain attack disturbance loss About the rain retention attack against disturbances The gradient of , sgn represents the sign of the extracted gradient, α controls the step size of each iteration, and Mask is a binary rain mask, where a value of 1 represents an area with rain and a value of 0 represents an area without rain;

[0094] Among them, the rain attack disturbance loss for:

[0095] The rain retention attack generated by the rain retention attack is used to counter the disturbance Add to Rain Chart X rain In the deraining network f rain Get the output rain map

[0096]

[0097] Then, measure the rain map X rain arrive The distance between them is taken as the perturbation loss

[0098]

[0099] Among them, 0 is a zero matrix, which is related to the rain map have the same size, ||·||2 means finding the bi-norm of the matrix;

[0100] The T RRA The result of the iteration is the rain attack against the disturbance Denoted as the rain-retaining attack against the disturbance δ RRA .

[0101] Step S2.2: Then calculate the task attack adversarial perturbation δ TA :

[0102]

[0103] δ TA =clip(δ TA , -∈,∈);

[0104] in, is the cross entropy loss L TA (f task (X clean +δ RRA ),y) about the non-rain image X clean The gradient of task (X clean +δ RRA ) is the rain map X eain Non-rain image X clean Add rain retention attack to counter the disturbance δ RRA Afterwards, through the semantic segmentation network ftask The result of the semantic segmentation task after y is used as the label of the semantic segmentation task, L TA (.,.) means calculating the cross entropy loss between the two, and clip is the clipping function.

[0105] Step S3: If the adversarial attack is Cascaded Adversarial Attack - Task to Rain (CAA-TR), then:

[0106] Step S3.1: First calculate the task attack adversarial perturbation δ TA :

[0107]

[0108] δ TA =clip(δ TA -∈,∈);

[0109] Step S3.2: Then the number of iterations t increases from 1 to T RRA Perform the rain retention attack counter-perturbation iteration to obtain the rain retention attack counter-perturbation δ RRA :

[0110]

[0111]

[0112] Among them, the rain attack disturbance loss for:

[0113] Task attack adversarial perturbation δ TA , the rain-retaining attack adversarial disturbance generated by the rain-retaining attack Add to Rain Chart X rain In the deraining network f rain Get the output rain map

[0114]

[0115] Then, measure the rain map X rain arrive The distance between them is taken as the perturbation loss

[0116]

[0117] Among them, 0 is a zero matrix, which is related to the rain map have the same size, ||·||2 means finding the bi-norm of the matrix;

[0118] The T RRA The result of the iteration is the rain attack against the disturbance Denoted as the rain-retaining attack against the disturbance δ RRA ; Step S4: Output adversarial sample image

[0119]

[0120]

[0121] Experimental verification

[0122] In order to ensure the versatility and practicality of the present invention and to suit the characteristics of drones, in this experimental verification, an integrated system is formed by using MPRNet (rain removal) which is widely used in real-world applications and PIDNet (semantic segmentation) which is famous for its real-time semantic segmentation capabilities, and the present invention is tested on the Cityscapes and RainCityscapes datasets. Since the present invention is the first to propose a method for simultaneously attacking two models with huge differences, no other method can be compared with it. Therefore, in this experimental verification, it can only be compared with the attack method that only attacks a single network. In Yu, Y., Yang, W., Tan, YP, & Kot, AC (2022). Towards robust rain removal against adversarial attacks: A comprehensive benchmark analysis and beyond. In Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition (pp. 6013-6022), several methods for attacking the rain removal network are proposed, and it is claimed that they can affect the downstream task network. For attacks on segmented networks, the task attack (TA) we implemented is a natural powerful attack algorithm. The comparison results are shown in Table 1:

[0123]

[0124] Table 1

[0125] Table 1 shows the results of various attack strategies. The best results are in bold.

[0126] In this experimental verification, the mean intersection over union (mIoU) is used as an evaluation metric to evaluate the performance of semantic segmentation. The smaller the mIoU value, the better the attack performance, and the more it can interfere with the execution of the segmentation task. We use the widely used quality indicators Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) to measure the concealment of the perturbation (the values ​​of PSNR and SSIM are calculated by comparing the adversarial sample with the clean rain image). To ensure the imperceptibility of the human eye, we set ||δ|| ∞ ≤∈, and ∈ is set to 4 / 255, α=∈ / 4. For the generation of binary mask, we generate it by subtracting the rain-free image from the input rain image. By setting a certain pixel threshold, we can easily generate a binary mask.

[0127] As can be seen from Table 1, attacking a single network alone is not enough to pose a serious threat to the entire integrated system. Our method has the best attack performance, which shows that it is necessary to attack both networks at the same time. This attack strategy can significantly reduce the performance of the final task. At the same time, it can be seen from the PSNR and SSIM values ​​that the changes in the image are almost imperceptible to human vision. Following the principle of concealment of adversarial attacks, it can also ensure that the input image of the drone is not detected after replacement.

[0128] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A cascade adversarial attack method for visual perception of unmanned aerial vehicles in rainy weather, characterized in that: The following steps are involved: (1) Initialization Sample the initial rain-retention attack adversarial perturbation from the uniform distribution U(-∈,∈) Among them, ∈ is the perturbation limit; (2) If the adversarial attack is a cascaded adversarial attack - from rain to task, then: 2.1) First, iterate the number of times t from 1 to T RRA The rain retention attack against the perturbation iteration: in, is the rain-retaining attack adversarial disturbance after t iterations, is the rain attack adversarial disturbance before t iterations, Proj(·) represents Projected into the norm limit (-∈,∈), and Project into the valid space (0,1); is the rain attack disturbance loss About the rain retention attack against disturbances The gradient of , sgn represents the sign of the extracted gradient, α controls the step size of each iteration, and Mask is a binary rain mask, where a value of 1 represents an area with rain and a value of 0 represents an area without rain; Among them, the rain attack disturbance loss for: The rain retention attack generated by the rain retention attack is used to counter the disturbance Add to Rain Chart X rain In the deraining network f rain Get the output rain map Then, measure the rain map X rain arrive The distance between them is taken as the perturbation loss Among them, 0 is a zero matrix, which is related to the rain map have the same size, ||·||2 means finding the bi-norm of the matrix; The T RRA The result of the iteration is the rain attack against the disturbance Denoted as the rain-retaining attack against the disturbance δ RRA ; 2.2) Then calculate the task attack adversarial disturbance δ TA : d TA =clip(δ TA ,-∈,∈); in, is the cross entropy loss L TA (f task (X clean +δ RRA ),y) about the non-rain image X clean The gradient of task (X clean +δ RRA ) is the rain map X rain Non-rain image X clean Add rain retention attack to counter the disturbance δ RRA Afterwards, through the semantic segmentation network f task The result of the semantic segmentation task after y is used as the label of the semantic segmentation task, L TA (.,.) means calculating the cross entropy loss between the two, and clip is the clipping function; (3) If the adversarial attack is a cascaded adversarial attack - from task to rain, then: 3.1) First, calculate the task attack adversarial disturbance δ TA : d TA =clip(δ TA -∈,∈); 3.2) Then the number of iterations t increases from 1 to T RRA Perform rain retention attack against disturbance iteration: Among them, the rain attack disturbance loss for: Task attack adversarial perturbation δ TA , the rain-retaining attack adversarial disturbance generated by the rain-retaining attack Add to Rain Chart X rain In the deraining network f rain Get the output rain map Then, measure the rain map X rain arrive The distance between them is taken as the perturbation loss Among them, 0 is a zero matrix, which is related to the rain map have the same size, ||·||2 means finding the bi-norm of the matrix; The T RRA The result of the iteration is the rain attack against the disturbance Denoted as the rain-retaining attack against the disturbance δ RRA ; (4) Output adversarial sample image

Citation Information

Patent Citations

  • Inhibitors of protein tyrosine phosphatase

    WO2000053583A1