A cross-task adversarial attack method, system, and storage medium for autonomous driving scenarios.

Abstract

This invention provides a cross-task adversarial attack method, system, and storage medium for autonomous driving scenarios. The method includes: Step S1, data preprocessing: performing image standardization and resolution compatibility processing on the collected raw image data; Step S2: optimizing adversarial examples based on the key features of different perception task models, including the following processes: feature extraction, feature destruction loss, feature sharing fusion, and feature-level perturbation optimization. Through multiple iterative optimizations, adversarial examples with deep feature destruction for multiple perception tasks are finally obtained; Step S3: further optimizing and generating the final adversarial examples based on the feature-level perturbations generated in Step S2 by constructing adaptive task weights. The beneficial effect of this invention is that it aims to propose a cross-task adversarial attack for complex systems with multiple perception models cooperating, enabling different perception modules of the system to simultaneously identify errors.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a cross-task adversarial attack method, system, and storage medium for autonomous driving scenarios. Background Technology

[0002] The perception layer of an autonomous driving system typically consists of multiple task modules working collaboratively, such as object detection, semantic segmentation, and depth estimation, to construct an accurate perception of the surrounding environment. Specifically, the object detection module is responsible for identifying and locating key objects such as vehicles, pedestrians, and traffic signs, outputting their category and bounding box information. The depth estimation module, based on monocular / binocular vision or LiDAR point cloud data, calculates the precise distances between the vehicle and other traffic participants, providing spatial information for subsequent obstacle avoidance and path planning. The semantic segmentation module performs pixel-level classification of scene images, dividing them into lane lines, drivable areas, curbs, and other scene elements to ensure the vehicle maintains the correct driving trajectory in complex environments. These modules do not operate independently but complement and enhance each other through multi-layered information fusion: the object location information from object detection can guide semantic segmentation to optimize boundary accuracy; for example, pixels within a vehicle detection bounding box are more likely to be classified as "car." The distance data provided by depth estimation can help object detection filter out unreasonable detection results (such as the impossibility of an oversized pedestrian in the distance). The drivable area information output by semantic segmentation can assist the trajectory planning module in eliminating obstacle areas, ensuring a safe path.

[0003] However, most existing adversarial attack methods only target a single perception task (such as deceiving an object detection model to miss a vehicle), and are difficult to disrupt the collaborative perception of multiple modules simultaneously. Because multi-task systems have cross-validation and error correction capabilities, even if an attacker successfully interferes with one module (such as causing the object detector to miss a vehicle ahead), other modules (such as depth estimation or semantic segmentation) may still detect anomalies (such as the presence of obstacles or abrupt depth changes in the area), thereby triggering system alarms or activating redundant perception strategies, causing the attack to fail.

[0004] Gao et al. used feature map data from the intermediate layers of the classifier to construct a loss function and generate targeted adversarial examples. Chang et al. used feature maps from multiple intermediate layers of a specified deep learning network and divided all feature maps into positive and negative feature regions based on the activation values ​​and gradients of the intermediate layers. They then used a weighted sum of different regions from all specified intermediate layers as the loss function for adversarial optimization. Wei et al. optimized adversarial examples by calculating the average gradient of model prediction results and intermediate layer features in image classification tasks, improving the transferability of adversarial examples across different classification models. Liu et al., in a single classification task model, directly transferred the perturbation space of adversarial perturbation from the pixel space to the feature space, improving the efficiency of destroying important features. They then performed feature-to-image inversion to obtain stronger transferability. Zhang et al. used a general patch to simultaneously mislead multiple perception task models within the autonomous driving perception module, resulting in a higher probability of system-wide failures, rather than targeting only one perception task within the autonomous driving perception module. They achieved this goal by treating the attention of the perception task as a higher-level feature representation instead of a traditional gradient-based attack. The target perception models they attacked mainly included obstacle detection, traffic sign detection, and road line detection, but they did not study the impact on semantic segmentation and depth estimation.

[0005] The main shortcomings of existing technology:

[0006] 1. Existing technologies mainly focus on adversarial attacks on single-task models, with limited research on cross-task attacks;

[0007] 2. Existing related studies have not adequately considered the impact on semantic segmentation and depth estimation. These studies mainly cover obstacle detection, traffic sign detection, and road line detection, but have not studied the impact on semantic segmentation and depth estimation.

[0008] 3. Existing research techniques use attention heatmaps to find perturbation regions, but do not combine key features of images on different tasks with the vulnerability of those tasks. Summary of the Invention

[0009] To address the problems in existing technologies, this invention provides a cross-task adversarial attack method for autonomous driving scenarios, comprising the following steps:

[0010] Step S1, Data Preprocessing: Perform image standardization and resolution compatibility processing on the collected raw image data;

[0011] Step S2, Key Feature Destruction for Multi-Sensory Tasks: Based on the key features of different perception task models, adversarial examples are optimized, including the following processes: feature extraction, feature destruction loss, feature sharing fusion, and feature-level perturbation optimization. Through multiple iterative optimizations, adversarial examples with deep feature destruction for multi-sensory tasks are finally obtained. ;

[0012] Step S3, Task-level perturbation of multi-sensory tasks: Based on the feature-level perturbation generated in step S2, further optimization is performed and the final adversarial sample is generated by constructing an adaptive task weight method.

[0013] As a further improvement of the present invention, step S1 further includes:

[0014] Step S101, Image Standardization: Standardize the original image data. Perform pixel value normalization processing. , Represent real numbers, H Indicates altitude, W Indicates width, C Indicates a channel;

[0015] Step S102, Resolution Compatibility: If the model requires different resolutions, use high-quality interpolation upsampling / downsampling, record the scaling factor, and ensure that the perturbation is applied to the original resolution image during the final evaluation.

[0016] As a further improvement of the present invention, step S2 further includes:

[0017] Step S201, Feature Extraction: Assume there exists Different perception task models , , It is the first The parameters of a perception task model will be the original image data. Input different perception task models respectively Obtain the latent key feature layer outputs of multiple different perception task models. ,in For the model The key feature layers are different for each model of a different perception task;

[0018] Step S202, Feature Destruction Loss: Construct the key feature destruction loss. , ,in Indicates adversarial examples, Indicates resistance to disturbance;

[0019] Step S203, Feature Sharing and Fusion: Optimize the loss of key features from multiple perceptual task models, whereby... As the initial task weights, The losses incurred in combating this situation are as follows:

[0020] ,

[0021] Step S204, Feature-level adversarial perturbation optimization: Perturbation optimization is performed using a multi-step iterative PGD method.

[0022] ,

[0023] in Indicates the first t +1 step adversarial example, This indicates a projection operation. Indicates the first t Adversarial examples of steps, The iteration step size, Indicated by x Centered on radius ball, The infinity norm is used to obtain adversarial examples of deep feature destruction in multi-sensory tasks through multiple iterations of optimization. .

[0024] As a further improvement of the present invention, step S3 further includes:

[0025] Step S301, Task-Specific Transformation: Adversarial Examples that Destroy Deep Features Perform specific image transformations based on different tasks to construct images with target adversarial semantics. ;

[0026] Step S302, Task adversarial loss: Convert the original image... and task conversion image Simultaneously input into the corresponding perception task model In this process, a task is constructed using the two outputs of each task model. Mission combat losses ;

[0027] Step S303, Dynamic weight update: Set different task weights for different tasks;

[0028] Step S304, Adversarial Example Iterative Update: Based on the adversarial loss of each task in step S302 and adaptive task weights To obtain the total losses of the entire attack mission for

[0029] ,

[0030] The PGD method is used for iterative updates, and its formula is as follows:

[0031] ,

[0032] in This indicates a projection operation. The iteration step size, Indicated by x Centered on radius The ball, through multiple iterations, generates the final adversarial examples.

[0033] As a further improvement of the present invention, step S301 further includes a target detection task step and / or a traffic sign detection task and / or a lane line detection task step, wherein:

[0034] The steps of the object detection task are as follows: obtain the bounding box of each object in the image based on the original label data or a model with high detection accuracy, perform grayscale masking on each bounding box, and use a pre-trained diffusion model to repair the image;

[0035] Traffic sign detection task steps: Perform grayscale masking on the traffic sign detection object, and repair it using a diffusion model;

[0036] Lane line detection task steps: perform grayscale masking and repair on the lane lines in the scene.

[0037] As a further improvement of the present invention, step S303 further includes:

[0038] Step S3031: Assess the vulnerability of different task models;

[0039] Step S3032: Set the gradient coordination factor, confirm the dominant attack direction, and then calculate the cosine similarity between the gradient of each task and the dominant gradient direction to obtain the gradient equalization factor.

[0040] Step S3033: Combine task vulnerability and coordination factor to form the final adaptive task weight.

[0041] ,

[0042] in, and These are hyperparameters used to control the relative importance of vulnerability and coordination factors. The softmax function ensures that the sum of the weights is 1.

[0043] As a further improvement of the present invention, in step S3031, the vulnerability of the task is assessed by calculating the percentage decrease in loss at the current iteration step;

[0044] In step S3032, the dominant attack direction is the average gradient direction, weighted aggregation direction, or maximum loss direction of each task.

[0045] The present invention also discloses a cross-task adversarial attack system for autonomous driving scenarios, comprising: a memory, a processor, and a computer program stored in the memory, wherein the computer program is configured to implement the steps of the method described in the present invention when invoked by the processor.

[0046] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the method described in the present invention when invoked by a processor.

[0047] The beneficial effects of this invention are: 1. Enhanced cross-task adaptability of adversarial attack research: Addressing the problem that existing technologies are mostly limited to single-task models, this invention explores cross-task adversarial attack methods, enabling a more comprehensive evaluation of the robustness of multi-task learning systems and providing a more effective verification method for the security of complex multi-task models in real-world scenarios; 2. Expanded impact analysis on key multi-task perception functions: Compared to existing research that only focuses on limited tasks such as obstacle detection and traffic sign detection, this invention further studies the impact of adversarial attacks on key visual tasks such as semantic segmentation and depth estimation, thereby more completely revealing the comprehensive harm of attacks in multi-task environments and improving the overall security assessment of the system; 3. Enhanced the targeting and effectiveness of adversarial perturbations: By combining key features of images in different tasks with the inherent vulnerabilities of the tasks themselves for attack design, this invention overcomes the limitations of previous methods that relied solely on attention heatmaps to determine perturbation regions, generating more task-relevant and destructive adversarial samples, thereby improving the accuracy of attacks and the realism of actual threat simulations. Attached Figure Description

[0048] Figure 1 This is a cross-task attack framework diagram of the present invention. Detailed Implementation

[0049] like Figure 1 As shown, this invention discloses a cross-task adversarial attack method for autonomous driving scenarios, including the following steps:

[0050] Step S1, Data Preprocessing: The acquired raw image data undergoes image standardization and resolution compatibility processing, specifically including:

[0051] Step S101, Image Standardization: Standardize the original image data. , (Height × Width × Channel, usually) ) Perform pixel value normalization (e.g., mean=[0.485, 0.456, 0.406], std=[0.229, 0.224,0.225]). Represents a real number.

[0052] Step S102, Resolution Compatibility: If the model requires a different resolution, use high-quality interpolation (e.g., bilinear / bicubic) for upsampling / downsampling. Record the scaling factor to ensure that the perturbation is applied to the original resolution image during the final evaluation.

[0053] Step S2, key feature destruction in multi-sensory tasks:

[0054] Adversarial example optimization is performed based on the key features of different perception task models. Specifically, it includes four parts: feature extraction, feature destruction loss, feature sharing fusion, and feature-level perturbation optimization.

[0055] Step S201, Feature Extraction: Assume that there exists Different perception task models , , It is the first The parameters of a perception task model. First, the raw image data... Input different perception task models To obtain the latent key feature layer outputs of its different perception task models ,in For the model The key feature layers are different for each model of a different perception task.

[0056] Step S202, Feature Destruction Loss: Construct the key feature destruction loss. , ,in Indicates adversarial examples, It indicates resistance to disturbances.

[0057] Step S203, Feature Sharing and Fusion: Optimize the loss of key features from multiple perceptual models, whereby... As the initial task weights, The losses incurred in combating this situation are as follows:

[0058] ,

[0059] Step S204: Feature-level adversarial perturbation optimization: Perturbation optimization is performed using a multi-step iterative PGD (projected gradient descent) method.

[0060] ,

[0061] This involves multi-step iterative optimization. Let represent the adversarial example at step t+1, where This indicates a projection operation. The iteration step size, Indicates that x is the center, radius ball, The infinity norm is used. Through multiple iterative optimizations, adversarial examples of deep feature destruction in multi-sensory tasks are finally obtained. .

[0062] Step S3, Task-level perturbation for multi-sensory tasks: The deep feature-level adversarial examples generated in step S204 may have little impact on some edge-sensory tasks. The main purpose of this step is to enhance the adversarial nature of the adversarial examples by constructing an adaptive task weight method to further optimize the adversarial examples based on the feature-level perturbations generated in the second step.

[0063] Step S301, Specific task conversion:

[0064] Deep feature-level adversarial examples Perform specific image transformations based on different tasks to construct an image with target adversarial semantics. For example: (1) The main task of object detection is to detect all target objects in the surrounding environment, locate and identify them. For this task, we first obtain the bounding box of each object in the image based on the original label data or a model with high detection accuracy, and then perform grayscale masking on each bounding box. Finally, we use a pre-trained diffusion model to repair the image; (2) The task of traffic sign detection is to identify and locate traffic lights, signs, etc. in the surrounding environment. For this task, we only perform grayscale masking on these objects, and then use a diffusion model to repair them; (3) The task of lane line detection is mainly to detect road dividing lines. For this task, we only perform grayscale masking on the lane lines in the scene and then repair them; other tasks are processed similarly to the above.

[0065] Step S302, Mission Countermeasure Loss:

[0066] Original image and task conversion image Simultaneously input into the corresponding perception task model In this process, a task is constructed using the two outputs of each task model. Mission combat losses .

[0067] Step S303, Dynamic Weight Update:

[0068] Because different tasks exhibit varying adversarial robustness, some tasks are easily affected by adversarial perturbations, while others require stronger perturbations to achieve the desired attack effect. Therefore, different task weights need to be assigned to different tasks. Specifically, we first assess the vulnerability of different task models by calculating the percentage decrease in loss at the current iteration step. Then, we set a gradient coordination factor to identify the dominant attack direction (which can be the average gradient direction of all tasks, the weighted aggregation direction, or the direction of maximum loss. In this case, we use the weighted gradient direction of the least vulnerable task in the Top-k list as the dominant attack direction). We calculate the cosine similarity between the gradient of each task and the dominant gradient direction to obtain the gradient balancing factor. Finally, we combine the task vulnerability and the coordination factor to form the final adaptive task weights.

[0069] ,

[0070] in, and These are hyperparameters that control the relative importance of fragility and coordination factors. Typically... This is because pushing the most difficult task is usually the primary goal. Using the softmax function ensures that the weights sum to 1, which conforms to the probability distribution property and is beneficial for optimizing stability.

[0071] Step S304: Adversarial Example Iterative Update

[0072] Based on the adversarial losses of each task in step S302 and adaptive task weights The total loss of the entire attack mission for

[0073] ,

[0074] Then, the PGD method is used for iterative updates.

[0075] ,

[0076] in This indicates a projection operation. The iteration step size, Indicated by x Centered on radius The ball. Through multiple iterations, the final adversarial examples are generated.

[0077] The method proposed in this invention aims to provide a cross-task adversarial attack for complex systems with multiple perception models working together, enabling different perception modules of the system to simultaneously identify errors.

[0078] The present invention also discloses a cross-task adversarial attack system for autonomous driving scenarios, comprising: a memory, a processor, and a computer program stored in the memory, wherein the computer program is configured to implement the steps of the method described in the present invention when invoked by the processor.

[0079] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the method described in the present invention when invoked by a processor.

[0080] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A cross-task adversarial attack method for autonomous driving scenarios, characterized in that, Includes the following steps: Step S1, Data Preprocessing: Perform image standardization and resolution compatibility processing on the collected raw image data; Step S2, Key Feature Destruction for Multi-Sensory Tasks: Based on the key features of different perception task models, adversarial examples are optimized, including the following processes: feature extraction, feature destruction loss, feature sharing fusion, and feature-level perturbation optimization. Through multiple iterative optimizations, adversarial examples with deep feature destruction for multi-sensory tasks are finally obtained. ; Step S3, Task-level perturbation of multi-sensory tasks: Based on the feature-level perturbation generated in step S2, further optimization and generation of final adversarial examples are achieved by constructing adaptive task weights. Step S2 further includes: Step S201, Feature Extraction: Assume there exists Different perception task models , It is the first The parameters of a perception task model will be the original image data. Input different perception task models respectively Obtain the latent key feature layer outputs of multiple different perception task models. ,in For the model The key feature layers are different for each model of a different perception task; Step S202, Feature Destruction Loss: Construct the key feature destruction loss. ,in Indicates adversarial examples, Indicates resistance to disturbance; Step S203, Feature Sharing and Fusion: Optimize the loss of key features from multiple perceptual task models, whereby... As the initial task weights, The losses incurred in combating this situation are as follows: ； Step S204, Feature-level adversarial perturbation optimization: Perturbation optimization is performed using a multi-step iterative PGD method. ， in Indicates the first t +1 step adversarial example, This indicates a projection operation. Indicates the first t Adversarial examples of steps, The iteration step size, Indicated by x Centered on radius ball, The infinity norm is used to obtain adversarial examples of deep feature destruction in multi-sensory tasks through multiple iterations of optimization. ; Step S3 further includes: Step S301, Task-Specific Transformation: Adversarial Examples that Destroy Deep Features Perform specific image transformations based on different tasks to construct images with target adversarial semantics. ; Step S302, Task adversarial loss: Convert the original image... and task conversion image Simultaneously input into the corresponding perception task model In this process, a task is constructed using the two outputs of each task model. The mission is to combat losses; Step S303, Dynamic weight update: Set different task weights for different tasks; Step S304, Adversarial Example Iterative Update: Based on the adversarial loss of each task in step S302 and adaptive task weights To obtain the total losses of the entire attack mission for: ， The PGD method is used for iterative updates, and its formula is as follows: ， in This indicates a projection operation. The iteration step size, Indicated by x Centered on radius The ball, through multiple iterations, generates the final adversarial examples.

2. The cross-task adversarial attack method according to claim 1, characterized in that, Step S1 further includes: Step S101, Image Standardization: Standardize the original image data. Perform pixel value normalization processing. , Represent real numbers, H Indicates altitude, W Indicates width, C Indicates a channel; Step S102, Resolution Compatibility: If the model requires different resolutions, use interpolation upsampling / downsampling, record the scaling factor, and ensure that the perturbation is applied to the original resolution image during the final evaluation.

3. The cross-task adversarial attack method according to claim 1, characterized in that, Step S301 further includes a target detection task step, a traffic sign detection task step, or a lane line detection task step, wherein: The steps of the object detection task are as follows: obtain the bounding box of each object in the image based on the original label data or a model with high detection accuracy, perform grayscale masking on each bounding box, and use a pre-trained diffusion model to repair the image; Traffic sign detection task steps: Perform grayscale masking on the traffic sign detection object, and repair it using a diffusion model; Lane line detection task steps: perform grayscale masking and repair on the lane lines in the scene.

4. The cross-task adversarial attack method according to claim 1, characterized in that, Step S303 further includes: Step S3031: Assess the vulnerability of different task models; Step S3032: Set the gradient coordination factor, confirm the dominant attack direction, and then calculate the cosine similarity between the gradient of each task and the dominant gradient direction to obtain the gradient equalization factor. Step S3033: Combine task vulnerability and coordination factor to form the final adaptive task weights: ， in, and These are hyperparameters used to control the relative importance of vulnerability and coordination factors. The softmax function ensures that the sum of the weights is 1.

5. The cross-task adversarial attack method according to claim 4, characterized in that, In step S3031, the vulnerability of the task is assessed by calculating the percentage decrease in loss at the current iteration step; In step S3032, the dominant attack direction is the average gradient direction, weighted aggregation direction, or maximum loss direction of each task.

6. A cross-task adversarial attack system for autonomous driving scenarios, characterized in that, include: A memory, a processor, and a computer program stored on the memory, the computer program being configured to implement the steps of the method of any one of claims 1-5 when invoked by the processor.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to implement the steps of the method according to any one of claims 1-5 when invoked by a processor.

Images