Selective Dropout Layer for Neural Network Adversarial Robustness

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Deep neural networks (DNNs) are susceptible to adversarial attacks, leading to inaccurate predictions and classifications due to noisy input, which affects their robustness in image understanding tasks.

Innovation Solution

A selective dropout layer in the neural network is introduced, where nodes are deactivated based on a comparison of adversarial image features with natural image features, using a sensitivity threshold, and the weights are updated via backpropagation to enhance robustness against such attacks.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional DNNs process all input features, then they maintain full computational capability, but they become susceptible to adversarial attacks causing inaccurate predictions

Engineering Contradiction:

Improverobustness against adversarial attacksVSAvoidvulnerability to noisy input

Core Design Contradiction:

ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The patent extracts and removes harmful adversarial features from the input by comparing adversarial image features with natural image features. The selective dropout layer identifies and deactivates nodes responding to adversarial patterns, effectively separating harmful signals from useful information to improve robustness against adversarial attacks

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies different processing treatments to different regions of the feature space. By selectively activating or deactivating specific nodes in the neural network based on their response to adversarial versus natural features, the system creates local quality variations that enhance robustness in vulnerable regions while preserving functionality in reliable regions

Inventive Principle:

Principle #3Local quality

2Reliability

If nodes are deactivated to filter adversarial features, then robustness improves, but computational complexity and network capacity decrease

Engineering Contradiction:

Improverobustness against adversarial attacksVSAvoidnetwork structure complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent performs preliminary classification of features as adversarial or natural before the main processing occurs. The selective dropout layer pre-identifies harmful features through comparison with natural image features, allowing the network to filter them out early in the processing pipeline before they can significantly impact the final prediction

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary selective dropout layer that acts as a mediator between the input features and the main neural network processing. This intermediary component compares adversarial features with natural features and selectively passes only beneficial features to subsequent layers, reducing the burden on the main network while improving robustness

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20230162039A1Selective dropout of features for adversarial robustness of neural network

Publication Date: 2023.05.25 GM GLOBAL TECHNOLOGY OPERATIONS LLC

AI summary

A system comprises a computer including a processor and a memory. The memory includes instructions such that the processor is programmed to: receive, at a selective dropout layer of a neural network, a plurality of adversarial image features and a plurality of natural image features, select one or more nodes within the selective dropout layer to deactivate based on a comparison of the plurality of adversarial image features with the plurality of natural image features, and deactivate the selected one or more nodes.