Certifying Robustness Bounds for Convolutional Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for certifying the adversarial robustness of convolutional neural networks (CNNs) face scalability issues and lack generality in supporting various architectures and activation functions, making them inefficient for moderate-sized networks and untested attacks.
Innovation Solution
A computer-implemented certification method that derives efficient upper and lower bounds on activation functions and applies these bounds to compute certified robustness, enabling robustness certification for a broad range of CNN architectures, including convolutional, max-pooling, and residual blocks, using a polynomial-time algorithm.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional verification-based approaches are used to provide robustness certification, then security guarantees are improved, but computational complexity increases making it inapplicable to moderate-sized networks
Solution Approach 1:
The patent segments the verification process into two distinct phases: (1) an efficient optimization phase that provides a lower bound on robustness using gradient-based methods, and (2) a certification phase that verifies the lower bound using linear programming. This segmentation allows the computationally intensive optimization to be performed once, while the certification can be efficiently repeated, resolving the contradiction between providing reliable certification and managing computational complexity
Solution Approach 2:
The patent performs preliminary optimization to compute a lower bound on the robustness before performing the actual certification. By pre-computing this lower bound through efficient gradient-based optimization, the subsequent certification process only needs to verify this bound using tractable linear programming, rather than solving the full NP-complete verification problem from scratch, thus reducing the computational complexity while maintaining certification reliability
2Reliability
If conventional techniques convert convolutional layers to fully-connected layers for verification, then robustness certification is achieved, but computational efficiency deteriorates
Solution Approach 1:
The patent introduces linear programming as an intermediary verification method that operates directly on the convolutional layer structure without requiring conversion to fully-connected layers. This linear programming formulation serves as a mediator that can certify robustness while preserving the computational efficiency of convolutional operations, avoiding the exponential blowup in parameters that occurs with full conversion to fully-connected layers
3Adaptability or versatility
If conventional techniques are used to support various network architectures, then generality is improved, but computational scalability worsens
Solution Approach 1:
The patent develops a universal verification framework based on linear programming that can handle multiple network architectures (convolutional layers, fully-connected layers, batch normalization, pooling, and various activation functions) through a single unified formulation. This universal approach avoids the need for architecture-specific optimizations and maintains computational scalability by using efficient linear programming solvers that work consistently across different network types
Data Source
AI summary
A certification method, system, and computer program product include certifying an adversarial robustness of a convolutional neural network by deriving an analytic solution for a neural network output using an efficient upper bound and an efficient lower bound on an activation function and applying the analytic solution in computing a certified robustness.