Neural Network Training via Local Linearity Regularization

The patent extracts only the essential property needed for adversarial robustness - the non-linearity of the loss function with respect to perturbations - and regularizes specifically this aspect. Instead of full adversarial training which requires generating and training against complex adversarial examples, the method isolates and regularizes the critical non-linear component, significantly reducing computational requirements while maintaining robustness.

The patent changes the parameter being regularized from the full loss function or adversarial loss to specifically the non-linearity measure of the loss function with respect to input perturbations. By parameterizing the regularization target as the non-linear component (using second-order derivatives or curvature measurements), the method achieves robustness with reduced computational overhead compared to conventional approaches.

The patent extracts the critical vulnerability factor - non-linearity of the loss function - and focuses regularization on this specific aspect. By taking out only the non-linear component that causes susceptibility to adversarial attacks and regularizing it, the method avoids the complexity of considering all possible perturbations while still achieving robustness against the most dangerous attack vectors.

Instead of training against all possible perturbations (excessive action), the patent applies partial regularization only to the non-linear component of the loss function. This partial action targeting the specific non-linear term that causes adversarial vulnerability achieves sufficient robustness without the prohibitive complexity of comprehensive perturbation-based training.

The patent merges the standard training objective (minimizing classification loss) with a regularization term that penalizes non-linearity with respect to perturbations. By combining these two objectives into a unified loss function, the method maintains the training speed and流程 of standard procedures while simultaneously improving robustness against adversarial attacks through the added regularization component.

The patent performs preliminary regularization during the training phase by incorporating the non-linearity penalty term into the loss function. This preliminary action of regularizing against non-linear vulnerabilities during training prepares the network in advance to resist adversarial attacks, rather than requiring separate post-training robustness enhancement steps.