Shared-Parameter Learning Device for Adversarial Robustness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models using ensemble learning for robustness against adversarial examples require large storage capacity due to the increased number of parameters, leading to processing delays and memory limitations.
Innovation Solution
Implementing a learning device that determines shared and non-shared parameters for multiple machine learning models, using a mask initialization and loss function calculation to update parameter values, reducing the overall number of parameters stored.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple machine learning models are used for ensemble learning to improve robustness against adversarial examples, then the reliability is improved, but the number of parameters increases leading to larger storage requirements and processing delays
Solution Approach 1:
The patent merges parameters across multiple machine learning models by identifying and sharing common parameters (e.g., weight matrices W1, W2, W3 and bias vectors b1, b2, b3 are shared across different models). This allows the system to maintain multiple models for ensemble learning while reducing the total number of unique parameters that need to be stored and processed, thereby resolving the contradiction between reliability and parameter quantity.
Solution Approach 2:
The patent implements a universal parameter sharing mechanism where the same parameters serve multiple functions across different machine learning models. The mask initialization and parameter updating mechanisms enable a single parameter set to be universally applied across multiple models, reducing storage requirements while maintaining the robustness benefits of ensemble learning.
2Quantity of substance
If the number of parameters is reduced through parameter sharing, then the storage requirements are reduced, but the complexity of managing shared and non-shared parameters increases
Solution Approach 1:
The patent segments parameters into two distinct categories: shared parameters that are common across multiple models and non-shared parameters that are specific to individual models. This segmentation is implemented through mask information that indicates which parameters should be shared and which should not, simplifying the management complexity by providing clear rules for parameter handling rather than requiring complex dynamic management.
Solution Approach 2:
The patent introduces mask information as an intermediary mechanism that mediates between the desire for parameter sharing and the need for model specificity. The mask acts as a mediator that automatically determines which parameters to share and which to keep separate, eliminating the need for complex manual parameter management and reducing the cognitive load on the system.
3Manufacturing precision
If parameter values are updated using loss function calculation, then the learning accuracy is improved, but the processing time increases
Solution Approach 1:
The patent combines the parameter updating operations for multiple models into a single unified process. By sharing parameters across models, the system updates the same parameters simultaneously for all models in one operation rather than performing separate updates for each model, thereby improving learning accuracy through consistent parameter optimization while reducing processing time through operational merging.
Data Source
AI summary
A learning device determines, for a plurality of parameters of a machine learning model having the plurality of parameters, mask information representing a distinction between a shared parameter provided for common use by a plurality of machine learning models, and a non-shared parameter that is provided individually to each machine learning model. The learning device calculates a value of a loss function with respect to training data. The loss function is based on the plurality of machine learning models to which the shared parameter, the non-shared parameter, and a parameter value indicated by the mask information have been applied. The learning device updates a value of the shared parameter and a value of the non-shared parameter by using the value of the loss function.