Machine Learning Score Obfuscation via Step Function Noise
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning model outputs can be intercepted and manipulated by malicious actors, allowing them to iteratively modify input data until a desired classification is achieved, which is not necessarily accurate, posing a threat in applications like malware detection.
Innovation Solution
The output score of a classification model is modified using a step function that applies noise, such as position-dependent noise, to obfuscate the true score, making it difficult for malicious actors to reverse-engineer or manipulate the classification, while maintaining the ultimate classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the classification model output is left unmodified, then the model provides accurate classifications, but malicious actors can intercept and manipulate the output to achieve desired classifications through iterative input modification
Solution Approach 1:
The patent introduces an intermediary component between the classification model and the consuming application that modifies the output score. This intermediary applies noise functions (such as Gaussian noise or uniform noise) to the raw classification score, transforming it into an obfuscated score that is then provided to the consuming application. This intermediary layer prevents malicious actors from directly observing and manipulating the true classification score while maintaining the system's security and reliability.
2Object-affected harmful factors
If noise is applied to the classification score to obfuscate it, then manipulation by malicious actors is prevented, but the true classification score becomes hidden
Solution Approach 1:
The patent changes the parameter of the classification score by applying noise functions that add random variations to the original score. The noise level and distribution characteristics can be adjusted as parameters to control the degree of obfuscation. This parameter transformation ensures that the true score information is not directly visible to malicious actors while still allowing the consuming application to make accurate classification decisions based on the obfuscated score.
3Object-affected harmful factors
If the output score is obfuscated using noise functions, then the true effect of input changes is masked, but additional processing complexity is introduced
Solution Approach 1:
The patent applies local quality by selectively applying different noise functions or noise levels to different regions or aspects of the classification output. For example, different types of noise (Gaussian, uniform, Laplacian) can be applied based on the specific classification context or score range. This localized approach provides effective obfuscation where needed while keeping the processing relatively simple and maintainable.
Data Source
AI summary
An artefact is received. Features are extracted from this artefact which are, in turn, used to populate a vector. The vector is then input into a classification model to generate a score. The score is then modified using a step function so that the true score is not obfuscated. Thereafter, the modified score can be provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.