Proactive defense of untrustworthy machine learning system

Abstract

Methods and systems for inducing model shift in a malicious computer's machine learning model is disclosed. A data processor can determine that a malicious computer uses a machine learning model with a boundary function to determine outcomes. The data processor can then generate transition data intended to shift the boundary function and then provide the transition data to the malicious computer. The data processor can repeat generating and providing the transition data, thereby causing the boundary function to shift over time.

Description

Background technique [0001] As machine learning systems become more robust, efficient, and accurate, machine learning has been applied to a growing number of academic, industrial, and security applications. Specifically, machine learning classifiers have been increasingly used to automate complex processes that require deliberate decision-making. [0002] A machine learning classifier is a machine learning model that learns to distinguish between input data belonging to multiple categories. For example, machine learning classifiers can be used to classify real versus fake news stories, legitimate email versus spam, various images (for example, between an image of a dog versus an image of a cat), or alphanumeric characters. distinguish. During the training phase, a machine learning classifier learns to recognize patterns in labeled training data. Later, in the output phase, machine learning classifiers can use these identified patterns to produce classified data correspondin...

AI Technical Summary

Problems solved by technology

Model shifting prevents malicious computers from performing malicious attacks using malicious machine learning models

Images