Anomaly Detection Layer Against Model Stealing in Driving Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing software systems, particularly those using machine learning models, are vulnerable to model theft attacks where attackers gain information about the system's internal workings to cause malfunction, sabotage, or extortion, posing risks in sectors like automotive where such failures can lead to accidents and reputational damage.
Innovation Solution
A computer-implemented method and system for preventing model theft attacks through anomaly detection in input data, including point-by-point, contextual, and time-series evaluations, with measures like documentation, altering functionality, and deactivation to thwart attackers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the software system processes input data without anomaly detection, then the system operates with high efficiency and productivity, but the system becomes vulnerable to model theft attacks where attackers can gain information about internal workings
Solution Approach 1:
The anomaly detection system is segmented into multiple independent evaluation components: point-by-point evaluation of individual data points, contextual evaluation of several data points, collective evaluation of groups of data points, and time-series evaluation. This segmentation allows the system to detect anomalies through multiple independent channels without requiring a single complex detection mechanism.
Solution Approach 2:
The patent introduces an intermediary anomaly detection layer that sits between the input data reception and the software system's core processing. This intermediary evaluates input data for anomalies before they reach the main system, preventing model theft attacks without requiring modification of the core software architecture.
2Reliability
If comprehensive anomaly detection is implemented including multiple evaluation types, then the system can detect various attack patterns, but the processing time and computational resources increase
Solution Approach 1:
The system implements partial anomaly detection by focusing on specific evaluation types based on the threat model. Rather than always performing all four evaluation types (point-by-point, contextual, collective, time-series), the system can selectively apply only the necessary evaluations for each situation, reducing processing overhead while maintaining detection accuracy for targeted threats.
3Reliability
If the system deactivates or alters functionality upon detecting anomalies, then the system prevents model theft attacks effectively, but the system loses operational availability during legitimate operations
Solution Approach 1:
The system applies local quality by selectively deactivating or altering only the specific software functionality that is under attack, rather than shutting down the entire system. This allows legitimate operations to continue uninterrupted while preventing model theft attacks on targeted components.
Solution Approach 2:
The anomaly detection system continuously monitors input data and provides feedback to the software system. When anomalies are detected, the system adjusts its response based on the severity and type of anomaly, dynamically altering functionality only when necessary and reversible when the threat ceases, maintaining operational availability during legitimate operations.
Data Source
Figure 1~2
Figure 3~4
Figure 5~7
AI summary
The invention relates to a computer-implemented method for preventing a model stealing attack on a software system (S), which receives input data (E) and provides output data, by means of which output data the software system (S) controls hardware parts, including hardware parts of a driving system, by means of closed-loop and/or open-loop control, said method comprising the steps of: receiving the input data (E) (V1); carrying out an anomaly detection on input data (E), comprising a point-by-point evaluation of individual data points, a contextual evaluation of a plurality of the data points, a collective evaluation of groups of the data points and/or a time-series evaluation of the data points, wherein anomalies are detected when there are outliers of the data points, including outliers in the data size of individual data points, outliers in the number of data points and/or outliers in the sequence of the data points (V2); and, if an anomaly is detected, ensuring information security of the software system (S), comprising documenting the attack, reporting the attack, changing the mode of functioning of the software system (S), extracting a watermark, executing the software system (S) in an emergency mode and/or deactivating the software system (S) (V3).