Analog CAM Soft Decision Tree for Robust Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Tree-based machine learning models are sensitive to adversarial attacks and hardware non-idealities, leading to instability and reduced trustworthiness, especially in analog content addressable memory (CAM) systems where they are inefficient due to high computational overhead.
Innovation Solution
An algorithm-hardware co-design approach that softens decision trees into analog CAM using knowledge distillation, leveraging non-idealities in emerging NVM devices to implement soft decision trees (SDTs) with improved accuracy and robustness, eliminating the need for additional hardware overhead and sensor amplifiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional decision trees with orthogonal boundaries are used in analog CAM, then the implementation is simple and fast, but the model is highly sensitive to noise and adversarial attacks around decision boundaries
Solution Approach 1:
The patent replaces orthogonal (straight-line) decision boundaries with curved sigmoid-like boundaries in the analog CAM system. This curvature allows for gradual transitions between decision regions, creating soft boundaries that are less sensitive to noise and adversarial attacks while maintaining the parallel inference capability of analog CAM
Solution Approach 2:
The patent changes the parameter representation from hard threshold values to soft sigmoid function parameters. By using sigmoid functions with adjustable parameters (alpha, beta) that control the curvature and transition width, the system achieves robustness against noise while maintaining computational efficiency in analog CAM
2Reliability
If soft decision trees with sigmoid boundaries are implemented in conventional digital hardware, then the robustness to adversarial attacks is improved, but the computational overhead becomes extremely expensive
Solution Approach 1:
The patent substitutes digital computational mechanisms with analog physical mechanisms. Sigmoid functions are implemented using analog circuit elements (such as transconductors, capacitors, and resistors) that naturally exhibit sigmoid-like transfer characteristics, eliminating the need for numerous multiplications and digital computations while maintaining the soft boundary robustness
Solution Approach 2:
The analog CAM cells inherently perform the sigmoid activation function through their physical characteristics. The memory cell's transfer function naturally provides the soft boundary behavior, eliminating the need for separate activation function computations and reducing overall computational overhead
3Measurement precision
If sensor amplifier circuits are added to analog CAM to make boundaries steeper, then the matching precision is improved, but the device complexity and energy consumption increase
Solution Approach 1:
The patent achieves steep boundaries through local optimization of the sigmoid function parameters in specific regions of the decision space, rather than uniformly increasing complexity across the entire system. This allows precise matching where needed while maintaining simplicity in other regions
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The proposed solution enhances the robustness and reliability of tree-based models against adversarial attacks and drifting noise, achieving higher accuracy and reduced latency, with a 1.7% drop in accuracy compared to traditional decision trees under adversarial attacks and only a 0.6% drop due to hardware noise, while significantly reducing energy consumption and inference time.
Implementation Method 1
The ACAM cell includes a first floating-gate transistor (FG-FET) and a second FG-FET
Data Source
AI summary
A SDT computation system is provided. The SDT computation system includes a SDT module, an ACAM array, and a mapping module. The SDT module is for outputting a final probability and configured to provide a SDT structure with a root node and deeper inner nodes for calculating probability, in which the final probability of each leaf node of the SDT structure is a product of all of node probabilities along a path from the root node to each leaf node, and wherein a final output of the SDT module is from a leaf with the highest probability. The mapping module is configured to map the SDT structure into the ACAM array, such that each of the ACAM cells in the ACAM array has a threshold programmed to be a parameter for a node in a path leading towards the leaf node of the SDT structure.