Knowledge-Distilled TSK Fuzzy Classifier for Compact Rule Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to effectively transfer fuzzy knowledge from complex neural networks to concise TSK fuzzy classifiers, and there is a challenge in obtaining effective features for epilepsy EEG signal recognition using machine learning techniques.
Innovation Solution
A born-again TSK fuzzy classifier based on knowledge distillation, denoted as CNNBaTSK, which utilizes a non-iterative learning method to optimize consequent parameters using a novel objective function, combining soft label information and original data to improve generalization and classification performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If knowledge distillation is used to transfer knowledge from complex CNN to TSK fuzzy classifier, then classification performance is improved, but the complexity of knowledge transfer process increases
Solution Approach 1:
The patent creates a simplified copy of the complex CNN knowledge by extracting soft labels from the trained CNN and using them to train a compact TSK fuzzy classifier. This copying approach transfers the essential classification knowledge while discarding the complex structure, achieving both performance transfer and complexity reduction.
Solution Approach 2:
The patent extracts only the critical knowledge components from the CNN - specifically the soft labels representing class probabilities - and separates them from the complex network architecture. This extraction allows knowledge transfer without transferring the cumbersome CNN structure, resolving the complexity issue.
2Device complexity
If TSK fuzzy classifier is designed to be concise, then model simplicity is improved, but the ability to capture complex patterns deteriorates
Solution Approach 1:
The patent introduces soft labels as an intermediary that carries complex pattern information from the CNN to the simple TSK classifier. The soft labels act as a bridge, encoding complex decision boundaries in a simplified format that the concise TSK model can effectively utilize without needing complex architecture.
Solution Approach 2:
The patent changes the parameter representation by using soft label probabilities instead of hard class labels. This parameter transformation allows the simple TSK classifier to access nuanced pattern information that would otherwise require complex model architecture, enabling a concise model to capture complex patterns.
3Ease of manufacture
If soft label information is partitioned into fixed fuzzy partitions, then rule construction is simplified, but flexibility in representing continuous data is reduced
Solution Approach 1:
The patent segments the continuous soft label space into discrete fuzzy partitions (e.g., five partitions for five classes). This segmentation simplifies rule construction by creating manageable discrete categories while the fuzzy membership functions maintain continuous representation capabilities, balancing both requirements.
Solution Approach 2:
The patent transforms the continuous probability space into a dimensional framework of fuzzy partitions. By introducing fuzzy membership degrees as an additional dimension, the system maintains continuous representation flexibility while organizing data into structured partitions that simplify rule construction.
Data Source
AI summary
This application provides a born-again TSK fuzzy classifier based on knowledge distillation. The born-again TSK fuzzy classifier based on knowledge distillation is denoted as CNNBaTSK, and a fuzzy rule of CNNBaTSK includes two parts: an antecedent part based on soft label information and a consequent part based on original data. A method for constructing the fuzzy rule of CNNBaTSK includes following steps: Step 1: taking, by the CNNBaTSK, the original data as input, obtaining a probability distribution of an output layer through a layer-by-layer neural expression, and introducing a distillation temperature to generate soft label information of DATASET; Step 2: partitioning the soft label information into five fixed fuzzy partitions to construct the fuzzy rule in a fuzzy part of the CNNBaTSK; Step 3: introducing the original data to calculate a consequent parameter, and optimizing the consequent parameter of CNNBaTSK using a non-iterative learning method.