Quasi-analytic directional wavelet packet filters for CNN feature extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Convolutional Neural Networks (CNNs) face challenges in image classification using small ('tiny') datasets due to overfitting and the lack of explainable features, requiring large datasets and numerous convolutional layers.
Innovation Solution
Replacing convolutional layers in CNNs with quasi-analytic directional wavelet packet (qWP)-based filters, which are generic, universal, and independent of specific images, allowing for efficient feature extraction from tiny datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If convolutional neural networks are used for image classification, then classification accuracy is improved, but the requirement for large training datasets increases
Solution Approach 1:
The patent applies preliminary action by pre-defining wavelet packet filters with known mathematical properties before the classification task. These filters are designed to capture specific image features (edges, textures, patterns) through their wavelet basis functions, eliminating the need for lengthy training processes and large datasets that conventional CNNs require to learn filters from scratch.
Solution Approach 2:
The patent changes the fundamental parameters of the feature extraction process by replacing learned convolutional kernels with wavelet packet filters characterized by mathematical parameters (scale, translation, orientation). This parameter-based approach allows the system to achieve classification accuracy with minimal training data, as the filters' behavior is determined by their mathematical definition rather than empirical learning from large datasets.
2Measurement precision
If numerous convolutional layers are used in CNN, then feature extraction capability is improved, but computational complexity and processing time increase
Solution Approach 1:
The patent extracts and isolates the most critical feature extraction function by using wavelet packet transforms that directly compute multi-scale, multi-orientation features in a single processing stage. This eliminates the need for stacking multiple convolutional layers, as the wavelet packet decomposition inherently provides hierarchical feature extraction through its multi-resolution analysis capability.
Solution Approach 2:
The patent substitutes the mechanical stacking of multiple convolutional layers with a mathematical transform-based approach. Wavelet packet filters use mathematical operations (convolution with predefined basis functions, downsampling, and reconstruction) to achieve feature extraction in fewer steps, replacing the iterative learning process of deep CNNs with a direct computational transform.
3Measurement precision
If conventional CNN filters are used, then image features are extracted, but the extracted features lack explainability and physical meaning
Solution Approach 1:
The patent applies universality by using wavelet packet filters that serve multiple functions simultaneously: they extract features, provide interpretability through their mathematical definition, and capture physically meaningful image characteristics (edges, textures, patterns). The same filter set can be applied across different images and tasks, maintaining consistency and interpretability while achieving robust feature extraction.
Solution Approach 2:
The wavelet packet filters are self-explanatory by their mathematical construction. Each filter corresponds to a specific wavelet basis function with known properties (support, oscillation frequency, vanishing moments), allowing the extracted features to be directly interpreted in terms of image structure without requiring external explanation or visualization of learned kernels.
Data Source
AI summary
Methods and systems that replace convolutional layers of a convolutional neural network (CNN) with quasi-analytic directional wavelet packet (qWP)-based filters, and which use the qWP-based filters to perform filtering and extract features from image data. The extracted features are then used by the CNN to perform a classification task. The results of the classification task are output to a user.