Sinc Activation Function for Hierarchical Classifier Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional artificial neural networks (ANNs) struggle with hierarchical classification, often producing erroneous results when faced with inputs outside their training domain, as they lack the capability to reject unfamiliar patterns and provide accurate hierarchical categorization beyond their training scope.
Innovation Solution
The use of a sinc function as an activation function in a hierarchical classifier, which allows for multiple training sets with increasingly specialized weights, enabling the network to recognize patterns across a broader hierarchy and reject unfamiliar inputs, thereby providing accurate classification up to more general categories.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional activation functions are used in ANN for hierarchical classification, then the network can process inputs within training domain, but it produces erroneous results when faced with inputs outside training domain and cannot reject unfamiliar patterns
Solution Approach 1:
The patent applies parameter changes by switching from conventional activation functions (sigmoid, tanh, ReLU) to the sinc function (sin(x)/x). This mathematical function has unique properties including oscillating behavior with decreasing amplitude and multiple local minima, which enable the neural network to distinguish between familiar and unfamiliar patterns. The sinc function's characteristic of having zeros at regular intervals allows it to naturally reject unknown inputs, thereby improving classification reliability while maintaining adaptability to hierarchical structures.
2Measurement precision
If the network is trained to recognize specific patterns, then it can classify those patterns accurately, but it cannot provide accurate hierarchical categorization beyond its training scope
Solution Approach 1:
The sinc activation function provides universality by enabling the neural network to perform multiple functions: it maintains high accuracy for trained patterns while simultaneously enabling hierarchical categorization for unseen patterns. The function's oscillating nature with global maximum at zero and systematic local minima creates a universal response pattern that works across different hierarchical levels, allowing the same network structure to handle both specific pattern recognition and general hierarchical classification.
Solution Approach 2:
The sinc function's characteristic of having multiple local minima segmented across its domain enables the network to segment the input space into distinct hierarchical categories. Each local minimum region can correspond to a different hierarchical level or category, allowing the network to systematically organize patterns from specific to general categories while maintaining recognition accuracy for each segment.
3Ease of manufacture
If conventional activation functions are used, then the training process is straightforward, but the network lacks the capability to stabilize weights for hierarchical classification across multiple training sets
Solution Approach 1:
The sinc function's mathematical properties create natural stability in weight composition during training. The function's oscillating behavior with predictable zeros and local minima provides inherent regularization, stabilizing the weight matrix across multiple training sets. This stability arises from the function's characteristic of having a global maximum at zero and systematically distributed local minima, which constrain weight values to meaningful ranges and prevent divergence during hierarchical classification training.
Data Source
AI summary
Cardinal sine function used as an activation function for a hierarchical classifier. Application of a sine function, or a cardinal sine function, for hierarchical classification of a subject within subject matter domains and sub-domains. Hierarchical classification or multi-level classification is improved through use of the cardinal sine function or even standard sine function. Some embodiments of the present invention focus on the usage of cardinal sine function as activation function and how to apply this cardinal sine function for hierarchical classification of a subject. Some embodiments include a technique by which hierarchical classification or multi-level classification can benefit from application of a cardinal sine function.