Neural Network Node Selection via Histogram Error Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for training artificial neural networks, particularly in unsupervised learning scenarios, face challenges in efficiently identifying and activating relevant nodes for pattern recognition and data classification, especially when dealing with unlabeled data.
Innovation Solution
A method involving inputting training data into a neural network, generating feature value distributions, calculating errors against a predetermined probability distribution, and selectively activating or deactivating hidden nodes based on these errors to optimize node weights and improve classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If all hidden nodes are activated in the neural network, then the network can process all possible features, but the training time and computational complexity increase significantly
Solution Approach 1:
The patent applies partial action by activating only a subset of hidden nodes based on their contribution to classification accuracy. Instead of all nodes being active, the system selectively activates nodes that provide meaningful feature extraction, reducing computational overhead while maintaining essential processing capabilities
Solution Approach 2:
The hidden layer is segmented into multiple nodes that can be independently evaluated and selectively activated. Each node processes specific features independently, allowing the system to divide the feature extraction task into manageable segments and activate only those segments that contribute to accurate classification
2Measurement precision
If more hidden nodes are activated to improve classification accuracy, then the network can capture more patterns, but the computational effort and complexity increase
Solution Approach 1:
The system uses partial action by activating only the necessary number of hidden nodes required to achieve satisfactory classification accuracy. The node selection process identifies the minimum subset of nodes that capture essential patterns, avoiding the complexity of maintaining all possible nodes in an active state
Solution Approach 2:
The patent changes the parameter of node activation state from binary (all on/off) to selective (individual on/off based on contribution). By evaluating each node's contribution to classification accuracy and adjusting activation states accordingly, the system optimizes the balance between accuracy and complexity
3Reliability
If the neural network processes all features from all hidden nodes, then comprehensive pattern recognition is achieved, but the training computational effort increases
Solution Approach 1:
The patent extracts and identifies the essential features that contribute most to reliable pattern recognition by evaluating each hidden node's contribution. Instead of processing all features equally, the system extracts only the meaningful features from selectively activated nodes, reducing computational effort while maintaining recognition reliability
Solution Approach 2:
The system changes the parameter of feature processing from comprehensive (all features) to selective (contributive features only). By adjusting which nodes are active based on their feature contribution to classification accuracy, the system reduces computational effort while preserving the reliability needed for accurate pattern recognition
Data Source
AI summary
Disclosed is a method of training at least a part of a neural network including a plurality of layers performed by a computing device according to an exemplary embodiment of the present disclosure. The method includes: inputting training data including normal data and abnormal data to an input layer of the neural network; making a feature value output from each of one or more hidden nodes of a hidden layer of the neural network for each training data into a histogram and generating a distribution of the feature value for each of the one or more hidden nodes; calculating an error between each distribution of the feature value and a predetermined probability distribution; and selecting at least one hidden node among the one or more hidden nodes of the hidden layer based on the error.