Multimodal Neural Classification Using Bidirectional BMU Associations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI systems face challenges in adapting to real-world environments due to limitations in multimodal data fusion and processing, often relying on supervised learning and centralized processing, which are inefficient and limited by the Von Neumann bottleneck.
Innovation Solution
A brain-inspired computational model for multimodal data classification using unsupervised learning, implemented in a distributed processing paradigm, that creates bidirectional lateral connections between co-activated Best Matching Units (BMUs) in ANNs to facilitate divergence and convergence mechanisms for labeling and classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If supervised learning is used for multimodal classification, then classification accuracy can be improved with labeled data, but the system complexity and data requirements increase significantly
Solution Approach 1:
The system performs self-labeling through unsupervised learning where the network automatically discovers patterns and structures in unlabeled multimodal data, eliminating the need for manual annotation while maintaining classification capability. The bidirectional lateral connections enable the system to self-organize and self-correct without external supervision.
Solution Approach 2:
The system divides the complex multimodal classification task into separate processing streams for different modalities (e.g., visual, auditory), each handled by dedicated neural networks with bidirectional lateral connections. This segmentation reduces the complexity of processing all modalities simultaneously while maintaining overall classification accuracy.
2Adaptability or versatility
If centralized processing is used for multimodal data fusion, then integration of multiple modalities is achieved, but the Von Neumann bottleneck limits processing speed and efficiency
Solution Approach 1:
The system implements distributed processing by maintaining separate neural networks for different modalities that process information in parallel, with bidirectional lateral connections enabling selective integration only when needed. This avoids the continuous data shuffling between memory and processor that causes the Von Neumann bottleneck.
Solution Approach 2:
The bidirectional lateral connections merge information from multiple modality-specific networks at the neuronal level, allowing seamless integration of multimodal data while maintaining the independence and parallel processing capabilities of each modality stream, thus achieving both versatility and high processing speed.
3Productivity
If brain-inspired unsupervised learning is implemented, then processing efficiency and adaptability improve, but the computational model complexity increases
Solution Approach 1:
The brain-inspired model uses self-organizing maps and unsupervised learning algorithms that automatically adapt to data distributions and learn features without manual intervention. The system self-regulates its learning process through bidirectional lateral connections that strengthen or weaken connections based on co-activation patterns, reducing the need for complex external control mechanisms.
Solution Approach 2:
The system dynamically adjusts connection weights and neuronal activation thresholds based on learning experiences and input patterns. The bidirectional lateral connections modify their strength according to the correlation between co-activated neurons, allowing the model to adapt its complexity to match the task requirements rather than maintaining fixed high complexity.
Data Source
AI summary
A computer implemented method is provided for multimodal data classification with brain-inspired unsupervised learning, and a neuromorphic computing hardware structure for implementing the method. In a preferred embodiment, the method comprises the steps of: training with unsupervised learning based on a multimodal training dataset each of a plurality of Artificial Neural Networks (ANNs); training with unsupervised learning based on the multimodal training dataset a multimodal association between the ANNs to generate a plurality of bidirectional lateral connections between co-activated Best Matching Units (BMUs); labeling the neurons of each of the at least two ANNs with a divergence algorithm; and electing a global BMU with a convergence algorithm.