JPAT Neural Network Architecture Reduces Training Time
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network architectures, particularly convolutional neural networks, face challenges in reducing training times and execution times for decision-making, and lack efficient intra-layer and inter-layer connections, which hinders their performance in pattern recognition and classification tasks.
Innovation Solution
The Joint Proximity Association Template (JPAT) integrates a convolutional neural network with a bidirectional associative memory matrix, forming a hierarchical architecture that reduces training and execution times by establishing parallel associative memory processes and connecting them through intra-layer matrices, emulating cognitive processes to enhance learning and decision-making efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a convolutional neural network is used for pattern recognition, then classification capability is improved, but training time increases significantly
Solution Approach 1:
The patent segments the neural network into distinct functional layers (convolutional layers, pooling layers, fully connected layers) that can be trained independently and in parallel. This segmentation allows different parts of the network to process different aspects of the input data simultaneously, reducing overall training time while maintaining classification accuracy.
Solution Approach 2:
The patent applies preliminary feature extraction and dimensionality reduction techniques before the main classification process. By pre-processing the input data to extract relevant features and reduce dimensionality, the network requires fewer training iterations to achieve the same classification accuracy, thereby reducing training time.
2Measurement precision
If more layers are added to increase processing capacity, then classification accuracy improves, but execution time increases
Solution Approach 1:
The patent extracts and removes redundant or less important layers from the neural network architecture. By identifying and eliminating layers that contribute minimally to classification accuracy, the network maintains its performance while reducing the number of computational steps required during execution.
Solution Approach 2:
The patent transforms the network architecture by changing the dimensionality of data representation at different layers. By using techniques such as batch normalization and dimensionality reduction, the network processes information more efficiently through transformed spaces, reducing execution time without sacrificing accuracy.
3Ease of manufacture
If traditional neural network architecture is used, then implementation is straightforward, but intra-layer and inter-layer connections are inefficient
Solution Approach 1:
The patent introduces intermediary components such as batch normalization layers, dropout layers, and activation functions that mediate between standard neural network layers. These intermediaries improve the flow of information and gradients through the network, enhancing training efficiency and convergence speed while maintaining architectural simplicity.
Solution Approach 2:
The patent combines different types of layers and processing mechanisms into a composite architecture that leverages the strengths of each component. By integrating convolutional layers, pooling layers, normalization layers, and fully connected layers in a optimized sequence, the network achieves both ease of implementation and high processing efficiency.
Data Source
AI summary
A technical solution is described for implementing a computer-executed system of association memory matrices to replace the proximal layers of a convolutional neural network (CNN). An example method includes configuring one Associative Memory Matrix (AMM) for each configured layer in the CNN. This one-to-one conversion method motivates the name to the product: the Joint Proximity Association Template (JPAT) for Neural Networks. The invention is a numerically stable soft-ware based implementation that (1) reduces the long training times, (2) reduces the execution time, and (3) produces bidirectional intra-layer connections and potentially, inter-layer connections as well. The method further includes, potentially, forming a single AMM, from the multiple AMMs corresponding to the multiple and proximal layers of the CNN, in anticipation of the well-known Universal Approximation Theorem.