Sparse-Matrix Training for Parallel Propositional Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing training methods for propositional logical neural networks lack parallelism capabilities, leading to inefficient training times due to sequential layer-by-layer inference without utilizing computing device parallelism.
Innovation Solution
Convert the weights and bounds of the propositional LNN into a sparse matrix, enabling concurrent updates using linear algebraic kernels on GPUs for distributed parallel training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If sequential layer-by-layer inference is used for training propositional LNN, then the training process maintains simplicity and ease of implementation, but the training time becomes excessively long due to lack of parallelism
Solution Approach 1:
The patent segments the training process by converting weights and bounds into sparse matrix representations, allowing the training data to be divided into batches that can be processed independently and in parallel across multiple computing units, thereby reducing overall training time while maintaining implementation simplicity
Solution Approach 2:
The patent transitions from sequential layer-by-layer processing to parallel batch processing by reformulating the training problem in terms of matrix operations that can be executed simultaneously across multiple dimensions of the computational space, effectively adding a parallel processing dimension to the training workflow
2Loss of time
If parallel processing is implemented for training propositional LNN, then the training time is reduced through concurrent weight updates, but the device complexity increases due to requirements for GPU acceleration and sparse matrix operations
Solution Approach 1:
The patent introduces sparse matrix structures as an intermediary representation between the neural network weights and the parallel processing operations, enabling efficient utilization of GPU computing resources while managing the complexity of parallel weight updates through structured data organization
Solution Approach 2:
The patent changes the representation parameters of the neural network weights from dense vectors to sparse matrices, and organizes training data into batch formats, thereby optimizing the parameters for parallel processing and reducing the computational complexity burden on the hardware
Data Source
AI summary
An embodiment may include a processor that identifies a plurality of weights from the propositional logical neural network. The embodiment may convert the plurality of weights into a sparse matrix. The embodiment may convert a training set into a plurality of bound vectors. The embodiment may update the sparse matrix using a graphical processing unit (GPU). The embodiment may compute a loss parameter and based on determining the loss function is below threshold, update the plurality of weights of the propositional neural network.