Modular Neural Network Training for Lower Computation Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large neural networks require significant computation time and resources for training, particularly due to the need for large amounts of data and many epochs, which hinders the advancement of artificial intelligence technologies.
Innovation Solution
A neural network is divided into modules, with a computer system tracking training process module-by-module and datum-by-datum, recording auxiliary information for efficient computation and error correction, allowing for module-specific adjustments and reducing computation time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large neural networks are used to solve difficult classification problems, then classification accuracy is improved, but computation time and training resources increase significantly
Solution Approach 1:
The patent divides a large neural network into multiple smaller modules that can be trained independently or in a coordinated manner. This segmentation allows the system to achieve high classification accuracy through ensemble methods while reducing the computation time required for training individual modules compared to training one large network.
Solution Approach 2:
The system performs preliminary training of individual modules separately before combining them into a complete classification system. This preliminary action allows each module to be optimized independently and trained on smaller subsets of data, reducing overall computation time while maintaining high accuracy when modules work together.
2Reliability
If large neural networks are trained with many epochs and large amounts of data, then model performance is improved, but training resources and computation time increase
Solution Approach 1:
The training process is segmented into module-level training tasks rather than training one large network end-to-end. Each module requires fewer computational resources and less data to achieve good performance, reducing overall training resource consumption while maintaining high model performance through the combined output of multiple modules.
Solution Approach 2:
The system trains each module to a sufficient level of performance individually rather than requiring all modules to reach maximum performance before combination. This partial action approach reduces training resources while still achieving high overall model performance when modules are integrated.
3Ease of operation
If the entire neural network is trained uniformly, then training is simple to manage, but problem diagnosis and correction become difficult
Solution Approach 1:
The neural network is segmented into multiple modules, each with its own training process and performance metrics. This segmentation makes it easier to identify which specific module is causing problems while maintaining manageable training operations through standardized module interfaces and coordinated training protocols.
Solution Approach 2:
Each module can be trained and evaluated with locally optimized parameters and data subsets tailored to its specific function. This local quality approach enables precise problem diagnosis at the module level while maintaining overall system coherence through standardized communication interfaces between modules.
4Reliability
If the entire neural network is trained as a monolithic structure, then the network can leverage all data and parameters, but interpretability of internal nodes becomes difficult
Solution Approach 1:
The network is divided into distinct modules with defined input-output relationships, making it easier to interpret what each module contributes to the final output. Each module can be analyzed independently while the complete network maintains full capability through modular composition.
Solution Approach 2:
Each module can be designed with locally optimized interpretability characteristics appropriate to its specific function. This allows different parts of the network to be interpreted in context-specific ways while maintaining overall network capability through the coordinated operation of specialized modules.
Data Source
AI summary
Machine-learning computer system breaks a neural network into a plurality of modules and tracks the training process module-by-module and datum-by-datum, recording auxiliary information during one iteration of the training process for retrieval during a later iteration. Based on this auxiliary information, the computer system can make decisions that can greatly reduce the amount of computation required by the training process. The auxiliary information allows the computer system to diagnose and fix problems that occur during the training process on a module-by-module and/or datum-by-datum basis.