Boosted Deep Convolutional Neural Networks Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training neural networks, particularly convolutional neural networks (CNNs), is a time-consuming and resource-intensive process due to the need for large datasets and complex architectures, which can lead to cumbersome models that are not optimally suited for all applications, resulting in inefficient processing and resource utilization.
Innovation Solution
The use of a boosting approach with an ensemble of weak learners to train neural networks, where an auxiliary network computes boosting weights and provides them to other networks, allowing for online computation and reducing complexity, training time, and resource requirements, while augmenting datasets to enhance training without overfitting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a complex neural network architecture is used to improve classification accuracy, then manufacturing precision is improved, but device complexity increases
Solution Approach 1:
The patent divides the complex neural network into multiple stages of training with different architectures. Initially, a simpler network is trained to establish basic classification capabilities, then progressively more complex networks are trained and combined. This segmentation allows the system to achieve high accuracy without requiring all complexity to be present simultaneously, resolving the contradiction between accuracy and complexity.
Solution Approach 2:
The patent implements preliminary training actions using simpler networks before deploying complex architectures. By pre-training with reduced complexity and progressively adding complexity in subsequent training stages, the system achieves high classification accuracy while managing device complexity through staged introduction of complex elements rather than immediate full complexity.
2Manufacturing precision
If extensive training with large datasets is performed to improve classification accuracy, then manufacturing precision is improved, but loss of time increases
Solution Approach 1:
The training process is segmented into multiple stages, each using different dataset sizes and network complexities. Instead of requiring all tens of millions of training samples from the outset, the system progresses through stages where earlier stages use smaller datasets with simpler networks, then combines these with additional training data in later stages. This segmentation dramatically reduces the time required to achieve high accuracy compared to traditional approaches that require full dataset training from the beginning.
Solution Approach 2:
Preliminary training actions are performed with smaller datasets and simpler architectures to establish baseline classification capabilities quickly. These preliminary results are then combined with additional training data in subsequent stages, allowing the system to achieve high classification accuracy without requiring the full extent of time-consuming training with all training samples simultaneously.
3Manufacturing precision
If a complex neural network is used to improve classification accuracy, then manufacturing precision is improved, but use of energy increases
Solution Approach 1:
The patent segments the classification task across multiple training stages with progressively increasing complexity. Rather than deploying a single highly complex network that requires substantial energy for processing, the system uses a combination of simpler networks trained in different stages, each requiring less energy. The segmented approach allows high classification accuracy to be achieved through coordinated simpler components rather than a single energy-intensive complex network.
Data Source
AI summary
Briefly, embodiments of methods and/or systems of training multiclass convolutional neural networks (CNNs) are disclosed. For one embodiment, as an example, an auxiliary CNN may be utilized to form an ensemble with the collection as a linear combination. The linear combination may be based, at least in part, on boost prediction error encountered during the training process.


