Neural Network Training With Sparse Targets for Faster Convergence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training neural networks for image processing using large datasets is time-consuming and requires significant storage and computational resources, especially in offline rendering scenarios where generating accurate target images is challenging.
Innovation Solution
Training neural networks using sparse target data, where only a selected subset of dense target data is used, reducing the need for generating and storing the entire dataset, and allowing real-time training with sparse input data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If traditional methods are used to train neural networks with dense target data, then training accuracy is maintained, but training time and storage requirements increase significantly
Solution Approach 1:
The patent divides the complete target data into sparse subsets, where only a portion of the target data is used for training each sample. This segmentation allows the system to reduce the amount of data processed while maintaining training effectiveness, directly addressing the contradiction between training time and accuracy.
Solution Approach 2:
The patent applies partial action by using only a subset of the complete target data for training purposes. Instead of processing all dense target data, the system selectively uses partial data points, which reduces computational load and training time while preserving sufficient training accuracy through strategic data selection.
2Reliability
If large datasets are used for training neural networks, then model accuracy improves, but storage capacity and computational resources requirements increase
Solution Approach 1:
The patent extracts only the necessary portions of target data for training, removing redundant information. By extracting sparse subsets from the complete dataset, the system maintains essential training information while significantly reducing storage requirements and the quantity of data that must be processed.
Solution Approach 2:
The system uses partial data action by training on subsets rather than the complete dataset. This approach provides sufficient training accuracy without requiring the full dataset, thereby reducing storage capacity and computational resource requirements while maintaining model performance.
3Measurement precision
If path traced images are generated for training, then training data quality improves, but data generation time and storage needs increase
Solution Approach 1:
The patent segments the complete target image data into sparse subsets, where only selected portions are used for training. This segmentation maintains the quality of training data by selecting representative samples while reducing the total data generation time and storage needs associated with generating all path traced images.
Solution Approach 2:
The system extracts only the necessary training data from the complete set of path traced images. By taking out and using only sparse subsets of the generated data, the patent maintains training data quality while significantly reducing the time and storage resources required for data generation and storage.
Data Source
AI summary
A method, computer readable medium, and system are disclosed for training a neural network model. The method includes the step of selecting an input vector from a set of training data that includes input vectors and sparse target vectors, where each sparse target vector includes target data corresponding to a subset of samples within an output vector of the neural network model. The method also includes the steps of processing the input vector by the neural network model to produce output data for the samples within the output vector and adjusting parameter values of the neural network model to reduce differences between the output vector and the sparse target vector for the subset of the samples.