Adaptive Neural Network Sampling for Nonlinear System Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training neural networks on non-linear, multi-variable complex systems require large amounts of training data and are inefficient, as they often rely on uniform random sampling and linear approximations, which are time-consuming and inaccurate.
Innovation Solution
A system and method that employs curriculum sampling and regression trees to determine regions of constant complexity, using K-dimensional trees and Z-score normalization to iteratively sample data points, allowing for adaptive sampling and efficient training of neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If uniform random sampling is used to generate training data, then the sampling process is simple and fast, but the accuracy of mapping highly complex input-output relationships deteriorates
Solution Approach 1:
The patent applies local quality by transitioning from uniform random sampling to adaptive sampling that concentrates data points in regions of high complexity (high curvature manifolds) while using fewer points in low complexity regions. The sampling density is locally adjusted based on the intrinsic curvature of the data manifold, achieving both efficiency and accuracy.
Solution Approach 2:
The patent changes the sampling parameter from uniform distribution to adaptive distribution based on manifold curvature. By estimating the curvature of the data manifold and adjusting sampling density accordingly, the system achieves more efficient data collection that balances speed and accuracy.
2Speed
If linear approximation is used to simulate physical systems, then the computation speed increases, but the accuracy deteriorates and the method works only for a small range of input domain
Solution Approach 1:
The patent applies dynamics by using a neural network that adapts its representation based on the input domain. Instead of using a fixed linear approximation, the neural network dynamically adjusts its internal parameters to capture non-linear relationships, allowing it to maintain high accuracy across the entire input domain while computing efficiently.
Solution Approach 2:
The patent replaces the mechanical linear approximation system with a neural network-based system. The neural network learns non-linear transformations from data, substituting the rigid linear model with a flexible non-linear model that can accurately represent complex physical systems across their full operating range.
3Measurement precision
If conventional simulation methods are used to generate training data, then the physical accuracy is maintained, but the time required for data generation becomes excessive
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing a comprehensive training dataset using neural networks that have learned from physical simulations. This pre-computed data can then be quickly queried during actual applications without requiring time-consuming real-time simulations, thus maintaining accuracy while reducing computation time.
Solution Approach 2:
The patent creates a digital copy of the physical system's behavior through neural network models trained on simulation data. This digital twin can be queried instantly to obtain system responses, replacing the need for time-consuming real-time physical simulations while maintaining accurate representations of system behavior.
4Measurement precision
If more training data is sampled from the entire input range, then the neural network accuracy improves, but the data generation becomes cumbersome and time-consuming
Solution Approach 1:
The patent applies local quality by concentrating sampling effort in regions of high manifold curvature where the data is most complex and informative, while using minimal sampling in low curvature regions. This adaptive sampling strategy achieves high neural network accuracy with significantly reduced total data points compared to uniform sampling across the entire input range.
Solution Approach 2:
The patent applies partial action by sampling only the necessary portion of the input space (regions of high complexity) rather than uniformly across the entire domain. This selective sampling approach provides sufficient training data for high accuracy while avoiding the time-consuming task of generating data for all input ranges.
Data Source
AI summary
The present disclosure generally relates to handling data of non-linear, multi-variable complex systems. More particularly, the present disclosure relates to methods and systems for training machine learning-based computing devices to ensure adaptive sampling of highly complex data packets. The present invention provides a robust and effective solution to implement a complexity-based sampling methodology that trains the neural network in complex mapping regions, by iteratively sampling the DBMS function and training the neural network in complex regions. The system (110) for training the complex and non-linear neural network may be equipped with a Machine Learning (ML) Engine (214) to solve the problem efficiently.