Neural Network Training via High-Dimensional Point Set Perturbation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional convolutional neural networks (CNNs) face challenges in training due to the need for large numbers of samples and a tendency to overfit, making them difficult to train effectively.
Innovation Solution
A training method that represents sample images as point sets in high-dimensional spaces, generates and smooths random perturbation matrices to perturb these images, and uses the perturbed sets to train the neural network, thereby generating more diverse training samples and reducing overfitting by adding noise to feature maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of layers and neurons in CNN is increased to improve performance, then the recognition accuracy is improved, but the training difficulty and risk of overfitting increase
Solution Approach 1:
The patent applies parameter changes by transforming the input image into a high-dimensional space and then applying random perturbation to generate diverse training samples. This changes the parameter space of the training data, allowing the CNN to learn more robust features without increasing model complexity. The perturbation is applied in the high-dimensional space, effectively increasing the diversity of training samples without adding more layers or neurons.
Solution Approach 2:
The patent introduces another dimension by transforming the 2D image into a high-dimensional space where random perturbation can be applied. This dimensional transformation allows for generating diverse training samples without increasing the spatial dimensions of the input images or the depth of the network. The high-dimensional space provides an additional degree of freedom for sample generation.
2Measurement precision
If the model size is increased to improve performance, then the recognition accuracy is improved, but the number of training samples required increases
Solution Approach 1:
The patent uses copying by generating synthetic perturbed versions of existing training samples. Instead of collecting more real samples, the method creates multiple copies of existing samples by applying random perturbation in the high-dimensional space. These perturbed copies serve as additional training data, effectively increasing the quantity of training samples without requiring more original data.
Solution Approach 2:
The patent changes the parameters of existing training samples by applying random perturbation in the high-dimensional space. This transformation creates new training samples from existing ones, increasing the sample quantity without requiring additional data collection. The parameter changes are applied systematically to generate diverse samples that maintain the original sample's characteristics while introducing variability.
3Measurement precision
If the model size is increased to improve performance, then the recognition accuracy is improved, but the tendency to overfit increases
Solution Approach 1:
The patent converts the harmful effect of overfitting into a benefit by intentionally introducing random perturbation to training samples. This perturbation simulates the effect of seeing varied versions of the same sample, which prevents the model from memorizing training data and encourages it to learn more generalizable features. The perturbation acts as a regularizer that reduces overfitting while maintaining model performance.
Solution Approach 2:
The patent applies preliminary action by pre-transforming images into high-dimensional space and pre-applying random perturbation before training the CNN. This preliminary processing of the training data creates a more robust representation that prevents overfitting during the training process. The perturbation is applied in advance to ensure that the model learns from diverse variations of each sample from the outset.
Data Source
AI summary
A training method and a training apparatus for a neutral network for image recognition are provided. The method includes: representing a sample image as a point set in a high-dimensional space, a size of the high-dimensional space being a size of space domain of the sample image multiplied by a size of intensity domain of the sample image; generating a first random perturbation matrix having a same size as the high-dimensional space; smoothing the first random perturbation matrix; perturbing the point set in the high-dimensional space using the smoothed first random perturbation matrix to obtain a perturbed point set; and training the neutral network using the perturbed point set as a new sample. With the training method and the training apparatus, classification performance of a conventional convolutional neural network is improved, thereby generating more training samples, reducing influence of overfitting, and enhancing generalization performance of the convolutional neural network.