Data Expansion Method for Training AI Models with Limited Samples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In the field of artificial intelligence, insufficient raw data poses a challenge for training data models effectively, as the amount of data directly impacts fitting accuracy and parameter settings, particularly in fields like military and archeology where data scarcity is prevalent.
Innovation Solution
A data generating method that involves obtaining first sample data, expanding it through techniques such as rotation, resizing, or generating synonyms for images, numbers, and text to create a verification set, which is then used to train and verify a data model, ensuring adequate data for model training and validation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data expansion techniques are applied to generate additional sample data, then the quantity of training data is improved, but the complexity of the data processing system increases
Solution Approach 1:
The patent applies copying by creating synthetic copies of existing sample data through data expansion techniques. Existing samples are replicated and transformed using methods like rotation, flipping, cropping, and adding noise to generate multiple variations. This allows the system to multiply the effective training data quantity without collecting new physical samples, directly resolving the contradiction between data quantity and system complexity.
Solution Approach 2:
The patent employs parameter changes by systematically varying data parameters during expansion. Image parameters include rotation angles, scaling factors, brightness levels, and noise intensities. Text parameters include synonym replacement, back-translation languages, and augmentation probabilities. By changing these parameters, the system generates diverse training samples from limited originals, improving data quantity while using standardized processing pipelines that control complexity.
2Measurement precision
If data expansion techniques are used to improve model training accuracy, then the fitting accuracy is improved, but the time required for data processing increases
Solution Approach 1:
The patent applies preliminary action by performing data expansion and augmentation during the data preparation phase before model training begins. All synthetic data generation, including image transformations and text augmentations, is completed in advance to create an expanded training dataset. This preliminary processing allows the actual training phase to use ready-to-go data, improving fitting accuracy while separating the time cost to the offline preparation stage rather than extending training time.
Solution Approach 2:
The patent replaces manual data collection and annotation processes with automated data expansion algorithms. Instead of mechanically gathering and labeling new samples, the system uses computational algorithms to automatically generate synthetic variations. This substitution of mechanical data collection with algorithmic generation improves accuracy by providing more diverse training examples while reducing the time investment required for manual data preparation.
Data Source
AI summary
A data generating method includes obtaining first sample data, determining a type of the first sample data and a corresponding data expansion method, expanding the first sample data according to the determined data expansion method to generate second sample data, and dividing the first sample data and the second sample data into a training set and a verification set according to a preset rule. A data model is trained according to the training set, and the data model is verified according to the verification set after training.


