Machine Learning Algorithm Verification Using Disjunct Test Data Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning methods for machine learning algorithms often result in overfitting, where models perform well on training data but poorly on new data, necessitating a method to verify generalization with minimal resource consumption and without complex algorithms.
Innovation Solution
A method involving the use of disjunct test data sets to assess similarity with training data sets, generating test results, and verifying the algorithm based on these similarities to deduce the generalization degree with lower resource consumption, particularly suitable for control units with limited resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If common methods for checking generalization are used, then the accuracy of generalization assessment is improved, but the resource consumption (memory and processor capacities) increases significantly
Solution Approach 1:
The patent extracts only the essential information needed for generalization assessment - the similarity values between test data sets and training data sets - while discarding unnecessary computational complexity. By focusing solely on similarity measurement and test result analysis, the method reduces resource consumption while maintaining assessment accuracy.
Solution Approach 2:
The patent uses simple, lightweight similarity measurement techniques that can be performed with minimal computational resources. Instead of employing complex, resource-intensive algorithms, the method uses straightforward similarity calculations that are computationally inexpensive but sufficient for verifying generalization.
2Reliability
If complex algorithms are used for checking generalization, then the reliability of verification is improved, but the device complexity increases
Solution Approach 1:
The patent segments the verification process into distinct, simple steps: providing test data sets, calculating similarity values, generating test results, and verifying the algorithm. This segmentation allows each step to be implemented with simple, straightforward logic rather than complex integrated algorithms, reducing overall system complexity while maintaining reliability.
Solution Approach 2:
The method uses the test data sets themselves to provide the verification information needed. By calculating similarity values between test and training data sets and using the algorithm's own test results, the system performs self-verification without requiring external complex verification mechanisms or additional sophisticated algorithms.
3Measurement precision
If large amounts of data are used for testing, then the accuracy of generalization assessment is improved, but the processing requirements and resource consumption increase
Solution Approach 1:
The patent changes the parameter used for assessment from raw data quantity to similarity values. Instead of analyzing large amounts of raw test data directly, the method transforms the data into similarity parameters that capture the essential relationships between test and training data sets. This parameter transformation allows accurate generalization assessment with significantly reduced data processing requirements.
Data Source
AI summary
A method for verifying a machine learning algorithm. The method includes: providing test data sets for testing a machine learning algorithm trained based on a training data set, wherein none of the test data sets has a common element with the training data set; respectively ascertaining, for each of the plurality of test data sets, a value of the similarity between the corresponding test data set and the training data set; respectively generating, for each of the plurality of test data sets, a test result by testing the machine learning algorithm based on the elements of the corresponding test data set; verifying the machine learning algorithm based on the values of the similarity of all of the plurality of test data sets and the test results of all of the plurality of test data sets in order to generate verification results; and providing the verification results.