Machine Learning Data Allocation for Limited Training Sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In situations where the quantity of training data is limited, reserving some potential training data for testing purposes is undesirable, as it can hinder the effectiveness of model training and testing, especially for machine learning models like neural networks and artificial intelligence.
Innovation Solution
A method for selecting and allocating training and test data from a limited dataset by assigning each data item a classification identifier and distributing them across multiple sets, ensuring that each data item is only used as test data for one set and training data for another, based on their classification identifiers, allowing for efficient training and testing of machine learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If some potential training data is reserved for testing purposes, then testing can be performed, but the quantity of training data is reduced which hinders model training effectiveness
Solution Approach 1:
The patent divides the limited data into multiple sets (first set, second set, third set) where each set contains different combinations of training and test data. This segmentation allows the same data to serve multiple purposes across different sets, resolving the contradiction between needing test data and preserving training data quantity.
Solution Approach 2:
The patent creates a system where data items can function as either training data or test data depending on which set they are assigned to. The same data item can contribute to training in one set while serving as test data in another set, making the data multi-functional and eliminating the need to permanently reserve data for testing.
2Reliability
If data items are allocated as test data for multiple sets, then testing coverage is improved, but data items are reused which may affect training data quality
Solution Approach 1:
The patent segments data items with specific identifiers (first data items, second data items, third data items) and assigns them to specific sets with clear rules. Each data item is allocated as test data for exactly one set, preventing reuse and maintaining training data quality while ensuring comprehensive testing coverage across multiple sets.
Data Source
AI summary
Methods and systems for analyzing machine-learned classifiers are disclosed herein. The method can include inputting a data item for processing by a machine-learned classifier model and receiving a plurality of confidence scores for a plurality of respective classes, the plurality of confidence scores having been generated by the machine-learned classifier model based on the data item. The method can also include determining a distance in dependence on a highest confidence score that is generated for the data item, and causing display of a class distribution diagram, where the class distribution diagram can illustrate a graphical representation corresponding to the data item located at said distance between the graphical representation of a first class and the graphical representation of a second class.