Training Data Collection List for Interruption Resumption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Collecting training data from multiple platforms for AI projects is time-consuming and prone to interruptions due to network and system maintenance, leading to higher time costs and resource wastage.
Innovation Solution
A method involving scanning multiple data sources to acquire information about training data, creating a collection list with identifiers and storage locations, and collecting the data based on this list, ensuring data tracking even if collection is interrupted.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data collection is performed from multiple platforms without interruption handling, then data collection can proceed, but time costs increase and resources are wasted due to interruptions
Solution Approach 1:
The patent applies preliminary action by creating a collection list that records storage locations of training data before actual collection begins. This pre-planning enables the system to quickly resume collection after interruptions without wasting time searching for data locations again, thus resolving the contradiction between reliability and time cost.
2Reliability
If data collection continues without tracking storage locations, then collection process is simple, but interruptions cause resource wastage and require restarting
Solution Approach 1:
The patent segments the data collection process into distinct components: a collection list that stores metadata about training data locations, and the actual data collection process. This segmentation allows the system to manage complexity by separating tracking functions from collection functions, enabling reliable resumption without overwhelming complexity.
3Productivity
If multiple data sources are scanned without organization, then data acquisition is thorough, but tracking and collection become inefficient
Solution Approach 1:
The patent applies copying by creating a collection list that contains copied metadata (storage locations) of training data from multiple data sources. Instead of managing the actual data and its locations, the system copies only the essential location information into the collection list, improving productivity while preventing loss of location information.
Data Source
AI summary
Embodiments of the present disclosure relate to a method, an electronic device, and a computer program product for collecting training data. The method for collecting training data provided in embodiments of the present disclosure includes: scanning a plurality of data sources to acquire information relating to a plurality of training data to be collected, and creating a collection list based on the information, the collection list including at least a plurality of identifiers of the plurality of training data and a plurality of storage locations of the plurality of training data in the plurality of data sources. The method further includes: collecting the plurality of training data from the plurality of data sources based at least on the collection list.


