Training Data Generation via Citation History Tracing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for accumulating training data in information processing systems are inefficient, resulting in low data generation rates, which hampers the accuracy of machine learning in providing appropriate answers to user inquiries.
Innovation Solution
A training data generation apparatus and method that utilizes citation history information to generate additional training data by tracing the citation relationships between inquiry and answer pairs, allowing for the creation of direct and indirect citation-based training data, thereby increasing the volume of training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional methods are used to accumulate training data, then the data generation process is simple, but the data generation rate is low resulting in insufficient training data volume
Solution Approach 1:
The system performs preliminary action by collecting and storing citation history information in advance. When generating training data, it retrieves this pre-collected citation history to efficiently create additional training data pairs without performing new citation analysis, thereby increasing data generation rate while maintaining data volume expansion.
Solution Approach 2:
The system creates copies of existing answer information by generating new training data pairs from cited answers. When an answer is cited in another answer, the system copies the cited answer information and creates new training data associations, effectively multiplying the utility of existing data without requiring new data collection.
2Quantity of substance
If citation history analysis is performed to generate additional training data, then the volume of training data increases, but the system complexity increases
Solution Approach 1:
The system segments the complex task of training data generation into distinct functional modules: a citation history information acquisition unit that collects citation data, a training data generation unit that processes the citations, and a storage unit that manages the generated data. This segmentation reduces overall system complexity by making each module independent and manageable.
Solution Approach 2:
The system introduces citation history information as an intermediary element between existing answer data and new training data pairs. This intermediary layer simplifies the generation process by providing structured citation relationships that the training data generation unit can directly process, avoiding the need for complex real-time analysis.
3Reliability
If only direct citation relationships are used for training data generation, then the data generation process is efficient, but the quality and diversity of training data is limited
Solution Approach 1:
The system adds another dimension to training data generation by incorporating indirect citation relationships in addition to direct citations. When generating training data, it not only processes direct citations but also traces back through the citation history to include indirectly cited answers, thereby expanding data diversity without significantly increasing processing complexity.
Data Source
AI summary
A non-transitory computer-readable storage medium storing a program that causes a computer to execute a process, the process including determining whether first answer information has been cited in a generation process of second answer information for a first inquiry based on history information, stored in a storage device, indicating a citation histories regarding pieces of answer information, and generating training data associating the first inquiry with the first answer information when the first answer information has been cited in a generation process.