Parameter Learning Apparatus for Predicate Relationship Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extracting predicate relationships from documents face challenges when the number of entities increases, leading to decreased frequency of text representations and potential negative effects on learning, while also encountering issues with using unlabeled text representations as learning data.
Innovation Solution
A parameter learning apparatus and method that extracts one-term or two-term document facts, calculates feature vectors, and updates parameters using a gradient method to prioritize scores for labeled and unlabeled data, enabling the use of unlabeled text representations and handling varying numbers of entities without performance degradation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the number of entities in predicate relationships increases, then the coverage of extracted relationships expands, but the frequency of text representations decreases leading to negative effects on learning performance
Solution Approach 1:
The patent segments the learning process into two distinct phases: a pre-training phase using labeled data to establish baseline parameters, and a fine-tuning phase using unlabeled data to adapt parameters for multi-entity scenarios. This segmentation allows the model to first learn from high-quality labeled examples and then adapt to diverse entity combinations without degradation, resolving the contradiction between expanded coverage and maintained learning performance
Solution Approach 2:
The patent performs preliminary pre-training on labeled data before fine-tuning on unlabeled data. This preliminary action establishes a solid foundation of learned parameters that can then be adapted to handle varying numbers of entities. The pre-training phase prepares the model to effectively utilize the subsequent unlabeled data, ensuring that learning performance is maintained even as the model adapts to more complex multi-entity relationships
2Quantity of substance
If unlabeled text representations are used as learning data, then the amount of available training data increases, but the accuracy of parameter learning may decrease due to lack of labels
Solution Approach 1:
The patent performs preliminary pre-training on labeled data to establish accurate parameter foundations before introducing unlabeled data for fine-tuning. This preliminary action ensures that the model has already learned correct parameter relationships from high-quality labeled examples, so that subsequent fine-tuning on larger volumes of unlabeled data can expand the training dataset without compromising the accuracy established in the pre-training phase
Solution Approach 2:
The patent implements a continuous two-stage learning process where pre-training on labeled data seamlessly transitions into fine-tuning on unlabeled data. This continuity ensures that the useful action of parameter learning is maintained throughout both phases, with the model continuously refining its parameters rather than discontinuing accurate learning when transitioning to unlabeled data. The continuous optimization process preserves measurement precision while expanding the quantity of training data
Data Source
AI summary
A parameter learning apparatus 100 extracts one entity in a document and a related text representation as a one-term document fact, outputs a one-term partial predicate fact including only the one entity using a predicate fact that includes entities and a predicate, calculates a first one-term score indicating the degree of establishment of the one-term document fact using a one-term partial predicate feature vector, a one-term text representation feature vector, and a one-term entity feature vector that are calculated from parameters, calculates a second one-term score with respect to a combination of one entity and a predicate or a text representation that is not extracted as the one-term partial predicate fact, updates the parameters such that the first one-term score is higher than the second one-term score, and calculates a score indicating the degree of establishment of the predicate fact and a score indicating the degree of establishment of a combination of entities and a predicate that is not obtained as the predicate fact using these scores.