Anomaly Detection for Relation-Based Classifier Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current question answer systems require extensive training data to accurately identify relations between entities, which can be time-consuming and inefficient, especially when dealing with multi-entity co-occurrences in large document sets.
Innovation Solution
An information handling system detects multi-entity co-occurrence anomalies within documents, identifies connecting verbs in document titles, and collects corresponding document segments to train a relation-based classifier, thereby enhancing the efficiency of relation extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If extensive training data is used to train relation-based classifiers, then the accuracy of identifying entity relationships improves, but the time and computational resources required for training increase significantly
Solution Approach 1:
The patent extracts only the essential and relevant features from training data - specifically multi-entity co-occurrences and their contextual relationships - rather than using extensive raw text data. This extraction approach maintains training accuracy while significantly reducing the volume of data that needs to be processed, thereby decreasing training time and computational resources.
Solution Approach 2:
The patent transforms the training data representation by changing parameters from raw text to structured features including entity pairs, co-occurrence counts, and contextual relationships. This parameter transformation enables the classifier to learn effective patterns with fewer training examples, resolving the contradiction between accuracy and training time.
2Quantity of substance
If all document segments containing entity pairs are collected for training, then the completeness of training data improves, but the data processing complexity and storage requirements increase
Solution Approach 1:
The patent applies local quality by focusing training data collection on specific local features - multi-entity co-occurrences with meaningful contextual relationships - rather than uniformly processing all document segments. This selective approach maintains data completeness for relevant patterns while reducing overall processing complexity and storage requirements.
Solution Approach 2:
The patent segments the training data collection process into distinct stages: identifying multi-entity co-occurrences, extracting contextual relationships, and filtering for relevance. This segmentation enables efficient processing by handling each aspect separately with appropriate methods, reducing overall complexity while maintaining completeness.
Data Source
AI summary
An approach is provided in which an information handling system detects a multi-entity co-occurrence anomaly within a set of documents that corresponds to an amount of times that a first entity and a second entity co-occur in the set of documents. The information handling system then determines that at least one of the documents includes a title having a verb that grammatically connects the first entity to the second entity. As such, the information handling system collects document segments from the set of documents that have the first entity, the second entity, and the connecting verb. In turn, the information handling system uses the collected document segments to train a relation-based classifier.