Computer System Data Element Evaluation via Co-occurrence Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data analysis systems struggle to accurately evaluate data containing morphemes not present in the training data, leading to insufficient evaluation of synonyms, near-synonyms, and similar terms, as they cannot calculate weights for these elements without additional training data, which may introduce irrelevant data and reduce evaluation accuracy.
Innovation Solution
A computer system that calculates similarity between data elements based on co-occurrence frequency and corrects evaluation values using a similarity matrix, allowing for accurate evaluation of unknown data elements without increasing the training data, by generating a word-context matrix, applying pointwise mutual information, and performing singular value decomposition to estimate evaluation values for morphemes and their synonyms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If training data is increased to include new morphemes, then evaluation coverage is improved, but evaluation accuracy deteriorates due to mixing in low-relevance data
Solution Approach 1:
The patent introduces a similarity calculation mechanism as an intermediary between existing training data and new unknown morphemes. By calculating similarity between unknown morphemes and known morphemes based on co-occurrence frequencies, the system can transfer evaluation values without directly adding new training data, thus avoiding the dilution effect of low-relevance data while still expanding evaluation coverage to synonyms and near-synonyms.
Solution Approach 2:
The patent copies evaluation values from known morphemes to unknown morphemes through similarity-based transfer. Instead of creating new training data, the system creates virtual copies of evaluation values by estimating similarity degrees, allowing unknown morphemes to inherit evaluation information from their semantically similar counterparts in the existing training data.
2Quantity of substance
If training data is increased to include synonyms and near-synonyms, then evaluation completeness is improved, but data quality deteriorates
Solution Approach 1:
The patent changes the parameter of evaluation from direct training data inclusion to similarity-based value transfer. By transforming the approach from adding morphemes to training data to calculating similarity parameters between morphemes, the system expands the number of evaluable morphemes while maintaining data quality through mathematical transformation rather than data expansion.
Solution Approach 2:
The patent introduces a new dimension of similarity calculation based on co-occurrence frequencies, moving the problem from a one-dimensional training data inclusion approach to a multi-dimensional space where morpheme relationships are captured through contextual co-occurrence patterns, enabling evaluation expansion without quality degradation.
3Measurement precision
If weight calculation is restricted to training data morphemes only, then evaluation precision is maintained, but evaluation applicability deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-calculating co-occurrence frequencies and similarity relationships between all morphemes before evaluation occurs. This preliminary preparation enables the system to quickly transfer evaluation values to unknown morphemes during actual evaluation, maintaining precision through pre-established relationships while expanding coverage to include synonyms and near-synonyms that were not in the original training data.
Data Source
AI summary
[Problem to be Solved]Provided is a computer system that can accurately evaluate data to be analyzed without adding training data.[Solution]The computer system forms, from a matrix based on a co-occurrence frequency of first data elements forming at least one piece of data out of a plurality of data and second data elements appearing in vicinity of the first data elements, vectors for a plurality of data elements as the first data elements, calculates similarities for the first data elements on the basis of the vectors, and sets evaluation values for the first data elements on the basis of evaluation values corrected in accordance with the similarities.