Word Selection Support Device for Dictionary Registration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for selecting unknown words for inclusion in a dictionary, such as those used in FAQ retrieval systems, are inefficient due to the need for manual verification and lack of expertise, leading to potential overlooking of useful words and deterioration in retrieval accuracy.
Innovation Solution
A word selection support device that derives statistical information from multiple corpora to calculate the appropriateness of unknown words for registration, allowing for mechanical selection of useful words to be registered in the dictionary.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If all extracted unknown words are added to the dictionary, then the dictionary coverage is improved, but the retrieval accuracy deteriorates due to inclusion of irrelevant words
Solution Approach 1:
The patent changes the parameter of word selection from binary inclusion/exclusion to a probabilistic approach based on statistical analysis. By calculating appropriateness scores using multiple corpus frequencies and statistical formulas, the system dynamically determines which unknown words to include, transforming the dictionary construction from a粗放 approach to a precision-oriented process that maintains both coverage and accuracy
Solution Approach 2:
The system implements feedback mechanisms by using statistical information from multiple corpora to evaluate and score each extracted unknown word. The appropriateness calculation incorporates frequency data and statistical measures that provide continuous feedback on word quality, enabling iterative refinement of the dictionary to balance coverage and retrieval accuracy
2Measurement precision
If manual verification is performed to select registered unknown words, then the retrieval accuracy is improved, but the work cost and time consumption increase enormously
Solution Approach 1:
The patent replaces the mechanical manual verification process with an automated computational system. Instead of human experts manually evaluating each extracted unknown word, the system uses statistical formulas and computer algorithms to automatically calculate appropriateness scores and select registered unknown words, dramatically improving productivity while maintaining or enhancing retrieval accuracy through objective statistical criteria
Solution Approach 2:
The system enables self-service by allowing the dictionary construction process to evaluate and select words autonomously based on statistical principles. The automated appropriateness calculation and selection mechanisms eliminate the need for human intervention in each word evaluation step, making the process both more efficient and scalable while consistently applying the same selection criteria
3Measurement precision
If manual selection of registered unknown words is performed, then useful words can be identified, but the process requires expert knowledge and may overlook useful words depending on worker skill
Solution Approach 1:
The patent transforms the complex subjective judgment process into an objective parameter-based system. By defining specific statistical parameters such as corpus frequency, appropriateness scores, and selection thresholds, the system replaces expert intuition with quantifiable metrics, eliminating variability due to worker skill while maintaining high selection accuracy through mathematically sound criteria
Data Source
AI summary
A word selection support device according to the present disclosure includes processing circuitry configured to derive, for each extracted unknown word that is a term that is extracted from a target corpus and is not registered in dictionary data, statistical information regarding the extracted unknown word in a plurality of corpuses including the target corpus, and calculate appropriateness as a registered unknown word possibility that is a possibility of an unknown word to be registered in the dictionary data for each of the extracted unknown word on the basis of the statistical information.