Lexical Association Scoring for Automated Word Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computational linguistics methods lack efficient automated systems for determining target words or n-grams with lexical relationships to provided cue words, relying on human intervention and limited statistical data.
Innovation Solution
A computer-implemented method and system that analyzes statistical lexical information from a corpus to identify candidate words and n-grams with lexical associations, using numerical values to calculate association scores and aggregate scores for selecting target words or n-grams based on their relationships to cue words.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated systems are implemented for determining target words with lexical relationships, then productivity is improved, but device complexity increases
Solution Approach 1:
The system segments the lexical relationship determination process into distinct modules: cue word reception, candidate word identification using statistical lexical information, association score calculation, and target word selection. This modular segmentation enables automated processing while managing system complexity through organized functional components.
Solution Approach 2:
Statistical lexical information serves as an intermediary between cue words and target words. The system uses pre-computed statistical data about word co-occurrences and lexical relationships as a mediator to automatically determine target words without requiring complex real-time analysis, thus improving productivity while controlling complexity.
2Measurement precision
If statistical lexical information from large-scale corpora is used, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The system performs preliminary action by pre-processing large-scale corpora to extract and store statistical lexical information before actual target word determination. This includes pre-computing word co-occurrence statistics, lexical relationships, and association scores, which are then readily available for quick querying during operation, thereby improving measurement precision without incurring processing time delays.
Solution Approach 2:
The system extracts only the essential statistical lexical information needed for target word determination from large-scale corpora. By extracting and storing key statistical features (word co-occurrence frequencies, association scores) in advance, the system achieves high measurement precision while minimizing the time required for actual processing by working with condensed, pre-extracted data rather than raw corpora.
3Reliability
If multiple statistical association scores are calculated for each candidate word, then reliability is improved, but use of energy increases
Solution Approach 1:
The system applies partial action by calculating association scores for only the most promising candidate words rather than all possible words in the corpus. By identifying and focusing computational resources on a limited set of high-probability candidates based on initial statistical filtering, the system achieves reliable target word selection while significantly reducing energy consumption compared to exhaustive calculation approaches.
Data Source
AI summary
Systems and methods are provided for identifying one or more target words of a corpus that have a lexical relationship to a plurality of provided cue words. The cue words and statistical lexical information derived from a corpus of documents are analyzed to determine candidate words that have a lexical association with the cue words. The statistical information includes numerical values indicative of probabilities of word pairs appearing together as adjacent words in a well-formed text or appearing together within a paragraph of a well-formed text. For each candidate word, a statistical association score between the candidate word and each of the cue words is determined. An aggregate score for each of the candidate words is determined based on the statistical association scores. One or more of the candidate words are selected to be the one or more target words based on the aggregate scores.