Context Information Generation for Word Embedding Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional word embedding techniques fail to accurately capture the context of words, leading to reduced search accuracy for similar sentences due to fixed relationships between word spellings and vectors, and difficulties in handling words with multiple meanings, spelling mistakes, or variants.
Innovation Solution
A context information generation method that computes a sum vector of surrounding word vectors and uses a machine learning model to predict these words, storing the resulting parameter as context information associated with each word vector to enhance context representation and distinguish different meanings or spellings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional word embedding techniques are used to associate a unique vector with each word spelling, then the relationship between word spellings and vectors remains fixed and simple, but the search accuracy for similar sentences degrades and the ability to handle context, multiple meanings, and spelling variations is reduced
Solution Approach 1:
The patent segments the single word vector representation into multiple context-specific vectors. For each word, it generates separate vectors for different contexts (e.g., different meanings, different positions in sentences), allowing the system to select the appropriate vector based on the specific context, thereby improving search accuracy while maintaining the ability to handle multiple meanings and variations
Solution Approach 2:
The patent introduces dynamic context information that allows word vectors to change based on their usage context. Instead of fixed one-to-one mappings, the system dynamically selects or generates vectors that reflect the actual context of word occurrence, enabling adaptability to different meanings, spelling variations, and positional contexts while improving measurement precision
2Productivity
If context information is not assigned to words with multiple meanings, then the processing is simpler and faster, but the words are incorrectly identified as having the same meaning and average vectors are computed without distinction
Solution Approach 1:
The patent performs preliminary action by pre-computing and storing context information for words during the training phase. Context vectors are prepared in advance for different meanings and usages, so that during actual processing, the system can quickly retrieve and use the appropriate pre-computed vectors without performing complex real-time analysis, thus maintaining high processing speed while improving identification accuracy
3Device complexity
If spelling mistakes or variants are treated as different words, then the statistical process generates different vectors for each spelling, but this results in dissimilar vectors being computed for what should be the same word
Solution Approach 1:
The patent introduces context information as an intermediary that bridges spelling variations. Instead of directly comparing word spellings, the system uses context vectors that capture the semantic meaning and usage patterns. Spelling variants with similar contexts generate similar context vectors, allowing the system to identify them as representing the same word meaning despite spelling differences, thus improving vector similarity while maintaining processing simplicity
4Measurement precision
If algorithms that consider the order of surrounding words are used, then context representation is more accurate, but the computational complexity increases significantly
Solution Approach 1:
The patent extracts essential context information without considering the full order of surrounding words. It identifies and extracts key contextual features (such as nearby words within a certain window, positional information, and semantic relationships) while discarding less important sequential details. This extraction approach maintains sufficient context representation accuracy while significantly reducing computational complexity compared to full sequence analysis
Data Source
AI summary
A memory stores therein a document and a plurality of word vectors that are word embeddings respectively computed for a plurality of words. A processor extracts, with respect to one of the words, two or more surrounding words within a prescribed range from one occurrence position where the one word occurs, from the document, and computes a sum vector by adding word vectors corresponding to the surrounding words. The processor determines a parameter such as to predict the surrounding words from the sum vector and the parameter using a machine learning model. The processor stores the parameter as context information for the one occurrence position, in association with the word vector corresponding to the one word.