Word Meaning Clustering via Vector Change Segmentation
Solution Overview
Problem
Existing word representation technologies, such as Word2Vec, struggle to accurately capture the multiple meanings of words in a distributed representation, as they either focus on surface-level representations or fail to account for semantic similarity, leading to poor accuracy in distinguishing between different meanings.
Innovation Solution
A clustering method that records initial and learned vector values for words, computes motion vectors based on changes in these values during learning, and clusters words to separate meanings, using techniques like Skip-gram and dimensionally-compressed vector spaces to account for both similarity and multiple meanings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Word2Vec or Skip-gram model is used to achieve distributed representation, then dimensional compression and word co-occurrence relations are achieved, but accuracy is poor for words with multiple meanings
Solution Approach 1:
The patent segments the distributed representation of a word with multiple meanings into multiple separate vectors, each representing a different meaning. This is achieved by clustering words based on the change between initial and learned vector values, and then generating separate vectors for each meaning cluster. This segmentation allows the system to distinguish between different meanings of the same word, resolving the contradiction between accuracy and adaptability.
Solution Approach 2:
The patent transitions from a single distributed representation vector to multiple separate vectors by introducing an additional dimension of meaning separation. Through clustering based on vector value changes and generating separate vectors for each meaning, the system adds a dimensional layer that captures multiple meanings without sacrificing the compressed representation benefit.
2Measurement precision
If distributed representation is obtained by accounting for multiple meanings and changing written expressions, then accuracy improves, but determining how to separate multiple meanings becomes difficult
Solution Approach 1:
The patent employs self-service by using the learning process itself to automatically separate multiple meanings. The system records initial and learned vector values, computes changes, and performs clustering without requiring manual intervention to determine meaning separations. The learning process automatically reveals the structure of multiple meanings through the pattern of vector changes, eliminating the need for complex manual meaning separation procedures.
Solution Approach 2:
The patent implements feedback by using the learned vector values to compute changes from initial values, and then using these changes as the basis for clustering. This feedback loop allows the system to automatically adjust and refine the separation of multiple meanings based on the learning process itself, making the complex task of meaning separation manageable and automated.
3Ease of manufacture
If surface layer representation is used, then processing is simplified, but semantic distance and meaning accuracy are compromised
Solution Approach 1:
The patent applies preliminary action by recording the initial vector values before the learning process begins. This preliminary recording allows the system to later compute the changes between initial and learned values, which becomes the basis for clustering and separating multiple meanings. This preliminary step enables the system to maintain simplicity in the representation process while achieving accurate semantic representation through the analysis of vector changes.
Data Source
AI summary
A clustering method, clustering program, and clustering device are described herein for clustering of words with multiple meanings based on generating vectors for each meaning of a word. The generated vectors provide a distributed representation of a word in a vector space that account for the multiple meanings of the word so as, for instance, to learn semantic representations with high accuracy.