Position Relation Skip-Gram Model for Context Prediction

The patent introduces position information as an additional dimension to the traditional Skip-Gram model. Instead of only considering contextual relations between words, the model now incorporates relative position data between input and output words. This dimensional expansion allows the model to capture both semantic context and positional relationships, thereby improving context prediction accuracy while maintaining a relatively straightforward model architecture through the use of position embeddings and augmented training samples.

The patent applies preliminary action by pre-processing training data to extract and encode position information before feeding it into the model. During the data preparation phase, position relationships between words are identified and represented as position embeddings. This preliminary structuring of position data simplifies the main training process, as the model directly receives pre-computed position embeddings rather than having to compute positional relationships during training, thus improving learning effectiveness while managing processing complexity.

The patent employs parameter changes by adjusting the dimensionality and structure of position embeddings to balance representation capability and computational efficiency. The position matrix embedding uses a configurable number of dimensions that can be optimized based on available computational resources. By tuning this parameter, the model achieves enhanced representation capability for capturing positional relationships while controlling the increase in computational resources required during training and inference.