Word Vector Generation via N-ary Character Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing natural language processing solutions for generating word vectors lack accuracy, as they do not effectively capture the semantic information and context of words.

Innovation Solution

The method involves segmenting a corpus to obtain words, establishing feature vectors based on n-ary characters, training a recurrent neural network using these feature vectors and context words, and generating word vectors through recurrent calculation and a specified loss function.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional word vector algorithms are used, then the generation process is simple, but the accuracy of capturing semantic information is insufficient

Engineering Contradiction:

Improveaccuracy of word vector generationVSAvoidcomplexity of generation process

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments words into n-ary character sequences (e.g., 2-grams, 3-grams) to create detailed feature vectors. This segmentation allows the model to capture local character patterns and morphological information, significantly improving semantic representation accuracy while maintaining a structured approach to handling complexity

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transitions from traditional word-level representations to multi-dimensional feature vectors that incorporate n-ary character sequences. By adding this additional dimensional layer of character n-grams, the model enriches the semantic space and captures more nuanced word meanings, resolving the accuracy limitation of simpler algorithms

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Measurement precision

If context information is not considered, then the processing is faster, but the semantic representation is less accurate

Engineering Contradiction:

Improvesemantic information captureVSAvoidprocessing speed

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent pre-processes the corpus to extract and encode n-ary character features before the main training phase. This preliminary action prepares rich semantic features in advance, allowing the recurrent neural network to focus on learning contextual relationships during training, thereby achieving both high semantic accuracy and efficient processing

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The recurrent neural network processes sequences of n-ary character features continuously, maintaining the temporal and contextual relationships among characters. This continuous processing ensures that semantic information is captured comprehensively while the model learns to process information efficiently through recurrent computation

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS10824819B2Generating word vectors by recurrent neural networks based on n-ary characters

Publication Date: 2020.11.03 ADVANCED NEW TECHNOLOGIES CO LTD

AI summary

Implementations of the present specification disclose methods, apparatuses, and devices for generating word vectors. The method includes: obtaining individual words by segmenting a corpus; establishing a feature vector of each word based on n-ary characters; training a recurrent neural network based on the feature vectors of the obtained words and feature vectors of context words associated with the obtained words in the corpus; and generating a word vector for each obtained word based on the feature vector of the obtained word and the trained recurrent neural network.