Neural Network Word Vector Generation for Text Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for natural language processing in artificial intelligence, particularly in Chinese text processing, face challenges due to the large number of Chinese vocabularies, resulting in high performance requirements for storage, network transmission, and memory, which reduces adaptability.
Innovation Solution
The method involves segmenting text into characters to generate character vectors, forming subsequences based on a preset vocabulary, and using pre-trained neural networks to obtain word vectors, reducing the need for static storage space by leveraging the smaller number of Chinese characters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a preset word vector matrix is used to obtain word vectors, then text processing can be performed, but the parameter size becomes very large due to the large number of Chinese vocabularies, resulting in higher requirements on physical storage, network transmission, and memory
Solution Approach 1:
The patent segments the text processing task into multiple stages: first segmenting text into characters to obtain character vectors, then segmenting the character vector sequence into multiple sub-sequences, and finally processing each sub-sequence through a neural network to generate word vectors. This segmentation approach breaks down the原本 monolithic word vector matrix into smaller, more manageable components.
Solution Approach 2:
The patent performs preliminary actions by pre-training the neural network model offline to learn the mapping from character vector sequences to word vectors. During actual text processing, the pre-trained model is used to generate word vectors without requiring the entire word vector matrix to be loaded into memory, thus reducing real-time resource requirements.
2Adaptability or versatility
If a large parameter size word vector matrix is used, then comprehensive text processing is achieved, but performance requirements for physical storage, network transmission, and memory increase
Solution Approach 1:
The patent replaces the mechanical lookup approach in a static word vector matrix with a computational neural network model. Instead of directly accessing pre-computed word vectors from a large matrix, the system uses a neural network to dynamically generate word vectors from character sequences, substituting a memory-intensive mechanical system with a computation-intensive but memory-efficient model.
Solution Approach 2:
The patent changes the fundamental parameter representation from direct word vector lookup to neural network-generated vectors. By transforming the input character vector sequences through a pre-trained neural network, the system generates word vectors on-demand with much smaller memory footprint, changing the parameter storage requirement from O(vocabulary size) to O(model parameters).
Data Source
AI summary
An artificial intelligence based method and apparatus for generating information are disclosed. The method in an embodiment includes: segmenting a to-be-processed text into characters to obtain a character sequence; determining a character vector for each character in the character sequence to generate a character vector sequence; generating a plurality of character vector subsequences by segmenting the character vector sequence based on a preset vocabulary; for each generated character vector subsequence, determining a sum of character vectors composing the character vector subsequence as a target vector, and inputting the target vector into a pre-trained first neural network to obtain a word vector corresponding to the each character vector subsequence, the first neural network used to characterize a correspondence between the target vector and the word vector; and analyzing the to-be-processed text based on the obtained word vector to generate an analysis result. This embodiment improves the adaptability of text processing.