training method of a Chinese word segmentation model based on a neural network

Abstract

The invention discloses a training method of a Chinese word segmentation model based on a neural network. Corresponding corpus feature vectors are set for multiple word segmentation specifications, after training corpuses of the multiple word segmentation specifications are obtained, vector representation of characters is determined according to embedded vectors of the characters and the corpus feature vectors, and finally the vector representation of each character in the text sentence is input into a Chinese word segmentation model, so that a prediction result is obtained, and model parameters are adjusted according to the prediction result to complete training. Therefore, the method does not need to change the model structure; According to the method, the corresponding corpus feature vectors only need to be added to the vector representations of the characters, and the vector representations are used for training the model, so that the purpose of expanding the training corpus is achieved, the model can learn the generality among different word segmentation specifications, and therefore the purpose of improving the word segmentation performance under the single word segmentationspecification is achieved. In addition, the invention further provides a training device and equipment of the Chinese word segmentation model based on the neural network and a computer readable storage medium, and the effects of the training device and equipment correspond to those of the method.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to a training method, device, equipment and computer-readable storage medium of a neural network-based Chinese word segmentation model. Background technique [0002] Chinese word segmentation is a process of dividing text sentences into word sequences. Relevant scholars have proposed a variety of word segmentation specifications, and manually marked the corresponding training corpus for different word segmentation specifications. With the development of neural networks, Chinese word segmentation by training neural network-based models is becoming more and more common. [0003] At present, most Chinese word segmentation models focus on using the training corpus of the same word segmentation specification to improve the word segmentation performance under the word segmentation specification. This method is limited by the number of training corpora, and it is difficult to imp...

AI Technical Summary

Problems solved by technology

[0009] The object of the present invention is to provide a kind of training method, device, equipment and computer-readable storage medium of the Chinese participle model based on neural network, in order to solve the traditional method of training the Chinese participle model through the training corpus of multiple participle norms , there is a problem of increasing the complexity of the model or reducing the performance of the model word segmentation

Images