capsule model Chinese word segmentation method based on multi-regularization combination

Abstract

The invention provides a capsule model Chinese word segmentation method based on multi-regularization combination. The capsule model migration is applied to a natural language processing NLP sequencelabeling task, namely a Chinese word segmentation task, by adding a capsule sliding window capsule split window, so that the technical problem that the capsule model is not suitable for the sequence labeling task is solved; A plurality of regularization items are combined to realize simple field migration, and a capsule model is adapted to a sequence labeling task to complete Chinese word segmentation with higher accuracy and help a more complex natural language processing task; Through combination of multiple regular items, the generalization ability of the model is improved, certain field migration is achieved, manual corpus labeling can be reduced, and the labor and time cost of manual corpus labeling during natural language processing research is reduced.

Description

technical field [0001] The invention relates to the field of Internet technology, and in particular to a Chinese word segmentation method based on a multi-regularization combined capsule model. Background technique [0002] With the development of information technology, machine learning and other technologies, the technology of automatic processing of information has gradually been applied to various scenarios, such as mining user preferences in movie reviews, shopping product reviews, and automatically generating a short summary of an article. Automate the processing of text, and as Chinese users become more active on the Internet and generate more and more information, it is even more necessary to automate the processing of textual information. The emergence of these situations makes the related technologies of natural language processing widely applied to all corners of society. For natural language processing technology, especially for the development of domestic natur...

AI Technical Summary

Problems solved by technology

[0023] 1. It is only for the task of handwritten digit recognition, and the overall model is not suitable for sequence labeling tasks;

[0024] 2. The regularization term of the reconstructed image is not suitable for Chinese word segmentation tasks;

[0026] 1. Average accuracy;

[0027] 2. The capsule model is not suitable for the task of sequence labeling;

[0028] 3. The generalization performance is average, that is, training and testing can only be performed on the same corpus. When the test corpus in other fields is replaced, the performance will drop sharply

Images