Model training method for text analysis, text classification method and device
A technology for model training and text analysis, applied in the computer field, which can solve problems such as limiting the robustness of the model and not considering the information of the back position.
Active Publication Date: 2019-07-23
ADVANCED NEW TECH CO LTD
View PDF0 Cites 15 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0003] The Transformer model only processes sequences from one direction. When processing each position in the sequence, it only considers the information of all previous positions, and does not consider the information of subsequent positions, which greatly limits the robustness of the model.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0069] The solutions provided in this specification will be described below in conjunction with the accompanying drawings.
[0070] figure 1 It is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification. This implementation scenario involves text classification, and the training of models for text analysis. refer to figure 1 , this implementation scenario involves three types of models: bidirectional converter model (also called bidirectional Transformer model), language model, and multi-classification model (also called multi-classifier). When training the model, multiple models can be jointly trained Form multi-task learning.
[0071] Among them, text classification: text classification refers to the task of classifying the text input by the user into one or more classes in several pre-defined classes.
[0072] Language model: The language model judges whether the sentence belongs to the correct natural language by calculating t...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The embodiment of the invention provides a model training method and device for text analysis and a text classification method and device, and the method comprises the steps: firstly obtaining a positive vector corresponding to each word in a first training statement by utilizing a first bidirectional converter model based on an initial word vector of the word and the superior text information ofthe word; then, for each word in the first training statement, obtaining a reverse vector corresponding to the word based on the initial word vector of the word and the text information of the word byusing a first bidirectional converter model; then, according to the position of each word in the first training statement, splicing the forward vector of the previous word at the position and the reverse vector of the next word at the position to serve as a target word vector corresponding to the position; and training the first bidirectional converter model and the first language model by usingthe first language model and aiming at the target word vector corresponding to each position, so that the operation speed is rapid, and the robustness of the model can be ensured.
Description
technical field [0001] One or more embodiments of this specification relate to the computer field, and in particular, relate to a model training method for text analysis, a text classification method and an apparatus. Background technique [0002] The Transformer model is a neural network model proposed by Google's Ashish Vaswani et al. in 2017, which can be used for deep modeling of sequence data, and can replace the long short term memory network (long short termmemory, LSTM) model, with the speed of operation Fast feature. [0003] The Transformer model only processes sequences from one direction. When processing each position in the sequence, it only considers the information of all previous positions, and does not consider the information of subsequent positions, which greatly limits the robustness of the model. [0004] Therefore, it is hoped that there will be an improved solution that can take advantage of the fast running speed of the Transformer model and ensure t...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/35
CPCG06F16/35
Inventor 蒋亮张家兴温祖杰梁忠平
Owner ADVANCED NEW TECH CO LTD