BILSTM-CRF product name identification method based on self-attention

Abstract

The invention discloses a BILSTM-CRF product name identification method based on self-attention. The identification method is characterized by comprising three parts of semi-automatic product title data labeling, model construction and training and model use. The semi-automatically labeling product title data part establishes an iterative process of preliminary labeling, learning, label prediction, manual correction, learning and label prediction, the model construction and training part includes performing N-dimensional dense vector coding on each word, inputting the N-dimensional dense vector coding into a BiLSTM layer to obtain text sequence characteristics, and obtaining a label probability of each word by utilizing a Softmax classification layer; extracting text local features by using the CRF layer, and training a model; the model use part is used for extracting text features and obtaining probability of all labels by utilizing a classification layer; and obtaining a corresponding tag by using a Viterbi algorithm, thereby identifying the product name. According to the method, the labor cost is greatly reduced, and the model accuracy and robustness are improved.

Description

technical field [0001] The invention relates to the field of electronic commerce, in particular to a self-attention-based BILSTM-CRF product name recognition method. Background technique [0002] As far as the e-commerce field is concerned, the product titles filled in by merchants contain a large number of descriptive words and redundant information, such as "acaricide factory direct sales Yuexiu plant protection acaricide acaricide wholesale purchase", in which the product name should be acaricide The redundant information is factory direct sales, Yuexiu plant protection, wholesale procurement, etc. These modifiers and redundant information have caused great difficulties in product name identification. At present, the identification method of product name is mainly to establish a product name dictionary, find candidate product names according to keyword matching, and find out the most suitable product according to product attributes provided by merchants. The disadvantage...

AI Technical Summary

Problems solved by technology

The disadvantage of this method is that it is difficult to build a complete product name dictionary and corresponding product attribute library, and the construction process is time-consuming and laborious. At the same time, it cannot better solve the problem of product name nesting, such as "flowers delivered directly in the same city", Flowers itself is a product

Images