Hierarchical label extraction method based on bidirectional Transformer model

Abstract

The invention discloses a hierarchical label extraction method based on a bidirectional Transformer model. The hierarchical label extraction method comprises the following steps: obtaining a feature vector through unsupervised pre-training text data; further adjusting and optimizing the feature vector by using a bidirectional Transformer model; in combination with a multi-layer label knowledge base system, performing supervised training on the nested Multi-Class classification model by utilizing softmax and a manual labeling label; and finally, outputting a hierarchical prediction label. According to the invention, a bidirectional Transformer model is adopted, and multi-level labels are combined and nested into a multi-class classification model for learning and training. According to the method, the learning of the multi-level classification labels of the text is realized, and the method can be used for automatic labeling (a plurality of specific labels in a certain field) of the network public opinion text and case analysis (layer-by-layer deepening of case skills) in a police service platform, so that the three-dimensional labeling of the text data is realized.

Description

technical field [0001] The invention relates to the fields of machine learning and text mining analysis, in particular to a method for extracting hierarchical labels based on a bidirectional Transformer model. Background technique [0002] With the rapid development of information technology, Internet information platforms and some business platforms (such as police comprehensive platforms) are flooded with massive data, which contains a lot of valuable information. How to quickly identify the subject tags corresponding to massive information is of great importance for efficient screening. Value information is of great significance. [0003] Labeling a piece of text is essentially a multi-class problem. In machine learning algorithms, SVM, Bayesian, and LSTM all play an important role, and statistical language models can also play a certain role. However, for the multi-layer label classification problem, that is, the classified label itself contains one level, two levels, o...

AI Technical Summary

Problems solved by technology

However, for the multi-layer label classification problem, that is, the classified label itself contains one level, two levels, or even more levels. At present, most of them adopt the pipeline method, first classify the labels level by level, and then connect multiple steps in sequence. However, end-to-end algorithm research is less

Images