Chinese entity extraction method and device

Abstract

The embodiment of the invention discloses a Chinese entity extraction method and device. The method comprises the steps that a target source statement is segmented into clauses; the words in the clause are vectorized to obtain a word vector; a probability matrix of each label corresponding to each word obtained by a long short-term memory network LSTM is determined according to the word vector andthe hierarchical bidirectional long short-term memory network BiLSTM; the probability matrix is input into a CRF model to obtain a label with the maximum probability in the labels corresponding to each word; and an entity consisting of the words corresponding to the label with the maximum probability is extracted. According to the embodiment of the invention, the target source statement is segmented into the clauses, so that subsequent semantic representation in the clauses learned at a word level and inter-clause semantic representation learned at a clause level are facilitated; through theCRF model, the tag with the maximum probability in the tags corresponding to each word is determined, and the Chinese entity composed of the words corresponding to the tag with the maximum probabilityis extracted, so that the Chinese entity identification accuracy is improved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a Chinese entity extraction method and device. Background technique [0002] With the advancement of science and technology and the digitization of information, tremendous changes and innovations have taken place in all walks of life. [0003] In recent years, entity recognition in specific fields has received continuous research attention. For example, in the field of food safety, NER (Named Entity Recognition, Named Entity Recognition) automatically recognizes entities related to food and generates structured data to help build Knowledge graph in the field of food. Domain-specific cases are usually recorded by recorders, but recorders sometimes use Chinese abbreviations, resulting in multiple expressions of the same entity. And for entities that are mixed with Chinese characters, letters, numbers and punctuation marks, it increases the difficulty of identifying entities. ...

AI Technical Summary

Problems solved by technology

The existing technology directly inputs long sentences into the BiLSTM-CRF (Bi-directional Long Short-Term Memory-Conditional Random Field) model, which does not consider enough semantic information of sentences

Images