A Chinese word extraction method, system, electronic device and storage medium

Abstract

The present application discloses a method for extracting Chinese special words, which includes: obtaining Chinese document data, and preprocessing and annotating the Chinese document data to obtain a training corpus data set; concatenating embedding layer features, vocabulary layer features, and sentence layer features to obtain a vocabulary feature representation of the Chinese document data; constructing a BiLSTM model and a document graph based on the vocabulary feature representation; constructing a special word extraction framework model using the BiLSTM model and the document graph; inputting the training corpus data set into the special word extraction framework model for training; if unknown Chinese document data is received, extracting Chinese special words from the unknown Chinese document data using the trained special word extraction framework model. The present application can reduce the extraction error of Chinese special words and improve the accuracy of entity recognition. The present application also discloses a Chinese special word extraction system, a storage medium, and an electronic device, which have the above beneficial effects.

Description

Technical Field

[0001] The present application relates to the fields of information extraction and natural language understanding, and in particular to a Chinese word extraction method, system, a storage medium and a storage medium. Background Art

[0002] Information extraction (IE) is to extract specific events or factual information from natural language texts, helping us to automatically classify, extract and reconstruct massive content. This information usually includes entities, relations and events. Correspondingly, information extraction mainly includes three subtasks: relation extraction (RE), named entity recognition (NER) and event extraction (EE).

[0003] Named Entity Recognition (NER) refers to the recognition of the boundaries and categories of proper nouns with specific meanings in text, mainly including names of people, places, institutions, proper nouns, etc., as well as text such as time, quantity, currency, and proportional values. Named entity recognition is a basic technology in text processing and is widely used in natural language processing, recommendation systems, knowledge graphs, and other fields. In the process of named entity recognition, specific vocabulary needs to be extracted. For data sets in special fields, it is extremely important to introduce special words (special vocabulary) information in vertical fields to improve the efficiency of entity recognition. The general method of introducing vocabulary information is to first segment the Chinese sentences in the data set, and then perform modeling training based on characters and vocabulary. The effect of this method is worse than that of the character-based method, because the error of mis-segmentation will occur during the segmentation process, and this error will be passed to the model.

[0004] Therefore, how to reduce the extraction error of Chinese words and improve the accuracy of entity recognition is a technical problem that technical personnel in this field currently need to solve. Summary of the invention

[0005] The purpose of this application is to provide a Chinese word extraction method, system, electronic device and storage medium, which can reduce the extraction error of Chinese words and improve the accuracy of entity recognition.

[0006] In order to solve the above technical problems, the present application provides a Chinese special word extraction method, which includes:

[0007] Acquire Chinese document data, and perform preprocessing and word tagging on the Chinese document data to obtain a training corpus data set;

[0008] Extracting embedding layer features, vocabulary layer features and sentence layer features of the Chinese document data respectively, and concatenating the embedding layer features, the vocabulary layer features and the sentence layer features to obtain a vocabulary feature representation of the Chinese document data;

[0009] Constructing a BiLSTM model and a document graph based on the vocabulary feature representation;

[0010] A special word extraction framework model is constructed using the BiLSTM model and the document graph; wherein the editor of the special word extraction framework model is the document graph, and the decoder is CRF;

[0011] Inputting the training corpus data set into the special word extraction framework model for training;

[0012] If unknown Chinese document data is received, the trained special word extraction framework model is used to extract Chinese special words from the unknown Chinese document data.

[0013] Optionally, the Chinese document data is preprocessed and word-tagged to obtain a training corpus data set, including:

[0014] Preprocessing the Chinese document data;

[0015] Using BIO annotation method to perform special word annotation on the pre-processed Chinese document data;

[0016] The Chinese document data after the special word annotation is divided into the training corpus data set and the test corpus data set according to a preset ratio.

[0017] Optionally, after inputting the training corpus data set into the special word extraction framework model for training, the method further includes:

[0018] The trained word extraction framework model is verified using the test data to obtain a model evaluation result.

[0019] Optionally, extracting embedding layer features of the Chinese document data includes:

[0020] A word embedding probability model is constructed, and the embedding layer features of the Chinese document data are obtained using the word embedding probability model.

[0021] Optionally, extracting vocabulary-level features of the Chinese document data includes:

[0022] Constructing a flat_lattice structure based on a special word list, and establishing a vocabulary relative position encoding matrix corresponding to the flat_lattice structure;

[0023] Constructing an XLNet attention network layer according to the vocabulary relative position encoding matrix;

[0024] The XLNet attention network layer is used to extract vocabulary-level features of the Chinese document data.

[0025] Optionally, extracting sentence-level features of the Chinese document data includes:

[0026] The features of context sentence information are integrated in the Chinese document data according to a preset sentence window to obtain sentence-level features of the Chinese document data.

[0027] Optionally, constructing a BiLSTM model and a document graph according to the vocabulary feature representation includes:

[0028] Building a BiLSTM model based on the vocabulary feature representation to obtain the vocabulary feature representation after local information enhancement;

[0029] An initial document graph corresponding to the Chinese document data is constructed with words as nodes and whether entities appear between sentences as the judgment criteria for establishing adjacent edges. The document graph is obtained by training the initial document graph using the word feature representation to obtain a word feature representation enhanced with global information.

[0030] The present application also provides a Chinese word extraction system, which includes:

[0031] The tagging module is used to obtain Chinese document data, and pre-process and tag the Chinese document data to obtain a training corpus data set;

[0032] A feature extraction module, used to extract embedding layer features, vocabulary layer features and sentence layer features of the Chinese document data respectively, and concatenate the embedding layer features, the vocabulary layer features and the sentence layer features to obtain a vocabulary feature representation of the Chinese document data;

[0033] A model building module, used to build a BiLSTM model and a document graph according to the lexical feature representation; and also used to build a special word extraction framework model using the BiLSTM model and the document graph; wherein the editor of the special word extraction framework model is the document graph, and the decoder is CRF;

[0034] A training module, used for inputting the training corpus data set into the special word extraction framework model for training;

[0035] The special word extraction module is used to extract Chinese special words in the unknown Chinese document data by using the trained special word extraction framework model if unknown Chinese document data is received.

[0036] The present application also provides a storage medium on which a computer program is stored. When the computer program is executed, the steps of the above-mentioned Chinese special word extraction method are implemented.

[0037] The present application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the steps of the above-mentioned Chinese special word extraction method are implemented.

[0038] The present application provides a method for extracting Chinese special words, which is characterized by comprising: obtaining Chinese document data, and preprocessing and special word annotating the Chinese document data to obtain a training corpus data set; respectively extracting embedding layer features, vocabulary layer features and sentence layer features of the Chinese document data, and concatenating the embedding layer features, the vocabulary layer features and the sentence layer features to obtain a vocabulary feature representation of the Chinese document data; constructing a BiLSTM model and a document graph according to the vocabulary feature representation; constructing a special word extraction framework model using the BiLSTM model and the document graph; wherein the editor of the special word extraction framework model is the document graph, and the decoder is CRF; inputting the training corpus data set into the special word extraction framework model for training; if unknown Chinese document data is received, extracting Chinese special words in the unknown Chinese document data using the trained special word extraction framework model.

[0039] This application extracts the embedding layer features, vocabulary layer features and sentence layer features of Chinese document data, and then concatenates the embedding layer features, vocabulary layer features and sentence layer features of Chinese document data to obtain the vocabulary feature representation of Chinese document data, and then uses the vocabulary feature representation to construct a BiLSTM model and a document graph. The BiLSTM model can enhance the local information of the vocabulary features, and the document graph can enhance the global information of the vocabulary features. The special word extraction framework model constructed using the BiLSTM model and the document graph can obtain a global semantic understanding. This application can reduce the extraction error of Chinese special words and improve the accuracy of entity recognition. This application also provides a Chinese special word extraction system, a storage medium and an electronic device, which have the above-mentioned beneficial effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 A flowchart of a Chinese word extraction method provided in an embodiment of the present application;

[0042] Figure 2 A flowchart of Chinese word extraction based on document-level global information provided in an embodiment of the present application;

[0043] Figure 3 A flowchart of a feature extraction method based on document modeling provided in an embodiment of the present application;

[0044] Figure 4 A flowchart of Chinese word extraction based on graph neural network provided in an embodiment of the present application;

[0045] Figure 5 A schematic diagram of the structure of a Chinese word extraction system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] See below Figure 1 , Figure 1 A flowchart of a Chinese word extraction method provided in an embodiment of the present application.

[0048] Specific steps may include:

[0049] S101: Acquire Chinese document data, and perform preprocessing and word tagging on the Chinese document data to obtain a training corpus data set;

[0050] Among them, the Chinese document data may include multiple Chinese sentences. After obtaining the Chinese document data, the Chinese document data may be preprocessed. The above preprocessing may include operations such as removing repeated sentences and punctuation marks in the Chinese document data. This step may also perform special word annotation on the preprocessed Chinese document data to obtain a training corpus data set. The above special annotation refers to the operation of annotating professional vocabulary in the document. Professional vocabulary refers to some words in a specific field. This embodiment can provide a professional vocabulary list. For example, the special words in the server professional vocabulary list include: migration, downtime, high availability, power on, etc.

[0051] Specifically, this step can preprocess the Chinese document data; use the BIO annotation method to perform special word annotation on the preprocessed Chinese document data; and divide the Chinese document data after special word annotation into the training corpus data set and the test corpus data set according to a preset ratio. The training corpus data set is used to train the special word extraction framework model, and the test corpus data set is used to detect the training effect of the special word extraction framework model.

[0052] S102: extracting embedding layer features, vocabulary layer features and sentence layer features of the Chinese document data respectively, and concatenating the embedding layer features, the vocabulary layer features and the sentence layer features to obtain a vocabulary feature representation of the Chinese document data;

[0053] Among them, this embodiment can extract embedding layer features in the following ways: construct a word embedding probability model, and use the word embedding probability model to obtain the embedding layer features of the Chinese document data. This embodiment can extract vocabulary layer features in the following ways: construct a flat_lattice structure based on a special word list, and establish a vocabulary relative position encoding matrix corresponding to the flat_lattice structure; construct an XLNet attention network layer according to the vocabulary relative position encoding matrix; and use the XLNet attention network layer to extract vocabulary layer features of the Chinese document data. This embodiment can extract sentence layer features in the following ways: fuse the features of context sentence information in the Chinese document data according to a preset sentence window to obtain sentence layer features of the Chinese document data.

[0054] On the basis of obtaining the embedding layer features, vocabulary layer features and sentence layer features, this embodiment can concatenate the embedding layer features, vocabulary layer features and sentence layer features to obtain the vocabulary feature representation of the Chinese document data.

[0055] S103: constructing a BiLSTM model and a document graph according to the vocabulary feature representation;

[0056] Among them, this embodiment can construct a BiLSTM model based on the lexical feature representation to obtain a lexical feature representation after local information enhancement; use vocabulary as nodes and whether entities appear between sentences as the judgment criteria for establishing adjacent edges to construct an initial document graph corresponding to the Chinese document data, and use the lexical feature representation to train the initial document graph to obtain the document graph, so as to obtain a lexical feature representation after global information enhancement.

[0057] S104: constructing a special word extraction framework model using the BiLSTM model and the document graph;

[0058] Among them, the editor of the special word extraction framework model is the document graph, and the decoder is CRF (Conditional Random Fields). In this embodiment, the document graph can be used as an encoder.

[0059] S105: inputting the training corpus data set into the special word extraction framework model for training;

[0060] Among them, after the training corpus data set is input into the special word extraction framework model for training, the trained special word extraction framework model can also be verified by using the test data to obtain a model evaluation result.

[0061] S106: If unknown Chinese document data is received, the trained special word extraction framework model is used to extract Chinese special words from the unknown Chinese document data.

[0062] The unknown Chinese document data is a Chinese document without special words extracted. After receiving the unknown Chinese document data, the unknown Chinese document data can be input into the trained special word extraction framework model to obtain Chinese special words in the unknown Chinese document data. After obtaining the Chinese special words, entity recognition can be performed, and then the dialogue system can be used to output the corresponding reply content according to the entity recognition result.

[0063] This embodiment extracts the embedding layer features, vocabulary layer features and sentence layer features of Chinese document data, and then splices the embedding layer features, vocabulary layer features and sentence layer features of Chinese document data to obtain the vocabulary feature representation of Chinese document data, and then uses the vocabulary feature representation to build a BiLSTM model and document graph. The BiLSTM model can enhance the local information of vocabulary features, and the document graph can enhance the global information of vocabulary features. The special word extraction framework model constructed using the BiLSTM model and document graph can obtain global semantic understanding. This embodiment can reduce the extraction error of Chinese special words and improve the accuracy of entity recognition.

[0064] The process described in the above embodiment is explained below through an embodiment in actual application.

[0065] NER technology is mainly implemented through modeling of sequence labeling problems, including entity labeling and entity recognition. Different data sets may use different entity labeling methods. The most common labeling methods are BIO, BIOES, BIOES+, and Markup. Entity recognition methods are mainly divided into four categories according to their development history: (1) rule-based methods, such as dictionaries and rule templates; (2) machine learning-based methods, such as hidden Markov model (HMM), maximum entropy Markov (MEMM), maximum entropy (ME), support vector machine (SVM), undirected graph conditional random field (CRF), etc.; (3) deep learning-based methods, such as BiLSTM-CRF, BiLSTM-CNN-CRF, etc.; (4) attention-based methods, such as Transformer-CRF, BERT-CRF, etc.

[0066] At present, the mainstream popular method is the method based on the attention mechanism, which is usually based on a single sentence for sequence labeling modeling, that is, only the context structure information within the sentence and the semantic information of the vocabulary within the sentence are considered when constructing vocabulary features. This may cause global information loss in the document-level word extraction scenario, because this type of method does not fully consider the information between sentences, such as the structure or semantics of adjacent sentences are generally highly similar, that is, there may be entities of the same type, and vice versa. If the same type of entity appears, then the two sentences are highly similar in semantics. Secondly, this type of method also does not take into account the distributed structural information at the level of word embedding within the vocabulary. In addition, for data sets in special fields, introducing word information in vertical fields is extremely important for improving entity recognition efficiency. The general method of introducing vocabulary information is to first segment the Chinese sentences in the data set, and then perform modeling training based on characters and vocabulary. The effect of this method is worse than that of the character-based method, because the error of misclassification will be generated during the word segmentation process, and the error will be transmitted to the model. Another way to introduce vocabulary information is to establish a word vector or a vocabulary list, that is, to use a word vector with good word segmentation results, or to use vocabulary and entity boundary information to extract relevant information through a graph network. There are two main modes of this method. One is to design a dynamic extraction framework that is compatible with vocabulary input, and the other is to build adaptive word embeddings based on vocabulary information, which is independent of the model framework.

[0067] See also Figure 2 , Figure 2A flowchart of Chinese word extraction based on document-level global information provided in the embodiment of the present application, the specific implementation steps are as follows:

[0068] Step 1: Obtain document data in a specific Chinese field, and perform data preprocessing and annotation to obtain training and test data sets.

[0069] Step 2: Based on the original data set, perform global modeling at three levels of the document to obtain the corresponding feature representation. Global modeling refers to modeling based on a document, rather than modeling a single sentence or a single word in the document.

[0070] Step 3: Based on the feature representation, build a BiLSTM model and document graph to enhance local and global information.

[0071] Step 4: Build a word extraction framework based on the global graph neural network as the encoder and CRF as the decoder.

[0072] Step 5: Input the training data into the model for training and save the model.

[0073] Step 6: Input the test data into the model for model verification and obtain the model evaluation results.

[0074] The above embodiment proposes a feature extraction method based on document modeling, which mainly integrates modeling information in three dimensions, namely, word embedding layer, vocabulary layer and sentence level: in the word embedding layer, the probability model is directly trained through the vocabulary window h to obtain the word embedding feature representation with distributed perception information of concentration k; in the vocabulary layer, the Chinese vocabulary is enhanced by introducing the Chinese flat-lattice structure, and then the final vocabulary layer feature representation is obtained through the XLNet attention layer; in the sentence level, the feature representation of the adjacent sentences of the current sentence is constructed through the sentence window k, and then it is integrated into the feature of the current sentence through the gating mechanism. This embodiment proposes a Chinese word extraction method based on a graph neural network, which first performs local information enhancement through a BiLSTM model, takes this feature representation as the basic entity as a node, and whether the same entity appears between different sentences as an adjacent edge, and constructs a document graph, which can enhance the model globally, and finally calculates through a global graph neural network and decodes through CRF. This method has a more global semantic understanding than the method based on a single sentence and an attention mechanism, and has a higher entity recognition accuracy.

[0075] As a further introduction to the above embodiment, step 1 may include the following steps:

[0076] Step 1.1: Obtain original document data in a specific Chinese field. The present invention uses data in an electronic invoice scenario and preprocesses the data.

[0077] Step 1.2: Use BIO annotation to annotate the data with special words, where B indicates the start of a special word, I indicates a special word, and O indicates a non-special word.

[0078] Step 1.3: Divide the dataset into training and test corpus datasets in a ratio of 9:1.

[0079] As a further introduction to the above embodiment, step 2 may include the following steps:

[0080] Step 2.1: Model feature extraction for word embedding layer.

[0081] Embedding layer feature extraction obtains distributed perception information by directly training the probability model. Among them, the probability distribution model assumes that the word embedding layer vector data is presented in the vMF distribution state, and the training loss is calculated as follows:

[0082]

[0083]

[0084] In the above formula, L represents the loss function, e i Represents the current word embedding vector, e i+j represents the embedding vector of adjacent words, k and v are the parameters of vMF distribution, which are the concentration coefficient k and the mean vector v respectively; v is the final word embedding representation. P is the probability symbol, D is the set of all vocabulary vectors, h is the sliding window, exp is the exponential function with e as the base, u is the training parameter, w is the vocabulary, w' is a certain vocabulary, and T is the vector transposition operation.

[0085] Step 2.2: The vocabulary layer feature extraction method mainly includes extracting the flat_lattice structure and building the XLNet attention network;

[0086] The lat_lattice structure is a flattened Chinese lattice structure in the FLAT model to facilitate the application of attention networks. The flat_lattice structure is constructed based on the word list, and the relative position encoding matrix of the corresponding words is established at the same time. The XLNet attention network layer is constructed based on the relative position encoding of the lattice structure. The calculation method is as follows:

[0087]

[0088]

[0089]

[0090] v j =(x j +p j)W V

[0091] The XLNet attention network layer is a self-attention mechanism network based on the XLNet relative position encoding method, which can fully obtain the relationship between words. In the above formula, u and v are trainable vector parameters, and w i represents the vector representation of the i-th word after attention calculation, where a i，j represents the attention score between the i-th word and the j-th word, that is, the correlation; W Q , W K , W V is a trainable matrix parameter, Relative position represents the relative distance between the token at position i and the token at position j in the matrix R, P i Represents the absolute position vector of the token at position j.

[0092] Step 2.3: The sentence-level feature extraction method is mainly achieved by fusing the features of adjacent sentences and the current sentence.

[0093] For the subsequent scalability of the model, the sentence window f is set to represent the number of adjacent sentences. In order to reduce the complexity of the model, f is taken as 2 here, that is, the two adjacent sentences before the current sentence are taken as the previous information features, and the two adjacent sentences after the current sentence are taken as the subsequent information features.

[0094] By calculating the relevance score with the current sentence, the feature representation of the final integrated context sentence information is obtained. The calculation formula is as follows, where s p，f represents the final fusion feature, s i Indicates the current sentence features, s p and f Represent the features of the previous sentence and the next sentence respectively, w T , and are all trainable parameter matrices, σ is the activation function, and b is the bias vector:

[0095] s p，f =softmax(f(s i ,s p ))s p +softmax(f(s i ,s f ))

[0096]

[0097] The context sentence features are fused with the current sentence features through a gating mechanism to obtain the final vocabulary feature representation:

[0098] si =λ·s i +(1-λ)·s p，f

[0099]

[0100] In the above formula, λ is used as a gating parameter to control the fusion strength of the current sentence and the adjacent sentences. and are three trainable parameter matrices, the subscript g represents the training parameters of the gate parameter λ, and finally, the word embedding feature representation e obtained in step 2.1 is i , the vocabulary feature representation W obtained in step 2.2 i , and the sentence feature representation S obtained in step 2.3 i , and fused into the final vocabulary feature representation in a concatenated manner.

[0101] See also Figure 3 , Figure 3 A flowchart of a feature extraction method based on document modeling provided in an embodiment of the present application. The feature extraction module based on document modeling mainly performs modeling feature extraction at three levels, namely, word embedding layer, vocabulary layer, and sentence layer. The word embedding layer obtains feature representations of corresponding concentrations by directly training a probability model, the vocabulary level performs vocabulary enhancement by introducing a flat-lattice structure, and then fully integrates the relative position information between words through the XLNet attention layer, while the sentence level models adjacent sentences through contextual relationships of a specific window size, and finally integrates them into the current sentence representation based on a gating mechanism.

[0102] See also Figure 4 , Figure 4 A flowchart of Chinese word extraction based on graph neural network provided in the embodiment of the present application is mainly divided into three steps. First, local and global information enhancement is performed by BiLSTM model and document graph construction, then encoding is performed by global GNN, and finally label decoding is performed by CRF. As a further introduction to the above embodiment, step 3 may include the following steps:

[0103] Step 3.1: Based on the feature representation extracted in step 2, build a BiLSTM model to obtain the feature representation h after local information enhancement i .

[0104] Step 3.2: Use words as nodes and whether the same entity appears between sentences as the criterion for establishing adjacent edges to construct the document graph G corresponding to the document sample. D = {V D , E D}, where V Drepresents a node set, E D represents the set of adjacent edges;

[0105] Step 3.3: Based on the document graph, calculate the feature representation a after global information enhancement i , the formula is as follows, N(i) represents the set of all adjacent nodes of the current node, W a and b a are trainable parameters:

[0106]

[0107] This embodiment can construct a special word extraction framework model based on the global GNN network (i.e., document graph) as an encoder and CRF as a decoder; input the training data into the model for training, and save the model.

[0108] As a further introduction to the above embodiment, the evaluation indicators include precision, recall and comprehensive evaluation indicator F1, which are calculated as follows, where TP indicates that a positive sample is predicted to be a positive sample, FP indicates that a negative sample is predicted to be a positive sample, and FN indicates that a positive sample is predicted to be a negative sample.

[0109] P(precision) = TP / (TP+FP);

[0110] R(Recall) = TP / (TP+FN);

[0111] F1 (comprehensive evaluation index) = 2PR / (P+R).

[0112] This embodiment can serve as the underlying support for many upper-level applications. For example, when intelligent customer service in a professional field conducts document Q&A, the user's intention can be accurately captured based on the entities extracted from the document. In addition, when building a knowledge graph in a specific field, many internal company documents are at the document level. Here, it is necessary to extract the relationships between entities based on the document-level data.

[0113] See also Figure 5 , Figure 5 A schematic diagram of the structure of a Chinese word extraction system provided in an embodiment of the present application, the system may include:

[0114] The tagging module 501 is used to obtain Chinese document data, and pre-process and tag the Chinese document data to obtain a training corpus data set;

[0115] A feature extraction module 502 is used to extract embedding layer features, vocabulary layer features and sentence layer features of the Chinese document data respectively, and concatenate the embedding layer features, the vocabulary layer features and the sentence layer features to obtain a vocabulary feature representation of the Chinese document data;

[0116] A model building module 503 is used to build a BiLSTM model and a document graph according to the lexical feature representation; and is also used to build a special word extraction framework model using the BiLSTM model and the document graph; wherein the editor of the special word extraction framework model is the document graph, and the decoder is CRF;

[0117] A training module 504 is used to input the training corpus data set into the specific word extraction framework model for training;

[0118] The special word extraction module 505 is used to extract Chinese special words from the unknown Chinese document data by using the trained special word extraction framework model if unknown Chinese document data is received.

[0119] This embodiment extracts the embedding layer features, vocabulary layer features and sentence layer features of Chinese document data, and then splices the embedding layer features, vocabulary layer features and sentence layer features of Chinese document data to obtain the vocabulary feature representation of Chinese document data, and then uses the vocabulary feature representation to build a BiLSTM model and document graph. The BiLSTM model can enhance the local information of vocabulary features, and the document graph can enhance the global information of vocabulary features. The special word extraction framework model constructed using the BiLSTM model and document graph can obtain global semantic understanding. This embodiment can reduce the extraction error of Chinese special words and improve the accuracy of entity recognition.

[0120] Furthermore, the annotation module 501 is used to preprocess the Chinese document data; it is also used to perform special word annotation on the preprocessed Chinese document data using the BIO annotation method; it is also used to divide the Chinese document data after special word annotation into the training corpus data set and the test corpus data set according to a preset ratio.

[0121] Furthermore, it also includes:

[0122] The evaluation module is used to verify the trained special word extraction framework model using the test data after the training corpus data set is input into the special word extraction framework model for training, so as to obtain a model evaluation result.

[0123] The further feature extraction module 502 comprises:

[0124] An embedding layer feature extraction unit, used to construct a word embedding probability model, and use the word embedding probability model to obtain the embedding layer features of the Chinese document data;

[0125] A vocabulary layer feature extraction module, used to construct a flat_lattice structure based on a special word list, and establish a vocabulary relative position encoding matrix corresponding to the flat_lattice structure; further used to construct an XLNet attention network layer according to the vocabulary relative position encoding matrix; and further used to extract vocabulary layer features of the Chinese document data using the XLNet attention network layer;

[0126] The sentence-level feature extraction module is used to fuse the features of context sentence information in the Chinese document data according to a preset sentence window to obtain the sentence-level features of the Chinese document data.

[0127] Furthermore, the model construction module 503 is used to construct a BiLSTM model based on the lexical feature representation so as to obtain a lexical feature representation after local information enhancement; it is also used to construct an initial document graph corresponding to the Chinese document data with vocabulary as nodes and whether entities appear between sentences as the judgment criteria for establishing adjacent edges, and use the lexical feature representation to train the initial document graph to obtain the document graph so as to obtain a lexical feature representation after global information enhancement.

[0128] Since the embodiments of the system part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the system part, which will not be repeated here.

[0129] The present application also provides a storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0130] The present application also provides an electronic device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the electronic device may also include various network interfaces, power supplies and other components.

[0131] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0132] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A Chinese word extraction method, characterized in that: include: Acquire Chinese document data, and perform preprocessing and word tagging on the Chinese document data to obtain a training corpus data set; Extracting embedding layer features, vocabulary layer features and sentence layer features of the Chinese document data respectively, and concatenating the embedding layer features, the vocabulary layer features and the sentence layer features to obtain a vocabulary feature representation of the Chinese document data; Constructing a BiLSTM model and a document graph based on the vocabulary feature representation; A special word extraction framework model is constructed using the BiLSTM model and the document graph; wherein the editor of the special word extraction framework model is the document graph, and the decoder is CRF; Inputting the training corpus data set into the special word extraction framework model for training; If unknown Chinese document data is received, the trained special word extraction framework model is used to extract Chinese special words from the unknown Chinese document data; wherein the unknown Chinese document data is a Chinese document without special words extracted; Wherein, constructing a BiLSTM model and a document graph according to the vocabulary feature representation includes: Building a BiLSTM model based on the vocabulary feature representation to obtain the vocabulary feature representation after local information enhancement; An initial document graph corresponding to the Chinese document data is constructed with words as nodes and whether entities appear between sentences as the judgment criteria for establishing adjacent edges. The document graph is obtained by training the initial document graph using the word feature representation to obtain a word feature representation enhanced with global information.

2. The Chinese word extraction method according to claim 1, characterized in that: The Chinese document data is preprocessed and labeled with specific words to obtain a training corpus data set, including: Preprocessing the Chinese document data; Using BIO annotation method to perform special word annotation on the pre-processed Chinese document data; The Chinese document data after the special word annotation is divided into the training corpus data set and the test corpus data set according to a preset ratio.

3. The Chinese word extraction method according to claim 2, characterized in that: After the training corpus data set is input into the special word extraction framework model for training, the method further includes: The trained word extraction framework model is verified using test data to obtain the model evaluation results.

4. The Chinese word extraction method according to claim 1, characterized in that: Extracting the embedding layer features of the Chinese document data includes: A word embedding probability model is constructed, and the embedding layer features of the Chinese document data are obtained using the word embedding probability model.

5. The Chinese word extraction method according to claim 1, characterized in that: Extracting vocabulary-level features of the Chinese document data includes: Constructing a flat_lattice structure based on a special word list, and establishing a vocabulary relative position encoding matrix corresponding to the flat_lattice structure; Constructing an XLNet attention network layer according to the vocabulary relative position encoding matrix; The XLNet attention network layer is used to extract vocabulary-level features of the Chinese document data.

6. The Chinese word extraction method according to claim 1, characterized in that: Extracting sentence-level features of the Chinese document data includes: The features of context sentence information are integrated in the Chinese document data according to a preset sentence window to obtain sentence-level features of the Chinese document data.

7. A Chinese word extraction system, characterized in that: include: The tagging module is used to obtain Chinese document data, and pre-process and tag the Chinese document data to obtain a training corpus data set; A feature extraction module, used to extract embedding layer features, vocabulary layer features and sentence layer features of the Chinese document data respectively, and concatenate the embedding layer features, the vocabulary layer features and the sentence layer features to obtain a vocabulary feature representation of the Chinese document data; A model building module, used to build a BiLSTM model and a document graph according to the lexical feature representation; and also used to build a special word extraction framework model using the BiLSTM model and the document graph; wherein the editor of the special word extraction framework model is the document graph, and the decoder is CRF; A training module, used for inputting the training corpus data set into the special word extraction framework model for training; A special word extraction module is used to extract Chinese special words from the unknown Chinese document data by using the trained special word extraction framework model if unknown Chinese document data is received; wherein the unknown Chinese document data is a Chinese document without special words extracted; Among them, the process of the model construction module constructing a BiLSTM model and a document graph according to the vocabulary feature representation includes: constructing a BiLSTM model according to the vocabulary feature representation to obtain a vocabulary feature representation after local information enhancement; using vocabulary as nodes and whether entities appear between sentences as judgment criteria for establishing adjacent edges, constructing an initial document graph corresponding to the Chinese document data, and using the vocabulary feature representation to train the initial document graph to obtain the document graph, so as to obtain a vocabulary feature representation after global information enhancement.

8. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the Chinese special word extraction method as claimed in any one of claims 1 to 6 when calling the computer program in the memory.

9. A storage medium, characterized in that: The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, the steps of the Chinese special word extraction method as claimed in any one of claims 1 to 6 are implemented.

Images