A method and electronic device for extracting entity attributes from colloquial short text

Abstract

The present invention provides a method and electronic device for extracting entity attributes of a spoken short text, including segmenting and part-of-speech tagging of the spoken short text, and performing naming subject recognition on each part-of-speech-tagged word to obtain entity words; mapping the spoken short text to a subject-predicate-object triple, obtaining the dependency relationship of each word in the subject-predicate-object triple, and using entity words to perform entity recognition on each word in the subject-predicate-object triple; when the subject-predicate-object in the subject-predicate-object triple satisfies a trigger rule, extracting the object phrase as an entity attribute. The present invention adopts a comprehensive method of part-of-speech tagging, dependency syntactic analysis, entity recognition, and combining trigger word part-of-speech rules to more specifically extract entity attribute information of spoken short data, enriching the structured information extraction method in the field of spoken short text.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a method and an electronic device for extracting entity attributes of a spoken short text. Background Art

[0002] With the rapid development of Internet applications, the amount of text data that can be obtained through the network has also grown exponentially. How to quickly and accurately analyze truly useful information from these massive text data is particularly critical and urgent, and this is exactly the problem that the research field of information extraction is trying to solve. The task of information extraction is to accurately and quickly obtain target information from a large amount of data and improve the utilization rate of information. The main function of information extraction is to extract specific factual information from text, which can be structured, semi-structured or unstructured data. Usually, information extraction uses machine learning, natural language processing (NLP) and other methods to extract specific information from the above text and save it in a structured database for user query and use. The specific implementation methods of information extraction can be divided into two categories: rule-based methods and statistical-based methods. The specific implementation process of information extraction depends on machine learning algorithms to a certain extent. In recent years, breakthroughs in some aspects of machine learning algorithms have provided direct support for the advancement of key technologies in information extraction.

[0003] Information extraction methods based on manually formulated rules have promoted significant progress in information extraction. The research results of information extraction for structured and semi-structured texts can basically achieve a high accuracy rate, and the extraction methods and processes are relatively easy. Early researchers usually created grammar-based language models and used manually written trigger word rules or grammars for qualitative information extraction. This method achieved good accuracy in many specific corpus contexts.

[0004] Dependency parsing is a method of analyzing the components of a sentence using statistical machine learning. Since it is a text mining method based on dependency parsing, it is necessary to explain the basic concepts of dependency parsing. For example, the result of dependency parsing of the sentence "I just ate an apple" is as follows: Figure 1 As shown. In the structure of dependency analysis, there is a direct dependency relationship between words, forming a dependency pair. Of the two words in a dependency pair, one is a core word, also called a dominant word; the other is a modifier, also called a subordinate word. The dependency relationship is represented by a directed arrow pointing to the father node of the word. In the present invention, the direction of the arrow is specified to be from the subordinate word to the dominant word. Figure 1 In the figure, each arrow has a label, called the relationship type, which indicates the dependency relationship between the two words in the dependency pair.

[0005] Named Entity Recognition (NER) is a fundamental task in information extraction. Its mission is to identify content such as person names, organization names, dates, times, locations, specific numerical forms, etc. from text and add corresponding annotation information, facilitating subsequent information extraction work. Given the inefficiency of rule-based methods, and the fact that writing rules usually takes domain experts several months or even longer to complete with unsatisfactory recall rates, people considered introducing statistical models into the implementation of named entity recognition, obtaining a domain knowledge base using machine learning methods, and then analyzing test texts. This approach has achieved good results, making up for the deficiencies of the original method to a certain extent. The research on named entity recognition is becoming increasingly mature, and the current research focus is on the application stage.

[0006] Because rule-based information extraction methods have their own limitations, such as: the process of manually formulating rules is complex, the rules obtained through machine learning are inefficient, and the system has poor generality, etc., subsequent research gradually turned back to statistical methods. Although statistical information extraction can make up for the defects of rule-based methods to a certain extent, with the in-depth research, people found that statistical methods are not perfect. At the same time, general and single-structured algorithms cannot be applied to the extraction of structured information from all domain texts. Especially for the data structure of specific topic corpora being special, a single method cannot extract more accurate and rich entity attribute information well in a production environment. Spoken short texts often contain a lot of unstructured information, but the data structure patterns are very different from those of standard written data, and the specific manifestations are as follows:

[0007] 1. Oral style: When communicating, both parties are often not as formal and rigorous as written expressions. The sentence pattern is a response mode, that is, one sentence is an information statement by one party, and the other party is the listener. The communication sentence pattern is also relatively single and casual.

[0008] 2. More noise: Spoken language corpora have more noise and obvious patterns, such as containing many meaningless words that only represent patterns, such as 'um', 'hello'.

[0009] 3. Shorter data length: Generally, the overall data length of spoken text data is shorter, usually only a few short sentences, and each word needs to be analyzed in detail during the information extraction process.

[0010] 4. High information concentration: In spoken texts, most of the time, the purpose is to convey information. In a short text length, when describing a certain object such as anything, there are relatively few rhetorical and redundant components. Therefore, there is a situation of high information concentration in a single sentence, and the information in a single sentence needs to be fully mined.

[0011] Although the spoken speech recognition method based on statistical models and grammatical rules of Chinese patent application CN101604520 uses multiple models to recognize spoken speech, it ignores the internal connection between the words in the sentence and has limited extraction accuracy.

[0012] In summary, there is an urgent need for a strategy that combines rule-based and statistical-based methods to find a more effective information extraction solution from these massive text data in different production environments. Summary of the invention

[0013] In view of the above problems, the present invention proposes a method and an electronic device for extracting entity attributes of spoken short texts, which adopt entity recognition, dependency syntax analysis, trigger word rules and other methods to jointly extract information, combine machine learning algorithms and formulate trigger word rule libraries, improve the richness and accuracy of structured information extraction of target data, and study Chinese trigger words and rules for corpora in specific environments. At the same time, they are modified according to data characteristics and applied to specific production environments in the later stage, thereby solving the problem that entity attribute information is difficult to extract due to the large number of noise words, short data length and high information concentration in spoken short text data, and achieving the purpose of extracting structured entity attribute information from spoken short texts.

[0014] Content of the present invention is as follows:

[0015] A method for extracting entity attributes from a spoken short text, the steps of which include:

[0016] 1) Segment and tag the colloquial short text, and identify the named subject of each tagged word to obtain the entity word;

[0017] 2) Map the colloquial short text into subject-predicate-object triples, obtain the dependency relationship of each word in the subject-predicate-object triples, and use entity words to perform entity recognition on each word in the subject-predicate-object triples;

[0018] 3) When the subject, predicate and object in the subject-predicate-object triplet meets a trigger rule, the object phrase is extracted as the entity attribute.

[0019] Furthermore, by deleting colloquial words and noise words without actual meaning in the colloquial short sentences, a colloquial short text is obtained; and by segmenting the colloquial long sentences by punctuation marks, a colloquial short sentence is obtained.

[0020] Furthermore, part-of-speech tagging is performed through the following steps:

[0021] 1) Segment the colloquial short text into words to obtain several words;

[0022] 2) Input each word into a hidden Markov labeling model trained with a number of sample words to label each word.

[0023] Further, parts of speech include adjectives, nouns, adverbs, verbs, conjunctions, and pronouns.

[0024] Furthermore, the named subject recognition is performed by inputting each word with a part of speech mark into an entity recognition model trained by a number of sample words and the CRF++ toolkit.

[0025] Furthermore, entity words include names of people, places and institutions.

[0026] Furthermore, the dependency relationship of each word in the subject-verb-object triple is obtained through the HanLP language processing tool.

[0027] Furthermore, the triggering rules include: the subject is a personal pronoun, the predicate is a controlling word of the verb part of speech, and the subject-object relationship of the controlling word is a noun phrase, or the subject is a personal pronoun, the predicate is an interrogative verb, and the object phrase contains a place noun.

[0028] A storage medium stores a computer program, wherein the computer program is configured to execute the method described above when running.

[0029] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer to execute the method described above.

[0030] Compared with the existing methods, the present invention has the following advantages:

[0031] 1) Perform differentiated and purposeful noise removal and sentence segmentation on colloquial short texts to reduce the interference of noise on the model and reduce the omission of important information;

[0032] 2) A joint entity attribute extraction method combining algorithms and rules is proposed. The comprehensive method of part-of-speech tagging, dependency syntactic analysis, entity recognition and trigger word part-of-speech rules is adopted to more specifically extract the entity attribute information of spoken short data, enriching the structured information extraction method in the field of spoken short text. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is an example diagram of the dependency parsing results of the prior art.

[0034] Figure 2 This is a network structure diagram of dependency parsing based on neural networks.

[0035] Figure 3 This is a schematic diagram of the CRF model

[0036] Figure 4This is a flow chart of training entity recognition model based on CRF++ tool

[0037] Figure 5 It is a flow chart of entity attribute information extraction from spoken short text

[0038] Figure 6 It is a schematic diagram of part-of-speech tagging and dependency syntactic analysis of a sentence in an embodiment of the present invention.

[0039] Figure 7 It is a schematic diagram of part-of-speech tagging and dependency syntactic analysis of another sentence in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, principle, technical solution and advantages of the present invention more clearly understood, the present invention will be described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0041] The present invention provides a method for extracting entity attributes from a spoken short text, comprising four aspects:

[0042] In the first aspect, we first removed specific noise words from the colloquial short text, and then performed word segmentation and part-of-speech tagging. For this special type of text, its noise words often show their own unique rules, so we specify corresponding rules to remove the frequently appearing high-probability noise words, which reduces the adverse interference in the information extraction process to a certain extent. At the same time, we use the hidden Markov tagging model to tag the word's part of speech, which includes adjectives, nouns, adverbs, verbs, conjunctions, pronouns and other parts of speech. The hidden Markov model (HMM) is a statistical analysis model that was created in the 1970s. The hidden Markov model is a probabilistic model about time series, which describes the process of randomly generating an unobservable random sequence of states by a hidden Markov chain, and then generating an observation from each state to generate a random sequence of observations. The sequence of states randomly generated by the hidden Markov chain is called a state sequence; each state generates an observation, and the random sequence of observations generated is called an observation sequence. At the same time, each position in the sequence can be regarded as a moment, so the hidden Markov model can be used for part-of-speech tagging tasks.

[0043] The second aspect is to analyze the sentence component structure in a targeted manner. Map the core information of a single sentence in a short spoken text into a structured information form of a triple [subject center word, predicate, object center word]. Dependency syntactic analysis part: To perform dependency syntactic analysis on the sentence and extract the sentence information of the training corpus, the sentence analysis level uses an open source language processing tool for production environments - HanLP, and uses a high-performance dependency syntactic analyzer based on a neural network in the tool. Different from the workshop-style of manually writing feature templates of traditional syntactic analyzers, the high-performance dependency syntactic analyzer of the neural network directly passes all the information of the sentence into the neural network model as input. Figure 2 As shown in the figure, the model includes input layer, hidden layer and softmax layer. The activation function of the hidden layer is special, using cube activation function, which is to take the cube. A configuration includes Stack, Buffer, and dependency arc set. Figure 2 As shown at the bottom, this is a specific configuration, which contains information about stack, buffer, and dependency arc set. The model extracts a feature vector based on the configuration information. This feature vector is composed of three vectors (words, POS tags, arc labels). This vector therefore contains the information of the corresponding configuration. The goal of the model is to input the feature vector and then predict the corresponding conversion type. After predicting the conversion type, the corresponding conversion operation is performed, so that the configuration information is updated, and then a new vector is obtained, which is then input into the model for prediction. In this way, the cycle is repeated, and finally the dependency arc set is obtained to find the dependency relationship in the sentence.

[0044] The way they are combined is no longer determined by manually written feature templates, but is automatically extracted by the hidden layer of the neural network model. Triple structured information extraction part: Combined with the spoken short text data, the subject, predicate, object and other sentence trunks after sentence analysis are extracted and output in the form of triple structures to prepare for further extraction of corresponding sentence information.

[0045] The third aspect is to perform entity recognition on each component of the triple. Named entity recognition is performed based on conditional random fields, such as Figure 3As shown, CRF (conditional random field) is a conditional probability distribution model of another set of output random variables under the condition of a given set of input random variables. Its characteristic is that it assumes that the output random variables constitute a Markov random field. A simpler conditional random field is a conditional random field defined on a linear chain, called a linear chain conditional random field. Linear chain conditional random fields can be used for problems such as sequence labeling, and the named entity recognition (NER) task that the present invention needs to solve can be solved by a sequence labeling method. In the conditional probability model P(Y|X), Y is the output variable, representing the tag sequence (or state sequence), and X is the input variable, representing the observation sequence that needs to be labeled. During learning, the conditional probability model p(Y|X) is obtained by maximum likelihood estimation or regularized maximum likelihood estimation using the training data set; during prediction, for a given input sequence x, the output sequence y0 with the largest conditional probability p(y|x) is obtained. As shown Figure 4 As shown, the present invention adopts the CRF++ toolkit and uses the open source People's Daily 1998 Chinese annotated corpus to train the conditional random field model to obtain an entity recognition model that can directly transfer and apply colloquial short texts. Finally, the model is applied to text data to identify entities such as names of people, places, and institutions contained in sentences.

[0046] The fourth aspect is the formulation of trigger word rules. Based on entity recognition, dependency syntactic analysis and trigger word part-of-speech rules, the extraction of entity attribute information related to unstructured data is completed. After these colloquial short texts are segmented, POS tagged and dependency syntactic analysis is performed, the next step is to judge the behavior of the subject by combining the trigger word rules and the dependency syntactic analysis results. The trigger word rule matching method is used here. For example, subject + predicate + object is one of the most common sentence patterns. Based on dependency syntactic analysis, a subject-predicate-object triple can be proposed. When a short text sentence is given, if it is necessary to judge whether it contains specific behavior information, when the syntactic analysis result satisfies the rule that the subject of the triple is a personal pronoun, the predicate is a dominant word of the verb part of speech, and the subject-object relationship of the dominant word is a noun phrase, the object phrase is judged to be text topic information; for example, when the subject of the triple is a personal pronoun, the predicate is an interrogative verb (such as "ask"), and the object phrase contains a place noun, the object phrase is judged to be topic information, which can also be called event information.

[0047] like Figure 5As shown in the figure, the overall process of extracting entity attribute information from colloquial short texts is presented. First, the corpus is preprocessed, including sentence segmentation based on punctuation marks, removing colloquial, meaningless noise words and sentence segmentation. Secondly, the trained part-of-speech tagging model is used to tag the corpus with parts of speech, identify the word parts of each component in the sentence, and prepare for the final trigger word rules. Then, the sentence is analyzed using the trained dependency syntax analysis model to find the core relation words, subject-predicate relation words, verb-object relation words, and attributive-central relation modifiers of verb-object relation words. These sentence components together constitute the backbone of a sentence, and the sentence triple information ["subject pronoun", "predicate verb", "object phrase"] can be extracted. At the same time, the trained named entity recognition model is used to identify the named entities contained in the sentence components. Finally, corresponding trigger word rules are specified for different entity attributes to extract the corresponding attribute information.

[0048] For example, in the sentence "Hello, I'm Li Si, a reporter from Broadcasting Company B. I'd like to ask you about the street explosion accident at place A" (the sample data is written according to the short text sentence structure, and this embodiment assumes that place A is an entity name), the relevant entity attribute information contained therein is extracted, and the specific steps are as follows.

[0049] Data preprocessing: Targetedly remove noise data, such as "Hello", segment the sentences with commas: "Hello", "I am Li Si, a reporter from Broadcasting Company B", "I want to ask you about the street explosion accident in place A", and then analyze the sentences after segmentation.

[0050] Dependency syntactic analysis: "Hello" is not short and cannot be analyzed for sentence structure, so it contains too little information and is discarded. The following is a schematic diagram of "I am Li Si, a reporter from Broadcasting Company B" after part-of-speech tagging and dependency syntactic analysis: Figure 6 The diagram of "I want to ask you about the street explosion accident in place A" after part-of-speech tagging and dependency syntax analysis is as follows Figure 7 As shown in the analysis results, it can be seen that each sentence has only one core word, namely <root>As a parent node, <root>The words "be" and "want to" with the core relationship, the doer is the personal pronoun "I", which has a subject-predicate relationship with the core verb. "Li Si" and "accident" are the core object words of the whole sentence.

[0051] Structured triple information extraction: For the sentence "I am Li Si, a reporter from B Broadcasting Company", the subject "I" is extracted in sequence, the predicate verb is "be", and since the object word "Li Si" contains multiple attributive relationships, it is extracted simultaneously to form the object noun phrase "Li Si, a reporter from B Broadcasting Company" for a complete information description; for the sentence "I want to ask you about the explosion accident on the street in Area A", the subject "I" is extracted in sequence, the predicate is composed of two verbs, namely "want to ask", and since the object word "accident" contains multiple attributive relationships, it is extracted simultaneously to form the object noun phrase "the explosion accident on the street in Area A" for a complete information description.

[0052] Named entity recognition: Since the extracted triple information contains a lot of entity information, the words are also identified for names, locations, and organizations. The name "Li Si", the location "Area A", and the organization name "B Broadcasting Company" are identified in the above data.

[0053] Trigger word rule formulation: For the sentence "I am Li Si, a reporter from B Broadcasting Company", the triple ["I", "be", "Li Si, a reporter from B Broadcasting Company"] is proposed. The judgment rule is that the subject is the first-person pronoun "I", the predicate is a single verb, and if the object phrase contains a name entity, it can be judged that the caller's name is ["Li Si"], which conforms to the data characteristics, so the judgment is relatively accurate; at the same time, analyzing the object noun phrase ["Li Si, a reporter from B Broadcasting Company"], the organization entity and the name have an attributive relationship and are the modifying components of the name, so the subordinate organization of this name is judged, so the caller's organization is ["B Broadcasting Company"]; at the same time, the noun "reporter" is the direct modifier of the name. Combining the part of speech and the distance, "reporter" is judged to be the identity word of this name. Extraction result: Caller: Li Si, Caller identity: Reporter, Caller organization: B Broadcasting Company.

[0054] For the sentence "I want to ask you about the explosion on the street in A", a triple ["I" "I want to ask", "explosion on the street in A"] is proposed. The judgment rule is that the subject is the first-person pronoun "I", the predicate is a verb, and the object is a long phrase. The object is most likely to be a topic event described by the personal pronoun. At the same time, because the personal pronoun is the first person, the topic event is the event described by the subject, that is, the topic event being discussed, which conforms to the characteristics of spoken short data, so the judgment is more accurate; at the same time, a secondary judgment is made to analyze the object noun phrase ["explosion on the street in A"]. The phrase contains the location entity "A", the verb "explosion" and the noun "accident", forming a descriptive phrase, further increasing the probability of being a topic event. The extraction result shows that the event is "explosion on the street in A" and the location of the event is "A".

[0055] This sentence, as a typical sentence example, elaborates in detail the process of extracting structured information from spoken short texts. The entire model framework includes but is not limited to this comprehensive judgment rule. However, this example has explained the technical path and ideas of the model, so other examples will not be shown one by one.

[0056] The above embodiments are only for better illustrating the purpose, principle, technical solution and beneficial effect of the present invention. It should be understood that the above embodiments are only specific embodiments of the present invention and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.< / root> < / root>

Claims

1. A method for extracting entity attributes from a spoken short text, the steps comprising: 1) After word segmentation of the colloquial short text, the hidden Markov tagging model is used to perform part-of-speech tagging, and the named subject recognition is performed on each part-of-speech-tagged word based on the conditional random field to obtain the entity word; 2) Based on the HanLP language processing tool, the colloquial short text is mapped into subject-predicate-object triples, and the dependency relationship of each word in the subject-predicate-object triples is obtained, and entity words are used to perform entity recognition on each word in the subject-predicate-object triples; 3) When the subject, predicate and object in the subject-predicate-object triplet satisfies a trigger rule, entity attribute information of the colloquial short text is obtained; wherein the trigger rule includes: When the subject of the triple is a personal pronoun, the predicate is a dominant word of the verb part of speech, and the verb-object relationship of the dominant word is a noun phrase, it is determined that the object phrase is entity attribute information of the spoken short text; and, When the subject of the triple is a personal pronoun, the predicate is an interrogative verb, and the object phrase includes a place noun, the object phrase is determined to be entity attribute information of the spoken short text.

2. The method according to claim 1, characterized in that By deleting colloquial words and meaningless noise words in colloquial short sentences, a colloquial short text is obtained; by segmenting colloquial long sentences based on punctuation marks, a colloquial short sentence is obtained.

3. The method according to claim 1, characterized in that Parts of speech include adjectives, nouns, adverbs, verbs, conjunctions, and pronouns.

4. The method according to claim 1, characterized in that Entity words include names of people, places and institutions.

5. A storage medium storing a computer program, wherein: The computer program is configured to execute the method according to any one of claims 1 to 4 when executed.

6. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 4.

Images