Noun phrase extraction method, system, storage medium and electronic device
By using the spacy model and entity recognition service, combining dependency relationships and extraction rules, extracting and integrating noun phrases in professional fields, the problem of incomplete extraction of noun phrases in the existing technology is solved, and a more accurate and rich noun phrases in professional fields is achieved. extract.
Patent Information
- Application Number
- CN202011414360.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-07
- Publication Date
- 2025-02-21
- Estimated Expiration
- 2040-12-07
AI Technical Summary
It is difficult for the prior art to effectively extract noun phrases in the professional field, resulting in fragmentation of individual words and incomplete expression of the meaning of words in the professional field.
The dependencies of the text to be extracted are extracted through the Chinese model of spacy, combined with entity recognition services, extraction rules are formulated, noun phrase sequences are extracted from the token sequence, and head-tail integration and entity merging are carried out to obtain the final noun phrase sequence.
It has achieved the extraction of richer and more accurate noun phrases from the perspective of professional fields, overcome the shortcomings of simple word participle results extraction, and can extract noun phrases containing special verb structures.
Smart Images

Figure CN112560488B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of text processing, and in particular to a method, system, storage medium and electronic device for extracting noun phrases oriented to professional fields. Background Art
[0002] Chinese word segmentation recombines continuous character sequences into word sequences according to certain specifications. Common word segmentation is generally based on general fields. Keyword extraction is to extract words that can express the core of the text from word sequences according to certain principles. It is performed on the word segmentation results in general fields and can only extract a series of single words. Single words are still fragmented and are not enough to express the complete meaning of words in professional fields. For example, "knowledge", "graph", "active", "learning", "technology", these single words cannot clearly express the "knowledge graph" and "active learning technology" in professional fields. Therefore, it is urgent to develop a method, system, storage medium and electronic device for extracting noun phrases for professional fields that overcome the above defects. Summary of the invention
[0003] In view of the above problems, the present invention provides a method for extracting noun phrases, which comprises:
[0004] Extraction step: extract the dependency relationship of the text to be extracted through the spacy Chinese model to obtain the token sequence, and identify the entities in the text to be extracted through the entity recognition service;
[0005] Extraction step: formulating extraction rules according to the dependency relationship, and extracting a noun phrase sequence from the token sequence according to the extraction rules;
[0006] Integration step: after judging the noun phrase sequence, integrating the noun phrases in the noun phrase sequence head to tail according to the judgment result to obtain a standard noun phrase sequence;
[0007] Merging step: merging the entity with the standard noun phrase sequence to obtain a final noun phrase sequence.
[0008] The above extraction method, wherein the extraction step comprises:
[0009] Dependency extraction step: performing dependency analysis on the text to be extracted through the Chinese model to obtain the token sequence with attribute values;
[0010] Entity acquisition step: identifying entities in the text to be extracted through the entity recognition service.
[0011] The above extraction method, wherein the extraction step comprises:
[0012] Length identification step: identifying the length of the token sequence and outputting the identification result;
[0013] Sequence output step: formulating the extraction rule according to the recognition result, and extracting the noun phrase sequence from the token sequence according to the extraction rule.
[0014] In the above extraction method, the integration step includes: integrating the token sequences in the noun phrase sequence that are greater than a threshold value from head to tail to obtain the standard noun phrase sequence.
[0015] In the above extraction method, the merging step includes: merging the standard noun phrase sequence and the entity according to a merging rule to obtain a final noun phrase sequence.
[0016] The present invention also provides a noun phrase extraction system, which includes:
[0017] The extraction unit extracts the dependency relationship of the text to be extracted through the spacy Chinese model to obtain a token sequence, and identifies the entities in the text to be extracted through the entity recognition service;
[0018] An extraction unit, formulating an extraction rule according to the dependency relationship, and extracting a noun phrase sequence from the token sequence according to the extraction rule;
[0019] An integration unit, after judging the noun phrase sequence, integrates the noun phrases in the noun phrase sequence head to tail according to the judgment result to obtain a standard noun phrase sequence;
[0020] A merging unit merges the entity with the standard noun phrase sequence to obtain a final noun phrase sequence.
[0021] The above extraction system, wherein the extraction unit comprises:
[0022] A dependency extraction module, which performs dependency analysis on the text to be extracted through the Chinese model to obtain the token sequence with attribute values;
[0023] The entity acquisition module identifies entities in the text to be extracted through the entity recognition service.
[0024] The above extraction system, wherein the extraction unit comprises:
[0025] A length recognition module, which recognizes the length of the token sequence and outputs a recognition result;
[0026] A sequence output module formulates the extraction rule according to the recognition result, and extracts the noun phrase sequence from the token sequence according to the extraction rule.
[0027] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the extraction methods described above when executing the computer program.
[0028] The present invention also provides a storage medium on which a computer program is stored, wherein when the program is executed by a processor, the extraction method as described in any one of the above is implemented.
[0029] In summary, the effectiveness of the present invention compared with the prior art lies in that: the present invention utilizes the word segmentation results to formulate a series of rules, which can not only extract noun phrases with conventional meanings from the perspective of professional fields, but also extract noun phrases containing special verb structures; thus, compared with simply extracting noun words based on the word segmentation results, it can extract richer and more accurate noun phrases related to professional fields.
[0030] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0032] Figure 1 is a flow chart of a method for extracting noun phrases of the present invention;
[0033] Figure 2 for Figure 1 A step-by-step flow chart of step S1;
[0034] Figure 3 for Figure 1 The step-by-step flow chart of step S2;
[0035] Figure 4 is an application flow chart of the method for extracting noun phrases of the present invention;
[0036] Figure 5A schematic diagram of the dependency relationship;
[0037] Figure 6 It is a structural schematic diagram of the noun phrase extraction system of the present invention;
[0038] Figure 7 It is a schematic structural diagram of the electronic device of the present invention. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] The exemplary embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. In addition, elements / components with the same or similar reference numerals used in the drawings and embodiments are used to represent the same or similar parts.
[0041] The terms “first”, “second”, “S1”, “S2”, etc. used in this document do not particularly refer to an order or sequence, nor are they used to limit the present invention. They are only used to distinguish elements or operations described with the same technical terms.
[0042] The directional terms used in this document, such as up, down, left, right, front or back, etc., are only used to refer to the directions of the drawings. Therefore, the directional terms used are used to illustrate and not to limit the invention.
[0043] The terms “include,” “including,” “have,” “contain,” etc. used in this document are open-ended terms, meaning including but not limited to.
[0044] As used herein, "and / or" includes any or all combinations of the items described.
[0045] As used herein, “plurality” includes “two” and “more than two”; as used herein, “plurality groups” includes “two groups” and “more than two groups”.
[0046] The terms "substantially" and "approximately" used herein are used to modify any quantity or error that may vary slightly, but these slight changes or errors do not change their essence. Generally speaking, the range of slight changes or errors modified by such terms may be 20% in some embodiments, 10% in some embodiments, 5% in some embodiments, or other values. Those skilled in the art should understand that the aforementioned values can be adjusted according to actual needs and are not limited thereto.
[0047] Certain terms used to describe the present application are discussed below or elsewhere in this specification to provide additional guidance to those skilled in the art regarding the description of the present application.
[0048] The present invention aims to propose a method for extracting noun phrases in professional fields, which can also be applied to document tag generation, database information query and other fields.
[0049] Please refer to Figure 1-Figure 3 , Figure 1 is a flow chart of a method for extracting noun phrases of the present invention; Figure 2 for Figure 1 A step-by-step flow chart of step S1; Figure 3 for Figure 1 The step-by-step flow chart of step S2 in FIG. Figure 1-Figure 3 As shown, the method for extracting noun phrases of the present invention comprises:
[0050] Extraction step S1: extract the dependency relationship of the text to be extracted through the Chinese model of spacy to obtain a token sequence, and identify the entities in the text to be extracted through the entity recognition service, where spacy is a library for natural language processing, which can be used for word segmentation, named entity recognition, part of speech recognition, etc.
[0051] The extraction step S1 includes:
[0052] Dependency extraction step S11: performing dependency analysis on the text to be extracted using the Chinese model to obtain the token sequence with attribute values;
[0053] Entity acquisition step S12: identifying entities in the text to be extracted through the entity recognition service.
[0054] Extraction step S2: formulating extraction rules according to the dependency relationship, and extracting noun phrase sequences from the token sequence according to the extraction rules, wherein the noun phrase sequences include regular noun phrase sequences and noun phrase sequences containing special verb structures.
[0055] Wherein, the extraction step S2 comprises:
[0056] Length identification step S21: identifying the length of the token sequence and outputting the identification result;
[0057] Sequence output step S22: formulating the extraction rule according to the recognition result, and extracting the noun phrase sequence from the token sequence according to the extraction rule.
[0058] Integration step S3: After judging the noun phrase sequence, the noun phrases in the noun phrase sequence are integrated head to tail to obtain a standard noun phrase sequence according to the judgment result, wherein the integration step S3 includes: integrating the token sequence greater than a threshold in the noun phrase sequence head to tail to obtain the standard noun phrase sequence.
[0059] Merging step S4: merging the entity with the standard noun phrase sequence to obtain a final noun phrase sequence, wherein the merging step S4 includes: merging the standard noun phrase sequence and the entity according to a merging rule to obtain a final noun phrase sequence.
[0060] Please refer to Figure 4-Figure 5 , Figure 4 is an application flow chart of the method for extracting noun phrases of the present invention; Figure 5 is a schematic diagram of the dependency relationship. Figure 4-Figure 5 As shown, the following combination Figure 4-Figure 5 The method for extracting noun phrases of the present invention is described as follows through a specific embodiment.
[0061] 1. Text Parsing
[0062] Spacy is a library for natural language processing, which can be used for word segmentation, named entity recognition, part-of-speech recognition, etc. The present invention uses this library to perform dependency analysis on the text to be processed. After analysis, a token sequence (i.e., a word sequence) can be obtained, and each token contains fixed attributes. Table 1 is a comparison table of the specific meanings of the attributes contained in the token, and Table 2 is a comparison table of the specific meanings of the tag_ attribute in the token.
[0063] Table 1: Comparison table of the specific meanings of token attributes
[0064]
[0065] Table 2: Comparison table of the specific meanings of common tag_ attributes
[0066]
[0067]
[0068] For example, the text "I love natural language processing." is parsed by spacy's dependency relationship to obtain the token sequence shown in Table 3 (one line is a token, and the header is part of the token's attributes). The dependency structure is as follows: Figure 5 As shown, it is a directed graph with arrows pointing from the central word to the subordinates, that is, from the head to the child.
[0069] Table 3: Token sequence example
[0070]
[0071] 2. Extracting noun phrase sequences according to rules
[0072] After obtaining the dependency relationship of the text to be extracted through spacy parsing, a series of rules are formulated based on experience to extract the regular noun phrase sequence nn_nps:
[0073] 1. If the length of the parsed token sequence is 1 and the tag_ attribute value of the token is NN or NR, add [current token] as a token sequence to the noun phrase sequence nn_nps.
[0074] 2. If the length of the parsed token sequence is greater than 1, traverse all tokens and handle them according to the following different situations.
[0075] (1) When all of the following conditions are met, [current token] is added to nn_nps as a token sequence:
[0076] a) The tag_ attribute value of the current token is JJ / AD / CD;
[0077] b) The tag_ attribute value of the next token is NN / NR;
[0078] c) The head attribute value of the current token is equal to the i attribute value of the next token or the head attribute value of the current token is equal to the head attribute value of the next token.
[0079] (2) If the tag_ attribute value of the current token is NN / NR / FW, the following situations are handled:
[0080] a) If the current token is the first token, or the length of the noun phrase sequence nn_nps is 0, add [current token] as a token sequence to nn_nps;
[0081] b) If the tag_ attribute value of the previous token is not NN / NR / JJ / AD / CD, then add [current token] as a token sequence to nn_nps;
[0082] c) If the tag_ attribute value of the previous token is NN / NR / JJ / AD / CD:
[0083] If the length of the last token sequence of nn_nps is greater than 0 and the last token of the last token sequence of nn_nps is the previous token of the current token, then the current token is added to the last token sequence of np_list; otherwise, [the current token] is directly added to nn_nps as a token sequence.
[0084] A series of rules are empirically formulated to extract noun phrase sequences vv_nps containing special verb structures:
[0085] 1. If the length of the token sequence is greater than 3, traverse from the first token to the second last token and handle according to the following different situations:
[0086] (1) If the tag_ attribute value of the previous token is NN / AD, and the tag_ attribute value of the current token is VV, and the head attribute value of the previous token is the i attribute value of the current token, and the tag_ attribute value of the next token is DEC / PU, then add [previous token, current token] as a token sequence to the noun phrase sequence vv_nps.
[0087] (2) If the tag_attribute value of the previous token is VV, the tag_attribute value of the current token is NN, and the tag_attribute value of the next token is ETC, then [previous token, current token] is added as a token sequence to the noun phrase sequence vv_nps.
[0088] (3) If the tag_ attribute value of the previous token is JJ, and the tag_ attribute value of the current token is JJ, and the tag_ attribute value of the next token is NN, and the head attribute value of the previous token is equal to the head attribute value of the current token is equal to the i attribute value of the next token, then add [previous token, current token, next token] as a token sequence to the noun phrase sequence vv_nps.
[0089] 2. If the length of the token sequence is greater than 4, traverse from the first token to the third last token and handle according to the following different situations:
[0090] (1) If the tag_attribute value of the previous token is AS / VV / CC, and the tag_attribute value of the current token is AD, and the tag_attribute value of the next token is VV, and the tag_attribute value of the next-next token is NN, and the head attribute value of the current token is equal to the i attribute value of the next token, and the head attribute value of the next token is equal to the i attribute value of the next-next token, then add [current token, next token, next-next token] as a token sequence to the noun phrase sequence vv_nps.
[0091] (2) If the tag_attribute value of the previous token is DEC, and the tag_attribute value of the current token is NN, and the tag_attribute value of the next token is VV, and the tag_attribute value of the next-next token is NN, and the head attribute value of the current token is equal to the head attribute value of the next token is equal to the i attribute value of the next-next token, then add [current token, next token, next-next token] as a token sequence to the noun phrase sequence vv_nps.
[0092] (3) If the tag_attribute value of the previous token is CC / PU / P, and the tag_attribute value of the current token is NN, and the tag_attribute value of the next token is VV, and the tag_attribute value of the next token is CC / PU, then add [current token, next token] as a token sequence to the noun phrase sequence vv_nps.
[0093] The extracted regular noun phrase sequence nn_nps and the noun phrase sequence vv_nps containing the special verb structure are combined to obtain all noun phrase sequences extracted according to the given rules. From the above rules, it can be seen that the noun phrase sequence is composed of several token sequences, where the length of each token sequence is different, which may be 1 or greater than 1.
[0094] Among them, if the current token is the nth token, the previous one is the n-1th token, the next one is the n+1th token, and the next one is the n+2th token.
[0095] 3. Noun phrase sequence integration
[0096] From the previous step, we know that the noun phrase sequence extracted according to the rule is composed of token sequences of varying lengths. If the length of the token sequence included is greater than 1, these token sequences need to be integrated head-to-tail to obtain the integrated standard noun phrase sequence.
[0097] For example, [active, learning, technology], where the position of "active" in the text to be extracted starts with index i and ends with index i+2; "learning" starts with index i+2 and ends with index i+4; "technology" starts with index i+4 and ends with index i+6, then it is integrated into [active learning technology], where "active learning technology" starts with index i and ends with index i+6 in the text to be extracted.
[0098] 4. Entity Recognition Results and Noun Phrase Merging
[0099] The above processing steps have obtained the noun phrases extracted according to the rules, including the content of the noun phrases and the start and end indexes of the noun phrases in the original text. Use the entity recognition service to identify the entities in the text to be extracted, including the content of the entity and the start and end indexes of the entity in the original text. Finally, merge the extracted noun phrases and the recognized entities to obtain the final noun phrase sequence np_list. Traverse the noun phrases and entities from the beginning. The merging rules are as follows:
[0100] 1. If the start index of the current entity is less than the end index of the current noun phrase, then process it according to different situations:
[0101] (1) If the start index of the current entity is equal to the start index of the current noun phrase, add the current noun phrase to np_list. Then traverse the entities backward until the start index of the entity is greater than or equal to the end index of the current noun phrase, and then traverse one more noun phrase backward.
[0102] (2) If the start index of the current entity is not equal to the start index of the current noun phrase, add the current entity to np_list. Then traverse the noun phrases backward until the end index of the entity is less than or equal to the start index of the noun phrase, and then traverse one entity backward.
[0103] 2. If the start index of the current entity is greater than the end index of the current noun phrase, add the current noun phrase to np_list. Then traverse the entities backward until the start index of the entity is greater than or equal to the end index of the noun phrase, and then traverse one noun phrase backward.
[0104] Repeat the above rules until the noun phrase or entity traversal ends. If the noun phrase priority traversal ends, the remaining entities are added to np_list. If the entity priority traversal ends, the remaining noun phrases are added to np_list.
[0105] After completing all the above steps, the np_list obtained is the final noun phrase for the professional field.
[0106] Please refer to Figure 6 , Figure 6 6 is a schematic diagram of the structure of the noun phrase extraction system of the present invention. As shown in FIG6 , the noun phrase extraction system of the present invention includes:
[0107] The extraction unit 11 extracts the dependency relationship of the text to be extracted through the spacy Chinese model to obtain a token sequence, and identifies the entities in the text to be extracted through the entity recognition service;
[0108] An extraction unit 12 formulates an extraction rule according to the dependency relationship, and extracts a noun phrase sequence from the token sequence according to the extraction rule;
[0109] An integration unit 13, after judging the noun phrase sequence, integrates the noun phrases in the noun phrase sequence from head to tail according to the judgment result to obtain a standard noun phrase sequence;
[0110] The merging unit 14 merges the entity with the standard noun phrase sequence to obtain a final noun phrase sequence.
[0111] Wherein, the extraction unit 11 comprises:
[0112] A dependency extraction module 111 performs dependency analysis on the text to be extracted through the Chinese model to obtain the token sequence with attribute values;
[0113] The entity acquisition module 112 identifies entities in the text to be extracted through the entity recognition service.
[0114] Further, the extraction unit 12 includes:
[0115] A length recognition module 121 recognizes the length of the token sequence and outputs a recognition result;
[0116] The sequence output module 122 formulates the extraction rule according to the recognition result, and extracts the noun phrase sequence from the token sequence according to the extraction rule.
[0117] The integration unit 13 integrates the token sequences in the noun phrase sequence that are greater than a threshold value from head to tail to obtain the standard noun phrase sequence.
[0118] The merging unit 14 merges the standard noun phrase sequence and the entity according to a merging rule to obtain a final noun phrase sequence.
[0119] Please refer to Figure 7 , Figure 7 Schematic diagram of the structure of the electronic device of the present invention. Figure 7 As shown, this embodiment discloses a specific implementation of an electronic device. The electronic device may include a processor 81 and a memory 82 storing computer program instructions.
[0120] Specifically, the processor 81 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0121] Among them, the memory 82 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 82 may be inside or outside the data processing device. In a specific embodiment, the memory 82 is a non-volatile memory. In a specific embodiment, the memory 82 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (ErasableProgrammable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically ErasableProgrammable Read-Only Memory, EEPROM for short), an electrically alterable ROM (ElectricallyAlterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (Static Random-Access Memory, abbreviated as SRAM) or a dynamic random access memory (Dynamic Random Access Memory, abbreviated as DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (Fast Page Mode Dynamic Random Access Memory, abbreviated as FPMDRAM), an extended data output dynamic random access memory (Extended Date Out Dynamic Random Access Memory, abbreviated as EDODRAM), a synchronous dynamic random access memory (Synchronous Dynamic Random-Access Memory, abbreviated as SDRAM), etc.
[0122] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .
[0123] The processor 81 implements any one of the extraction methods in the above embodiments by reading and executing computer program instructions stored in the memory 82 .
[0124] In some embodiments, the electronic device may further include a communication interface 83 and a bus 80. Figure 7 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.
[0125] The communication interface 83 is used to realize the communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication port 83 can also realize data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage and image / data processing workstations.
[0126] The bus 80 includes hardware, software or both, and couples the components of the electronic device to each other. The bus 80 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 80 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.
[0127] In addition, in combination with the processing method in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the extraction methods in the above embodiment is implemented.
[0128] In summary, the present invention designs a rule-based, professional field-oriented noun phrase extraction method, system, storage medium and electronic device. By using the word segmentation results, a series of rules are formulated, which can not only extract noun phrases with conventional meanings, but also extract noun phrases containing special verb structures from the perspective of professional fields. Compared with simply extracting noun words based on word segmentation results, it can extract richer and more accurate noun phrases related to professional fields.
[0129] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for extracting noun phrases, characterized in that: include: Extraction step: extract the dependency relationship of the text to be extracted through the spacy Chinese model to obtain a token sequence, and identify the entity in the text to be extracted through the entity recognition service, wherein the entity includes the content of the entity and the start and end indexes of the entity in the text to be extracted; Extraction step: formulating extraction rules according to the dependency relationship, and extracting a noun phrase sequence from the token sequence according to the extraction rules, wherein the noun phrase sequence includes the content of the noun phrase and the start and end indexes of the noun phrase in the text to be extracted; Integration step: after judging the noun phrase sequence, integrating the noun phrases in the noun phrase sequence head to tail according to the judgment result to obtain a standard noun phrase sequence; Merging step: merging the entity with the standard noun phrase sequence to obtain a final noun phrase sequence; Wherein, the extraction step comprises: Dependency extraction step: performing dependency analysis on the text to be extracted using the Chinese model to obtain the token sequence with attribute values; Entity acquisition step: identifying entities in the text to be extracted through the entity recognition service; The integration step includes: integrating the token sequences in the noun phrase sequence whose length is greater than a threshold from head to tail to obtain the standard noun phrase sequence; Wherein, the merging step includes: merging the standard noun phrase sequence and the entity according to a merging rule to obtain a final noun phrase sequence; The merging rule is: if the start index of the entity is less than the end index of the noun phrase, the noun phrase is added to the final noun phrase sequence; if the start index of the entity is greater than the end index of the noun phrase, the noun phrase is added to the final noun phrase sequence.
2. The extraction method according to claim 1, characterized in that The extraction step comprises: Length identification step: identifying the length of the token sequence and outputting the identification result; Sequence output step: formulating the extraction rule according to the recognition result, and extracting the noun phrase sequence from the token sequence according to the extraction rule.
3. A noun phrase extraction system, characterized in that: include: An extraction unit, which extracts the dependency relationship of the text to be extracted by using the spacy Chinese model to obtain a token sequence, and identifies the entity in the text to be extracted by using an entity recognition service, wherein the entity includes the content of the entity and the start and end indexes of the entity in the text to be extracted; An extraction unit, which formulates an extraction rule according to the dependency relationship, and extracts a noun phrase sequence from the token sequence according to the extraction rule, wherein the noun phrase sequence includes the content of the noun phrase and the start and end indexes of the noun phrase in the text to be extracted; an integration unit, after judging the noun phrase sequence, integrating the noun phrases in the noun phrase sequence head to tail according to the judgment result to obtain a standard noun phrase sequence; A merging unit, merging the entity with the standard noun phrase sequence to obtain a final noun phrase sequence; Wherein, the extraction unit comprises: A dependency extraction module, which performs dependency analysis on the text to be extracted through the Chinese model to obtain the token sequence with attribute values; An entity acquisition module, which identifies entities in the text to be extracted through the entity recognition service; The integration unit comprises: integrating the token sequences in the noun phrase sequence whose length is greater than a threshold from head to tail to obtain the standard noun phrase sequence; Wherein, the merging unit comprises: merging the standard noun phrase sequence and the entity according to a merging rule to obtain a final noun phrase sequence; The merging rule is: if the start index of the entity is less than the end index of the noun phrase, the noun phrase is added to the final noun phrase sequence; if the start index of the entity is greater than the end index of the noun phrase, the noun phrase is added to the final noun phrase sequence.
4. The extraction system according to claim 3, characterized in that The extraction unit comprises: A length recognition module, which recognizes the length of the token sequence and outputs a recognition result; A sequence output module formulates the extraction rule according to the recognition result, and extracts the noun phrase sequence from the token sequence according to the extraction rule.
5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the extraction method according to any one of claims 1 to 2 is implemented.
6. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the extraction method according to any one of claims 1 to 2 is implemented.
Citation Information
Patent Citations
Chinese entity relation extraction method based on keyword and verb dependency
CN109241538A
Apparatus and method for extracting noun phrase by using sentence structure information, especially for generating a noun phrase of two prosodic words by using structure information of each short sentence
KR1019990042430A