Named entity extraction system, named entity extraction method, and named entity extraction program

Abstract

To add a highly consistent named entity label to a text.SOLUTION: A named entity extraction system for extracting a named entity from an input text includes: a named entity estimation unit which estimates a named entity to be added to each word included in the input text; a word pair extraction unit which extracts a word pair from the input text; a feature quantity measurement unit which measures feature quantities of words of the word pair extracted by the word pair extraction unit; and a named entity consistency estimation unit which estimates, on the basis of the feature quantities measured by the feature quantity measurement unit, whether the named entity to be added to each word and estimated by the named entity estimation unit is consistent or not.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a named entity extraction system, a named entity extraction method, and a named entity extraction program. [Background technology]

[0002] There have been attempts to extract named entities from documents using statistical approaches such as machine learning. Input text is divided into morphemes, and after identifying part-of-speech information for the morphemes and the dependencies between the morphemes, named entities are extracted. For example, in the input text "I took two tablets of stomach medicine," "two tablets" is identified as "QUANTITY" (a named entity) and extracted. For example, to perform this process, a learning model is generated using data that has been assigned information to be identified for the text as training data, and named entities are then extracted from documents using this learning model (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-046775 [Patent Document 2] Japanese Patent Application Laid-Open No. 2007-148785 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the method disclosed in Patent Document 1 does not always output correct results. Statistical approaches, such as machine learning, may correctly label one word with a named entity, but not label another word that should be labeled with the same named entity. For example, when the input phrase "took two tablets of stomach medicine" is entered, the "two tablets" will be labeled with the named entity label "QUANTITY," while the "one tablet" in the phrase "took one tablet of cold medicine" will not be labeled with the named entity label "QUANTITY." For example, when the input phrases "2008," "2010," and "2014" are entered, the named entity label "DATE" will be assigned to "2008" and "2010," but the named entity label "DATE" will not be assigned to "2014." Thus, there is a problem of inconsistency in the extraction and labeling of named entities.

[0005] In this regard, the method disclosed in Patent Document 2 attempts to maintain consistency of named entities by extracting named entities based on weights of named entities that are based on the frequency of appearance of the named entities. However, with the method disclosed in Patent Document 2, even named entities that should be extracted are not extracted if their frequency of appearance is low, and there is still a problem of low consistency in the extraction and labeling of named entities.

[0006] The present invention has been made in view of the above, and one object of the present invention is to assign named entity labels to text with higher consistency. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, one aspect of the present invention is a named entity extraction system that extracts named entities from input text, characterized in that it includes a named entity estimation unit that estimates a named entity to be assigned to each word included in the input text, a word pair extraction unit that extracts word pairs from the input text, a feature measurement unit that measures a feature of each word in the word pair extracted by the word pair extraction unit, and a named entity consistency estimation unit that estimates, based on the feature measured by the feature measurement unit, whether or not there is consistency in the named entities to be assigned to each word in the word pair estimated by the named entity estimation unit. [Effects of the Invention]

[0008] According to one aspect of the present invention, for example, it is possible to assign more consistent named entity labels to text. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a hardware diagram of a computer that realizes a named entity extraction system according to an embodiment. [Figure 2] FIG. 1 is a block diagram showing a configuration of a named entity extraction system. [Figure 3] FIG. 2 is a diagram illustrating an example of the configuration of data in a character feature DB. [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of data in a position feature amount DB. [Figure 5] FIG. 2 is a diagram illustrating an example of the configuration of data in a semantic feature DB. [Figure 6] FIG. 10 is a diagram illustrating an example of the configuration of data in an input text DB. [Figure 7] FIG. 10 is a diagram illustrating an example of the configuration of data in a separate text DB. [Figure 8] FIG. 10 is a diagram illustrating an example of the configuration of data in a statistical information DB. [Figure 9] 10 is a flowchart illustrating a procedure of a named entity extraction process of the named entity extraction system according to the embodiment. [Figure 10] 10 is a flowchart showing a detailed procedure of a text analysis process. [Figure 11] 10 is a flowchart showing the detailed steps of a named entity estimation process. [Figure 12] 10 is a flowchart showing a detailed procedure of a word pair extraction process. [Figure 13] 10 is a flowchart showing a detailed procedure of another text generation process. [Figure 14] 10 is a flowchart showing a detailed procedure of a character feature amount measurement process. [Figure 15] 10 is a flowchart showing a detailed procedure of a position feature amount measuring process. [Figure 16] 10 is a flowchart showing a detailed procedure of a semantic feature amount measurement process. [Figure 17] 10 is a flowchart showing a detailed procedure of a named entity consistency estimation process. [Figure 18] 10 is a flowchart showing a detailed procedure of a word pair visualization process. [Figure 19] FIG. 10 is a diagram illustrating an operation result of a word pair visualization process. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the embodiments described below, including the drawings, are merely examples and do not limit the disclosed technology of the present application. Furthermore, not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention. Furthermore, illustrations and descriptions of well-known elements that are essential to the configuration of the invention may be omitted. Furthermore, the number of elements shown in each figure is an example and is not limited to the number shown.

[0011] In the following description, a program may be installed in a device such as a computer, or may be stored in, for example, a program distribution server or a computer-readable (e.g., non-transitory) recording medium. Also, in the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0012] In the following description, the term "CPU (Central Processing Unit)" refers to one or more CPUs. It is not limited to microprocessors, such as CPUs, but may also refer to other types of processors, such as GPUs (Graphics Processing Units). The CPU may be single-core or multi-core. The CPU may also be replaced by a broader processor, such as a hardware circuit (for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)) that performs some or all of the processing. In the following description, the term "xxx unit" refers to a functional unit realized by program execution in cooperation with the CPU and memory.

[0013] In the following explanation, information is described in the form of a database table, such as "xxxDB (Data Base)," but the information may be data of any structure. Therefore, "xxxDB" can be referred to as "xxx information." In the following explanation, the structure of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be combined into one table.

[0014] (Hardware configuration of computer 100) 1 is a hardware diagram of a computer 100 that realizes a named entity extraction system S according to this embodiment. In this embodiment, the named entity extraction system S is realized by executing programs corresponding to each process using computer hardware resources such as a CPU 110, a memory 120, and a hard disk drive 130. For example, the named entity extraction process is realized by the CPU 110 executing a named entity extraction program according to this embodiment stored in the hard disk drive 130.

[0015] The input device 140 is a keyboard or other input device. The output device 150 is a display or other output device that outputs various processing results. The network device 160 is a visual interface through which the computer 100 communicates with external devices and computers. Note that the implementation of the named entity extraction system S is not limited to that shown in FIG. 1, and may be configured to include multiple computers 100 connected to each other so as to be able to communicate with each other.

[0016] (Configuration of named entity extraction system S) 2 is a block diagram showing the configuration of a named entity extraction system S. The named entity extraction system S performs text processing using a computer. Specifically, the named entity extraction system S accepts input text from a user terminal 200, generates another text from the input text, measures character features, position features, and semantic features of words in the input text and the another text, estimates whether there is consistency between named entities in a word pair, and visualizes the estimation result and statistical information on the status of past named entity label assignment. Note that in this embodiment, character features, position features, and semantic features are listed as word features, but the features are not limited to these as long as they represent the characteristics of the word.

[0017] The named entity extraction system S includes a feature DB (Data Base) 210, a text information DB 220, a statistical information DB 230, a control unit 240, and a word pair visualization unit 250.

[0018] The feature information DB 210 stores data associated with features used to determine the consistency of word pairs. The feature information DB 210 includes a character feature DB 211 that stores character features of word pairs, a position feature DB 212 that stores position features of word pairs, and a semantic feature DB 213 that stores semantic features of word pairs.

[0019] The text information DB 220 stores data associated with an input text and another text generated from the input text. The text information DB 220 includes an input text DB 221 that stores the named entity assignment results for the input text, and another text DB 222 that stores the named entity assignment results for the other text generated from the input text.

[0020] The control unit 240 has a text analysis unit 241, a word pair extraction unit 242, a separate text generation unit 243, a character feature measurement unit 244, a position feature measurement unit 245, a semantic feature measurement unit 246, a named entity estimation unit 247, and a named entity consistency estimation unit 248.

[0021] The text analysis unit 241 receives input text input from the user terminal 200, breaks the input text down into words to generate a set of words contained in the input text, identifies the part of speech information and dependency relationships of each word, and stores them in the input text DB 221.

[0022] The word pair extraction unit 242 selects word pairs from the set of words in the input text and stores them in the input text DB 221 .

[0023] The separate text generation unit 243 generates separate text using the input text and stores it in the separate text DB 222. For example, the separate text generation unit 243 generates as much separate text as possible that includes word pairs extracted from the input text by collecting it from external information sources such as internet crawling or by generating it using a sentence generation model.

[0024] The amount of information contained in the input text alone is insufficient. Therefore, by generating as many separate texts as possible that contain word pairs extracted from the input text and examining more similarities based on the features of the word pairs in the separate text, the accuracy of determining word similarity, which is used to judge the consistency of named entities, can be improved.

[0025] The character feature amount measurement unit 244 measures the character feature amount of a word and stores the measured character feature amount in the character feature amount DB 211. The position feature amount measurement unit 245 measures the position feature amount of a word and stores the measured position feature amount in the position feature amount DB 212. The semantic feature amount measurement unit 246 measures the semantic feature amount of a word and stores the measured semantic feature amount in the semantic feature amount DB 213.

[0026] The named entity estimation unit 247 estimates named entities included in the input text and stores the estimated named entities in the input text DB 221. The named entity consistency estimation unit 248 estimates named entity consistency of word pairs using the feature information DB 210 and the text information DB 220. The word pair visualization unit 250 visualizes word pairs for which named entity consistency has been recognized, as well as statistical information about the word pairs, on a screen. The word pair visualization unit 250 may be integrated with the control unit 240.

[0027] (Data structure of character feature DB211) 3 is a diagram showing an example of the data configuration of the character feature DB 211. The data of the character feature DB 211 is configured to include one or more "character feature" records. The "character feature" record is configured with multiple fields such as "word pair ID," "word content," "extraction source input text ID," "extraction source text ID," "character feature (text)," and "character feature (vector)."

[0028] "Word pair ID" holds a value that uniquely identifies the word pair to which a word belongs. "Word content" holds the content of the word. "Source input text ID" holds the ID of the input text if the corresponding word was extracted from the input text, and holds an empty value if the corresponding word was extracted from another text. "Source text ID" holds the ID of the other text if the corresponding word was extracted from another text, and holds an empty value if the corresponding word was extracted from the input text.

[0029] "Character feature (text)" holds features expressed in text among features measured from the corresponding word. For example, "character feature (text)" holds the pronunciation of characters, but is not limited to this. "Character feature (vector)" holds features expressed as vectors among features measured from the corresponding word. For example, "character feature (vector)" holds the distributed representation of characters measured using a neural network, but is not limited to this.

[0030] 4 is a diagram showing an example of the data configuration of the position feature DB 212. The data of the position feature DB 212 is configured to include one or more "position feature" records. The "position feature" record is configured from multiple fields such as "word pair ID," "word content," "extraction source input text ID," "extraction source specific text ID," "position feature (numerical value)," and "position feature (vector)."

[0031] The “word pair ID,” “word content,” “extraction source input text ID,” and “extraction source specific text ID” in the position feature DB 212 are the same as the “word pair ID,” “word content,” “extraction source input text ID,” and “extraction source specific text ID,” respectively, in the character feature DB 211.

[0032] "Position feature (numeric value)" holds a feature expressed as a numerical value among the position features measured from the corresponding word. For example, "Position feature (numeric value)" holds information on the position of the corresponding word from the beginning of the corresponding text, but is not limited to this. "Position feature (vector)" holds a feature expressed as a vector among the position features measured from the corresponding word. For example, "Position feature (vector)" holds a distributed representation of words that are dependent on the corresponding word, measured using a neural network, but is not limited to this.

[0033] 5 is a diagram showing an example of the data configuration of the semantic feature DB 213. The data of the semantic feature DB 213 is configured to include one or more "semantic feature" records. The "semantic feature" record is configured from multiple fields such as "word pair ID," "word content," "extraction source input text ID," "extraction source text ID," "semantic feature (numerical value)," and "semantic feature (vector)."

[0034] The “word pair ID,” “word content,” “extraction source input text ID,” and “extraction source specific text ID” in the semantic feature DB213 are the same as the “word pair ID,” “word content,” “extraction source input text ID,” and “extraction source specific text ID” in the character feature DB211 and the position feature DB212, respectively.

[0035] "Semantic features (numeric)" holds features expressed as numerical values ​​among semantic features measured from the corresponding word. For example, when the corresponding word is "one tablet," the "semantic features (numeric)" holds "1" as the value that is considered to be a quantity, but is not limited to this. "Semantic features (vector)" holds features expressed as vectors among semantic features measured from the corresponding word. For example, "semantic features (vector)" holds distributed representations measured for the corresponding word using a neural network, but is not limited to this.

[0036] 6 is a diagram showing an example of the data configuration of the input text DB 221. The data of the input text DB 221 is configured to include one or more "word information" records. The "word information" record is configured with multiple fields such as "text ID," "word ID," "word pair ID," "word content," "in-text word index," "part of speech information," "dependency information," and "estimated named entity label."

[0037] The "text ID" holds a value for uniquely identifying the input text input from the user terminal 200. The "word ID" holds a value for uniquely identifying the word extracted from the corresponding input text. The "word pair ID" and "word content" of the input text DB221 are the same as the "word pair ID" and "word content" of the character feature DB211, the position feature DB212, and the semantic feature DB213.

[0038] The "word index in text" holds information about the position of the word in question from the beginning of the input text. The "part of speech information" holds the part of speech type of the word in question. The "dependency information" holds words in the same input text that are dependent on the word in question. The "estimated named entity label" holds information about the named entity label estimated for the word in question.

[0039] 7 is a diagram showing an example of the data configuration of the separate text DB 222. The data of the separate text DB 222 is configured to include one or more "word information" records. The "word information" record is configured with multiple fields: "text ID," "word ID," "word pair ID," "word content," "in-text word index," "part of speech information," "dependency information," and "estimated named entity label."

[0040] The "text ID" holds a value for uniquely identifying another text generated based on the input text input from the user terminal 200. The "word ID" holds a value for uniquely identifying a word extracted from the corresponding other text. The "word pair ID" and "word content" of the other text DB 222 are the same as the "word pair ID" and "word content" of the input text DB 221.

[0041] The "word index in text" holds information about the position of the word from the beginning in the other text. The "part of speech information" holds the part of speech of the word. The "dependency information" holds words in the same text that are dependent on the word. The "estimated named entity label" holds information about the named entity label estimated for the word.

[0042] 8 is a diagram showing an example of the data configuration of the statistical information DB 230. The data in the statistical information DB 230 is configured to include one or more "statistical information" records. The "statistical information" record is configured with multiple fields such as "statistical information ID," "word content," "estimated named entity label," and "finally assigned named entity label."

[0043] "Statistical Information ID" holds a value for uniquely identifying a "Statistical Information" record. "Word Content" holds the content of the word. "Estimated Named Entity Label" holds information on the named entity estimated for the word. "Final Assigned Named Entity Label" holds information on the named entity that was finally assigned by operation from the user terminal 200.

[0044] 9 is a flowchart showing a named entity extraction processing procedure in the named entity extraction system S according to the embodiment. First, in S11, the named entity extraction system S accepts input of an input text from the user terminal 200 and performs text analysis processing on the input text. Details of the text analysis processing will be described later with reference to FIG.

[0045] Next, in S12, the named entity extraction system S performs a named entity estimation process to estimate a named entity for each word in the word set obtained in S901. Details of the named entity estimation process will be described later with reference to Fig. 11. Next, in S13, the named entity extraction system S performs a word pair extraction process to select and extract word pairs from the input text received in S901. Details of the word pair extraction process will be described later with reference to Fig. 12.

[0046] Next, in S14, the named entity extraction system S performs a separate text generation process to generate a separate text using the input text accepted in S11. Details of the separate text generation process will be described later with reference to Fig. 13. Next, in S15, the named entity extraction system S performs a character feature measurement process to measure character features for each word of the word pair extracted in S903. Details of the character feature measurement process will be described later with reference to Fig. 14.

[0047] Next, in S16, the named entity extraction system S performs a position feature amount measurement process to measure the position feature amount of each word of the word pair extracted in S13. Details of the position feature amount measurement process will be described later with reference to Fig. 15. Next, in S17, the named entity extraction system S performs a semantic feature amount measurement process to measure the semantic feature amount of each word of the word pair extracted in S13. Details of the semantic feature amount measurement process will be described later with reference to Fig. 16.

[0048] Next, in S18, the named entity extraction system S performs a named entity consistency estimation process to estimate the consistency of named entities of the word pairs extracted in S13. Details of the named entity consistency estimation process will be described later with reference to FIG.

[0049] Next, in S19, the named entity extraction system S performs a visualization process of displaying, on the display screen of the user terminal 200, the eigenvalue expression labels to be assigned to the word pairs extracted in S13 so that the user can determine them. Details of the visualization process will be described later with reference to FIGS. 18 and 19.

[0050] Finally, in S20, if the named entity extraction system S receives input of additional information from the user terminal 200 (S20 YES), it returns the process to S11 and executes the text analysis process again, whereas if no additional information is input (S20: NO), it terminates the named entity extraction process.

[0051] (Text analysis process details) 10 is a flowchart showing the detailed procedure of the text analysis process S11 (FIG. 9). The text analysis process S11 is executed by the text analysis unit 241 (FIG. 2).

[0052] First, in S11a, the text analysis unit 241 receives input text input from the user terminal 200. Next, in S11b, the text analysis unit 241 generates a set of words from the input text. In S11b, an existing method such as morphological analysis is used to extract words from the input text, but the method is not limited to this.

[0053] Next, in S11c, the text analysis unit 241 determines whether or not there are any words among the words extracted in S11b for which part-of-speech information and dependency relationship information have not been specified. If there are any words for which part-of-speech information and dependency relationship information have not been specified (S11cYES), the text analysis unit 241 proceeds to S11d, and if there are no words for which part-of-speech information and dependency relationship information have not been specified (S11cNO), the text analysis process ends.

[0054] In S11d, the text analysis unit 241 identifies the part-of-speech information of the words for which part-of-speech information was determined not to be specified in S11c. The part-of-speech information can be identified using an existing method such as morphological analysis, but this is not limited to this. Next, in S11e, the text analysis unit 241 identifies the dependency relationships of the words for which dependency information was determined not to be specified in S11c. The word dependency relationships can be identified using an existing method such as syntactic analysis, but this is not limited to this. Next, in S11f, the text analysis unit 241 associates all of the words extracted in S11b, including duplicates, with their part-of-speech information and dependency information, and stores them in the input text DB 221 (FIGS. 2 and 6) together with the ID of the input text, which is the source.

[0055] (Details of named entity prediction process) 11 is a flowchart showing the detailed procedure of the named entity estimation process S12 (FIG. 9). The named entity estimation process S12 is executed by the named entity estimation unit 247 (FIG. 2).

[0056] First, in S12a, the named entity estimation unit 247 receives input of the text ID of the input text from the user terminal 200. Next, in S12b, the named entity estimation unit 247 acquires a set of words from the input text DB 221. Next, in S12c, the named entity estimation unit 247 determines whether there are any words for which named entities have not been estimated. If there are any words for which named entities have not been estimated (S12c YES), the named entity estimation unit 247 proceeds to step S12d, and if named entities have been estimated for all words (S12c NO), the named entity estimation process ends.

[0057] In S12d, the named entity estimation unit 247 estimates a named entity of an undetermined word. Identifying a named entity of a word can be done using an existing method such as named entity extraction technology, but is not limited to this. Next, in S12e, the named entity estimation unit 247 stores the named entity of the word estimated in S12d or a named entity label that identifies the named entity in the input text DB 221 (FIGS. 2 and 6) in association with the relevant word. When S12e ends, the named entity estimation unit 247 proceeds to S12c.

[0058] (Details of word pair extraction process) 12 is a flowchart showing the detailed procedure of the word pair extraction process S13 (FIG. 9). The word pair extraction process S13 is executed by the word pair extraction unit 242 (FIG. 2).

[0059] First, in S13a, the word pair extraction unit 242 receives an input of the text ID of the input text from the user terminal 200. Next, in S13b, the word pair extraction unit 242 acquires a set of words from the input text DB 221. Next, in S13c, the word pair extraction unit 242 determines whether there are any words for which no word pair ID has been input in the input text DB 221. If there are any words for which no word pair ID has been input (S13c YES), the word pair extraction unit 242 proceeds to step S13d, and ends the word pair extraction process if all word pair IDs have been input (S13c NO).

[0060] In S13d, the word pair extraction unit 242 selects, from the word set acquired in S13b, a word that will pair with a word for which a word pair ID has not been input. One method for selecting words that will pair is to narrow down the search to words with the same part-of-speech information. For example, for an input text of "Go to Tokyo. Return to Kyoto," the same verb "Go home" is selected as a pair for "Go." However, the specific procedure is not limited to this, and word pairs may be selected based on word attribute information other than part-of-speech information.

[0061] Next, in S13e, the word pair extraction unit 242 associates the word IDs of the paired words selected in S13d with the corresponding words and stores them in the input text DB 221 (FIGS. 2 and 6). After S13e is completed, the word pair extraction unit 242 proceeds to S13c.

[0062] 13 is a flowchart showing the detailed procedure of the separate text generation process S14 (FIG. 9). The separate text generation process S14 is executed by the separate text generation unit 243 (FIG. 2). .

[0063] First, in S14a, the separate text generation unit 243 receives an input of a word pair ID from the user terminal 200. The input word pair IDs are the word pair IDs of all word pairs extracted by the word pair extraction unit 242 in the word pair extraction process S13. Next, in S14b, the separate text generation unit 243 acquires word pairs corresponding to the word pair IDs input in S14a from the input text DB 221. Next, in S14c, the separate text generation unit 243 acquires from the input text DB 221 a group of "word information" records having the same input text ID as the input text (word pair extraction source text) to which the word pair acquired in S14b belongs.

[0064] Next, in S14d, the separate text generation unit 243 generates separate text using the word pair acquired in S14b and the "word information" record group acquired in S14c. For example, when "Tokyo" and "Kyoto" are acquired as word pairs and "Tokyo" "go" and "Kyoto" "go" are acquired as "word information" records of the source text for word pair extraction, the separate text can be generated by swapping the positions of "Tokyo" and "Kyoto" to generate "Go to Kyoto. Return to Tokyo." However, the method is not limited to such word swapping or replacement, and other operations on words, such as adding new words or deleting words, can also be used. Alternatively, the separate text can be generated using a language model.

[0065] Next, in S14e, the separate text generation unit 243 determines whether there is a word for which part of speech information, dependency information, and named entity information have not been specified for each word in the separate text generated in S14d. If there is a word for which part of speech information, dependency information, and named entity information have not been specified (S14e YES), the separate text generation unit 243 proceeds to S14f, and if the part of speech information, dependency information, and named entity information have been specified for all words (S14e NO), the separate text generation process ends.

[0066] Next, in S14f, the separate text generation unit 243 identifies part-of-speech information of the words determined not to be identified in S14e. Next, in S14g, the separate text generation unit 243 identifies dependency relationship information of the words determined not to be identified in S14e. Next, in S14h, the separate text generation unit 243 identifies named entity information of the words determined not to be identified in S14e. The methods for identifying part-of-speech information, dependency relationship information, and named entity information are as described above.

[0067] Next, in S14i, the separate text generation unit 243 associates the part-of-speech information, dependency information, and named entity information identified in S14f, S14g, and S14h with the relevant word and stores them in the separate text DB 222 (FIGS. 2 and 7). When S14i ends, the separate text generation unit 243 ends the separate text generation process.

[0068] 14 is a flowchart showing the detailed procedure of the character feature amount measurement process S15 (FIG. 9). The character feature amount measurement process is executed by the character feature amount measurement unit 244 (FIG. 2).

[0069] First, in S15a, the character feature measurement unit 244 receives an input of a word pair ID from the user terminal 200. The input word pair IDs are the word pair IDs of all word pairs extracted by the word pair extraction unit 242 in the word pair extraction process S13. Next, in S15b, the character feature measurement unit 244 acquires, from the input text DB 221, each "word information" record of the word pair of the word pair ID input in S15a. Next, in S15c, the character feature measurement unit 244 acquires, from the input text DB 221, character features of each word of the word pair acquired in S15b. Methods for acquiring character features include, but are not limited to, acquiring the pronunciation of characters or measuring embedded representations of characters using a neural network. In S15d, the character feature measurement unit 244 stores the character features acquired in S15c in the character feature DB 211 (FIGS. 2 and 3) in association with each word ID.

[0070] Next, in S15e, the character feature measurement unit 244 acquires, from the other text DB 221, each "word information" record of the word pair of the word pair ID input in S15a. Next, in S15f, the character feature measurement unit 244 acquires, from the other text DB 222, the character feature of each word of the word pair acquired in S15e. The method of acquiring the character feature here is the same as in S15c. In S15g, the character feature measurement unit 244 stores the character feature acquired in S15f in the character feature DB 211 (FIGS. 2 and 3) in association with each word ID.

[0071] 15 is a flowchart showing the detailed procedure of the position feature amount measuring process S15 (FIG. 9). The position feature amount measuring process is executed by the position feature amount measuring unit 245 (FIG. 2).

[0072] First, in S16a, the position feature measurement unit 245 receives an input of a word pair ID from the user terminal 200. The input word pair IDs are the word pair IDs of all word pairs extracted by the word pair extraction unit 242 in the word pair extraction process S13. Next, in S16b, the position feature measurement unit 245 acquires, from the input text DB 221, each "word information" record of the word pair of the word pair ID input in S16a. Next, in S16c, the position feature measurement unit 245 acquires, from the input text DB 221, a position feature of each word of the word pair acquired in S16b. Methods for acquiring the position feature include, for example, evaluating the position of a word from the beginning of the text, or measuring a distributed representation of words that have a dependency relationship with the word using a neural network, but are not limited to these. In S16d, the position feature amount measurement unit 245 stores the position feature amounts acquired in S16c in the position feature amount DB 212 (FIGS. 2 and 4) in association with each word ID.

[0073] Next, in S16e, the position feature amount measurement unit 245 acquires, from the other text DB 221, each "word information" record of the word pair of the word pair ID input in S16a. Next, in S16f, the position feature amount measurement unit 245 acquires, from the other text DB 222, the position feature amount of each word of the word pair acquired in S16e. The method of acquiring the position feature amount here is the same as in S16c. In S16g, the position feature amount measurement unit 245 stores the position feature amount acquired in S16f in the position feature amount DB 212 (FIGS. 2 and 4) in association with each word ID.

[0074] 16 is a flowchart showing the detailed procedure of the semantic feature amount measurement process S17 (FIG. 9). The semantic feature amount measurement process is executed by the semantic feature amount measurement unit 246 (FIG. 2).

[0075] First, in S17a, the semantic feature measurement unit 246 receives an input of a word pair ID from the user terminal 200. The input word pair IDs are the word pair IDs of all word pairs extracted by the word pair extraction unit 242 in the word pair extraction process S13. Next, in S17b, the semantic feature measurement unit 246 acquires, from the input text DB 221, each "word information" record of the word pair of the word pair ID input in S17a. Next, in S17c, the semantic feature measurement unit 246 acquires, from the input text DB 221, the semantic feature of each word of the word pair acquired in S17b. A method for acquiring the semantic feature may be, for example, but is not limited to, a method of measuring a distributed representation of words using a neural network. In S17d, the semantic feature measurement unit 246 stores the semantic feature acquired in S17c in the semantic feature DB 213 (FIGS. 2 and 5) in association with each word ID.

[0076] Next, in S17e, the semantic feature measurement unit 246 acquires, from the other text DB 221, each "word information" record of the word pair of the word pair ID input in S17a. Next, in S17f, the semantic feature measurement unit 246 acquires, from the other text DB 222, the position feature of each word of the word pair acquired in S17e. The method of acquiring the semantic feature here is the same as in S17c. In S17g, the semantic feature measurement unit 246 stores the semantic feature acquired in S17f in the semantic feature DB 213 (FIGS. 2 and 5) in association with each word ID.

[0077] 17 is a flowchart showing the detailed procedure of the named entity consistency estimation process S18 (FIG. 9). The named entity consistency estimation process is executed by the named entity consistency estimation unit 248.

[0078] First, in S18a, the named entity consistency estimation unit 248 receives input of word pair IDs from the user terminal 200. The input word pair IDs are the word pair IDs of all word pairs extracted by the word pair extraction unit 242 in the word pair extraction process S13. Next, in S18b, the named entity consistency estimation unit 248 acquires, from the input text DB 221, each "word information" record of the word pair of the word pair ID input in S18a. Next, in S18c, the named entity consistency estimation unit 248 acquires, from the other text DB 221, each "word information" record of the word pair of the word pair ID input in S18a.

[0079] Next, in S18d, the named entity consistency estimation unit 248 acquires character features of each word acquired in S18b and S18c using the character feature DB 211 (FIG. 3). Next, in S18e, the named entity consistency estimation unit 248 acquires position features of each word acquired in S18b and S18c using the position feature DB 212. Next, in S18f, the named entity consistency estimation unit 248 acquires semantic features of each word acquired in S18b and S18c using the semantic feature DB 213.

[0080] Next, in S18g, the named entity consistency estimation unit 248 estimates whether the word pair acquired in S18b has named entity consistency based on the "word information" records of each word acquired in S18b and S18c, the character features acquired in S18d, the position features acquired in S18e, and the semantic features acquired in S18f. The method of estimating named entity consistency includes, for example, evaluating the similarity of each feature between words based on the inner product of feature vectors, and estimating that the word pair has named entity consistency if the similarity of at least one feature exceeds a threshold value or if the weighted sum of the similarities of two or more feature values ​​exceeds a threshold value. This is based on the idea that if two words have similar characters, similar positions in the text, or similar meanings, the same named entity should be assigned. However, the specific procedure is not limited to this.

[0081] 18 is a flowchart showing the detailed procedure of the word pair visualization process S19 (FIG. 2). The word pair visualization process is executed by the word pair visualization unit 250.

[0082] In S19a, the word pair visualization unit 250 receives word pair IDs of word pairs whose named entities have been recognized as having consistency in S18g (FIG. 17). For example, the input word pair IDs are the word pair IDs of all word pairs whose named entities have been recognized as having consistency in the named entity consistency estimation process S18 by the named entity consistency estimation unit 248 but whose assigned named entity labels have inconsistencies. Next, in S19b, the word pair visualization unit 250 acquires, from the input text DB 221, a "word information" record for each word of the word pair received in S19a. Next, in S19c, the word pair visualization unit 250 acquires, from the statistical information DB 230, a "statistical information" record for each word of the word pair. Next, in S19d, the word pair visualization unit 250 visualizes the word pairs and their statistical information by, for example, displaying them on a screen. A specific example of visualization will be described later with reference to FIG. 19.

[0083] Next, in S19e, the word pair visualization unit 250 receives an input of the finally assigned named entity label from the user terminal 200. Then, in S19f, the word pair visualization unit 250 stores the result of the input of the finally assigned named entity label received in S19e in the statistical information DB 230 (FIGS. 2 and 8).

[0084] 19 is a diagram showing the operation result of the word pair visualization process S19 (FIG. 2). As shown in FIG. 19, the word pair visualization unit 250 displays the word pair (e.g., word 1: “one tablet” (label: none), word 2: “two tablets” (label: quantity)) estimated to have consistency of named entities by the process of S18g in FIG. 17 on a display screen 250D of a display which is one of the output devices 150 connected to the word pair visualization unit 250.

[0085] Word 1: "one tablet" displayed in display area 250D1 is a word extracted from the input text "Took one tablet of medicine for colds" or another text with ID=3. When the statistical information DB230 is tallied for word content: "one tablet," out of the total 13 occurrences of "one tablet," there are three "unlabeled" occurrences with no "estimated named entity label" assigned, and ten occurrences with "label: QUANTITY" assigned.

[0086] On the other hand, word 2: "two tablets" displayed in display area 250D2 is a word extracted from the input text "Took two tablets of stomach medicine" or another text with ID=8. When statistical information DB230 is tallied for word content: "two tablets," of the total 17 occurrences of "two tablets," two are "unlabeled," meaning no "estimated named entity label," and 15 are assigned the label "QUANTITY." The "statistical information" displayed in display areas 250D1 and 250D2 is a history of named entity labels previously assigned to each word in the word pair.

[0087] According to the display on display screen 250D, for the word pair "one tablet" and "two tablets," which are estimated to have named entity consistency, the named entity labels currently vary between "no label," "QUANTITY," and "QUANTITY." The user looks at the contents displayed on display screen 250D and decides whether to assign "QUANTITY" to all of the word pairs "one tablet" and "two tablets," which have named entity consistency, or to leave all of them "no label." The user presses button 250D3 if they want to assign "QUANTITY" to all of them, or presses button 250D3 if they want to leave all of them "no label."

[0088] When the word pair visualization unit 250 detects that button 250D3 has been pressed, it unifies the values ​​of the column of the finally assigned named entity label of all records of the word content: "one tablet" and "two tablets" in the statistical information DB 230 to "QUANTITY." When the word pair visualization unit 250 detects that button 250D4 has been pressed, it unifies the values ​​of the column of the finally assigned named entity label of all records of the word content: "one tablet" and "two tablets" in the statistical information DB 230 to NULL (no label).

[0089] In the example of Figure 19, of the word pair ("one pill" and "two pills") that have named entity consistency, "two pills" is a similar word to "one pill," but the named entity consistency is higher than the threshold with the label:QUANTITY, which is higher than "one pill." Therefore, "one pill," which has relatively low named entity consistency, is considered to be assigned the named entity label:QUANTITY, just like "two pills." In this case, as in correction candidate display 250D11, it may be displayed that word 1 "one pill" is a candidate for correcting the named entity label to "QUANTITY."

[0090] In this way, when there is fluctuation in the estimated named entity labels assigned to each word of a word pair that is estimated to have named entity consistency in the statistical information DB 230, the word pair visualization unit 250 unifies the named entity label of the word with low consistency with the named entity label of the word with high consistency. That is, the word pair visualization unit 250 determines which estimated named entity label to unify as the finally assigned named entity label based on the history of named entity labels assigned to each word of the word pair, and unifies the named entity labels based on the determination result. However, the criteria for determining which label to unify are subject to appropriate design changes.

[0091] Furthermore, the word pair visualization unit 250 may accumulate a history of user revisions of named entity labels assigned to each word of a word pair, and display (visualize) the revision history on the display screen 250D. The revision history includes information on the revision date and time and from which label the revision was made to which label. Based on this revision history, it may determine which estimated named entity label to unify the finally assigned named entity label with, and unify the named entity labels according to the determination result. For example, it may determine that the estimated named entity label assigned most frequently in past revisions should be unified, but the criteria for determining which label to unify are matters that can be changed as appropriate by design.

[0092] In the above-described embodiment, first, a named entity is estimated for each word in the input text, then character features, position features, and semantic features are measured for word pairs in the input text, and the consistency of the named entity to be assigned to the word pair is determined using these features. Therefore, even if a correct named entity estimation result is not obtained in the initial estimation, a named entity with high consistency can be assigned.

[0093] Furthermore, in the above-described embodiment, the consistency of named entities can be estimated using a variety of features, since at least one of the character feature, semantic feature, and position feature of each word in a word pair is used.

[0094] In the above-described embodiment, whether or not there is consistency in the named entities assigned to the word pairs is estimated based on the feature quantities of each word in the word pairs extracted from the input text and another text generated based on the input text. Therefore, by increasing the amount of information used to determine the consistency of the named entities, it is possible to improve the accuracy of determining the consistency of the named entities.

[0095] In addition, in the above-described embodiment, by performing predetermined operations such as replacement, substitution, addition, deletion, etc. on each word of a word pair extracted from the input text, a separate text can be generated based on each word of the word pair, thereby easily generating a separate text.

[0096] Furthermore, in the above-described embodiment, by collecting texts containing word pairs of the input text from external information sources such as by crawling the Internet, it is possible to easily increase the variety of other texts.

[0097] Furthermore, in the above-described embodiment, by using a sentence generation model for generation, it is possible to easily increase the variety of alternative texts.

[0098] Furthermore, in the above-described embodiment, by visualizing each word in a word pair that is estimated to have consistency in the named entities to be assigned, and the history of the named entity labels assigned to each word, and correcting and unifying the named entity labels, it is possible to assign named entities that are highly consistent, reflecting past performance.

[0099] In addition, in the above-described embodiment, by unifying the labels of named entities to be assigned in accordance with user operations, it is possible to assign named entities with high consistency, reflecting judgments based on the user's knowledge.

[0100] In addition, in the above-described embodiment, by automatically unifying the labels of named entities based on the history of named entity labels assigned to each word, the burden on the user of correcting labels can be reduced and named entities with high consistency can be assigned.

[0101] Furthermore, in the above-described embodiment, the revision history of named entity labels is accumulated and visualized, allowing the user to grasp the past revision trends of named entity labels.

[0102] In addition, in the above-described embodiment, the label is automatically corrected based on the correction history of the named entity label, thereby reflecting the past correction trends of named entity labels, reducing the burden on the user, and making it possible to assign named entities with high consistency.

[0103] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, as long as there is no contradiction, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of one embodiment to the configuration of another embodiment. Furthermore, it is possible to add, delete, replace, integrate, or distribute part of the configuration of each embodiment. Furthermore, the configurations and processes described in the embodiments can be distributed, integrated, or replaced as appropriate based on processing efficiency or implementation efficiency. [Explanation of symbols]

[0104] S: named entity extraction system, 100: computer, 241: text analysis unit, 242: word pair extraction unit, 243: separate text generation unit, 244: character feature measurement unit, 245: position feature measurement unit, 246: semantic feature measurement unit, 247: named entity estimation unit, 248: named entity consistency estimation unit, 250: word pair visualization unit.

Claims

1. A named entity extraction system that extracts named entities from input text, comprising: a named entity estimation unit that estimates a named entity label that indicates to which named entity each word included in the input text corresponds; a word pair extraction unit that extracts pairs of words having the same attribute information from the input text as word pairs; a feature measurement unit that measures character features by vectorizing characters of each word of the word pairs extracted by the word pair extraction unit, position features by vectorizing positions of each word in the input text, and semantic features by vectorizing meanings of each word; a named entity consistency estimation unit that estimates that the named entity labels assigned to the words of the word pair estimated by the named entity estimation unit should be the same when a predetermined value representing the similarity of each word based on the character feature, the position feature, and the semantic feature of the each word measured by the feature measurement unit exceeds a threshold; and A named entity extraction system comprising:

2. The named entity extraction system according to claim 1, The feature amount is at least one of a character feature amount, a semantic feature amount, and a position feature amount within the input text of each word of the word pair. A named entity extraction system characterized by:

3. The named entity extraction system according to claim 1, a separate text generation unit that generates separate text different from the input text, the separate text including each word of the word pair extracted from the input text by the word pair extraction unit; the word pair extraction unit extracts word pairs from the other text; the feature amount measurement unit measures the character feature amount, the position feature amount, and the semantic feature amount of each word of a word pair extracted from the separate text by the word pair extraction unit; The named entity consistency estimation unit estimates that the named entity labels assigned to the words of a word pair should be the same based on the character feature, the position feature, and the semantic feature of each word of the word pair extracted from the input text and the separate text, respectively, measured by the feature measurement unit. A named entity extraction system characterized by:

4. The named entity extraction system according to claim 3, The separate text generation unit performs a predetermined operation on each word of a word pair extracted from the input text by the word pair extraction unit, thereby generating separate text based on each word of the word pair. A named entity extraction system characterized by:

5. The named entity extraction system according to claim 3, The other text generation unit collects, from an external information source, the other text including each word of the word pair extracted from the input text by the word pair extraction unit. A named entity extraction system characterized by:

6. The named entity extraction system according to claim 3, The separate text generation unit generates the separate text including each word of the word pair extracted from the input text by the word pair extraction unit, using a sentence generation model. A named entity extraction system characterized by:

7. The named entity extraction system according to claim 1, a word pair visualization unit that visualizes each word of the word pair for which the named entity labels assigned by the named entity consistency estimation unit should be the same, and a history of the named entity labels assigned to each word; The word pair visualization unit corrects and unifies the named entity labels assigned to each word of the word pair that are estimated to have the same named entity label. A named entity extraction system characterized by:

8. The named entity extraction system according to claim 7, The word pair visualization unit unifies the named entity labels assigned to the words of the word pair that are estimated to have the same named entity labels in response to a user operation. A named entity extraction system characterized by:

9. The named entity extraction system according to claim 7, The word pair visualization unit determines, based on the history, which label to unify the named entity labels with, and unifies the named entity labels in accordance with a determination result. A named entity extraction system characterized by:

10. The named entity extraction system according to claim 7, The word pair visualization unit accumulates a revision history of the named entity label and visualizes the revision history. A named entity extraction system characterized by:

11. The named entity extraction system according to claim 10, The word pair visualization unit determines which label to unify the named entity labels with based on the correction history, and unifies the named entity labels according to a determination result. A named entity extraction system characterized by:

12. A named entity extraction method performed by a named entity extraction system that extracts named entities from input text, comprising: a named entity estimation step of estimating a named entity label that indicates to which named entity each word included in the input text corresponds, the named entity label being assigned to each word included in the input text; a word pair extraction step of extracting pairs of words having the same attribute information from the input text as word pairs; a feature measurement step of measuring character features obtained by vectorizing characters of each word of the word pair extracted by the word pair extraction step, position features obtained by vectorizing positions of each word in the input text, and semantic features obtained by vectorizing meanings of each word; a named entity consistency estimation step of estimating that the named entity labels assigned to the words of the word pair estimated by the named entity estimation step should be the same when a predetermined value representing the similarity of each word based on the character feature, the position feature, and the semantic feature of the each word measured by the feature measurement step exceeds a threshold; A named entity extraction method comprising:

13. A named entity extraction program for causing a computer to function as a named entity extraction system that extracts named entities from input text, comprising: The computer a named entity estimation unit that estimates a named entity label that is assigned to each word included in the input text and indicates to which named entity each word corresponds; a word pair extraction unit that extracts pairs of words having the same attribute information from the input text as word pairs; a feature amount measurement unit that measures character feature amounts by vectorizing characters of each word of the word pairs extracted by the word pair extraction unit, position feature amounts by vectorizing positions of the words in the input text, and semantic feature amounts by vectorizing meanings of the words; a named entity consistency estimation unit that estimates that the named entity labels assigned to the words of the word pair estimated by the named entity estimation unit should be the same when a predetermined value representing the similarity of each word based on the character feature, the position feature, and the semantic feature of the each word measured by the feature measurement unit exceeds a threshold. A named entity extraction program to function as a

Images