Extraction device

Abstract

To extract unknown words or word candidates of technical terms.SOLUTION: An extraction apparatus 1 includes a relation extraction word extraction unit 15 which acquires relation information on a relation between named entities extracted by relation extraction for named entities extracted by named entity extraction from an input sentence, and extracts and outputs, as an unknown word or a word candidate of technical term, a character string in the input sentence based on two named entities having a relation indicated by the acquired relation information. The character string may be a character string formed by combining words between the two named entities in the input sentence, in the order of the words, from one named entity closer to the beginning of the input sentence to the other named entity, out of the related two named entities. The relation extraction word extraction unit 15 may divide the input sentence by included predetermined parts of speech, and further extract and output, as word candidates, subdivided character strings excluding the predetermined parts of speech.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] One aspect of this disclosure relates to an extraction device for extracting word candidates for unknown words or technical terms. [Background technology]

[0002] To utilize natural language processing technologies such as speech recognition or machine translation in specific domains or situations, it is crucial to register unknown words or technical terms. For example, in speech recognition or machine translation, adding unknown words or technical terms to their respective dictionaries enables recognition and translation processing. Patent Document 1 below discloses a method for learning unknown words in an automatic speech recognition system. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Special Publication No. 2022-522926 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] While the aforementioned listing of unknown words or technical terms is generally done manually, this process becomes difficult when the number of words is large or the usage frequency is high. Therefore, there is a need to extract potential unknown words or technical terms. [Means for solving the problem]

[0005] An extraction device relating to one aspect of this disclosure includes an extraction unit that acquires relational information regarding the relationships between named entities extracted by relational extraction of named entities extracted by named entity extraction from an input sentence, and extracts and outputs strings in the input sentence based on two related named entities indicated by the acquired relational information as candidate words for unknown words or technical terms.

[0006] In this respect, strings within the input sentence based on two related named entities extracted from the input sentence are extracted and output as potential unknown words or technical terms. In other words, potential unknown words or technical terms can be extracted. [Effects of the Invention]

[0007] According to one aspect of this disclosure, it is possible to extract potential words that are unknown or technical terms. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the functional configuration of the extraction apparatus according to the embodiment. [Figure 2] This diagram summarizes the input and output information in the processing performed by the extraction device according to the embodiment. [Figure 3] This figure shows an example of input data. [Figure 4] This figure shows an example of extracted text data. [Figure 5] This figure shows an example table of morphological delimiters and a list of parts of speech. [Figure 6] This figure shows an example table of morphological delimiters and a list of parts of speech and named entity labels. [Figure 7] This figure shows an example table of morphological delimiters, part-of-speech and named entity labels, and a list of related word IDs. [Figure 8] This figure shows an example of a table for the extraction candidate list. [Figure 9] This figure shows an example of a table of notation and named entity label lists. [Figure 10] This figure shows an example of a table for a list of parts of speech to be excluded from extraction. [Figure 11] This figure shows an example of a table for the extraction candidate list. [Figure 12] This figure shows an example table of named entity labels to be excluded from extraction. [Figure 13] This figure shows an example of a table for a list of unknown words and technical terms. [Figure 14] This is a diagram illustrating an example table of a word occurrence frequency dictionary. [Figure 15] This is a diagram illustrating an example table of a registration target word list. [Figure 16] This is a flowchart illustrating an example of processing executed by the extraction device according to the embodiment. [Figure 17] This is a diagram illustrating an example of the hardware configuration of a computer used in the extraction device according to the embodiment. [Modes for Carrying Out the Invention]

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the description of the drawings, the same reference numerals are assigned to the same elements, and duplicate descriptions are omitted. In addition, the embodiments of the present disclosure in the following description are specific examples of the present invention, and the present invention shall not be limited to these embodiments unless otherwise stated to specifically limit the present invention.

[0010] Figure 1 is a diagram illustrating an example of the functional configuration of the extraction device 1 according to the embodiment. The extraction device 1 is a computer device that extracts and outputs word candidates for unknown words or technical terms from input data. An unknown word is an unknown phrase, such as a phrase that is not registered in a dictionary in natural language processing. A technical term is a specialized term, such as a phrase used or commonly accepted in a specific field.

[0011] As shown in Figure 1, the extraction device 1 is configured to include an information storage unit 10, a text acquisition unit 11, a morpheme analysis unit 12, a named entity extraction unit 13, a relation extraction unit 14, a relation extraction word extraction unit 15 (extraction unit), a morpheme analysis word extraction unit 16 (extraction unit), a word screening unit 17 (type screening unit), a model registration target word screening unit 18 (frequency screening unit), and a dictionary registration unit 19 (registration unit).

[0012] Each functional block of the extraction device 1 is intended to function within the extraction device 1, but is not limited to this. For example, some of the functional blocks of the extraction device 1 may function within a computer device separate from the extraction device 1, connected to the extraction device 1 via a network, while appropriately sending and receiving information with the extraction device 1. Furthermore, some functional blocks of the extraction device 1 may be omitted, multiple functional blocks may be integrated into a single functional block, or a single functional block may be decomposed into multiple functional blocks.

[0013] Figure 2 is a diagram that organizes the input and output information in the process performed by extraction device 1. Details of each input and output will be described later, but here we will explain the overall flow following Figure 2.

[0014] The input data (see Figure 3 for an example) is input to the text acquisition unit 11, and text (see Figure 4 for an example) is output. This text is input to the morphological analysis unit 12, and a list of morphological delimiter notations and parts of speech (see Figure 5 for an example) is output. This list of morphological delimiter notations and parts of speech is input to the named entity extraction unit 13, and a list of morphological delimiter notations, parts of speech, and named entity labels (see Figure 6 for an example) is output. This list of morphological delimiter notations, parts of speech, and named entity labels is input to the relation extraction unit 14, and a list of word IDs related to the morphological delimiter notations, parts of speech, and named entity labels (see Figure 7 for an example) is output. This list of word IDs related to the morphological delimiter notations, parts of speech, and named entity labels is input to the relation extraction word extraction unit 15, and a list of extraction candidates (see Figure 8 for an example) and a list of notations and named entity labels (see Figure 9 for an example) are output.

[0015] The list of morphological delimiters and parts of speech output from the morphological analysis unit 12, and the list of parts of speech excluded from extraction (a specific example is shown in Figure 10) pre-stored in the information storage unit 10, are input to the morphological analysis word extraction unit 16, and an extraction candidate list (a specific example is shown in Figure 11) is output. The extraction candidate list and the list of notations and named entity labels output from the relational extraction word extraction unit 15, the extraction candidate list output from the morphological analysis word extraction unit 16, and the list of named entity labels excluded from extraction (a specific example is shown in Figure 12) pre-stored in the information storage unit 10 are input to the word selection unit 17, and an unknown word / technical term list (a specific example is shown in Figure 13) is output. The unknown word / technical term list and the word frequency dictionary pre-stored in the information storage unit 10 (a specific example is shown in Figure 14) are input to the model registration target word selection unit 18, and a registration target word list (a specific example is shown in Figure 15) is output. The list of words to be registered is entered into the dictionary registration unit 19 and registered in the dictionary.

[0016] The following describes the functions of the extraction device 1 shown in Figure 1.

[0017] The information storage unit 10 stores arbitrary information used in calculations by the extraction device 1, as well as the results of calculations by the extraction device 1. The information stored in the information storage unit 10 may be referenced as appropriate by each function of the extraction device 1. The information storage unit 10 pre-stores, for example, a list of parts of speech not to be extracted, a list of named entity labels not to be extracted, and a word frequency dictionary (details of each will be described later).

[0018] The text acquisition unit 11 extracts text from the input data, which is text data input to the extraction device 1 (or previously stored by the information storage unit 10), and outputs the extracted text. More specifically, the text acquisition unit 11 takes input data in which unknown words or technical terms appear as input and outputs extracted text, which is text with a line break after each sentence. The input data may include, for example, meeting materials for a company meeting, past meeting minutes, or related internal company documents if the target is a company meeting. There are no restrictions on the file format of the input data. When extracting text, the text acquisition unit 11 uses a corresponding text extraction method (existing technology) to extract text from the file. The text acquisition unit 11 then adds line breaks to the extracted text at the unit of line break codes and periods, and outputs it as extracted text. The text acquisition unit 11 may output the extracted text to other functional blocks such as the morphological analysis unit 12, or it may be stored by the information storage unit 10.

[0019] Figure 3 shows an example of input data. The input data shown in Figure 3 includes the text "First meeting with Company A in the Heisei era," followed by two line breaks (newline characters), and then the text "Proposed a new service B related to Hirobumi Ito. As a result, there were positive opinions."

[0020] Figure 4 shows an example of extracted text data. The extracted text shown in Figure 4 includes the texts "First meeting with Company A in the Heisei era," "Proposing a new service B related to Hirobumi Ito," and "As a result, there were positive opinions.", each with a line break.

[0021] The morphological analysis unit 12 performs morphological analysis on the extracted text. More specifically, the morphological analysis unit 12 performs morphological analysis (existing technology) on the extracted text input from the text acquisition unit 11 (or previously stored by the information storage unit 10) using a morphological analyzer (existing technology), and outputs the notation of morphemes and a list of parts of speech. Morphological analysis is a technology that analyzes natural language sentences and enables division into morphemes and determination of parts of speech. Any technology may be used as the morphological analyzer. The morphological analysis unit 12 may output the notation of morphemes and the list of parts of speech to other functional blocks such as the named entity recognition unit 13 and the morphological analysis word extraction unit 16, or it may be stored by the information storage unit 10.

[0022] Figure 5 shows an example table of morphological delimiter notations and parts of speech lists. In the list of morphological delimiter notations and parts of speech shown in Figure 5, each notation obtained by morphological delimiting a portion of the extracted text (the first two lines) shown in Figure 4 corresponds to the part of speech of that notation. More specifically, for the text "First meeting with Company A in the Heisei era" included in the extracted text, the list of morphological delimiter notations and parts of speech is output, corresponding to the notation "Heisei" and part of speech "noun", notation "first" and part of speech "adverb", notation "no" and part of speech "particle", notation "A" and part of speech "noun", notation "company" and part of speech "noun", notation "to" and part of speech "particle", notation "no" and part of speech "particle", and notation "meeting" and part of speech "noun". Furthermore, for the text "We propose a new service B concerning Hirobumi Ito." included in the extracted text, a list of morphological delimiters and their parts of speech are output, corresponding to the following, in order from the beginning of the sentence: "Ito" and part of speech "noun", "Hirobumi" and part of speech "noun", "ni" and part of speech "particle", "kansuru" and part of speech "verb", "shin" and part of speech "prefix", "service" and part of speech "noun", "B" and part of speech "noun", "o" and part of speech "particle", "o" and part of speech "particle", "proposal" and part of speech "noun", and "." and part of speech "period".

[0023] The named entity recognition unit 13 performs named entity recognition on the morphological analysis results. Named entity recognition is an existing technology that analyzes natural language sentences or morphological analysis results and automatically extracts words that are classified as named entities, such as names of people, places, organizations, products, quantities, dates, and times. Any technology may be used for named entity recognition (or the named entity recognition system). Since many unknown words or technical terms are composed of named entities, using named entity recognition makes it possible to extract words more robustly than when using morphological analysis alone.

[0024] More specifically, the named entity recognition unit 13 performs named entity recognition on a list of morphological delimiter notations and parts of speech input from the morphological analysis unit 12 (or pre-stored by the information storage unit 10). For words determined to be named entities, the named entity recognition unit 13 assigns a label indicating the type of named entity and outputs a list of morphological delimiter notations, parts of speech, and named entity labels. The named entity recognition unit 13 may output the list of morphological delimiter notations, parts of speech, and named entity labels to other functional blocks such as the relational extraction unit 14, or it may be stored by the information storage unit 10.

[0025] Figure 6 shows an example table of morphological delimiter notations and labels for parts of speech and named entities. The list of morphological delimiter notations and labels for parts of speech and named entities shown in Figure 6 was created by performing named entity extraction on the list of morphological delimiter notations and parts of speech shown in Figure 5, and assigning a label for the type of named entity to words that were determined to be named entities. More specifically, the notation "Heisei" was determined to be a named entity and assigned the label "Date" as the type of named entity; the notation "Ito" was determined to be a named entity and assigned the label "Person's Name" as the type of named entity; and the notation "Hirofumi" was determined to be a named entity and assigned the label "Person's Name" as the type of named entity.

[0026] The relationship extraction unit 14 performs relationship extraction on the named entity extraction results. Relationship extraction is an existing technique that analyzes natural language sentences and determines the relationships (associations) between named entities, or automatically extracts each word and its relationship when there are dependencies between words classified as named entities. Relationship extraction makes it possible to detect named entities that have a high degree of relationship. Any technique may be used for relationship extraction (or the relationship extraction system). Unknown words or technical terms may include compound words of multiple named entities, or compound words of multiple named entities and common words. Since the multiple named entities that make up a compound word are often related, there is a high probability that compound words can be detected by using relationship extraction.

[0027] More specifically, the relationship extraction unit 14 performs relationship extraction on named entities in the list of morphological delimiter notations, parts of speech, and named entity labels input from the named entity extraction unit 13 (or previously stored by the information storage unit 10). For words determined to be related, the relationship extraction unit 14 assigns the ID of the related word as a label and outputs a list of morphological delimiter notations, parts of speech, named entity labels, and the ID of the related word. The relationship extraction unit 14 may output the list of morphological delimiter notations, parts of speech, named entity labels, and the ID of the related word to other functional blocks such as the relationship extraction word extraction unit 15, or it may be stored by the information storage unit 10.

[0028] Figure 7 shows an example table of morphological delimiter notations, part-of-speech and named entity labels, and a list of related word IDs. The list of morphological delimiter notations, part-of-speech and named entity labels and related word IDs shown in Figure 7 was created by performing relationship extraction on named entities in the list of morphological delimiter notations, part-of-speech and named entity labels shown in Figure 6, and assigning the ID of the related word as a label to words that were determined to be related. More specifically, the ID "2," which is the ID of the word "Hirobumi" that was determined to be related to the notation "Ito," is assigned, and the ID "1," which is the ID of the word "Ito" that was determined to be related to the notation "Hirobumi," is assigned. No words were determined to be related to the notation "Heisei," so no ID was assigned. Note that the word IDs are assigned in ascending order from "1" for each morpheme in the extracted text, starting from the beginning of the sentence. For example, the ID for the notation "Ito" is "1", the ID for the notation "Hirofumi" is "2", the ID for the notation "ni" is "3", the ID for the notation "kansuru" is "4", and so on. Furthermore, although the embodiment considers a case where one notation is related to another notation, it is not limited to this, and one notation may be related to multiple other notations.

[0029] Although the morphological analysis unit 12, named entity recognition unit 13, and relation extraction unit 14 described above were each described as separate (independent) functional blocks, the processing performed by the morphological analysis unit 12, named entity recognition unit 13, and relation extraction unit 14 may be executed simultaneously as a single functional block.

[0030] The relational word extraction unit 15 obtains relational information (a list of morphological delimiter notation, part of speech, label of the named entity, and IDs of related words) regarding the relationships between named entities extracted by relational extraction of named entities extracted by named entity extraction on the input sentence (input data or extracted text), and extracts and outputs a string from the input sentence based on the two related named entities indicated by the obtained relational information as a candidate for an unknown word or technical term. The string may be a string obtained by concatenating the words in the input sentence in word order from the named entity closer to the beginning of the input sentence to the other named entity. A detailed explanation follows below.

[0031] The relational word extraction unit 15 creates and outputs a list of extraction candidates and a list of notations and named entity labels based on the named entity information and the results of relational extraction. More specifically, the relational word extraction unit 15 performs the following processes (1) to (3) on the list of morphologically separated notations, part-of-speech and named entity labels and related word IDs (relationship information) input from the relational extraction unit 14 (or previously stored by the information storage unit 10). (1) In the list of morphological delimiter notations, part of speech, and related word IDs, the word that has been assigned the label of a named entity is w i Add to the list of candidates for extraction. (2) If an ID for a related word exists, then word w i Words related to that lol j And the word in between lol i+1 , w i+2 ...w j-2 , w j-1 Combine them and add them to the list of extraction candidates. (3) Output a list of extraction candidates and a list of labels for the notation and named entities.

[0032] Note that while process (2) assumes the combination of two related words, it is not limited to this; three or more related words may also be combined.

[0033] By combining related named entities with the words between them, it is highly likely that unknown words or technical terms composed of compound words of multiple named entities, or compound words of multiple named entities and common words, can be extracted. Since the relational word extraction unit 15 combines words while also considering the relationships between them, it is possible to extract candidate unknown words or technical terms more robustly compared to combining surrounding words using only the morphological analysis results.

[0034] The relational word extraction unit 15 may output the extraction candidate list and the list of labels for notation and named entities to other functional blocks such as the word selection unit 17, or store them in the information storage unit 10, or transmit them to other devices via the communication device 1004 described later, or display them on the output device 1006 (such as a display) described later.

[0035] Figure 8 shows an example of a table for the extraction candidate list. The extraction candidate list shown in Figure 8 includes the notations "Heisei", "Ito", and "Hirobumi" added by processing (1) of the list of morphological delimiters, part of speech, and named entity labels and related word IDs shown in Figure 7, as well as the notation "Ito Hirobumi" added by combining them through processing (2).

[0036] Figure 9 shows an example of a table of labels for spellings and named entities. The list of labels for spellings and named entities shown in Figure 9 is extracted from the list of word IDs related to the morphological delimiters, spellings, parts of speech, and named entity labels shown in Figure 7, specifically the rows for spellings and named entities.

[0037] The morphological analysis word extraction unit 16 divides the input sentence (input data or extracted text) into predetermined parts of speech, and further extracts and outputs each of the divided strings that do not contain the predetermined part of speech as word candidates. This will be explained in detail below.

[0038] The morpheme analysis word extraction unit 16 creates and outputs an extraction candidate list for the morpheme analysis information and the non-extraction target part-of-speech. More specifically, the morpheme analysis word extraction unit 16 performs the following processes (4) to (7) on the morpheme-separated notation and part-of-speech list input from the morpheme analysis unit 12. (4) A word corresponding to a part-of-speech in the non-extraction target part-of-speech list (previously stored by the information storage unit 10) is defined as a delimiter word. The part-of-speech of the non-extraction target part-of-speech list can be set arbitrarily, and may be a part-of-speech that is rarely included in unknown words or technical terms. For example, the parts-of-speech in the non-extraction target part-of-speech list include "particle", "verb", "adverb", "conjunctive", "adjective", "prefix noun", "suffix", "comma", "period", "parenthesis" and the like. (5) The list of morpheme-separated notations and parts-of-speech is split by words that correspond to the delimiter words in process (4). For example, if the delimiter word is w i , the list is split in the form of l1={w1,…,w i-1}, l2={w i+1 ,…,w N} (where N in one sentence is the number of morphemes). (6) For each list split in process (5), the words contained therein are concatenated. In the above case, a word formed by concatenating w1,…,w i-1 and a word formed by concatenating w i+1 ,…,w N are created. (7) Each word concatenated in process (6) is added to the extraction candidate list, and the extraction candidate list is output.

[0039] Some unknown words or technical terms are not composed of proper nouns. The constituent parts-of-speech are often, for example, nouns or alphabetic strings. By removing words of parts-of-speech that are rarely used in unknown words or technical terms based on the part-of-speech information of the morpheme analysis result, words composed of compound words can also be extracted.

[0040] The morphological analysis word extraction unit 16 may output the extraction candidate list to other functional blocks such as the word selection unit 17, store it in the information storage unit 10, transmit it to another device via the communication device 1004 described later, or display it on the output device 1006 (such as a display) described later.

[0041] Figure 10 shows an example of a table for a list of parts of speech to be excluded from extraction. The list of parts of speech to be excluded from extraction shown in Figure 10 includes particles, verbs, adverbs, adverbs, adjectives, injunctions, suffixes, commas, and periods as parts of speech.

[0042] Figure 11 shows an example of a table for the extraction candidate list. The extraction candidate list shown in Figure 11 includes the notation "Heisei", "Company A", "Meeting", "Hirobumi Ito", "New Service B", and "Proposal", which are split and combined from the list of morpheme-delimited notations and parts of speech shown in Figure 5 by the notation "first time" which is an adverb (included in the list of parts of speech excluded from extraction shown in Figure 10), the notation "no" which is a particle (included in the list of parts of speech excluded from extraction shown in Figure 10), the notation "to" which is a particle, the notation "no" which is a particle, the notation "ni" which is a particle, the notation "sekisuru" which is a verb (included in the list of parts of speech excluded from extraction shown in Figure 10), the notation "o" which is a particle, and the notation "." which is a period (included in the list of parts of speech excluded from extraction shown in Figure 10).

[0043] The word selection unit 17 outputs word candidates that are of a predetermined type of named entity, excluding those extracted by the relational word extraction unit 15. This will be explained in detail below.

[0044] The word selection unit 17 performs list normalization and deletion of words containing named entities that are not to be extracted, on the extraction candidate list output by the relational word extraction unit 15 and the extraction candidate list output by the morphological analysis word extraction unit 16. More specifically, the word selection unit 17 takes the extraction candidate list output by the relational word extraction unit 15, the extraction candidate list output by the morphological analysis word extraction unit 16, and the label list of spelling and named entity as input and performs the following processes (8) to (12). (8) Combine the two candidate lists and remove any duplicate elements. If a compound word exists in the candidate list, it is possible that only a part of the compound word may be spoken, so (if the compound word contains the same word) it is not considered a duplicate element. For example, if one list contains "Ito" and "Hirobumi", and the other list contains "Ito Hirobumi", combining them will result in a list that contains "Ito", "Hirobumi", and "Ito Hirobumi" (i.e., all elements from both lists). (9) Normalize the characters included in the extraction candidate list. The targets for normalization can be set arbitrarily, but strings that are unlikely to be found in unknown words or technical terms may be removed. For example, words consisting of only one character, numbers, or symbols may be removed. (10) Refer to the list of candidate words and the list of labels for spellings and named entities, and label each word in the list of candidate words as a named entity. (11) If a word in the extraction candidate list contains a named entity label in the excluded named entity label list (stored in advance by the information storage unit 10), the word is removed from the extraction candidate list. The named entity labels in the excluded named entity label list can be set arbitrarily, but they may be named entity labels that are rarely included in unknown words or technical terms. For example, the named entity labels in the excluded named entity label list include "quantity," "date," and "time." (12) The list (extraction candidate list) created in process (11) is output as an unknown word / technical term list.

[0045] By normalizing the strings of named entities or strings that are less likely to constitute unknown words or technical terms, or by excluding words with named entity labels from the list of excluded named entity labels, it becomes possible to extract unknown words or technical terms with high accuracy.

[0046] The word selection unit 17 may output the unknown word / specialized term list to other functional blocks such as the model registration target word selection unit 18, store it in the information storage unit 10, transmit it to another device via the communication device 1004 described later, or display it on the output device 1006 (such as a display) described later.

[0047] Figure 12 shows an example table of a list of named entity labels to be excluded from extraction. The list of named entity labels to be excluded from extraction shown in Figure 12 includes "quantity," "date," and "time" as named entity labels.

[0048] Figure 13 shows an example of a table for the unknown word / technical term list. The unknown word / technical term list shown in Figure 13 is the result of combining the extraction candidate list shown in Figure 8 and the extraction candidate list shown in Figure 11, removing the duplicate elements of the notation "Heisei" (one of them) and the notation "Hirobumi Ito" (one of them), and removing the notation "Heisei" (the other one remaining after the aforementioned duplication was removed) which is assigned the named entity label "Date" and is included in the non-extraction named entity label list shown in Figure 12 from the notation and named entity label list shown in Figure 9. It includes the notations "Company A", "Meeting", "Ito", "Hirobumi", "Hirobumi Ito", "New Service B", and "Proposal".

[0049] The model registration target word selection unit 18 outputs word candidates extracted by the relation extraction word extraction unit 15, excluding those whose frequency of occurrence in a predetermined dictionary (such as a word frequency dictionary) is higher than a predetermined standard. The predetermined dictionary may be a dictionary in which the word candidates output by the model registration target word selection unit 18 are registered. A detailed explanation follows below.

[0050] The model registration target word selection unit 18 outputs only words with low occurrence frequency (occurrence frequency lower than a predetermined standard) from the word occurrence frequency dictionary (pre-stored by the information storage unit 10) as a list of words to be registered. More specifically, the model registration target word selection unit 18 performs the following processes (13) and (14) on the unknown word / technical term list input from the word selection unit 17 (or stored by the information storage unit 10). (13) For each word w in the list of unknown words and technical terms, refer to the word frequency dictionary. If the word exists, perform (a) below; otherwise, perform (b) below. (a) The word frequency P of word w in the word frequency dictionary w <If the threshold T is met, add word w to the list of words to be registered (or, if the word frequency P is met) w Words w with a threshold T above this threshold will not be added to the list of words to be registered (they will be deleted). (b) Add word w to the list of words to register. (14) Output the list of words to be registered.

[0051] The word frequency dictionary used is one included in the natural language processing model, such as speech recognition or machine translation, where the words are registered. If the model does not have a word frequency dictionary in its format, the extractor (e.g., extractor 1) will create one based on the text data used to build the model. Furthermore, if the word frequency can be calculated, there are no restrictions on the format of the word frequency dictionary.

[0052] To utilize natural language processing technologies such as speech recognition or machine translation in specific domains or situations, it is necessary to register not only unknown words but also less frequently occurring words in the corresponding dictionary. This allows for highly accurate use of natural language processing technologies. Therefore, words with low frequency (lower than a predetermined threshold) are also included in the word frequency dictionary for registration.

[0053] The model registration target word selection unit 18 may output the list of target words to another functional block such as the dictionary registration unit 19, store it in the information storage unit 10, transmit it to another device via the communication device 1004 described later, or display it on the output device 1006 (such as a display) described later.

[0054] Figure 14 shows an example of a word frequency dictionary table. The word frequency dictionary shown in Figure 14 includes the word "meeting" with a word frequency of "-2.4", the word "proposal" with a word frequency of "-3.5", the word "Ito" with a word frequency of "-4.6", the word "Hirobumi" with a word frequency of "-6.2", and the word "Hirobumi Ito" with a word frequency of "-8.4". As shown in Figure 14, the threshold is set to "-5.0".

[0055] Figure 15 shows an example of a table for the list of words to be registered. The list of words to be registered shown in Figure 15 includes the words "Hirobumi" and "Ito Hirobumi," which are from the list of unknown words and technical terms shown in Figure 13, and whose word frequency in the word frequency dictionary shown in Figure 14 is less than the threshold "-5.0," as well as the words "Company A" and "New Service B," which are not present in the word frequency dictionary (words "meeting," "proposal," and "Ito," which have a word frequency of "-5.0" or higher, have been removed).

[0056] The dictionary registration unit 19 registers the word candidates extracted by the relational word extraction unit 15 into a dictionary to be used in a predetermined natural language processing operation. The predetermined natural language processing operation is, for example, speech recognition or machine translation.

[0057] Next, an example of the process (extraction process) performed by the extraction device 1 will be explained with reference to Figure 16. Figure 16 is a flowchart showing an example of the process performed by the extraction device 1 according to this embodiment.

[0058] First, the text acquisition unit 11 acquires text (extracted text) from the input data (step S1). Next, the morphological analysis unit 12 performs morphological analysis on the text acquired in S1 (step S2). Next, the named entity extraction unit 13 performs named entity extraction on each morpheme extracted in S2 (step S3). Next, the relation extraction unit 14 performs relation extraction on each named entity extracted in S3 (step S4) and outputs the named entities as an extraction candidate list (step S5). Next, the relation extraction word extraction unit 15 determines whether or not there are related words to the named entities extracted in S3 (step S6).

[0059] If it is determined in S6 that there is a relationship (S6:YES), the relationship extraction word extraction unit 15 combines the related words with the words in between and outputs it as an extraction candidate list (step S7). If it is determined in S6 that there is no relationship (S6:NO), or following S7, the morphological analysis word extraction unit 16 divides the morphological analysis result in S2 into words that are not to be extracted and outputs the divided strings as an extraction candidate list (step S8). Next, the word selection unit 17 combines the extraction candidate lists output in S7 and S8, removes duplicate elements (step S9), normalizes the characters included in the extraction candidate list (step S10), and removes words containing named entities that are not to be extracted (step S11). Next, the model registration target word selection unit 18 outputs only the words with an occurrence frequency of less than T from the unknown word / technical term list that has been processed in S9 to S11 in list format (step S12).

[0060] Note that S8 may be executed at any time after S2 and before S9. S8 may be omitted. S9 may be omitted. S10 may be omitted. S11 may be omitted. S12 may be omitted. After S12, the dictionary registration unit 19 may register the word output in S12 in the dictionary.

[0061] Next, the operation and effects of the extraction apparatus 1 according to this embodiment will be described.

[0062] The extraction device 1 includes a relation extraction word extraction unit 15 that acquires relation information regarding the relationships between named entities extracted by relation extraction to named entities extracted from the input sentence, and extracts and outputs strings in the input sentence based on the two related named entities indicated by the acquired relation information as word candidates for unknown words or technical terms. With this configuration, strings in the input sentence based on the two related named entities extracted from the input sentence are extracted and output as word candidates for unknown words or technical terms. In other words, it is possible to extract word candidates for unknown words or technical terms.

[0063] Furthermore, in the extraction device 1, the string may be a string obtained by concatenating the words in the input sentence in word order, from the named entity closer to the beginning of the input sentence to the other named entity among two related named entities. With this configuration, a string obtained by concatenating the words in the input sentence in word order, from the named entity closer to the beginning of the input sentence to the other named entity among two related named entities, is extracted and output as a candidate for an unknown word or technical term. By concatenating related named entities with the words between them, it becomes possible to extract unknown words or technical terms that consist of compound words of multiple named entities, or compound words of multiple named entities and common words. Because the concatenation is performed while considering the relationships between words, it is possible to extract unknown words or technical terms more robustly compared to concatenating surrounding words using only the morphological analysis results.

[0064] Furthermore, according to the extraction device 1, the relational word extraction unit 15 may divide the input sentence into predetermined parts of speech, and further extract and output each of the divided strings excluding the predetermined part of speech as word candidates. Unknown words or technical terms may include words that are not composed of proper nouns. The constituent parts of speech are often nouns or alphabetic sequences. By removing words of parts of speech that are rarely used in unknown words or technical terms based on the part of speech information from the morphological analysis results, words composed of compound words can also be extracted as unknown words or technical terms.

[0065] Furthermore, the extraction device 1 may further include a word selection unit 17 that outputs word candidates that contain words of a predetermined type of named entity from among the word candidates extracted by the related word extraction unit 15. By excluding words that have been assigned a named entity label in the list of excluded named entity labels (word candidates that contain words of a predetermined type of named entity) from named entities or strings that are unlikely to constitute unknown words or technical terms, unknown words or technical terms can be extracted with high accuracy.

[0066] Furthermore, the extraction device 1 may further include a model registration target word selection unit 18 that outputs word candidates extracted by the relational word extraction unit 15, excluding those word candidates whose frequency of appearance in a predetermined dictionary is higher than a predetermined standard. The predetermined dictionary may be a dictionary that registers the word candidates output by the model registration target word selection unit 18. In order to use natural language processing technologies such as speech recognition or machine translation in a specific domain or situation, registering not only unknown words but also words that do not appear often in the corresponding dictionary (the predetermined dictionary) will enable the use of natural language processing technologies with high accuracy. For this reason, words with low frequency of appearance in the word frequency dictionary (the predetermined dictionary) may also be included as targets for word registration.

[0067] Furthermore, the extraction device 1 may also include a dictionary registration unit 19 that registers the word candidates extracted by the relational word extraction unit 15 into a dictionary used in a predetermined natural language processing method. By registering the word candidates extracted by the relational word extraction unit 15 into a predetermined natural language processing method such as speech recognition or machine translation, the accuracy of each method can be improved.

[0068] Extraction device 1 relates to an unknown word and technical term extraction device and extraction method.

[0069] In the prior art described in the patent document "Japanese Patent Publication No. 2006-155528," it is difficult to extract unknown words if they are compound words of known words or if they are not correctly identified as unknown words during morphological analysis. According to the extraction device 1, for example, unknown word candidates are extracted based on part-of-speech information, so even words that are not identified as unknown words during morphological analysis can be extracted. Furthermore, since unknown words or technical terms often contain named entities, it is possible to extract unknown words or technical terms more robustly by using named entity recognition.

[0070] The prior art described in the patent document "Japanese Patent Publication No. 2006-139686" is difficult to use for words for which there is no corresponding text data or for which there is little or no corresponding text data. According to the extraction device 1, for example, unknown words or technical terms can be extracted even from text in which the unknown word appears in only one place.

[0071] In the prior art described in Patent Document No. 2022-42033, it is difficult to extract words with the correct delimiters depending on the accuracy of speech recognition. More specifically, when a phoneme sequence containing unknown words is converted into a word sequence by a language model, the accuracy of the conversion to phonemes of common words decreases due to the influence of the unknown word portion, so there is a high possibility that phonemes of common words will also be converted into incorrect words. As a result, when comparing morphological sequences and word sequences, there is a possibility that incorrect delimiters will be output as unknown words. For example, the phoneme sequence of the text "Reiwa ni wa" becomes "Reiwa Niwa" (phoneme sequence is written in moras for readability), the word sequence becomes "0 wa 2 wa", and the text "Reiwa ni wa" is output as an unknown word. According to the extraction device 1, words can be extracted robustly regardless of the performance of the natural language processing model to which the words are registered.

[0072] The extraction device 1 may perform, for example, the following processes (A) to (F). (A) Extract text from the input data. (B) Morphological analysis, named entity recognition, and relational analysis were performed on the text. (C) Create a list of candidate names from the results of named entity recognition and relation recognition. (D) Create a list of candidate extractors from the morphological analysis results and the list of parts of speech to be excluded from extraction. (E) Create a list of unknown words and technical terms from the extraction candidate list and the list of named entities to be excluded from the extraction in processes (C) and (D). (F) Create a list of words to be registered from the word frequency list associated with the machine learning model to be registered.

[0073] According to the extraction device 1, the present invention reduces the burden on the user by automatically extracting unknown words or technical terms from text data in which such words appear. Furthermore, by registering the words extracted by the extraction device 1 into natural language processing technologies such as speech recognition or machine translation, the accuracy of each technology can be improved.

[0074] The unknown words and technical terms created by the extraction device 1 can be used as word registration data for natural language processing technologies such as speech recognition or machine translation. There are no restrictions on the format of the algorithms or models used in speech recognition or machine translation. The unknown words and technical terms extracted by the extraction device 1 can be used, for example, in the technology (word extraction unit) described in the patent document "Japanese Patent Application Publication No. 2022-74852".

[0075] The extraction device 1 of this disclosure may have the following configuration.

[0076] [a] A text extraction unit that extracts the text portion from the input data, A morphological analysis unit that performs morphological analysis on text data, A named entity recognition unit performs named entity recognition on the morphological analysis results, A relation extraction unit performs relation extraction on the named entity recognition results, A relational extraction and word extraction unit creates a list of candidate words for extraction from named entity information and the results of relational extraction. A morphological analysis word extraction unit creates a list of extraction candidates based on morphological information and parts of speech not to be extracted, A word selection unit performs normalization and removes named entities that are not to be extracted from the candidate lists of the relational word extraction unit and the morphological analysis word extraction unit. Based on a word frequency dictionary, the model selects words to be registered, outputting only low-frequency words as a list of words to be registered. A method and apparatus for extracting unknown words and technical terms, comprising:

[0077] [i] Text extraction unit Extract the text portion from the input data.

[0078] [U] Morphological Analysis Unit Morphological analysis is performed on the text data.

[0079] [E] Named entity recognition unit Named entity recognition is performed on the morphological analysis results.

[0080] [O] Relationship extraction unit Relationship extraction is performed on the named entity recognition results.

[0081] [Ka] Related word extraction unit Based on the extracted named entities and the results of relation extraction, words are combined to create a list of extraction candidates.

[0082] [Ki] Morphological analysis word extraction unit Based on the morphological analysis results and the parts of speech excluded from extraction, the word list is split and combined to create a list of extraction candidates.

[0083] [Ku] Word selection section The candidate lists for the relational word extraction unit and the morphological analysis word extraction unit are normalized, and named entities that are not to be extracted are removed.

[0084] [Ke] Select word to register as model Based on a word frequency dictionary, only words with low frequency are output as a list of words to be registered.

[0085] The extraction device 1 of this disclosure may have the following configuration.

[0086] [1] An extraction device comprising an extraction unit that acquires relational information regarding the relationships between named entities extracted by relational extraction of named entities extracted by named entity extraction from an input sentence, and extracts and outputs strings in the input sentence based on two related named entities indicated by the acquired relational information as word candidates for unknown words or technical terms.

[0087] [2] The aforementioned string is a string obtained by concatenating the words in the input sentence in word order, from the named entity closer to the beginning of the input sentence to the other named entity among two related named entities. [1] The extraction apparatus described above.

[0088] [3] The extraction unit divides the input sentence into predetermined parts of speech, and further extracts and outputs each of the divided strings excluding the predetermined part of speech as word candidates. The extraction apparatus described in [1] or [2].

[0089] [4] The system further includes a type selection unit that outputs word candidates from the extraction unit that include words of a predetermined type of named entity. An extraction apparatus as described in any one of items [1] to [3].

[0090] [5] The system further includes a frequency selection unit that outputs word candidates extracted by the extraction unit, excluding those whose frequency of appearance in a predetermined dictionary is higher than a predetermined standard. An extraction apparatus as described in any one of items [1] to [4].

[0091] [6] The aforementioned predetermined dictionary is a dictionary for registering the word candidates output by the frequency selection unit. [5] The extraction apparatus described above.

[0092] [7] The system further includes a registration unit that registers the word candidates extracted by the extraction unit into a dictionary used in a predetermined natural language processing system. An extraction apparatus as described in any one of items [1] to [6].

[0093] The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining the above one device or the above multiple devices with software.

[0094] Functions include, but are not limited to, judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. As mentioned above, the method of implementation is not particularly limited.

[0095] For example, the extraction device 1 in one embodiment of the present disclosure may function as a computer that processes the extraction method of the present disclosure. Figure 17 is a diagram showing an example of the hardware configuration of the extraction device 1 according to one embodiment of the present disclosure. The above-described extraction device 1 may be physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0096] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of extraction device 1 may include one or more of the devices shown in the figure, or it may be configured to omit some of the devices.

[0097] Each function in the extraction device 1 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which allows the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of the reading and writing of data in the memory 1002 and storage 1003.

[0098] The processor 1001 controls the entire computer, for example, by running the operating system. The processor 1001 may be composed of a central processing unit (CPU) that includes interfaces with peripheral devices, control units, arithmetic units, registers, etc. For example, the above-mentioned text acquisition unit 11, morphological analysis unit 12, named entity recognition unit 13, relation extraction unit 14, relation extraction word extraction unit 15, morphological analysis word extraction unit 16, word selection unit 17, model registration target word selection unit 18, and dictionary registration unit 19 may be implemented by the processor 1001.

[0099] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, the text acquisition unit 11, morphological analysis unit 12, named entity recognition unit 13, relation extraction unit 14, relation extraction word extraction unit 15, morphological analysis word extraction unit 16, word selection unit 17, model registration target word selection unit 18, and dictionary registration unit 19 may be stored in the memory 1002 and implemented by a control program that operates on the processor 1001, and other functional blocks may be implemented similarly. The above-described various processes have been explained as being executed by one processor 1001, but they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0100] Memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 1002 may also be called a register, cache, main memory, etc. Memory 1002 can store executable programs (program code), software modules, etc., for carrying out a wireless communication method according to one embodiment of the present disclosure.

[0101] Storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. Storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003.

[0102] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc. The communication device 1004 may be configured to include, for example, a high-frequency switch, duplexer, filter, frequency synthesizer, etc., in order to implement at least one of frequency division duplex (FDD) and time division duplex (TDD). For example, the above-mentioned text acquisition unit 11, morphological analysis unit 12, named entity recognition unit 13, relation extraction unit 14, relation extraction word extraction unit 15, morphological analysis word extraction unit 16, word selection unit 17, model registration target word selection unit 18, and dictionary registration unit 19 may be implemented by the communication device 1004.

[0103] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0104] Furthermore, each device, such as the processor 1001 and memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.

[0105] Furthermore, the extraction device 1 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.

[0106] The notification of information is not limited to the manner / embodiments described herein and may be carried out by other means.

[0107] Each aspect / embodiment described in this disclosure may be applied to at least one of the following systems: LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (new Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark), and other appropriate systems, as well as next-generation systems extended based thereon. Furthermore, multiple systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A with 5G).

[0108] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described herein may be reordered, provided they are consistent with each other. For example, the methods described herein present various step elements in an exemplary order and are not limited to that specific order.

[0109] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be transmitted to other devices.

[0110] The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0111] Each aspect / embodiment described herein may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0112] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0113] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.

[0114] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technology (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technology (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0115] The information, signals, etc. described in this disclosure may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0116] In addition, terms used in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meaning.

[0117] The terms “system” and “network” as used in this disclosure are interchangeable.

[0118] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0119] The names used for the parameters described above are not restrictive in any way. Furthermore, the formulas and other expressions using these parameters may differ from those expressly disclosed in this disclosure.

[0120] As used in this disclosure, the terms “determining” and “determining” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiry (e.g., searching in a table, database, or other data structure), and ascertaining. “Determining” may also include, for example, receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, and accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0121] The terms “connected,” “coupled,” or any variation thereof, mean any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are “connected” or “coupled” with each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, “connection” may be reinterpreted as “access.” As used in this disclosure, two elements may be considered to be “connected” or “coupled” with each other using at least one of one or more wires, cables, and printed electrical connections, and, in some non-limiting and non-exclusive examples, electromagnetic energy having wavelengths in the radio frequency domain, microwave domain, and optical (both visible and invisible) domain.

[0122] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0123] Any reference to elements using the designations “first,” “second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the first and second elements do not imply that only two elements may be employed, or that the first element must precede the second element in any way.

[0124] In the configuration of each of the above devices, "means" may be replaced with "part," "circuit," "device," etc.

[0125] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0126] In this disclosure, if articles are added by translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0127] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different." [Explanation of Symbols]

[0128] 1... Extraction device, 10... Information storage unit, 11... Text acquisition unit, 12... Morphological analysis unit, 13... Named entity recognition unit, 14... Relationship extraction unit, 15... Relationship extraction word extraction unit, 16... Morphological analysis word extraction unit, 17... Word selection unit, 18... Model registration target word selection unit, 19... Dictionary registration unit, 1001... Processor, 1002... Memory, 1003... Storage, 1004... Communication device, 1005... Input device, 1006... Output device, 1007... Bus.

Claims

1. An extraction unit that obtains relational information regarding the relationships between named entities extracted by relational extraction of named entities extracted by named entity extraction from the input sentence, and extracts and outputs strings in the input sentence based on two related named entities indicated by the obtained relational information as word candidates for unknown words or technical terms. A frequency selection unit outputs the word candidates extracted by the extraction unit, excluding those whose frequency of appearance in a predetermined dictionary is higher than a predetermined standard. An extraction device equipped with the following features.

2. The aforementioned string is a string obtained by concatenating the words in the input sentence in word order, from the named entity closer to the beginning of the input sentence to the other named entity among two related named entities. The extraction apparatus according to claim 1.

3. The extraction unit divides the input sentence into predetermined parts of speech, and further extracts and outputs each of the divided strings excluding the predetermined part of speech as word candidates. The extraction apparatus according to claim 1.

4. The system further includes a type selection unit that outputs word candidates from the extraction unit that include words of a predetermined type of named entity. The extraction apparatus according to claim 1.

5. The aforementioned predetermined dictionary is a dictionary for registering the word candidates output by the frequency selection unit. The extraction apparatus according to claim 1.

6. The system further includes a registration unit that registers the word candidates extracted by the extraction unit into a dictionary used in a predetermined natural language processing system. The extraction apparatus according to claim 1.

Images