Language processing device, machine learning method, estimation method and program

Abstract

To improve classification performance for low-frequency words.SOLUTION: A language processing device 3 comprises a supervised knowledge corpus storage unit 21 that stores a plurality of supervised knowledge sentences with text data containing unique expressions and label information indicating whether or not each word in the text data is a unique expression, the language processing device 3 obtains an input word series containing a predetermined unique expression and an output label series containing label information indicating whether or not each word in the input word series is a predetermined unique expression, searches the supervised knowledge corpus storage unit 21 based on the predetermined unique expression and reads out a predetermined supervised knowledge sentence containing the same unique expression as the predetermined unique expression (32), then inserts the given supervised knowledge sentence into the input word series to generate an input word series with the knowledge sentence inserted and passes it to a language processing unit 34, and inserts the predetermined supervised knowledge sentence into the output label series to generate an output label series with the knowledge sentence inserted and passes it to a loss calculation unit 35 (33).SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to natural language processing using natural language models, and in particular to named entity recognition. [Background technology]

[0002] In recent years, research into natural language processing using natural language models such as BERT (Bidirectional Encoder Representations from Transformers) has progressed (see Non-Patent Document 1). Natural language processing includes a task called named entity recognition (NER). A named entity is a word or phrase that indicates a specific thing, and generally refers to a proper noun, specifically a noun. Examples of proper nouns include place names such as "Tokyo" and personal names such as "Suzuki." Named entity recognition is a task that extracts named entities contained in a given sentence (text data) and classifies the named entities into predefined labels (Entity Categories in Figure 9).

[0003] For example, as shown in Fig. 10, the language processing unit 200 receives an input word sequence (NTT / What is / Nippon Telegraph and Telephone / Corporation / No / Koto / Desu) and assigns a label indicating whether or not each input character (word) is a named entity, thereby outputting an output label sequence (B-ORG / O / B-ORG / I-ORG / O / O / O). In this case, the label "O" is a word that is not a named entity, "B-XXX" is the first word in category X in Fig. 9, and the label "I-XXX" is a word in category X that belongs to the same named entity as the previous word (for example, "Nippon Telegraph and Telephone" is a named entity that is paired with "Corporation" and forms "Nippon Telegraph and Telephone Corporation"). In Fig. 10, ORG is an abbreviation for Organization. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (https: / / arxiv.org / abs / 1810.04805) Summary of the Invention [Problem to be solved by the invention]

[0005] However, many named entities have low frequencies. Hereafter, low-frequency named entities will be referred to simply as "low-frequency words." If the language processor has never dealt with low-frequency words, it must classify the low-frequency words into the correct labels based only on the input word sequence. This creates the problem of poor classification performance for low-frequency words.

[0006] The present invention has been made in view of the above points, and has as its object to improve classification performance for low-frequency words. [Means for solving the problem]

[0007] In order to solve the above problem, the invention of claim 1 is a machine learning method executed by a language processing device that trains a machine learning model in a learning phase, the language processing device comprising a supervised knowledge corpus storage unit that stores a plurality of supervised knowledge sentences each having text data including named entities and label information indicating whether or not each word in the text data is the named entity, the language processing device acquiring an input word sequence including a predetermined named entity and an output label sequence including label information indicating whether or not each word in the input word sequence is the predetermined named entity, searching the supervised knowledge corpus storage unit based on the predetermined named entity to read out predetermined supervised knowledge sentences each including the same named entity as the predetermined named entity; a knowledge insertion process for generating an input word sequence with knowledge sentences inserted by inserting the predetermined supervised knowledge sentence into a sequence, and for generating an output label sequence with knowledge sentences inserted by inserting the predetermined supervised knowledge sentence into the output label sequence; a language process for converting the input word sequence with knowledge sentences inserted generated by the knowledge insertion process into a score vector sequence using the machine learning model; a loss calculation process for calculating a loss function value using the score vector sequence converted by the language processing and the output label sequence with knowledge sentences inserted generated by the knowledge insertion process; and a parameter update process for updating parameters of the machine learning model based on the loss function value calculated by the loss calculation process. [Effects of the Invention]

[0008] As described above, the present invention has the effect of improving classification performance for low-frequency words compared to the prior art. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic diagram of a communication system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a diagram illustrating the hardware configuration of a language processing device and a communication terminal. [Figure 3]FIG. 2 is a functional configuration diagram of the language processing apparatus of the present embodiment in a learning phase. [Figure 4] 10 is a table showing the relationship between an input word sequence and each knowledge sentence. [Figure 5] FIG. 2 is a functional configuration diagram of the language processing apparatus of the present embodiment in an estimation phase. [Figure 6] 10 is a flowchart showing the processing executed by the language processing device in the learning (training) phase. [Figure 7] 10 is a flowchart showing the processing executed by the language processing device in the estimation (inference) phase. [Figure 8] FIG. 10 is a diagram showing evaluation results obtained by the conventional method and the method of the present embodiment. [Figure 9] 1 is a table showing predefined labels for named entities. [Figure 10] FIG. 1 is a conceptual diagram illustrating natural language processing. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] [System configuration of the embodiment] First, an outline of the configuration of a communication system 1 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a schematic diagram of a communication system according to an embodiment of the present invention.

[0012] 1, the communication system 1 of this embodiment is constructed by a language processing device 3 and a communication terminal 5. The communication terminal 5 is managed and used by a user Y.

[0013] Furthermore, the language processing device 3 and the communication terminal 5 can communicate with each other via a communication network 100 such as the Internet. The communication network 100 may be connected wirelessly or via a wired connection.

[0014] The language processing device 3 is configured by one or more computers. When the language processing device 3 is configured by multiple computers, it may be referred to as a "language processing device" or a "language processing system."

[0015] The language processing device 3 is a device that performs natural language processing, which is a task called named entity recognition (NER), using AI (Artificial Intelligence). The language processing device 3 of this embodiment outputs result data indicating a predicted label sequence (described below) that improves classification performance for low-frequency words, even for text data including low-frequency named entities (hereinafter referred to as "low-frequency words") that BERT has not handled. Examples of output methods include transmitting the result data to the communication terminal 5, thereby displaying or printing a graph or the like related to the classification data on the communication terminal 5 side, displaying the graph or the like on a display connected to the language processing device 3, or printing the graph or the like on a printer or the like connected to the language processing device 3.

[0016] The communication terminal 5 is a computer, and in FIG. 1, a notebook computer is shown as an example, but it is not limited to a notebook computer and may be a desktop computer. The communication terminal may also be a smartphone or a tablet terminal. In FIG. 1, a user Y operates the communication terminal 5.

[0017] [Hardware configuration of language processing device and communication terminal] [Hardware configuration] Next, the electrical hardware configuration of the language processing device 3 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the electrical hardware configuration of the language processing device.

[0018] As shown in FIG. 2, the language processing device 3 is a computer and includes a CPU (Central Processing Unit) 3101, a ROM (Read Only Memory) 302, a RAM (Random Access Memory) 303, an SSD (Solid State Drive) 304, an external device connection I / F (Interface) 305, a network I / F 306, a media I / F 309, and a bus line 310.

[0019] Of these, the CPU 301 controls the overall operation of the language processing device 3. The ROM 302 stores programs such as an IPL (Initial Program Loader) used to drive the CPU 301. The RAM 303 is used as a work area for the CPU 301.

[0020] The SSD 304 reads or writes various data under the control of the CPU 301. Note that instead of the SSD 304, a hard disk drive (HDD) may be used.

[0021] The external device connection I / F 305 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB (Universal Serial Bus) memory, and a printer.

[0022] The network I / F 306 is an interface for performing data communication via a communication network such as the physical network 300 .

[0023] The media I / F 309 controls reading and writing (storing) of data from and to a recording medium 309m such as a flash memory, etc. The recording medium 309m includes a DVD (Digital Versatile Disc) and a Blu-ray Disc (registered trademark).

[0024] The bus line 310 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 301 shown in FIG.

[0025] The communication terminal 5 has the same configuration as the language processing device 3, and therefore a description of the hardware configuration will be omitted.

[0026] [Functional configuration of language processing device] Next, the functional configuration of the language processing device 3 according to this embodiment in the learning phase and estimation (inference) phase will be described.

[0027] <Functional configuration of the learning (training) phase> FIG. 3 is a functional configuration diagram of the language processing apparatus of this embodiment in the learning (training) phase.

[0028] (Storage unit, etc.) In the RAM 303 or the SSD 304 in FIG. 2, a training data storage unit 20, a supervised knowledge corpus storage unit 21, an unsupervised knowledge corpus storage unit 22, and a machine learning model 40 are constructed.

[0029] The training data storage unit 20 stores a plurality of pairs of data of input word sequences and data of output label sequences as training (teaching) data.

[0030] The supervised knowledge corpus storage unit 21 stores a supervised knowledge corpus. The supervised knowledge corpus includes text data that is decomposed into a word sequence and includes named entities, and label information indicating whether each word in the text data is a named entity or not is associated with the text data. The text data represents, for example, "NTT / means / Telecom Corporation / of / Japan" decomposed into a word sequence. The corresponding label information represents, for example, "B-ORG / O / B-ORG / I-ORG / O / O / O". In this case, the label "O" represents a word that is not a named entity, "B-XXX" represents the first word of category X in FIG. 9, and the label "I-XXX" represents a word in category X that belongs to the same named entity as the previous word (for example, "Telecom Corporation" and "of Japan" form a single named entity "Telecom Corporation of Japan" as a set). Note that ORG is an abbreviation of Organization.

[0031] The unsupervised knowledge corpus storage unit 22 stores an unsupervised knowledge corpus. The unsupervised knowledge corpus includes text data that is decomposed into a word sequence and includes named entities, similar to the case of the supervised knowledge corpus, but does not include label information.

[0032] As described above, although the knowledge sentence is arbitrary text data (and label information), for example, it is expected to satisfy the following conditions. · Text data that refers to named entities in the input word sequence · Text data that shares many words with the input word sequence Thus, it is expected that data of the input word sequence can be accurately classified by complementing named entities that often lack necessary information for classification, such as low-frequency words.

[0033] The machine learning model 40 represents a mechanism for deriving a result (output) for input data in machine learning and is constructed by a number of (model) parameters.

[0034] (Each functional configuration) 3, the language processing device 3 includes a sampling unit 30, a search unit 32, a knowledge insertion unit 33, a language processing unit 34, a loss calculation unit 35, and a parameter update unit 36. Each of these units has a function realized by an instruction from the CPU 301 in FIG. 2 based on a program. Among these units, the sampling unit 30 samples predetermined training data (pairs of an input word sequence including a predetermined named entity and an output label sequence including label information indicating whether or not each word in the input word sequence is the predetermined named entity) from the training data storage unit 20, and outputs the data to the search unit 32 and the knowledge insertion unit 33.

[0035] The search unit 32 acquires an input word sequence including a predetermined named entity from the sampling unit 30, searches the supervised knowledge corpus storage unit 21 based on the predetermined named entity, and reads out data of predetermined supervised knowledge sentences including the same named entity as the predetermined named entity. The search unit 32 also searches the unsupervised knowledge corpus storage unit 22 based on the predetermined named entity under predetermined conditions, and reads out data of predetermined unsupervised knowledge sentences including the same named entity as the predetermined named entity. Specifically, the search unit 32 uses a predetermined named entity in the input word sequence as a query, and reads out data of knowledge sentences including the same named entity from the supervised knowledge corpus storage unit 21 and the unsupervised knowledge corpus storage unit 22.

[0036] The named entities are determined in advance by extracting them from all text data in the training data, the supervised knowledge corpus, and the unsupervised knowledge corpus. The named entities are extracted using a machine learning model (named entity extraction model) separate from the machine learning model 40. The search unit 32 holds a table as a database for managing named entities contained in a sentence (text data). In the learning phase, the search unit 32 reads out knowledge sentences containing a specified named entity that is the same as a specified named entity in the input word sequence from the supervised knowledge corpus storage unit 21 (including the unsupervised knowledge corpus storage unit 22 under specified conditions).

[0037] In this way, the search unit 32 always reads out supervised knowledge sentences containing a predetermined named entity from the supervised knowledge corpus storage unit 21, but may not read out unsupervised knowledge sentences containing a predetermined named entity from the unsupervised knowledge corpus storage unit 22. This is performed under the following conditions. That is, when the search unit 32 retrieves a predetermined number or more of supervised knowledge sentences containing the same named entity as the predetermined named entity included in the input word sequence by searching the supervised knowledge corpus storage unit 21, the search unit 32 randomly retains a predetermined number of knowledge sentences from the supervised knowledge sentences and discards the other read supervised knowledge sentences. On the other hand, when the number of retrieved knowledge sentences is less than the predetermined number, the search unit 32 first retains all supervised knowledge sentences containing the predetermined named entity, and then reads out unsupervised knowledge sentences containing the same named entity as the predetermined named entity included in the input word sequence from the unsupervised knowledge corpus storage unit 22, and then randomly retains unsupervised knowledge sentences until the predetermined number is reached and discards the other read unsupervised knowledge sentences.

[0038] The knowledge insertion unit 33 converts data of supervised knowledge sentences and (under predetermined conditions) unsupervised knowledge sentences into a format understandable by the language processing unit 34. To this end, the knowledge insertion unit 33 generates a knowledge-sentence-inserted input word sequence by inserting predetermined supervised knowledge sentences (and unsupervised knowledge sentences under predetermined conditions) into the input word sequence acquired from the sampling unit 30. Furthermore, the knowledge insertion unit 33 generates a knowledge-sentence-inserted output label sequence by inserting predetermined supervised knowledge sentences (and unsupervised knowledge sentences under predetermined conditions) into the output label sequence acquired from the sampling unit 30.

[0039] Here, a detailed description will be given of the knowledge insertion unit 33. First, strictly speaking, there are three types of input data to BERT, which is the language processing unit 34: (1) Word ID sequence: A sequence of predefined words converted into fixed IDs using fixed IDs (e.g., [1, 3101, 4124, 103,…]). (2) Position ID sequence: A sequence representing word positions (e.g., [1, 2, 3, 4, …]) (3) Token type ID series: A series in which a predefined series type is converted into a fixed ID by using a fixed ID (e.g., [1, 1, 1, 1, …]). In the case of (3), in a typical named entity recognition task, all the same sequence types are used, such as [1, 1, 1, 1,…]. Also, in a task of determining sentence similarity, the first sentence is assigned a “1” and the second sentence is assigned a “2.” BERT only has two sequence types: “1” and “2.”

[0040] In this embodiment, by using a unique token type id, label information included in a supervised knowledge sentence is given to the language processing unit 34 as input data as shown in (a), (b), and (c) below. (a) The token type id of the input word sequence is set to “1”. (b) The token type id of the unsupervised knowledge sentence is set to “2”. (c) The token type id of the supervised knowledge is set to the output label id “+3” described below.

[0041] This process (c) makes it possible to express the label information attached to the supervised knowledge sentence as a token type id. Then, the knowledge insertion unit 33 generates an input word sequence with inserted knowledge sentences by concatenating (i) the input word sequence, the word ID sequence of the supervised knowledge sentences (and the knowledge sentences without knowledge), (ii) the position ID sequence, and (iii) the token type ID sequence into one long word ID sequence, position ID sequence, and token type ID sequence.

[0042] Furthermore, in order to facilitate context understanding by the language processing unit 34, the knowledge insertion unit 33 performs masking processing on all named entities in the input word sequence into which knowledge sentences have been inserted with a predetermined probability when generating the input word sequence into which knowledge sentences have been inserted. That is, the knowledge insertion unit 33 performs masking processing on word IDs corresponding to a certain named entity. In this case, the knowledge insertion unit 33 replaces the word ID of the named entity with a masking ID prepared in advance. Without masking processing, the label of a certain named entity can be classified by a kind of rote memorization (such as the company name for NTT). Furthermore, masking processing makes it necessary to perform classification based only on surrounding context information. Therefore, the language processing unit 34 can promote learning of the machine learning model 40 by using context information for classification. In the experimental example described below, named entities were masked with a probability of 30%.

[0043] Furthermore, the knowledge insertion unit 33 converts predefined labels into fixed label ids to represent the output label sequence. In this case, the label id is equal to or greater than 0. However, if the label id is 0, the word corresponding to that label is considered to be outside the scope of classification, is not treated as input data to the loss calculation unit, and the loss calculation described below is not performed.

[0044] The output label sequence with knowledge sentences inserted output by the knowledge insertion unit 33 is a label ID sequence of the same length as the input word sequence with knowledge sentences inserted. Of the output label sequence with knowledge sentences inserted, the parts corresponding to the words in the input word sequence use the IDs of the output label sequence as they are, and the parts corresponding to the knowledge sentences are represented by 0.

[0045] The language processing unit 34 converts the input word sequence into which knowledge sentences have been inserted into a score vector sequence using the "BERT + linear transformation" model (see Non-Patent Document 1), which is widely used in language processing tasks. The dimension of the vector is the number of labels, and represents the score of a word being classified into that label.

[0046] Furthermore, the language processing unit 34 of this embodiment adds a mechanism for processing knowledge sentences at high speed to BERT. Here, high-speed processing will be described in detail.

[0047] Generally, BERT models dependencies between words by performing self-attention processing. The dependencies between words are expressed as an L × L matrix, where L is the length of the input sequence. If n is the number of knowledge sentences inserted into the input word sequence, the length of the input word sequence with knowledge sentences inserted and input to the language processing unit 34 is approximately (n + 1) times, so the calculation cost of the matrix is ​​(n + 1) times. 2 The problem arises that it will cost twice as much.

[0048] Therefore, in this embodiment, in order to reduce this increase in cost, a process is performed in which the dependency relationships across knowledge sentences (parts without circles) are ignored and not calculated, as shown in Figure 4. This reduces the calculation cost by (3n+1) times.

[0049] The loss calculation unit 35 typically uses Cross-Entropy Loss. The loss calculation unit 35 calculates a loss function value using the score vector sequence obtained from the language processing unit 34 and the output label sequence with knowledge sentences inserted obtained from the knowledge insertion unit 33.

[0050] The parameter update unit 36 ​​uses the backpropagation gradient method to calculate the gradient of each (model) parameter of the neural network machine learning model 40 based on the loss function value calculated by the loss calculation unit 35, and updates the parameters.

[0051] This completes the description of the functional configuration of the language processing device 3 in the learning phase.

[0052] <Functional configuration of the estimation (inference) phase> 5 is a functional configuration diagram of the language processing device of this embodiment in the estimation (inference) phase. Unlike the learning phase, the language processing device 3 in the estimation phase does not have the learning data storage unit 20 and the parameter update unit 36, but instead newly has an input unit 31 and an output unit 39. Note that functional configurations similar to those in the learning phase shown in FIG. 3 are assigned the same reference numerals, and their description will be omitted.

[0053] However, the search unit 32 in the estimation phase cannot extract named entities in advance from the input word sequence. Therefore, in the learning phase, the search unit 32 extracts a predetermined named entity from the input word sequence each time, and then reads out a knowledge sentence containing the same predetermined named entity as the predetermined named entity in the input word sequence from the supervised knowledge corpus storage unit 21 (which includes the unsupervised knowledge corpus storage unit 22 under predetermined conditions). Note that the language processing device 3 in the estimation phase has a trained machine learning model.

[0054] The input unit 31 inputs an input word sequence as data to be classified (evaluated). For example, the input unit 31 inputs data of an input word sequence that a user Y has transmitted from a communication terminal 5 to the language processing device 3 into the language processing device 3.

[0055] The output unit 39 calculates the dimension with the maximum score for all words (vectors) in the score vector sequence acquired from the language processing unit 34, and classifies all the words into the label corresponding to the maximum score dimension. The output unit 39 also outputs a sequence of the labels of the classification results as a predicted label sequence.

[0056] This completes the description of the functional configuration of the language processing device 3 in the estimation phase.

[0057] [Processing or Operation of the Embodiment] Next, the processing or operation of this embodiment will be described in detail with reference to Figures 6 and 7. The description will be divided into a learning phase and an estimation phase.

[0058] <Learning (training) phase processing> First, the learning phase will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the processing executed by the language processing device in the learning phase.

[0059] S11: The sampling unit 30 samples the input word sequence and the output word sequence from the training data stored in the training data storage unit 20.

[0060] S12: The search unit 32 searches the supervised knowledge corpus memory unit 21 based on a predetermined named entity included in the input word sequence to read out a predetermined supervised knowledge sentence (under predetermined conditions, the search unit 32 searches the unsupervised knowledge corpus memory unit 22 to read out a predetermined unsupervised knowledge sentence).

[0061] S13: The knowledge insertion unit 33 generates an input word sequence with inserted knowledge sentences by inserting a predetermined supervised knowledge sentence (a predetermined unsupervised knowledge sentence under predetermined conditions) into the input word sequence, and generates an output label sequence with inserted knowledge sentences by inserting a predetermined supervised knowledge sentence (a predetermined unsupervised knowledge sentence under predetermined conditions) into the output label sequence.

[0062] S14: The language processing unit 34 uses the machine learning model 40 to convert the input word sequence into which the knowledge sentence has been inserted into a score vector sequence.

[0063] S15: The loss calculation unit 35 calculates a loss function value using the output label sequence into which the knowledge sentence has been inserted and the score vector sequence.

[0064] S16: The parameter update unit 36 ​​updates the (model) parameters of the machine learning model 40 using the loss function value.

[0065] S17: If the process has not been completed the predetermined number of times (NO), the process returns to step S11. On the other hand, if the process has been completed the predetermined number of times (YES), the process of the learning phase shown in FIG. 6 ends.

[0066] <Inference phase processing> Next, the learning phase will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the processing executed by the language processing device in the estimation (inference) phase.

[0067] S21: The input unit 31 inputs an input word sequence including a predetermined named entity from the communication terminal 5 etc. Note that the input word sequence does not necessarily include a named entity.

[0068] S22: The search unit 32 searches the supervised knowledge corpus memory unit 21 based on a predetermined named entity included in the input word sequence to read out a predetermined supervised knowledge sentence (under predetermined conditions, the search unit 32 searches the unsupervised knowledge corpus memory unit 22 to read out a predetermined unsupervised knowledge sentence).

[0069] S23: The knowledge insertion unit 33 generates a knowledge-sentence-inserted input word sequence by inserting a predetermined supervised knowledge sentence (a predetermined unsupervised knowledge sentence under a predetermined condition) into the input word sequence.

[0070] S24: The language processing unit 34 uses the trained machine learning model 40 to convert the input word sequence into which the knowledge sentence has been inserted into a score vector sequence.

[0071] S25: The output unit 39 calculates the dimension with the maximum score for all words in the score vector sequence, classifies all the words into labels corresponding to the dimension with the maximum score, and outputs a predicted label sequence that lists the label information of the classification results.

[0072] This completes the learning phase process shown in FIG.

[0073] [Experimental Example] Next, an example of an experiment using the method of this embodiment will be described with reference to Fig. 8. Fig. 8 is a diagram showing the evaluation results of the conventional method and the method of this embodiment.

[0074] First, to evaluate the method of this embodiment, we performed an evaluation in the Politics domain of the Cross-NER dataset (see Reference 1). <Reference 1> CrossNER: Evaluating Cross-Domain Named Entity Recognition (https: / / arxiv.org / abs / 2012.04373) Before training on the Cross-NER dataset, we also trained on a pseudo-NER dataset created following the procedure of NER-BERT (see Reference 2). <Reference 2> NER-BERT: A Pre-trained Model for Low-Resource Entity Tagging (https: / / arxiv.org / abs / 2112.00405) In this case, the specified number of knowledge items was set to "2" when learning in the NER pseudo dataset, and to "9" when learning in the Cross-NER dataset. The evaluation metric was F1 (harmonic mean of Precision and Recall), which is common in NER, and the average and standard deviation of five results were recorded.

[0075] Figure 8 shows the evaluation results using development data. For comparison, the evaluation results using a machine learning model trained according to NER-BERT (see Reference 2) are also shown. Since the method of this embodiment and the above-mentioned NER-BERT method were conducted using the same experimental settings and available data, it was shown that the method of this embodiment improves the performance of NER by using knowledge sentences.

[0076] [Major Effects of the Embodiments] As described above, according to this embodiment, it is possible to improve classification performance by supplementing knowledge sentences even for named entities that often lack the information necessary for classification, such as low-frequency words.

[0077] 〔supplement〕 As described above, the present invention is not limited to the above-described embodiment, and various modifications and applications are possible, for example, as shown below.

[0078] (1) The language processing device 3 can be realized by a computer and a program, but this program can also be recorded on a (non-transitory) recording medium or provided via a communication network such as the Internet.

[0079] (2) The CPU 301 may be multiple, not just single. [Explanation of symbols]

[0080] 1. Communication Systems 3 Language Processing Unit 5. Communication terminals 30 Sampling section 31 Input section 32 Search section 33 Knowledge Insertion Section 34 Language Processing Unit 35 Loss calculation section 36 Parameter update section 39 Output section 20 Learning data storage unit 21 Supervised knowledge corpus memory 22 Unsupervised Knowledge Corpus Memory 40 Machine Learning Models

Claims

1. A machine learning method executed by a language processing device that trains a machine learning model in a learning phase, comprising: the language processing device includes a supervised knowledge corpus storage unit that stores a plurality of supervised knowledge sentences each having text data including a named entity and label information indicating whether or not each word in the text data is the named entity; The language processing device, a search process of acquiring an input word sequence including a predetermined named entity and an output label sequence including label information indicating whether or not each word in the input word sequence is the predetermined named entity, searching the supervised knowledge corpus storage unit based on the predetermined named entity, and reading out a predetermined supervised knowledge sentence including the same named entity as the predetermined named entity; a knowledge insertion process for generating a knowledge-sentence-inserted input word sequence by inserting the predetermined supervised knowledge sentence into the input word sequence, and for generating a knowledge-sentence-inserted output label sequence by inserting the predetermined supervised knowledge sentence into the output label sequence; a language processing step of converting the input word sequence into which the knowledge sentence has been inserted, which has been generated by the knowledge insertion process, into a score vector sequence using the machine learning model; a loss calculation process that calculates a loss function value using the score vector sequence converted by the language processing and the knowledge sentence-inserted output label sequence generated by the knowledge insertion process; a parameter update process for updating parameters of the machine learning model based on the loss function value calculated by the loss calculation process; Machine learning methods that perform

2. 2. The machine learning method of claim 1, the language processing device includes an unsupervised knowledge corpus storage unit that stores a plurality of unsupervised knowledge sentences having text data including named entities; when the search process is unable to read out a specified number of the predetermined supervised knowledge sentences from the supervised knowledge corpus storage unit, the search process includes a process of searching the unsupervised knowledge corpus storage unit based on the predetermined named entity and reading out predetermined unsupervised knowledge sentences including the same named entity as the predetermined named entity; the knowledge insertion process includes a process of generating an input word sequence into which the knowledge sentence has been inserted by inserting the predetermined unsupervised knowledge sentence in addition to the predetermined supervised knowledge sentence into the input word sequence, and a process of generating an output label sequence into which the knowledge sentence has been inserted by inserting the predetermined unsupervised knowledge sentence in addition to the predetermined supervised knowledge sentence into the output label sequence, Machine learning methods.

3. A language processing device that trains a machine learning model in a learning phase, a supervised knowledge corpus storage unit that stores a plurality of supervised knowledge sentences each having text data including a named entity and label information indicating whether or not each word in the text data is the named entity; an unsupervised knowledge corpus storage unit that stores a plurality of unsupervised knowledge sentences having text data including named entities; a search unit that acquires an input word sequence including a predetermined named entity and an output label sequence including label information indicating whether or not each word in the input word sequence is the predetermined named entity, searches the supervised knowledge corpus storage unit based on the predetermined named entity to read out predetermined supervised knowledge sentences including the same named entity as the predetermined named entity, and when a predetermined number of the predetermined supervised knowledge sentences cannot be read out from the supervised knowledge corpus storage unit, searches the unsupervised knowledge corpus storage unit based on the predetermined named entity to read out predetermined unsupervised knowledge sentences including the same named entity as the predetermined named entity; a knowledge insertion unit that inserts the predetermined supervised knowledge sentence into the input word sequence, and when the predetermined number of the predetermined supervised knowledge sentences cannot be read from the supervised knowledge corpus storage unit, further inserts the predetermined unsupervised knowledge sentence to generate a knowledge-sentence-inserted input word sequence, and that inserts the predetermined supervised knowledge sentence into the output label sequence, and when the predetermined number of the predetermined supervised knowledge sentences cannot be read from the supervised knowledge corpus storage unit, further inserts the predetermined unsupervised knowledge sentence to generate a knowledge-sentence-inserted output label sequence; a linguistic processing unit that converts the knowledge sentence-inserted input word sequence generated by the knowledge insertion unit into a score vector sequence using the machine learning model; a loss calculation unit that calculates a loss function value using the score vector sequence converted by the language processing unit and the knowledge sentence-inserted output label sequence generated by the knowledge insertion unit; a parameter update unit that updates parameters of the machine learning model based on the loss function value calculated by the loss calculation unit; A language processing device having:

4. A program that causes a computer to execute the method according to claim 1 or 2.

5. An estimation method executed by a language processing device that estimates a word classification in an estimation phase, comprising: the language processing device includes a supervised knowledge corpus storage unit that stores a plurality of supervised knowledge sentences each having text data including a named entity and label information indicating whether or not each word in the text data is the named entity; The language processing device comprises: an input process for inputting an input word sequence including a predetermined named entity; a search process for searching the supervised knowledge corpus storage unit based on the predetermined named entity included in the input word sequence input by the input process, and reading out a predetermined supervised knowledge sentence including the same named entity as the predetermined named entity; a knowledge insertion process for generating a knowledge-sentence-inserted input word sequence by inserting the predetermined supervised knowledge sentence into the input word sequence; A language process that converts the input word sequence into a score vector sequence using a trained machine learning model; an output process of calculating the dimension with the maximum score for all words in the score vector sequence, classifying all the words into labels corresponding to the dimension with the maximum score, and outputting a predicted label sequence in which label information of the classification results is arranged; Estimation method to perform.

6. 6. The estimation method according to claim 5, the language processing device includes an unsupervised knowledge corpus storage unit that stores a plurality of unsupervised knowledge sentences having text data including named entities; when the search process is unable to read out a specified number of the predetermined supervised knowledge sentences from the supervised knowledge corpus storage unit, the search process includes a process of searching the unsupervised knowledge corpus storage unit based on the predetermined named entity and reading out predetermined unsupervised knowledge sentences including the same named entity as the predetermined named entity; the knowledge insertion process includes a process of generating the knowledge-sentence-inserted input word sequence by inserting the predetermined unsupervised knowledge sentence into the input word sequence in addition to the predetermined supervised knowledge sentence, Estimation method.

7. A language processing device that estimates a word classification in an estimation phase, comprising: a supervised knowledge corpus storage unit that stores a plurality of supervised knowledge sentences each having text data including a named entity and label information indicating whether or not each word in the text data is the named entity; an unsupervised knowledge corpus storage unit that stores a plurality of unsupervised knowledge sentences having text data including named entities; an input unit for inputting an input word sequence including a predetermined named entity; a search unit that searches the supervised knowledge corpus storage unit based on the predetermined named entity included in the input word sequence input by the input unit, reads out predetermined supervised knowledge sentences including the same named entity as the predetermined named entity, and when a predetermined number of the predetermined supervised knowledge sentences cannot be read out from the supervised knowledge corpus storage unit, searches the unsupervised knowledge corpus storage unit based on the predetermined named entity, and reads out predetermined unsupervised knowledge sentences including the same named entity as the predetermined named entity; a knowledge insertion unit that inserts the predetermined supervised knowledge sentences into the input word sequence, and when the predetermined number of the predetermined supervised knowledge sentences cannot be read from the supervised knowledge corpus storage unit, further inserts the predetermined unsupervised knowledge sentences to generate an input word sequence with knowledge sentences inserted; a linguistic processing unit that converts the input word sequence into which the knowledge sentence has been inserted into a score vector sequence using a trained machine learning model; an output unit that calculates the dimension with the maximum score for all words in the score vector sequence, classifies all the words into labels corresponding to the dimension with the maximum score, and outputs a predicted label sequence in which label information of the classification results is arranged; A language processing device having:

8. A program that causes a computer to execute the method according to claim 5 or 6.

Images