Information processing device, information processing method, and program

Abstract

To provide an information processor, a method for processing information, and a program that can accurately interpret technical terms or expressions unique to a specific field.SOLUTION: An information processor extracts a domain specific word as an expression unique to a field by using sample data, and acquires a sentence including the domain specific word as learning target data. After that, the information processor constructs a new language model by mechanically learning the domain specific word on the basis of the acquired learning target data. In that way, it becomes possible to accurately interpret technical terms or expressions unique to a specific field and the like.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Natural language analysis techniques have been known for some time. For example, Patent Document 1 discloses a training data collection device that collects data related to a specific field to be used as training data for machine learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-124617 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 leaves room for improvement in terms of its ability to interpret technical terms and industry-specific expressions, and is still insufficient in terms of accurately interpreting technical terms and industry-specific expressions.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide an information processing device, an information processing method, and a program that are capable of accurately interpreting technical terms and industry-specific expressions. [Means for solving the problem]

[0006] In order to achieve the above object, an information processing device according to a first aspect of the present invention comprises: Domain-specific terms, which are expressions specific to the industry, Contains sentences using expressions specific to the industry An extraction means for extracting using sample data; a learning target data acquisition means for acquiring, as learning target data, sentences including the domain-specific words extracted by the extraction means; a new language model construction means for constructing a new language model by machine learning the domain-specific words based on the learning object data acquired by the learning object data acquisition means. 、 The extraction means extracts difficult words, which are parts with low accuracy when predicting words in sentences included in the sample data using a language model that has been machine-learned in advance, as the domain-specific words.

[0007] The extraction means extracts, as the domain-specific words, difficult words that are parts of a sentence with low accuracy when a part of the sentence is predicted using a language model that has been machine-learned in advance. This may be done.

[0008] The extraction means identifies the difficult words using the sample data, and extracts the difficult words that satisfy a set condition from the identified difficult words. This may be done.

[0009] the new language model construction means constructs a new language model by machine learning the domain-specific words included in the training target data similar to the sample data, from the training target data acquired by the training target data acquisition means; This may be done.

[0010] the new language model construction means identifies the training object data similar to the sample data from the training object data acquired by the training object data acquisition means based on the respective document vectors; This may be done.

[0011] In order to achieve the above object, an information processing method according to a second aspect of the present invention comprises: An information processing method by an information processing device, Domain-specific terms, which are expressions specific to the industry, Contains sentences using expressions specific to the industry an extraction step of extracting using sample data; a learning target data acquisition step of acquiring, as learning target data, sentences including the domain-specific words extracted in the extraction step; a new language model construction step of constructing a new language model by machine learning the domain-specific words based on the learning target data acquired in the learning target data acquisition step. 、 In the extraction step, when predicting words in sentences included in the sample data using a language model that has been machine-learned in advance, difficult words that have low accuracy are extracted as the domain-specific words.

[0012] In order to achieve the above object, a program according to a third aspect of the present invention comprises: Computer, Domain-specific terms, which are expressions specific to the industry, Contains sentences using expressions specific to the industry extraction means for extracting using sample data; a learning target data acquisition means for acquiring, as learning target data, sentences including the domain-specific words extracted by the extraction means; and functioning as a new language model construction means for constructing a new language model by machine learning the domain-specific words based on the learning target data acquired by the learning target data acquisition means. 、 The extraction means extracts difficult words, which are parts with low accuracy when predicting words in sentences included in the sample data using a language model that has been machine-learned in advance, as the domain-specific words. [Effects of the Invention]

[0013] According to the present invention, technical terms and expressions specific to an industry can be interpreted with high accuracy. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram illustrating an example of an information processing device according to an embodiment of the present invention. [Figure 2] 10 is a flowchart illustrating an example of a model construction process. [Figure 3] FIG. 10 is a diagram illustrating an example of a difficult word extraction process. [Figure 4] FIG. 10 is an explanatory diagram for explaining identification of difficult words; [Figure 5] FIG. 10 is an explanatory diagram for explaining extraction of difficult words; [Figure 6] FIG. 10 is a diagram illustrating an example of clustering. DETAILED DESCRIPTION OF THE INVENTION

[0015] (Embodiment 1) An information processing device, an information processing method, and a program according to an embodiment of the present invention will be described in detail below with reference to the drawings. Note that the same or corresponding parts in the drawings are designated by the same reference numerals.

[0016] First, the configuration of an information processing device 100 according to the present embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the information processing device 100 according to the embodiment is an information terminal connected to a network 210 such as the Internet. The information processing device 100 is capable of transmitting and receiving various data to and from other information terminals via the network 210. For example, the information processing device 100 may obtain a sample document 113, which will be described later, from another information terminal via the network 210.

[0017] The information processing device 100 is an information terminal such as a mobile phone, smartphone, tablet, or PC (Personal Computer) that constructs a language model for natural language analysis, and is connected to a network 210 such as the Internet. The information processing device 100 according to this embodiment regards technical terms and expressions specific to an industry as domain-specific terms, and constructs a new language model by learning the domain-specific terms.

[0018] As shown in FIG. 1, the information processing device 100 includes a storage unit 110, a control unit 120, an input / output unit 130, a communication unit 140, and a system bus (not shown) that interconnects these units.

[0019] The storage unit 110 includes a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The ROM stores a program 111 to be executed by a CPU (Central Processing Unit) of the control unit 120, a language model 112, sample documents 113, and various data (not shown) required in advance for executing the program 111.

[0020] The program 111 is a program for executing a model construction process, which will be described later. Specifically, the program 111 is a program for realizing the functions of a difficult word extraction unit 121, a target sentence collection unit 122, a clustering unit 123, and a model construction unit 124, which will be described later, and is stored in the storage unit 110 in advance.

[0021] The language model 112 is a general-purpose model obtained by pre-training document data through unsupervised learning, such as BERT (Bidirectional Encoder Representations from Transformers), GPT-2 (Generative Pre-Training), or T5 (Text-To-Text Transfer Transformer), and is pre-stored in the storage unit 110. Note that the language model 112 may be stored in the storage unit 110 from another information terminal via a network 210 such as the Internet.

[0022] Sample document 113 is sample data of a business document to be processed, and includes multiple sentences (sample sentences) containing industry terminology. In this embodiment, the following description will be given taking as an example a case where sample document 113 is a business document in the financial field and is stored in advance in storage unit 110. As described above, sample document 113 may also be stored in storage unit 110 from another information terminal via network 210.

[0023] The control unit 120 is configured with a CPU, an ASIC (Application Specific Integrated Circuit), etc. The control unit 120 operates in accordance with a program 111 stored in the storage unit 110, and executes processing in accordance with the program. The control unit 120 includes a difficult word extraction unit 121, a target sentence collection unit 122, a clustering unit 123, and a model construction unit 124 as main functional units provided by the program 111 stored in the storage unit 110.

[0024] The difficult word extraction unit 121 is a functional unit that performs a difficult word extraction process, which will be described later. The difficult word extraction process is a process of extracting, as difficult words, words that have low accuracy when predicting words using the language model 112 stored in the storage unit 110. The functions of the difficult word extraction unit 121 include a morphological analysis function that performs morphological analysis of sample sentences included in the sample document 113, a difficult word identification function that identifies difficult words using a masked language model (MLM), and a difficult word extraction function that extracts difficult words that meet certain conditions from the identified difficult words. While this embodiment illustrates an example in which the difficult word extraction unit 121 extracts difficult words as an example of domain-specific words, the extraction is not limited to difficult words as long as domain-specific words can be extracted. For example, the difficult word extraction unit 121 may extract domain-specific words based on the frequency of occurrence of words in sample sentences included in the sample document 113, the difference in word distribution between sentences included in general documents and the sample sentences included in the sample document 113, or words that do not appear in other industries. The general document may be, for example, a document randomly acquired via the network 210, or may be input by a user using any input method and stored in the storage unit 110. Furthermore, domain-specific words are not limited to words, but also include phrases and other expressions that are part of a sentence. In other words, the difficult word extraction process also includes a process of extracting, as domain-specific words, phrases that have low accuracy when phrases are predicted using the language model 112 stored in the storage unit 110.

[0025] The morphological analysis function performs morphological analysis of a sample sentence contained in the sample document 113, thereby dividing the sample sentence into morphemes and allocating information such as a part of speech to each morpheme.

[0026] The difficult word identification function has a function of masking nouns included in sample sentences that have been morphologically analyzed by the morphological analysis function, and predicting the masked nouns using MLM to identify difficult words. As will be described in detail later, in this embodiment, nouns with low prediction scores in MLM are identified as words with low accuracy, i.e., difficult words.

[0027] The difficult word extraction function has a function of extracting difficult words that satisfy preset conditions from among the difficult words identified by the difficult word identification function. As will be described in detail later, in this embodiment, difficult words that satisfy conditions such as the frequency being greater than 2 or the score and frequency being greater than a certain level are extracted based on the frequency of appearance of the difficult words in sample document 113 and the average score.

[0028] The target sentence collection unit 122 is a functional unit that acquires sentences containing the difficult words extracted by the difficult word extraction unit 121 as sentences to be learned (target sentences) via the network 210. For example, the target sentence collection unit 122 acquires the target sentences by performing an internet search using the difficult words extracted by the difficult word extraction unit 121 as keys. Note that, when domain-specific words are extracted based on the frequency of word appearance in the sample sentences included in the sample documents 113, the difference in word distribution between sentences included in general documents and the sample sentences included in the sample documents 113, or words that do not appear in other industries, the target sentences can be acquired using the domain-specific words as keys.

[0029] The clustering unit 123 is a functional unit that performs clustering to classify the target sentences acquired by the target sentence collection unit 122 into groups based on the similarity between the target sentences and the sample sentences included in the sample documents 113.

[0030] The model construction unit 124 is a functional unit that constructs a new language model by having the language model 112 learn target sentences that have a high similarity to the sample sentences included in the sample document 113 from the target sentences clustered by the clustering unit 123 as learning data.

[0031] The input / output unit 130 is a device that is configured with a keyboard, a mouse, a camera, a microphone, a liquid crystal display, an organic EL (Electro-Luminescence) display, and the like, and is used to input and output various types of data.

[0032] The communication unit 140 is a device that enables the information processing device 100 to communicate with other information terminals via the network 210 .

[0033] These functional units work together to enable the information processing device 100 to realize a function for constructing a new language model that can accurately interpret technical terms and expressions specific to an industry.

[0034] The above is the configuration of the information processing device 100. Next, the operation of the information processing device 100 will be described with reference to FIGS. 2 to 6. In the example shown below, the sample document 113 is a business document in the financial field, and the sample sentences shown in FIGS. 4 and 5 will be used as examples. In the example shown below, a case where difficult words are extracted as domain-specific words will be described as an example. As described above, as long as domain-specific words can be extracted based on the frequency of word appearance in the sample sentences included in the sample document 113, the difference in word distribution between sentences included in general documents and the sample sentences included in the sample document 113, words that do not appear in other industries, etc., they are not limited to difficult words.

[0035] Execution of the model construction process is started when the user operates the input / output unit 130. Fig. 2 is a flowchart showing an example of the model construction process.

[0036] 2 starts, the control unit 120 first executes a difficult word extraction process using the function of the difficult word extraction unit 121 (step S101). FIG. 3 is a flowchart showing an example of the difficult word extraction process executed in the process of step S101 in FIG. 2. The difficult word extraction unit 121 that executes the process of step S101 corresponds to the extraction means, and step S101 corresponds to the extraction step. As described above, this embodiment shows an example in which difficult words are extracted as domain-specific words using the difficult word extraction process. However, it is sufficient if domain-specific words can be extracted. For example, using the difficult word extraction process, the difficult word extraction unit 121 may extract domain-specific words based on the frequency of word appearance in sample sentences included in the sample document 113, the difference in word distribution between sentences included in general documents and the sample sentences included in the sample document 113, words that do not appear in other industries, etc.

[0037] 3, the difficult word extraction unit 121 first uses the morphological analysis function to perform morphological analysis of the sample sentence included in the sample document 113 (step S201). Specifically, in step S201, a sample sentence such as "The minimum amount of the margin deposit is XX million yen" shown in Fig. 4 is divided into morphemes such as "entrustment," "guarantee," "money," "of," "minimum," "amount," "is," "XX million," "yen," and "desu," and information such as a part of speech is assigned to each morpheme as shown in the figure.

[0038] After executing the process of step S201 shown in Fig. 3, the difficult word extraction unit 121 uses the difficult word identification function to identify difficult words using MLM (step S202). Specifically, in step S202, first, as shown in Fig. 4, for the sample sentence "The minimum amount of margin is XX million yen," the morphemes assigned as nouns in step S201 are masked. Then, the masked words are predicted, and words that do not meet the prediction are identified as difficult words.

[0039] In the example shown in Figure 4, if the masked word is in the top 5 predicted words, it is evaluated as True, and if not, it is evaluated as False. ” is used as the MASK word. ...

[0040] Furthermore, as shown in FIG. 5, the processing of step S201 and the processing of step S202 are performed on multiple sample sentences (sample sentences 1 to 7 in the illustrated example) included in sample document 113, and multiple difficult words are identified. The example shown in FIG. 5 shows a case where five difficult words, such as "entrust," "guarantee," "credit," "securities," and "transaction," are identified. In the processing of step S202, as shown in FIG. 5, when a difficult word is identified, its average score and its frequency of appearance (frequency) are counted. The frequency indicates the cumulative number of times the difficult word appears in all sample sentences. For example, the frequency "7" shown in FIG. 5 indicates that "entrust" was counted seven times among sample sentences 1 to 7.

[0041] After executing the process of step S202 shown in Fig. 3, the difficult word extraction unit 121 uses the difficult word extraction function to extract difficult words that satisfy the set condition from among the difficult words identified in the process of step S202 (step S203), and ends the difficult word extraction process. In the example shown in Fig. 5, three difficult words, "entrust," "guarantee," and "trust," are extracted under the condition that the frequency is not 2 or less. Note that the condition can be changed arbitrarily, and may be any condition based on a combination of average score and frequency, such as a condition that the average score is 8 or more or a frequency is 3 or more, or a condition that the average score is 6 or more and the frequency is not 2 or less.

[0042] Returning to FIG. 2, after executing the process of step S101, the control unit 120, using the function of the target sentence collection unit 122, performs an Internet search via the network 210 using the difficult words extracted in the process of step S101 as a key to acquire target sentences (step S102). As described above, a target sentence refers to a sentence that can serve as a learning target for the information processing device 100 to build a new language model. Specifically, in the process of step S102, an Internet search can be performed using each of the multiple difficult words extracted in the process of step S101 as a key, as well as an Internet search using a combination of the difficult words as a key. It is sufficient that the user can select which difficult words to use as keys and the combinations. For example, in the example shown in FIG. 5, for the three difficult words "entrust," "guarantee," and "trust," it is sufficient that a search using each of "entrust," "guarantee," or "trust" as a key (logical OR) as well as a search using a logical AND such as "entrust," "guarantee," and "trust" as a key can be selected. By executing the process of step S102, a sentence containing the key is acquired as a target sentence. Note that the target sentence collection unit 122 that executes the process of step S102 corresponds to the learning target data acquisition means, and step S102 corresponds to the learning target data acquisition step. Also, the target sentence corresponds to the learning target data.

[0043] After executing the process of step S102 shown in FIG. 2 and acquiring the target sentence, the control unit 120 uses the function of the clustering unit 123 to cluster the target sentence and the sample sentences included in the sample document 113, as shown in FIG. 6, based on the similarity between the acquired target sentence and the sample sentences included in the sample document 113 (step S103). Specifically, in the process of step S103, a Gaussian mixture model is used to assign a probability of belonging to each cluster to the sentence vectors of the target sentence and the sample sentences. Here, the probability of belonging to a cluster is estimated based on the distance between the sentence vector and the average of each cluster. For example, as shown in FIG. 6, the distance between sentence vector 1 and the average n1 of cluster A is the probability of belonging to cluster A. Similarly, the distance between sentence vector 1 and the average n2 of cluster B is the probability of belonging to cluster B, and the distance between sentence vector 1 and the average n3 of cluster C is the probability of belonging to cluster C. Note that the average of each cluster is the sentence vector of the most typical sentence in each cluster. In the process of step S103 shown in Fig. 2, if there are three clusters as shown in Fig. 6, for example, three belonging probabilities are calculated for each of the sentence vectors of all the target sentences acquired in step S102. Also, in the process of step S103, the clustering model is applied to the sentence vectors of all the sample sentences included in the sample document 113, thereby mapping them as shown in Fig. 6.

[0044] After executing the process of step S103 shown in FIG. 2, the control unit 120 constructs a new language model by using the function of the model construction unit 124 to train the language model 112 with the target sentences clustered by the clustering unit 123 as training data (step S104), and ends the model construction process. Specifically, in the process of step S104, of the target sentences clustered in the process of step S103, target sentences that have a high similarity to sample sentences included in the sample document 113 are trained as training data in the language model 112 to construct a new language model. For example, in the example shown in FIG. 6, clusters A and B, which are clusters close to the sentence vectors of the sample sentences, are set as clusters with a high similarity to the sample sentences, and sentences belonging to clusters A and B are trained as training data in the language model 112. Note that the model construction unit 124 that executes the process of step S104 corresponds to a new language model construction means, and step S104 corresponds to a new language model construction step.

[0045] In this way, in the information processing device 100 according to this embodiment, the model construction process shown in Fig. 2 is executed to extract difficult words such as technical terms and industry-specific expressions that have low interpretation performance in the current language model 112. Then, a new language model is constructed by acquiring sentences containing the extracted difficult words and learning them as training data. Therefore, the interpretation performance of technical terms and industry-specific expressions can be improved.

[0046] Furthermore, in the information processing device 100 according to this embodiment, only difficult words that satisfy set conditions are extracted, thereby preventing unnecessary learning and improving interpretation performance. Furthermore, for sentences that include difficult words, instead of using all of them as training data, only those that have a high similarity to the sample sentences are used as training data, making it possible to build a language model that is more in line with reality.

[0047] (Variation) It should be noted that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible. For example, the information processing device 100 according to the above-described embodiment does not need to have all of the technical features described above, but may have some of the configurations described in the above-described embodiment so as to solve at least one problem in the prior art. Furthermore, at least a portion of each of the following modifications may be combined.

[0048] In the above embodiment, the sample document 113 is a business document in the financial field, but this is just one example. The sample document 113 may be a document from any industry, such as a business document in the medical field, the automotive industry, or the broadcasting industry, as long as it contains technical terms and industry-specific expressions. Even in the financial field, the sample document 113 may be different for each field, such as banking, securities, or insurance. This allows for the extraction of difficult words with more specific content, thereby further improving interpretation performance.

[0049] Furthermore, in the above embodiment, an example is shown in which the sample document 113 is in Japanese, but the sample document 113 is not limited to Japanese, and may be in a language other than Japanese, such as English, Chinese, or Spanish.

[0050] In the above embodiment, as shown in Figure 6, clusters A and B, which are close to the sentence vector of the sample sentence, are set as clusters with high similarity to the sample sentence, and all sentences belonging to clusters A and B are used as training data, but this is just one example. Alternatively, only sentences that satisfy a set condition, such as those whose distance from the nearest sample sentence is within a set value, may be used as training data. This allows for the construction of a language model that is more in line with reality.

[0051] In the above embodiment, the target sentence collection unit 122 acquires, via the network 210, sentences containing the difficult words extracted by the difficult word extraction unit 121 as learning target sentences (target sentences). However, this is merely an example. Alternatively, the target sentence collection unit 122 may acquire learning target sentences by extracting sentences containing the difficult words extracted by the difficult word extraction unit 121 from miscellaneous collected text data. The text data may be input by a user using any input method and stored in the storage unit 110. Alternatively, instead of training the language model 112 with all sentences belonging to cluster A and cluster B as learning data, the learning data may be selected by weighting the sentences according to their importance and selecting some sentences stochastically (randomly). This reduces the processing load, improves processing speed, and enables efficient learning.

[0052] The information processing device 100 according to the above embodiment can be realized using a normal computer, not a dedicated device. For example, the information processing device 100 that executes the above processes may be configured by installing a program for executing any of the above processes on a computer from a recording medium storing the program. Also, one information processing device 100 may be configured by multiple computers operating in cooperation with each other.

[0053] Furthermore, when the above-mentioned functions are realized by sharing the functions between an OS (Operating System) and an application, or by cooperation between the OS and the application, only the parts other than the OS may be stored on the medium.

[0054] It is also possible to superimpose the program on a carrier wave and distribute it via a communication network. For example, the program may be posted on a bulletin board system (BBS) on the communication network and distributed via the network. These programs may then be started and run under the control of an operating system in the same way as other application programs, thereby enabling the above-mentioned processing to be performed. [Explanation of symbols]

[0055] 100 Information processing device, 110 Memory unit, 111 Program, 112 Language model, 113 Sample document, 120 Control unit, 121 Difficult word extraction unit, 122 Target sentence collection unit, 123 Clustering unit, 124 Model construction unit, 130 Input / output unit, 140 Communication unit, 210 Network

Claims

1. An extraction means for extracting domain-specific words, which are expressions specific to an industry, using sample data including sentences using the expressions specific to the industry; a learning target data acquisition means for acquiring, as learning target data, sentences including the domain-specific words extracted by the extraction means; a new language model construction means for constructing a new language model by machine learning the domain-specific words based on the learning object data acquired by the learning object data acquisition means, The extraction means extracts, as the domain-specific words, difficult words that are parts with low accuracy when predicting words in sentences included in the sample data using a language model that has been machine-learned in advance. Information processing device.

2. The extraction means identifies the difficult words using the sample data, and extracts the difficult words that satisfy a set condition from the identified difficult words. The information processing device according to claim 1 .

3. the new language model construction means constructs a new language model by machine learning the domain-specific words included in the training target data similar to the sample data, from the training target data acquired by the training target data acquisition means; 3. The information processing device according to claim 1.

4. the new language model construction means identifies the training object data similar to the sample data from the training object data acquired by the training object data acquisition means based on the respective document vectors; The information processing device according to claim 3 .

5. An information processing method by an information processing device, An extraction step of extracting domain-specific terms, which are expressions specific to an industry, using sample data including sentences using the expressions specific to the industry; a learning target data acquisition step of acquiring, as learning target data, sentences including the domain-specific words extracted in the extraction step; a new language model construction step of constructing a new language model by machine learning the domain-specific words based on the learning target data acquired in the learning target data acquisition step, In the extraction step, difficult words, which are parts with low accuracy when predicting words in sentences included in the sample data using a language model that has been machine-learned in advance, are extracted as the domain-specific words. Information processing methods.

6. Computer, An extraction means for extracting domain-specific words, which are expressions specific to an industry, using sample data including sentences using the expressions specific to the industry; a learning target data acquisition means for acquiring, as learning target data, sentences including the domain-specific words extracted by the extraction means; functioning as a new language model construction means for constructing a new language model by machine learning the domain-specific words based on the learning object data acquired by the learning object data acquisition means; The extraction means extracts, as the domain-specific words, difficult words that are parts with low accuracy when predicting words in sentences included in the sample data using a language model that has been machine-learned in advance. program.

Images