Inference device, learning device, inference method, method for generating a trained model, inference program and learning program

Abstract

The goal is to obtain an inference device capable of generating an ethically acceptable sentence. The inference device according to the present disclosure comprises: a mask data retrieval unit for obtaining a string containing a masked section; a word sequence retrieval unit for segmenting the string into words and obtaining a word sequence containing a plurality of words; a control information retrieval unit for obtaining an adjectival expression representing a characteristic or condition of a thing as a control word; and an inference unit for inferring a candidate word for the masked section from the control word and the word sequence and for outputting the string in which the masked section is replaced by the candidate word.

Description

[Technical field]

[0001] The present disclosure relates to an inference device, a learning device, an inference procedure, a method for generating a trained model, an inference program and a learning program. [Technological background]

[0002] In recent years, language models utilizing artificial intelligence (AI) have achieved remarkable improvements in accuracy; for example, patent document 1 proposes a language model with an attention-based sequence transformation network. [References to the state of the art][Patent documents]

[0003] [Patent document 1] Japanese Patent No. 6884871 [Summary of the invention][Problems to be solved by the invention]

[0004] Learning a language model requires a large amount of textual data. However, if the data is collected randomly and used for learning, the language model becomes biased by a sense of human discrimination based on race, gender, ethnicity, culture, etc., and thus the inferential device using the language model may produce an ethically inappropriate sentence.

[0005] The present disclosure was created to solve the above problem and to obtain an inference device that produces ethically appropriate texts. [Means of solving the problems]

[0006] An inference device according to the present disclosure comprises: a mask data retrieval unit for retrieving a string containing a masked section; a word sequence retrieval unit for segmenting the string into words and retrieving a word sequence containing a plurality of words; a control information retrieval unit for retrieving an adjectival expression representing a characteristic or condition of a thing as a control word; and an inference unit for inferring a candidate word for the masked section from the control word and the word sequence and for outputting the string in which the masked section is replaced by the candidate word. [Effects of the invention]

[0007] The inference device according to the present disclosure comprises the control information retrieval unit for obtaining the adjectival expression representing a characteristic or condition of a thing, as the control word; and the inference unit for inferring the candidate word for the masked section from the control word and the word sequence and for outputting the string in which the masked section is replaced by the candidate word, so that an ethically acceptable sentence can be generated by inferring the candidate word on the basis of the control word. [Brief description of the drawings] Fig. Figure 1 is a representation showing a configuration of a speech processing system 10 according to embodiment 1. Fig. Figure 2 is a representation showing a configuration of a learning facility 100 according to embodiment 1. Fig. Figure 3 is a conceptual representation showing a concrete example of the processing of a morpheme analysis unit 121 according to embodiment 1. Fig. Figure 4 is a conceptual representation showing a concrete example of the processing of a lexicalization unit 122 according to embodiment 1. Fig. Figure 5 is a conceptual representation showing an example of a word sequence. Fig. Figure 6 is a conceptual representation showing an example of a trained model. Fig. Figure 7 is a conceptual representation that shows a concrete example of the adjectival expression term PMI. Fig. Figure 8 is a representation showing an example of a hardware configuration of a computer that implements the learning facility 100 according to embodiment 1. Fig. Figure 9 is a flowchart showing the operation of the learning facility 100 according to embodiment 1. Fig. Figure 10 is a representation showing a configuration of a language model storage device 300 according to embodiment 1. Fig. Figure 11 is a representation showing a configuration of an inference device 200 according to embodiment 2. Fig. Figure 12 is a conceptual representation showing a concrete example of a word sequence that has a masked section. Fig. Figure 13 is a conceptual representation that shows a concrete example of a forward n-gram likelihood. Fig. Figure 14 is a conceptual representation that shows a concrete example of a backward n-gram likelihood. Fig. Figure 15 is a conceptual representation to illustrate a concrete processing example in which a first inference unit 251 obtains mutual information point by point using the trained model. Fig. Figure 16 is a representation showing an example of a hardware configuration of a computer that implements the inference device 200 according to embodiment 1. Fig. Figure 17 is a flowchart illustrating the operation of the inference device 200 according to embodiment 1. Fig. Figure 18 is a representation showing a configuration of a speech processing system 2010 according to embodiment 2. Fig. Figure 19 is a representation showing a configuration of a learning facility 2100 according to embodiment 2. Fig. Figure 20 is a conceptual representation to illustrate a specific example of the processing of a distortion correction unit according to embodiment 2. [Embodiments for carrying out the invention]Embodiment 1

[0008] Fig. Figure 1 is a representation showing a configuration of a speech processing system 10 according to embodiment 1. The speech processing system 10 comprises a learning device 100, an inference device 200, and a speech model storage device 300.

[0009] Speech processing system 10 automatically generates a sentence and can be used, for example, for a chatbot or an automated speech dialogue system. As described later, speech processing system 10 generates a sentence by inferring a word that fits into a masked section within a string containing the masked section. Here, a string and text are treated as synonymous.

[0010] First, a learning phase is described in which the learning institution 100 creates a trained model, and then an inference phase is described in which the inference institution 200 makes inferences using the trained model.

[0011] In the present disclosure, the trained model refers to a model of the adjectival expression term PMI (pointwise mutual information), which will be described later. <lernphase>

[0012] Fig. Figure 2 is a representation showing a configuration of the learning device 100 according to embodiment 1. The learning device 100 comprises a learning data acquisition unit 110, a word sequence acquisition unit 120, a species identification unit 130, an N-gram model generation unit 140, and a learning unit 150.

[0013] The training data acquisition unit 110 acquires a string D1 as training data, which is contained in a training dataset. More precisely, the training data acquisition unit 110 acquires a multitude of strings by segmenting text data into sentences, with the text data being input as training data. That is, in the following, it is assumed that one string corresponds to one sentence.

[0014] The training data set is stored in a (not shown) storage device, and the learning data retrieval unit 110 retrieves the training data from the storage device when the learning is performed.

[0015] Word sequence retrieval unit 120 segments the string D1, which is retrieved by learning data retrieval unit 110, into words and retrieves a word sequence D3 containing a large number of words. Here, word sequence D3 represents a set of words with accompanying lexical category information.

[0016] Segmenting a string into words includes not only direct segmentation of a string into words, but also segmentation of a string first into larger units such as phrases and their subsequent segmentation into words, as well as segmentation of a string first into smaller units such as morphemes and their subsequent concatenation to form words.

[0017] In embodiment 1, the word sequence procurement unit 120 comprises a morpheme analysis unit 121 and a lexicalization unit 122.

[0018] The morpheme analysis unit 121 segments the string D1, which is procured by the learning data acquisition unit 110, into morphemes that are grammatically minimal units, and procures a morpheme sequence D2 that contains a variety of morphemes with the attached lexical category information.

[0019] A concrete example of processing the morpheme analysis unit 121 is given using Fig. 3 described. Fig. Figure 3 is a conceptual representation that shows a concrete example of the processing of morpheme analysis unit 121.

[0020] When a string, for example, "Karasu taisaku ni netto wo setchi suru no ha koka teki desu." (It is effective to install a net as a measure against crows.) is entered, the Morpheme Analysis Unit 121 segments it into: "karasu (crow)", "taisaku (measure)", "ni (particle)", "netto (net)", "wo (particle)", "setchi (installation)", "suru (do)", "no (particle)", "ha (particle)", "koka (effectiveness)", "teki (suffix)", "desu (auxiliary verb)", ". (auxiliary symbol)". The lexical category information is appended so that "karasu (crow)" is noun and "ni" is particle. It is also possible to append more detailed grammatical information, such as general noun and case particle. For lexical category information, the UniDic classification can be used, for example. UniDic is an electronic dictionary for Japanese texts.An electronic dictionary other than UniDic may be used if it is possible to include the lexical category information similar to UniDic.

[0021] Lexicalization unit 122 generates the word sequence D3 by concatenating the morphemes contained in the morpheme sequence D2 output by morpheme analysis unit 121, based on lexical category information. Lexicalization refers to the process of creating a word by concatenating a preceding and a subsequent morpheme in a morpheme sequence. Since UniDic uses a morphological unit—that is, a linguistic unit uniformly defined based on a minimal unit—the lexicalization described above is applied to process word meanings.

[0022] In the following, a word is defined as one that consists of one or more morphemes. In other words, this includes both a single morpheme that is not linked but functions as a word, and a linking of morphemes that functions as a word.

[0023] The processing of lexicalization unit 122 is described in detail below. Lexicalization unit 122 receives the morpheme sequence output by morpheme analysis unit 121, checks a lexical category for each morpheme, concatenates the morphemes that can be concatenated, and outputs the lexicalized morpheme sequence.

[0024] When a morpheme is preceded by a prefix morpheme or followed by a suffix morpheme, the lexicalization unit 122 concatenates the morphemes. The lexicalization unit 122 replaces the lexical category of the last concatenated morpheme with the lexical category of the morpheme obtained through the concatenation.

[0025] A concrete example of the processing of lexicalization unit 122 is given using Fig. 4 described. Fig. Figure 4 is a conceptual representation that shows a concrete example of the processing of the lexicalization unit 122. For example, if a Fig. When the morpheme sequence described in section 3, namely “Karasu taisaku ni netto wo setchi suru no ha koka teki desu.” (It is effective to install a net as a measure against crows.) is entered, the lexicalization unit 122 concatenates “koka (effectiveness)” before “teki (suffix)” and “teki (suffix)” to generate a vocabulary “koka-teki (effective, suffix)”, since the lexical category of “teki” is a suffix.

[0026] Another example: When morpheme analysis is performed on a string, “Karasu taisaku ni netto wo setchi suru no ha hi koka teki desu.” (It is ineffective to install a net as a measure against crows.), the string is first segmented into morphemes, “Karasu (crow)”, “taisaku (measure)”, “ni (particle)”, “netto (net)”, “wo (particle)”, “setchi (installation)”, “suru (do)”, “no (particle)”, “ha (particle)”, “hi (prefix)”, “koka (effectiveness)”, “teki (suffix)”, “desu (auxiliary verb)”, “. (auxiliary symbol)”, by morpheme analysis unit 121.

[0027] The lexicalization unit 122 concatenates “koka (effectiveness)” to “hi (prefix)” and “hi (prefix)” to obtain “hi-koka (ineffectiveness)” since “hi” is a prefix, and further concatenates “hi-koka (ineffectiveness)” and “teki (suffix)” to generate a word “hi-koka-teki (ineffective, suffix)”.

[0028] The type identification unit 130 identifies a sentence type D4, which is specified by the word sequence D3 obtained by the word sequence acquisition unit 120. The type identification unit 130 outputs the identified sentence type D4 to the learning unit 150. The sentence type D4, used here to classify a sentence, is either an affirmative sentence, a negative sentence, or a question.

[0029] The processing of the species identification unit 130 is described in detail below. The species identification unit 130 determines the sentence type according to the following rules, based on lexical category information and the notation of the words contained in a word sequence. If a sentence ends with "?" (auxiliary symbol), the sentence type is determined to be an interrogative sentence. If the last word of a sentence, except for an auxiliary symbol at the end of the sentence, is "nai" (particle), the sentence type is determined to be a negative sentence. Any sentence type other than those mentioned above is determined to be an affirmative sentence.

[0030] The species identification unit 130 may use another existing identification procedure. For example, an automatic detection procedure for a negative element and a point of negation within a sentence may be used to identify a negative sentence. Determining whether a sentence is a question may be based on whether a phrase frequently used in questions, such as "kana," "kane," "ka," "noka," or "daroka"—which are sentence-terminating particles—is used at the end of the sentence.

[0031] The N-gram model generation unit 140 generates an N-gram model D5 based on the word sequence D3 obtained by the word sequence acquisition unit 120. The N-gram model generation unit 140 outputs the generated N-gram model D5 to an N-gram model storage unit 310, which is contained in the language model storage device 300.

[0032] The N-gram model is a language model in which the probability of occurrence of each word depends only on the N-1 words that immediately precede or follow the word in question.

[0033] In the word sequence with m words, as in Fig. As shown in Figure 5, the N-gram model generation unit 140 calculates the forward N-gram likelihood for one N-gram of a word W. i , “w i-N+1 , ..., w i-1 , w i ", using the following expression. Forward−N−gram−likelihood=log2(P(wi|wi−N+1i−1))

[0034] The N-gram model generation unit 140 calculates the backward N-gram likelihood for an N-gram "w i , w i+1, ..., w i+N-1 ", which is ordered backwards from the end of a sentence, using the following expression. Backward−N−gram−likelihood=log2(P(wi|wi+N−1i+1))

[0035] The N-gram model generating unit 140 generates an N-gram model by combining a pair of the N-gram “w i-N+1 , ..., w i-1 , w i "and the corresponding forward n-gram likelihood as well as a pair of the n-gram "w i , w i+1 , ..., w i,N-1 "and the corresponding backward N-gram likelihood are stored. Hereinafter, the log likelihoods obtained from the N-gram model, i.e., the forward N-gram likelihood and the backward N-gram likelihood, are collectively referred to as the N-gram likelihood."

[0036] The learning unit 150 generates a trained model D6 that receives an adjectival expression representing a characteristic or state of something and two words to output the pointwise mutual information between the adjectival expression and the two words, based on the occurrence probabilities in the training data set for the learning words contained in the word sequence D3.

[0037] Specifically, learning unit 150 generates the trained model D6 based on the probability of occurrence of a first word, which is an adjectival expression and is contained in the word sequence D3, in the training data set, the probability of occurrence of a second word, which is contained in the word sequence D3, in the training data set, the probability of occurrence of a third word, which is contained in the word sequence D3, in the training data set, and a probability of the simultaneous occurrence of the first word, the second word, and the third word in the training data set.

[0038] The adjectival expression is either an adjective, an adjectival verb, or an adjectival noun. Here, the adjectival noun refers to a noun that is transformed into an adjectival verb when followed by "na" (an adnominal form of the auxiliary verb "da"), as in "anzen (na)" and "shimpai (na)," meaning "safe" and "disturbing," respectively. According to the UniDic classification, the adjectival expression can be further subdivided into an adjective, an adjectival noun, a noun (general.adjectival), a noun (general.verbal.adjectival), a suffix (nominal.adjectival), a suffix (adjectival_noun), or a suffix (adjectival_i).

[0039] In embodiment 1, the learning unit 150 also generates the trained model D6, which, in addition to the adjectival expression and the two words, also receives the sentence type and outputs the point-by-point mutual information.

[0040] More precisely, learning unit 150 generates a fourth-order tensor as a trained model D6, which receives the sentence type determined by the type-determining unit, the adjectival expression and the two words, and outputs the pointwise mutual information between the adjectival expression and the two words.

[0041] The following describes in detail the processing of the tensor learning performed by learning unit 150. First, learning unit 150 classifies the word sequence D3 according to sentence type D4. Then, learning unit 150 counts the total number of words Z and the number of occurrences c(w) for each word w for each sentence type D4.

[0042] The learning unit 150 then counts the number of simultaneous occurrences c(A, w) x , w y ) and c(A, w y , w x ) in relation to an adjectival expression A and the words w X and w y for each sentence type D4. More precisely, the adjectival expression (A) and the two words (w) are x , w y ) arbitrarily extracted from each word sequence, and the number of occurrences is taken for each occurrence of a word sequence (A, w x , w y ) and a word sequence (A, w y , w x ) is increased by one. This processing is performed for all word sequences. Here, occurrences of words in reverse order are also added to make the tensor robust against rarity. However, as described later, it can happen that occurrences of words in reverse order are not added, and an asymmetric tensor can be generated. A word that appears more than once in a word sequence may only be counted once, and a diagonal component for which w x = w y Depending on the purpose, a diagonal component can be treated differently than a non-diagonal component.

[0043] Subsequently, learning unit 150 calculates a value of an adjectival expression term PMI (the pointwise mutual information) for each of the stored triplets of A, w x and w y using expressions 3 to 7. P(A)=c(A)Z P(wz)=c(wx)Z P(wy)=c(wy)Z P(A,wx,wy)=c(A,wx,wy)Z PMI(A,wx,wy)=log2P(A,wx,wy)P(A)P(wx)P(wy)

[0044] Here, w = A is the first word of the adjectival expression, w = w x , the second word, w = w y the third word, and P(A), P(w) x ), P(w y ) and P(A, w x , w y ) are the respective probabilities of occurrence. However, if there is no triplet of A, w x , w y occurs or when the PMI (A, w x , w y ) are negative, the PMI to be stored (A, w x , w y ) replaced by 0.

[0045] Finally, learning unit 150 generates the trained model D6 by integrating the values ​​of the adjectival expression term PMI of the triplet for each sentence type D4 into a fourth-order tensor based on equation 8. PMI(G,A,wx,wy)=log2P(G,A,wx,wy)P(G,A)P(G,wx)P(G,wy)

[0046] Fig. Figure 6 is a conceptual representation that shows an example of the generated trained model. Fig. 6. Due to limitations of representation on paper, the fourth-order tensor is represented as three cuboids, and the pointwise mutual information is given at each point of the cuboids, which is designated by the sentence type and the three words. The dimension of the sentence type G (the first argument) is 3, the dimension of the adjectival expression A (the second argument) is p, and the dimensions of the words w x (the third argument) and w y (the fourth argument) are q. Both p and q are positive integers, where p is the number of adjectival expressions in the dictionary (the vocabulary size of adjectival expressions) and q is the number of all words in the dictionary (the total vocabulary size).

[0047] For example, if the sentence type is "affirmative sentence", the adjectival expression A "yoi (good)" and the word w x “hito (person)” is when used as w y , a word is entered that is strongly related to "hito (person)" and "yoi (good)", such as "aisuru (love)", "seicho-suru (ripe)", "mamoru (protect)" and "hanei-suru (thrive)", PMI (G, A, w x , w y ) then assume a large value. In a case where, for example, the sentence type is "negative sentence", the adjectival expression A is "yoi (good)" and the word w x “hito (person)” is and if as w y If a word like "shinu (to die)", "kizutsuku (to injure)", "nakusu (to lose)" and "kanashimu (to mourn)" is entered, then take PMI (G, A, w x , w y ) a great value. In a case where, for example, the sentence type is "affirmative sentence" and the adjectival expression A is "yoi (good)", when a word pair such as "zembu (all)" and "naoru (heal)", "kofuku (luck)" and "yobu (wave)", and "warui (bad)" and "naoru (heal)" for the words w x and w y If entered, then take PMI (G, A, w). x , w y ) is of great value. In a case where, for example, the sentence type is also "affirmative sentence" and the adjectival expression A is "yoi (good)", if for the words w x and w y If a word pair such as "kizu (wound)" and "aru (have)", "hito (person)" and "shinu (die)", and "chiryo (treatment)" and "naru (undergo)" is entered, then take PMI (G, A, w x , w y ) a small value. By being able to calculate the pointwise mutual information regarding "yoi (good)" not only on the basis of a single word but also of a word pair, the probability or likelihood can be precisely determined according to the context of an input sentence. The training example for the pointwise mutual information calculated according to Equation 8 is in Fig. 7 shown.

[0048] Next, a hardware configuration of the learning facility 100 according to embodiment 1 is described. Each of the functions of the learning facility 100 is implemented by a computer. Fig. Figure 8 is a representation showing an example of the hardware configuration of the computer that implements the learning facility 100 according to embodiment 1.

[0049] The in Fig. 8 Hardware shown includes a processing device 1000, such as a central processing unit (CPU), and a storage device 1001, such as a read-only memory (ROM) and a hard disk.

[0050] The in Fig. The two depicted learning data acquisition units 110, 120, 130, 140, and 150 are implemented by a program stored in the storage device 1001 and executed by the processing device 1000. The above configuration is not limited to a configuration implemented by a single processing device 1000 and a single storage device 1001, but can also be implemented by a plurality of processing devices 1000 and storage devices 1001.

[0051] The method for implementing each function of the learning device 100 is not limited to that carried out by the combination of hardware and program described above, but can also be implemented solely by hardware, e.g., a large-format integrated circuit (LSI) in which a program is implemented in a processing unit. Alternatively, a configuration is also possible in which some of the functions are implemented by dedicated hardware and the remaining functions are implemented by a combination of a processing unit and a program.

[0052] The learning facility 100 according to embodiment 1 is configured as described above.

[0053] The operation of the learning facility 100 according to embodiment 1 is described below. Fig. Figure 9 is a flowchart showing the operation of the learning device 100 according to embodiment 1. The operation of the learning device 100 corresponds to a generation procedure of the trained model, and the program that causes a computer to perform the operation of the learning device 100 corresponds to a learning program.

[0054] The operation of the learning data acquisition unit 110 corresponds to a learning data acquisition process, the operation of the word sequence acquisition unit 120 corresponds to a word sequence acquisition process, the operation of the species identification unit 130 corresponds to a species identification process, the operation of the N-gram model generation unit 140 corresponds to an N-gram model generation process, and the operation of the learning unit 150 corresponds to a learning process.

[0055] In step S1, the learning data acquisition unit 110 first obtains the string D1 as the training data contained in the training data set.

[0056] In step S2, the morpheme analysis unit 121 next segments the string D1, which is procured by the learning data acquisition unit 110 in step S1, into morphemes that are grammatically minimal units, and procures the morpheme sequence D2, which contains the multitude of morphemes with the attached lexical category information.

[0057] In step S3, the lexicalization unit 122 next obtains the word sequence D3 by concatenating the morphemes contained in the morpheme sequence D2 obtained by the morpheme analysis unit 121 in step S2, based on the lexical category information.

[0058] In step S4, the species identification unit 130 next determines the sentence type D4 specified by the word sequence D3 on the basis of the lexical category information obtained by the word sequence procurement unit 120 and the notation of the words contained in the word sequence D3.

[0059] In step S5, the N-gram model generation unit 140 then counts the number of occurrences of the N-gram word sequence that is contained in the word sequence D3 procured by the word sequence procurement unit 120.

[0060] In step S6, learning unit 150 then counts the number of occurrences of each word contained in the word sequence D3, and the number of occurrences of the triplet of the adjectival expression and the two other extracted words contained in the word sequence D3.

[0061] Next, in step S7, the training data acquisition unit 110 determines whether the training dataset contains training data to be processed next. If the training data acquisition unit 110 determines that there is data to be processed next, the process returns to step S1 to acquire the next training data. If it determines that there is no data to be processed next, the process proceeds to step S8 so that the N-gram model generation unit 140 can generate the N-gram model D5, and then to step S9 so that the training unit 150 can generate the trained model D6.

[0062] In step S8, the N-gram model generation unit 140 calculates the forward N-gram likelihood and the backward N-gram likelihood for each N-gram based on the number of occurrences of the N-gram's word sequence counted in step S5, and generates the N-gram model D5 by storing the mapping of each N-gram to the forward N-gram likelihood and the backward N-gram likelihood.

[0063] In step S9, the learning unit 150 calculates the probability of occurrence of each word and the probability of occurrence of each triplet based on the number of occurrences of each word and the number of occurrences of each triplet counted in step S6. The learning unit 150 then generates the trained model D6 by calculating the pointwise mutual information for each triplet based on the probability of occurrence of each word and each triplet, and by storing the mapping of sentence type, triplet, and pointwise mutual information.

[0064] By performing the above operation, the learning device 100 according to embodiment 1 can obtain a trained model capable of inferring a candidate word, which has a large amount of pointwise mutual information, with the adjectival expression that has a good meaning and is therefore ethically appropriate, because the trained model is configured to receive an adjectival expression representing a characteristic or state of something and two words, and outputs the pointwise mutual information between the adjectival expression and the two words. With the help of such a trained model, an ethically appropriate sentence can be created.

[0065] Furthermore, the learning device 100 according to embodiment 1 can receive a trained model that is able to infer the candidate word more accurately by outputting the pointwise mutual information for each sentence type, since the sentence type specified by the word sequence is determined and the trained model is configured to receive the sentence type in addition to the adjectival expression and the two words and output the pointwise mutual information.

[0066] Furthermore, according to embodiment 1, the learning device 100 segments a string into morphemes, which are grammatically minimal units, obtains a morpheme sequence containing a multitude of morphemes with attached lexical category information, concatenates the morphemes contained in the morpheme sequence based on the lexical category information, and obtains a word sequence. Thus, by generating a word that summarizes the meanings of the morphemes, a trained model can be created that can infer the candidate word more accurately.

[0067] For example, if "koka (effectiveness)" and "teki (suffix)" are processed separately, it can happen that "teki (suffix)" is identified as an adjectival expression, and the pointwise mutual information relating to "teki (suffix)" is learned because "koka (effectiveness)" is a noun and "teki" is a suffix of an adjectival noun. However, if "koka (effectiveness)" and "teki (suffix)" are concatenated as "koka-teki (adjectival noun-suffix)" to be processed as a single word, the pointwise mutual information for the word whose meaning is easier for humans to understand can be learned. The pointwise mutual information of antonyms can be correctly learned and processed by concatenating a prefix, such as "fu" and "hi," with a following morpheme. <inferenzphase>

[0068] Next, an inference phase is described in which a sentence is generated using the trained model (a model for an adjectival expression term PMI) that was created in the learning phase.

[0069] Before describing the inference device 200, the language model storage device 300 will first be described. Fig. Figure 10 is a representation showing a configuration of the speech model storage device 300 according to embodiment 1. The speech model storage device 300 comprises the N-gram model storage unit 310, a trained model storage unit 320, and a neural speech model storage unit 330.

[0070] The N-gram model storage unit 310 stores an N-gram model generated by the N-gram model generation unit 140.

[0071] The trained model storage unit 320 stores the trained model generated by the learning unit 150.

[0072] The Neural Language Model Storage Unit 330 stores a neural language model generated by a different learning device (not shown) than learning device 100. Details of the neural language model will be described later.

[0073] The language model storage device 300 is implemented by a storage device such as a read-only memory (ROM) and a hard disk.

[0074] The speech model storage facility 300 can be implemented by a single server, by a multitude of servers distributed in a cloud, or as part of an edge-based storage system. For example, if the speech processing system 10 is used for an automatic response system of a robot, the models can be stored on a server that collectively manages the robot or on the robot itself.

[0075] The inference facility 200 is described below. Fig. Figure 11 is a representation showing a configuration of the inference unit 200 according to embodiment 1. The inference unit 200 comprises a mask data acquisition unit 210, a word sequence acquisition unit 220, a type identification unit 230, a control information acquisition unit 240, and an inference unit 250.

[0076] The mask data acquisition unit 210 acquires a string D11 containing a masked section. The masked section refers to a segment within the string where a word to be inserted is missing and is replaced by a special word [MASK]. A word that can replace the special word [MASK] is the inference target of the inference unit 200. The masking data consists of text data of the string D11 containing the masked section.

[0077] The word sequence procurement unit 220 segments the string D11, which is procured by the mask data procurement unit 210, into words and procures a word sequence D13 which has a large number of words, which is a similar process to the process carried out by the word sequence procurement unit 120 of the learning facility 100.

[0078] The word sequence procurement unit 220 comprises a morpheme analysis unit 221 and a lexicalization unit 222.

[0079] The morpheme analysis unit 221 segments the string D11, which is procured by the mask data retrieval unit 210, into morphemes that are grammatically minimal units and procures a morpheme sequence D12 that has a variety of morphemes with the attached lexical category information, which corresponds to a process similar to the process carried out by the morpheme analysis unit 121.

[0080] The lexicalization unit 222 concatenates the morphemes contained in the morpheme sequence based on the lexical category information and procures the word sequence D13, which corresponds to a process similar to the process carried out by the lexicalization unit 122.

[0081] The species identification unit 230 determines a sentence type D14 specified by a word sequence 13, which corresponds to a process carried out by the species identification unit 130 of the learning institution 100.

[0082] The tax information retrieval unit 240 retrieves tax information D15 through user input. More precisely, the tax information retrieval unit 240 retrieves the adjectival expression representing a characteristic or state of something as the tax word, as well as weighting factors of the pointwise mutual information and the N-gram likelihoods as tax intensities. The weighting factors of the pointwise mutual information and the N-gram likelihoods can be understood as weighting factors for the model for the adjectival expression concept PMI and the N-gram model.

[0083] The tax information retrieval unit 240 retrieves the tax information D15 by input via an input device (not shown) such as a keyboard or touch panel from a user or designer of the speech processing system 10. The input from a user or designer can occur during the execution of the inference, or the pre-entered and stored tax information D15 can be read during the execution of the inference.

[0084] The tax intensity can be set for each sentence type. For example, the tax intensity can be set to 0.8 for an affirmative sentence, 0.2 for a negative sentence, and 0.5 for a question, with a higher value for an affirmative sentence, a lower value for a negative sentence, and an intermediate value for a question.

[0085] Inference unit 250 infers the candidate word for the masked section and outputs a string D19 in which the masked section is replaced by the inferred candidate word. Inference unit 250 comprises a first inference unit 251, a second inference unit 252, and an inference result integration unit 253.

[0086] As described later, in inference unit 250 in embodiment 1 infers the candidate word for the masked section from the control word obtained by a control information acquisition unit 30 and the word sequence D13 obtained by the word sequence acquisition unit 220.

[0087] The inference unit 250 outputs the string in which the masked section is replaced by the inferred candidate word to a display or speaker in order to transmit the string, i.e. the generated text, to the user.

[0088] The first inference unit 251, the second inference unit 252 and the inference result integration unit 253, which are contained in the inference unit 250, are described below.

[0089] The first inference unit 251 infers candidate words for the masked section from the control word obtained by the control information retrieval unit 30 and the word sequence D13 obtained by the word sequence retrieval unit 220.

[0090] Specifically, the first inference unit 251 infers the candidate words using the trained model, which receives the control word, the words contained in the word sequence and candidate words, and outputs the pointwise mutual information of the control word, the words contained in the word sequence and the candidate words.

[0091] The first inference unit 251 also infers the candidate words using the pointwise mutual information obtained from the trained model and the N-gram likelihoods from the N-gram model.

[0092] For example, if the length of the word sequence is m and the masked section is the nth word in the word sequence, the first inference unit 251 can infer the candidate words according to equation 9 below.

[0093] α is the tax intensity and β is the soil coefficient. Wn^=arg maxW((1−α)(β+∑i=1nlog(P(wi|wi−N+1i−1))+∑i=mnlog(P(wi|wi+N−1i+1)))+α∑i=1mPMI(G,A,wi,wn))

[0094] The processing by the first inference unit 251 is described in detail below. If equation 9 is used unchanged, the candidate words are limited to one, but in the following processing, the first inference unit 251 selects a plurality of the candidate words with the highest likelihoods in the argmax function of equation 9 from above and outputs the selected set of the plurality of candidate words to the inference result integration unit 253 as a first candidate word group D16.

[0095] First, the first inference unit 251 obtains the sentence type D14 from the type determination unit 230, the word sequence D13 from the word sequence procurement unit 220, and the control information D15 (the control word and the control intensity) from the control information procurement unit 240.

[0096] The first inference unit then inserts 251 N-1 special words [NULL], taking into account sentence-initial prefixation and sentence-final suffixation.

[0097] The first inference unit 251 finds the special word [MASK] in the word sequence and defines it as w n .

[0098] The first inference unit 251 takes w n as well as N-1 words before and after w n as the words of the N-gram are expelled in both forward and backward directions.

[0099] The first inference unit 251 determines a word that can be placed in the section of [MASK] from the dictionary and obtains the forward N-gram likelihood and the backward N-gram likelihood using the N-gram model.

[0100] A concrete example of processing the recovery of the N-gram likelihood from the N-gram model is given using the Fig. Sections 12 to 14 are described. For simplicity, N = 3 applies here.

[0101] Fig. Figure 12 is a conceptual representation that provides a concrete example of the word sequence containing the masked section. For example, it is assumed that the words and word sequence from which [MASK] is to be inferred are “netto (net)”, “wo (particle)”, “[MASK]”, “suru (do)”, “.(auxiliary symbol)”.

[0102] In this case, the forward n-grams are as in Fig. 13 are shown in descending order of the forward N-gram likelihood, where the likelihood of “tsukatt (use)” is obtained as 9.397643358, for example, and the likelihood of “tsuji (through)” is obtained as 9.146110803.

[0103] Similarly, the backward n-grams are as in Fig. 14 are shown in descending order of the backward N-gram likelihood, where the likelihood of "sonzai (existence)" is obtained as 8.709336799 and the likelihood of "shokai (introduction)" is obtained as 8.144842576.

[0104] Next, the inference of the candidate word using the trained model is described. The first inference unit 251 retrieves the trained model D6 from the trained model storage unit 320.

[0105] The first inference unit 251 generates a triplet of words, where the first word is the control word, the second word is a word from the word sequence, and the third word is a word from the dictionary.

[0106] The first inference unit 251 receives the pointwise mutual information of the generated triplet in sentence type D14 using the trained model.

[0107] A concrete example of processing the acquisition of pointwise mutual information from the trained model is given using Fig. 15 described.

[0108] As in the case of the n-gram model, the word sequence is in Fig. 12 was used as input in the description.

[0109] It is assumed that “yoi (good)” is entered as the control word.

[0110] In this case, where sentence type G is "affirmative sentence", in addition to the adjectival expression A "yoi (good)", which is entered as a control word, and "netto (net)", "wo (particle)", "suru (do)", ".(auxiliary symbol)", which stand in for the word w x words that are included in the dictionary, such as "setchi (installation)", "sakusei (creation)" and "jokyo (removal)", are entered as candidate words for w y The inputs are entered, and then the point-by-point mutual information is obtained for each of the inputs.

[0111] The first inference unit 251 calculates the N-gram likelihood and the pointwise mutual information for all words in the dictionary and then sorts the candidate words in descending order of the sum of the N-gram likelihoods and the pointwise mutual information (hereinafter referred to as the first likelihood).

[0112] The first inference unit 251 outputs the candidate words whose first likelihoods are greater than or equal to a predetermined threshold from the candidate words sorted by first likelihood to the inference result integration unit 253 as the first candidate word group D16.

[0113] The second inference unit 252 infers candidate words for the masked section using a neural language model D17. A neural language model is a language model that uses a neural network. Existing neural language models include, for example, a language model that uses a recurrent neural network and a language model that uses an attention mechanism such as Transformers and BERT (Bidirectional Encoder Representations from Transformers).

[0114] As an example, a case is described here in which the neural feedforward language model is used based on the neural feedforward network.

[0115] The neural feedforward language model predicts the next word based on a chain of N-1 words, similar to the N-gram model. If the nth word is the masked segment, the words from the (n-N+1)th to the (n-1)th position are converted into a one-hot vector and fed into the neural feedforward network.

[0116] A linear transformation is applied to the vector output by the neural feedforward network to convert it into a vector with the same dimensions as the vocabulary size. The converted vector is then fed into a softmax function.

[0117] The vector output by the above softmax function is a probability distribution for the nth word, and each of the vector elements corresponds to the probability of occurrence of each word in the dictionary.

[0118] Therefore, the candidate words for the masked section can be inferred by performing a maximum likelihood estimation using the vector output by the softmax function above.

[0119] Although the neural feedforward network is described here as an example, an existing neural language model can be used, such as a language model using a recurrent neural network, or a language model that uses an attention mechanism like Transformer and BERT (Bidirectional Encoder Representations from Transformers).

[0120] The second inference unit 252 outputs the candidate words whose likelihoods obtained from the neural language model (hereinafter referred to as the second likelihood) are greater than or equal to a given threshold to the inference result integration unit 253 as a second candidate word group D18 from the candidate words inferred using the neural language model D17.

[0121] The inference result integration unit 253 determines the candidate word that is common to both the first candidate word group D16, which is a set of the candidate words inferred by the first inference unit 251, and the second candidate word group D18, which is a set of the candidate words inferred by the second inference unit 252, as the candidate word to replace the masked section.

[0122] For example, if the candidate words in the first candidate word group are "shokai (introduction)", "setchi (installation)" and "katsuyo (use)" and the candidate words in the second candidate word group are "kogeki (attack)", "hakai (destruction)" and "setchi (installation)", an output integration unit 270 determines that the word to be placed in the masked section is "setchi (installation)" because the word that is common to both the first candidate word group and the second candidate word group is "setchi (installation)".

[0123] If both the first candidate word group and the second candidate word group contain more than one candidate word, the candidate word with the highest second likelihood in the second candidate word group can be determined as the candidate word to replace the masked section of the jointly contained candidate words.

[0124] This is because the inferential accuracy of the neural language model is generally higher, so the candidate words included in the second candidate word group are more likely to be contextually appropriate, and if a candidate word is also included in the first candidate word group, it cannot be considered ethically inappropriate. In other words, the process performed in this case is equivalent to excluding ethically inappropriate words from the candidate words inferred by the neural language model using the PMI model for adjectival expression terms.

[0125] Next, a hardware configuration of the inference device 200 according to embodiment 1 is described. Each of the functions of the inference device 200 is implemented by a computer. Fig. Figure 16 is a representation showing an example of a computer hardware configuration that implements the inference device 200.

[0126] The in Fig. 16 Hardware shown includes a processing device 1100, such as a central processing unit (CPU), and a storage device 1101, such as a read-only memory (ROM) and a hard disk.

[0127] The in Fig. The mask data acquisition unit 210, the word sequence acquisition unit 220, the type determination unit 230, the control information acquisition unit 240, and the inference unit 250 shown in Figure 11 are implemented by a program stored in the storage device 1101, which is executed by the processing device 1100. The above configuration is not limited to a configuration implemented by a single processing device 1100 and a single storage device 1101, but can also be implemented by a plurality of processing devices 1100 and storage devices 1101.

[0128] The method for implementing each function of the inference device 200 is not limited to that performed by the hardware and program combination described above, but can also be implemented solely by hardware, e.g., a large-format integrated circuit (LSI) in which a program is implemented in a processing unit. Alternatively, a configuration is also possible in which some of the functions are implemented by dedicated hardware and the remaining functions are implemented by a combination of a processing unit and a program.

[0129] The inference device 200 according to embodiment 1 is configured as described above.

[0130] Next, the operation of the inference device 200 according to embodiment 1 will be described. Fig. Figure 17 is a flowchart illustrating the operation of the inference device 200 according to embodiment 1. The operation of the inference device 200 corresponds to an inference procedure, and the program that causes a computer to perform the operation of the inference device 200 corresponds to an inference program.

[0131] The operation of the mask data acquisition unit 210 corresponds to a mask data acquisition process, the operation of the word sequence acquisition unit 220 corresponds to a word sequence acquisition process, the operation of the species determination unit 230 corresponds to a species determination process, the operation of the tax information acquisition unit 240 corresponds to a tax information acquisition process, and the operation of the inference unit 250 corresponds to an inference process.

[0132] First, in step S11, the mask data acquisition unit 210 procures the string D11, which contains the masked section.

[0133] In step S12, the morpheme analysis unit 221 next segments the string D11, which is procured by the mask data retrieval unit 210 in step S11, into morphemes that are grammatically minimal units, and procures the morpheme sequence D12, which contains the multitude of morphemes with the attached lexical category information.

[0134] In step S13, the lexicalization unit 222 next concatenates the morphemes contained in the morpheme sequence D12 obtained by the morpheme analysis unit 221 in step S12, based on the lexical category information, and obtains the word sequence D13.

[0135] In step S14, the species identification unit 230 next determines the sentence type D14 specified by the word sequence on the basis of the lexical category information obtained by the word sequence procurement unit 220 in the word sequence D13 and the notation of the words.

[0136] In the next step S15, the tax information retrieval unit 240 obtains the tax information D15 (the tax word and the tax intensity) through user input.

[0137] Next, in step S16, the first inference unit 251 retrieves the trained model D6 from the trained model storage unit 320 and the N-gram model D5 from the N-gram model storage unit 310 and infers the candidate words for the masked section using the pointwise mutual information obtained from the trained model D6 and the N-gram likelihoods obtained from the N-gram model D5. The first inference unit 251 outputs a set of candidate words whose first likelihoods are greater than or equal to a predetermined threshold to the inference result integration unit 253 as the first candidate word set D16.

[0138] Next, in step S17, the second inference unit 252 retrieves the neural language model D17 from the neural language model storage unit 330 and infers the candidate words for the masked section using the retrieved neural language model D17. The second inference unit 252 outputs a plurality of the candidate words whose second likelihoods are greater than or equal to a predetermined threshold to the inference result integration unit 253 as the second candidate word group D18.

[0139] Finally, in step S18, the inference result integration unit 253 compares the first candidate word group D16 and the second candidate word group D18 and determines a common candidate word as the final inference result. The inference result integration unit 253 outputs the string D19, obtained by replacing the masked section with the candidate word determined as the final inference result, to an external display or speaker.

[0140] Through the operation described above, the inference device 200 according to embodiment 1 can determine an ethically appropriate candidate word and generate an ethically appropriate sentence by inferring the candidate word for the masked section from the adjectival expression representing a characteristic or condition of a thing and a word sequence. In this case, the string obtained by replacing the masked section with the candidate word is the generated sentence.

[0141] For example, if "yoi (good)" or "utsukushii (beautiful)" is entered as a control word, a word strongly associated with "yoi (good)" or "utsukushii (beautiful)" will be inferred as the candidate word. The word highly associated with "yoi (good)" or "utsukushii (beautiful)" can be considered an ethically appropriate word, so the inference unit 200 can infer an ethically appropriate word as a candidate word and generate an ethically appropriate sentence by having the user or designer enter a control word with positive connotations.

[0142] The inference device 200 according to embodiment 1 infers the candidate word using the trained model, which receives the control word, the words contained in the word sequence and the candidate words and outputs the pointwise mutual information between the control word, the words contained in the word sequence and the candidate words, so that the candidate word, which is highly related to both the control word and the words contained in the word sequence, can be inferred and thus a natural and ethically appropriate sentence can be generated.

[0143] Furthermore, the inference device 200 according to embodiment 1 can infer the candidate word more accurately, since it determines the sentence type specified by the word sequence and infers the candidate word using the trained model, which is set up to receive the sentence type in addition to the adjectival expression, the words contained in the word sequence and the candidate words, and to output the pointwise mutual information.

[0144] Furthermore, the inference device 200 according to embodiment 1 segments a character string into morphemes, which are grammatically minimal units, obtains a morpheme sequence containing a multitude of morphemes with attached lexical category information, concatenates the morphemes contained in the morpheme sequence based on the lexical category information, and obtains a word sequence. Thus, by generating a word that summarizes the meanings of the morphemes, the candidate word can be inferred more precisely. <modifikation>

[0145] A modification of the speech processing system 10 according to embodiment 1 is described.

[0146] The learning device 100 and the inference device 200 according to embodiment 1 are configured to determine the sentence type in order to infer the candidate word with greater accuracy. However, if accuracy is not required or if the calculation is to be simple, it is possible to omit the sentence type determination or to perform the learning only for affirmative sentences.

[0147] Learning facility 100 and inference facility 200 are configured to obtain a sentence as a string and execute the processes sentence by sentence. However, if the sentence type is not to be specified, a multitude of sentences can be processed as a single string.

[0148] In embodiment 1, the inference device 200 is configured to use not only the PMI adjectival expression model, but also the N-gram model and the neural language model to infer the candidate word with higher accuracy. However, if accuracy is not required, or if computational simplicity is desired, the candidate word can be inferred using only the PMI adjectival expression model, or, in addition to the PMI adjectival expression model, using either the N-gram model or the neural language model. If the N-gram model is not to be used, only the control word should be obtained as the control information, since control intensity is not required.

[0149] Inference device 200 is configured to procure a single word as the control word, but it can also be configured to procure a multitude of words for this purpose. For example, if more than one word is used as the control words, the PMI in Equation 9 should be the sum of the PMI values ​​for all of the control words.

[0150] The order of the steps in the flowcharts can be changed as needed. For example, the process for generating the trained model in step S9 can be performed before the process for generating the N-gram model in step S8, or the processes of step S8 and step S9 can be performed simultaneously.

[0151] For example, the process performed by the second inference unit 252 in step S17 can be performed before the process performed by the first inference unit 251 in step S16, or the processes of step S16 and step S17 can be performed simultaneously. The same applies to the other steps.

[0152] The trained model is described for the case where w x and w y They are symmetrical, but they can also be asymmetrical. Regarding the word order of w x and w y As described above, the backward order of the words is added during the learning process to obtain a learning outcome that is robust against rarity. However, the backward order should not be added to make the meaning given by the word order more distinctive. Design 2

[0153] Next, embodiment 2 is described. In embodiment 2, a configuration with a distortion elimination unit for eliminating distortions between two words is described, in addition to that of embodiment 1.

[0154] In today's world, where diversity is paramount, it can be ethically inappropriate to ascribe superiority or inferiority to two things. A learning device 2100 according to embodiment 2 eliminates the bias between things specified by two words by performing bias-removal processing between the two words with respect to a trained model, in order to prevent the generation of an unethical sentence.

[0155] A configuration of a speech processing system 2010 according to embodiment 2 is described. Fig. Figure 18 is a diagram showing a configuration of the speech processing system 2010 according to embodiment 2. The inference unit 2200 of the speech processing system 2010 infers the candidate word by using a de-squared trained model instead of the trained model according to embodiment 1, and the other configurations are the same as those of the inference unit 200 according to embodiment 1. A speech model storage unit 2300 stores the de-squared trained model instead of the trained model according to embodiment 1, and the other configurations are the same as those of the speech model storage unit 300 according to embodiment 1.

[0156] A configuration of the learning facility 2100 according to embodiment 2 is described. Fig. Figure 19 is a representation showing a configuration of the learning device 2100 according to embodiment 2. The learning device 2100 comprises a learning data acquisition unit 2110, a word sequence acquisition unit 2120, a species identification unit 2130, an N-gram model generation unit 2140, a learning unit 2150, and a distortion removal unit 2160.

[0157] The learning data acquisition unit 2110, the word sequence acquisition unit 2120, the species identification unit 2130, the N-gram model generation unit 2140 and the learning unit 2150 perform similar processes to those in embodiment 1.

[0158] The hardware configuration of learning facility 2100 is the same as the hardware configuration of learning facility 100, as described in Fig. 8 embodiment 1, and the distortion elimination unit 2160 is implemented by a program stored in the storage device 1001, which is executed by the processing device 1000.

[0159] The distortion removal unit 2160 eliminates the distortion between two words by performing distortion removal processing on the trained model D6 generated by the learning unit 2150.More precisely, if it is assumed that the distortion between the two words, i.e., a first distortion word and a second distortion word, is to be removed, the distortion removal unit 2160 performs distortion removal processing on the trained model D6 to balance the pointwise mutual information output by the trained model D6 when the first distortion word is input to the trained model D6, and the pointwise mutual information output by the trained model D6 when the second distortion word, which differs from the first distortion word, is input to the trained model D6, in order to produce a trained model D26 with the distortion removed.

[0160] In embodiment 2, the distortion elimination unit 2160 forms the average of the values ​​of the two sets of pointwise mutual information according to equations 10 to 13 below. PMI(G,A,wx,X)=12(PMI(G,A,wx,X)+PMI(G,A,wx,Y)) PMI(G,A,wx,Y)=12(PMI(G,A,wx,X)+PMI(G,A,wx,Y)) PMI(G,A,X,wy)=12(PMI(G,A,X,wy)+PMI(G,A,Y,wy)) PMI(G,A,Y,wy)=12(PMI(G,A,X,wy)+PMI(G,A,Y,wy))

[0161] A specific example of processing the distortion correction unit 2160 is given using Fig. 20 described. Fig. Figure 20 is a conceptual representation to illustrate a concrete example of the processing of the distortion correction unit 2160.

[0162] For example, it is assumed that the first distortion word is "inu (dog)" and the second distortion word is "neko (cat)". Here, by replacing X = "inu (dog)" and Y = "neko (cat)" in equations 10 to 13, the pointwise mutual information when "inu (dog)" is included in the triplet of words of the trained model, and the pointwise mutual information when "neko (cat)" is included in the triplet of words of the trained model, can be balanced.

[0163] The learning device 2100 according to embodiment 2 can generate a trained model without superiority or inferiority between the two words by generating the de-distorted trained model using the de-distortion removal unit 2160, as described above, and thus the inference device 2200 can avoid a discriminatory representation between the two things by inferring the candidate word using the de-distorted trained model.

[0164] The modification in embodiment 1 is applicable to the speech processing system 2010 according to embodiment 2.

[0165] So far, the description has been based on the example of a case where the string to be processed is in Japanese, but it is not limited to this. Both embodiments 1 and 2 can also be applied to languages ​​other than Japanese, such as English. In this case, it is possible to implement the inference device, the learning device, and the language processing system for languages ​​other than Japanese by using the training dataset, dictionary, etc., for the language to which the embodiments are to be applied. [Industrial applicability]

[0166] The learning device and the inference device according to the present disclosure are suitable for use in a system that automatically generates sentences and conducts conversations with humans, such as a chatbot or an automated speech dialogue system. [List of reference symbols] 10, 2010 Speech processing system, 100, 2100 learning facility, 200, 2200 Inference device, 300, 2300 language model storage device, 110, 2110 Learning Data Acquisition Unit, 120, 2120 Word sequence procurement unit, 121,2121 Morpheme analysis unit, 122, 2122 Lexicalization unit, 130.2130 species identification unit, 140.2140 N-gram model generating unit, 150, 2150 learning unit, 210 Mask data acquisition unit, 220 word sequence procurement unit, 221 Morpheme analysis unit, 222 Lexicalization unit, 230 species identification unit, 240 Tax Information Procurement Unit, 250 inference units, 251 first inference unit, 252 second inference unit, 253 Inference result integration unit, 310 N-gram model storage unit, 320 Trained Model Memory Unit, 330 Neural Language Model Storage Unit QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 6884871

[0003] < / modifikation> < / inferenzphase> < / lernphase>

Claims

[1] Inference facility, comprising: a mask data acquisition unit for obtaining a string that contains a masked section; a word sequence procurement unit for segmenting the string into words and for procuring a word sequence containing a large number of words; a tax information procurement unit for obtaining an adjectival expression that represents a characteristic or condition of something, as a tax word; and an inference unit for inferring a candidate word for the masked section from the control word and the word sequence and for outputting the string in which the masked section is replaced by the candidate word. [2] Inference device according to claim 1, wherein the word sequence procurement unit comprises: a morpheme analysis unit for segmenting the string into morphemes that are grammatically minimal units, and for procuring a morpheme sequence that contains a variety of morphemes with attached lexical category information; and a lexicalization unit for linking the morphemes contained in the morpheme sequence on the basis of the lexical category information and for constructing the word sequence. [3] Inference device according to claim 1 or 2, wherein the inference unit infers the candidate word using a trained model that receives the control word, words contained in the word sequence and candidate words and outputs pointwise mutual information between the control word, the words contained in the word sequence and the candidate words. [4] Inference device according to claim 3, further comprising a unit of identification for determining a sentence type specified by the word sequence, wherein the inference unit infers the candidate word using the trained model, which receives the sentence type in addition to the control word, the words contained in the word sequence and the candidate words, and outputs the pointwise mutual information. [5] Inference device according to claim 3 or 4, wherein the inference unit infers the candidate word using the pointwise mutual information obtained from the trained model and an N-gram likelihood obtained from an N-gram model. [6] Inference device according to claim 5, wherein The tax information procurement unit obtains weighting factors of the point-wise mutual information and the N-gram likelihood as tax intensities, and The inference unit infers the candidate word by performing a maximum likelihood estimation of a sum of the pointwise mutual information and the N-gram likelihood based on the tax intensities. [7] Inference device according to any one of claims 3 to 6, wherein the inference unit comprises: a first inference unit for inferring candidate words for the masked section using the trained model; a second inference unit for inferring candidate words for the masked section using a neural language model; and an inference result integration unit for determining a candidate word that is common to both a first candidate word group, which is a set of the candidate words inferred by the first inference unit, and a second candidate word group, which is a set of the candidate words inferred by the second inference unit, as the candidate word to replace the masked section. [8] Learning institution, comprehensive: a training data acquisition unit for obtaining a string as training data contained in a training data set; a word sequence procurement unit for segmenting the string into words and for procuring a word sequence containing a large number of words; and A learning unit for generating a trained model that receives an adjectival expression representing a characteristic or state of something and two words, and outputs pointwise mutual information between the adjectival expression and the two words based on the probability of occurrence of the words contained in the word sequence in the training data set. [9] Learning device according to claim 8, wherein the learning unit generates the trained model on the basis of a probability of occurrence of a first word, which is an adjectival expression and is contained in the word sequence, in the training data set, a probability of occurrence of a second word, which is contained in the word sequence, in the training data set, a probability of occurrence of a third word, which is contained in the word sequence, in the training data set, and a probability of the simultaneous occurrence of the first word, the second word and the third word in the training data set. [10] Learning facility according to claim 8 or 9, furthermore, a unit of identification for determining a sentence type specified by the word sequence, the learning unit generates the trained model, which, in addition to the adjectival expression and the two words, receives the sentence type and outputs the point-by-point mutual information. [11] Learning device according to one of claims 8 to 10, wherein the word sequence procurement unit comprises: a morpheme analysis unit for segmenting the string into morphemes that are grammatically minimal units, and for procuring a morpheme sequence that contains a multitude of the morphemes with attached lexical category information; and a lexicalization unit for linking the morphemes contained in the morpheme sequence on the basis of the lexical category information and for constructing the word sequence. [12] Learning device according to any one of claims 8 to 11, further comprising a distortion removal unit for performing distortion removal processing for the trained model in order to balance the pointwise mutual information output by the trained model when a first distortion word is input into the trained model and the pointwise mutual information output by the trained model when a second distortion word, different from the first distortion word, is input into the trained model. [13] Inference methods, including: a mask data acquisition step to obtain a string that contains a masked section; a word sequence procurement step to segment the string into words and to obtain a word sequence containing a large number of words; a tax information procurement step to obtain an adjectival expression that represents a characteristic or condition of something, as a tax word; and an inference step to infer a candidate word for the masked section from the control word and the word sequence and to output the string in which the masked section is replaced by the candidate word. [14] Inference program that causes a computer to perform all the steps according to claim 13. [15] Method for generating a trained model, wherein the method comprises: a training data acquisition step to obtain a string as training data contained in a training data set; a word sequence procurement step to segment the string into words and to obtain a word sequence containing a large number of the words; and a learning step to generate a trained model that receives an adjectival expression representing a characteristic or state of something and two words, and outputs pointwise mutual information between the adjectival expression and the two words based on occurrence probabilities in the training data set of words contained in the word sequence. [16] Learning program that causes a computer to perform all the steps according to claim 15.

Images