Learning device, learning method, and learning program

Abstract

A training device (1) comprises: a first input reception unit (11) that receives inputs of text, a template of a feedback comment to the text, and a masked template in which zero or at least one word in the template is masked; and a first trained model generation unit (12) that generates a first trained model which outputs a masked template when text and a template are inputted, by using the text, the template, and the masked template as first teacher data.

Description

[Technical Field]

[0001] This invention relates to a learning device, a learning method, and a learning program. [Background technology]

[0002] Among natural language processing techniques, there is a text generation technique that uses a neural network to generate a target text, which is a text that has undergone a predetermined transformation from the source text, based on the text of a specified template.

[0003] Examples of the aforementioned technologies include tasks such as neural machine translation, which involves inputting a source text consisting of a sequence of words (word sequence) containing one or more words into a trained model such as a neural network and converting it into a target text consisting of a word sequence in another language. In such tasks, a neural sequence conversion model (neural sequence conversion model) that utilizes a neural network, such as an encoder-decoder model, is typically used.

[0004] Among tasks that use neural sequence transformation models (sequence transformation tasks), there is a known task that generates feedback comments (comments) for sentences with grammatical errors. In this task, if a sentence with a grammatical error is input as source text, comments that help language learners correct the grammatical error are output as target text. For example, in this task, if the input source text is "He agrees the opinion.", the target text might be something like, "Agree is an intransitive verb, so it requires an appropriate preposition. Look up 'agree' in the dictionary."

[0005] In more flexible tasks like generating comments on source text, external knowledge related to what is written in the source text (in this case, the knowledge that "agree" is an intransitive verb and that intransitive verbs require a preposition) is required, making it more difficult to generate the target text from scratch, just as in a normal sequence conversion task.

[0006] Therefore, a method has been proposed in which existing target text is input into a neural sequence conversion model, and new target text is generated to rewrite it. For example, by inputting "He agrees the opinion." and "adapt is an intransitive verb, so it requires an appropriate preposition. Let's look up adapt in the dictionary.", and outputting "agree is an intransitive verb, so it requires an appropriate preposition. Let's look up agree in the dictionary.", there is a method that effectively rewrites adapt to agree (see, for example, Non-Patent Document 1).

[0007] In this way, by using existing text as a template for comments on the source text and generating new text by rewriting that template, complex generation that would otherwise require external knowledge becomes easier. This method is useful not only for tasks that generate comments, but also for tasks that can acquire new target text by modifying a template, such as generating opinions on a given document. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Tatsunori B Hashimoto, Kelvin Guu, Yonatan Oren, and Percy Liang. A Retrieve-and-Edit Framework for Predicting Structured Outputs. In Proc. NeurIPS, pp. 10052-10062, 2018. [Overview of the Initiative] [Problems that the invention aims to solve]

[0009] However, the aforementioned method of using a template to rewrite that template was prone to excessive changes to the template. For example, although this method aims to transform only a portion of the template, it often results in problems such as rewriting the entire template. This is because the performance of this method varies greatly depending on the template selected for training the neural network.

[0010] Furthermore, if the template and target text are almost identical, the model only needs to learn to rewrite a portion of it. However, if the template and target text are significantly different, trained models such as neural sequence conversion models tend to learn to rewrite many words. Thus, with conventional techniques, the scope and amount of template rewriting varies depending on the template, and it is difficult to prepare only templates similar to the target text. As a result, it is difficult to appropriately learn or predict the scope and amount of rewriting to be done, and excessive changes to the template are likely to occur. [Means for solving the problem]

[0011] To solve the above-mentioned problems and achieve the objective, the learning device is characterized by comprising: a first input receiving unit that accepts input of text, a template for feedback comments on the text, and a mask template in which zero or one or more words of the template are masked; and a first trained model generation unit that uses the text, template, and mask template accepted by the first input receiving unit as first training data to generate a first trained model in which the mask template is output when the text and template are input. [Effects of the Invention]

[0012] According to the present invention, it is possible to make it less likely for excessive changes to the template to occur. [Brief explanation of the drawing]

[0013] [Figure 1] Figure 1 shows an example of the configuration of a learning device. [Figure 2] Figure 2 is a diagram illustrating the outlines of the first and second production units. [Figure 3] Figure 3 shows an example of the configuration of the first generation unit. [Figure 4] Figure 4 shows an example of the configuration of the second generation unit. [Figure 5] Figure 5 is a flowchart showing an example of the processing flow by the learning device. [Figure 6] Figure 6 is a flowchart showing an example of the flow of feedback comment generation processing by a learning device. [Figure 7] Figure 7 shows an example of a computer running a learning program.

[0014] Embodiments of the present invention will be described in detail below with reference to the drawings. However, the present invention is not limited to the following embodiments. In addition, the same parts are denoted by the same reference numerals in the drawings, and redundant explanations are omitted.

[0015] [1. Example of a learning device configuration] The learning device 1 according to the embodiment will be described using Figure 1. Figure 1 is a diagram showing an example of the configuration of the learning device 1 according to the embodiment. When the learning device 1 receives text and a template of feedback comments for the text as input, it generates a first trained model in which a masked template is output in which zero or one or more words of the template are masked.

[0016] Here, "text" refers to source text, while "feedback comments" refer to target text. "Mask templates" include templates where the parts to be changed are replaced with special tokens called mask tokens, or templates where the parts to be changed are specified by a mask. Regarding mask templates, if the number of masked words in the template is 0, that is, if none of the words in the template are masked, the mask template becomes the template itself. Thus, the term "mask template" here includes templates where not all words are masked. A mask token is a special token used in Masked Language Models, such as BERT (Bidirectional Encoder Representations from Transformers), which predict only the words replaced by mask tokens in the text. For details on mask tokens, please refer to Reference 1 below. Reference 1: Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv: 1810.04805, 2018.

[0017] For example, learning device 1 generates a first trained model that outputs a masked template in which words are replaced with masked tokens by masking only the words that need to be changed in the template. As an example, if learning device 1 is given the text "He agrees the opinion." and the template "agree is an intransitive verb, so it needs an appropriate preposition. Look up agree in the dictionary.", it generates a first trained model that outputs a masked template "[MASK] is an intransitive verb, so it needs an appropriate preposition. Look up [MASK] in the dictionary."

[0018] Examples of pre-trained models include neural network-based sequence labeling models (neural sequence labeling models) and neural sequence transformation models, and any one of these models can be used. Examples of neural sequence labeling models include those using bidirectional LSTM (Long Short Term Memory). Examples of neural sequence transformation models include those using attention-based encoder-decoder models and Transformers. For details on these models, please refer to references 2-4 below. Reference 2: Ji Ma, Kuzman Ganchev, and David Weiss. State-of-the-art Chinese Word Segmentation with Bi-LSTMs. In Proc. EMNLP, pp.4902-4908, 2018. Reference 3: Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. Effective Approaches to Attention-based Neural Machine Translation. In Proc. EMNLP, pp. 1412-1421, 2015. Reference 4: A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, AN Gomez, L. Kaiser, and I. Polosukhin. Attention Is All You Need. In Proc. NIPS, pp. 5998-6008, 2017.

[0019] As described above, learning device 1 is applicable when performing the task of generating feedback comments for grammatically incorrect text translated from Japanese into English. However, learning device 1 can also be applied to other tasks, such as generating feedback comments for text in other languages, such as Japanese, and generating feedback comments for articles on websites. In this case, learning device 1 can perform the other tasks in the same manner as in the example above, except that it uses different text, templates, mask templates, and feedback comments than in the example above.

[0020] To perform this processing, first, the learning device 1 generates a first trained model that outputs a mask template when text and a template of feedback comments for the text are input. Next, the learning device 1 generates a second trained model that outputs feedback comments when text and a mask template are input. Examples of the second trained model include a neural sequence labeling model and a neural sequence transformation model, and are not particularly limited to these models. As shown in Figure 1, the learning device 1 has a storage unit 2 and a control unit 3.

[0021] The memory unit 2 stores the OS (Operating System) and various programs executed by the learning device 1, as well as various data such as text, templates, mask templates, feedback comments, the first trained model, and the second trained model. Examples of storage devices for the memory unit 2 include HDDs (Hard Disk Drives), SSDs (Solid State Drives), and optical discs. The memory unit 2 may also be a rewritable semiconductor memory such as RAM (Random Access Memory), flash memory, and NVSRAM (Non-Volatile Static Random Access Memory). The memory unit 2 stores the OS (Operating System) and various programs executed by the learning device 1.

[0022] The control unit 3 controls the entire learning device 1. The control unit 3 is, for example, an electronic circuit such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), and GPU (Graphics Processing Unit), as well as an integrated circuit such as an ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array). The control unit 3 also has an internal memory for storing programs that define various processing procedures and control data, and executes each process using the internal memory. Furthermore, the control unit 3 functions as various processing units as various programs are run. As shown in Figure 1, the control unit 3 has a first generation unit 10 and a second generation unit 20.

[0023] Before describing the configuration examples of the first generation unit 10 and the second generation unit 20, we will explain the overview of the first generation unit 10 and the second generation unit 20 using Figure 2. Figure 2 is a diagram illustrating the overview of the first generation unit 10 and the second generation unit 20. First, the first generation unit 10 is set D, which will be described later. m or set D gThe text, template, and mask template included in the data are used as the primary training data, and the model parameter Θ m The first trained model is trained until it is optimized. The first generation unit 10 also generates the optimized model parameters Θ of the first trained model. m Based on the output information from the first trained model, a mask template is generated by probability calculation according to the formula. Next, the second generation unit 20 generates a mask template based on the set D described later. m or set D g The text, mask template, and feedback comments included in the data are used as the second training data to set the model parameter Θ. g The second trained model is trained until the optimized model parameters Θ of the second trained model are obtained. g Based on the output information from the second trained model, a feedback comment is generated through probability calculations according to the formula. The second generation unit 20 recursively performs the above process by training the second trained model using the feedback comment output from the second trained model as second training data.

[0024] [1-1. Example of the configuration of the first generation unit] The first generation unit 10 generates a first trained model and a mask template. As shown in Figure 1, the first generation unit 10 includes a first input receiving unit 11, a first trained model generation unit 12, and a mask template generation unit 13.

[0025] [1-1-1. Example of configuration of the first input receiving unit] The first input receiving unit 11 accepts input of text, a template for a feedback comment on the text, and a mask template in which one or more words of the template are masked. The first input receiving unit 11 may accept these inputs from outside the learning device 1, or it may accept input of data selected from the storage unit 2.

[0026] [1-1-2. Example of the configuration of the first pre-trained model generation unit] The first trained model generation unit 12 generates a first trained model using the text, template, and mask template received by the first input reception unit 11 as first teacher data. For example, the first trained model generation unit 12 generates the first trained model by machine learning the first trained model until the model parameters of the first trained model are optimized using the first teacher data.

[0027] The first trained model generation unit 12 may use any text as the first teacher data. The first trained model generation unit 12 may use any mask template as the first teacher data as long as the unmasked words all include the words of the feedback comment. That is, the first trained model generation unit 12 may generate the first trained model using, as the first teacher data, a mask template that includes only the words of the feedback comment except for the words at the masked positions. For example, the first trained model generation unit 12 may use, as the first teacher data, a mask template in which the masked words are selected by manual labeling. Also, the first trained model generation unit 12 may use, as the first teacher data, a mask template in which the masked words are selected according to a rule-based method of masking only the words of a specific part of speech using part-of-speech information such as verbs and nouns.

[0028] The first trained model generation unit 12 may generate the first trained model using first teacher data in which at least one of first semantic information indicating the meaning of one or more words and second semantic information indicating the meaning of the entire sentence is included in at least one of the text and the template.

[0029] Hereinafter, the first trained model generation unit 12 will be described in detail using mathematical formulas. The first trained model generation unit 12 is a sequence of words (word sequence) X = {x1,..., x n} and a word sequence Y = {y1,..., y n} and a sequence of mask labels (mask label sequence) L = {l1, ..., l n Set D of pairs with} m The data is input to the first trained model as the first training data, and the model parameters Θ m The first pre-trained model is used for machine learning until it is optimized.

[0030] Here, if the word sequence consists of English words, the first trained model generation unit 12 may, for example, consider a unit separated by a space as one word. If the word sequence consists of Japanese words, the first trained model generation unit 12 may, for example, consider a unit automatically divided by morphological analysis, or a unit divided into character units, as one word.

[0031] Optimized model parameter Θ m The following equation (1) applies: Model parameters Θ generated by the first trained model generation unit 12 m The optimization of can be done using optimization methods common in neural network training. Furthermore, the first generation unit 10 is represented by the following equation (1) Θ m The optimized model parameter Θ m It shall be used as such.

[0032]

number

[0033] In the example where the above formula is used, the first trained model generation unit 12 does not directly output the mask template, but instead outputs information about the mask level, indicating which word in the mask template is masked by the mask label, starting from the first word. However, the first trained model generation unit 12 may generate a first trained model in which the mask template is output instead of the mask label sequence. In this case, the first trained model generation unit 12 may use the mask template as the first training data instead of the mask label sequence. In this case as well, the first trained model generation unit 12 can generate the first trained model in the same way as in the example above. Furthermore, when the first trained model generation unit 12 uses the mask template as the first training data instead of the mask label sequence, the mask template generation unit 13, described later, can generate the mask template by using the output information from the first trained model as is.

[0034] Below, an example of the first trained model generation unit 12 will be explained in more detail using Figure 3. Figure 3 is a diagram showing an example of the configuration of the first generation unit 10. In the example described above, the first trained model generation unit 12 is described in the case where it generates one first trained model using one set of first training data. However, as shown in Figure 3, if the mask template generation unit 13 has multiple members, the first trained model generation unit 12 may generate multiple trained models corresponding to each of the multiple members. In this case, the model parameter Θ m is, Θ m ={Θ h , Θ u , Θ w , Θ p It is represented as}.

[0035] For example, the first trained model generation unit 12 may generate a trained model for calculating text word embedding vectors, which is used in the calculation of text word embedding vectors by the text word embedding vector calculation unit 131. Here, a text word embedding vector is a vector into which first semantic information indicating the meaning of one or more words in a text is embedded. This first semantic information is used to determine the locations of words in the template that correspond to words in the text, which are locations to be masked in the template. The number of dimensions of the text word embedding vector and the method for determining the number of dimensions are not particularly limited, but examples of the number of dimensions include 512 dimensions, and examples of methods for determining the number of dimensions include manual determination. Thus, the larger the number of dimensions, the more information the text word embedding vector contains.

[0036] As an example, the first trained model generation unit 12 generates a trained model for calculating text word embedding vectors using text and text word embedding vectors in which the first semantic information is embedded in the text as training data for calculating text word embedding vectors. The first trained model generation unit 12 generates a trained model for calculating text word embedding vectors by machine learning the trained model for calculating text word embedding vectors using the training data for calculating text word embedding vectors until the model parameters of the trained model for calculating text word embedding vectors are optimized.

[0037] The first pre-trained model generation unit 12 may generate a pre-trained model for calculating text word sequence embedding vectors, which is used in the calculation of text word sequence embedding vectors by the text word sequence embedding vector calculation unit 132. Here, a text word sequence embedding vector is a vector that contains semantic information about a word sequence, which is an aggregation of first semantic information indicating the meaning of each word in the text, i.e., second semantic information indicating the meaning of the entire text. This second semantic information is used to determine the locations to be labeled for the word sequence of the next template. The number of dimensions of the text word sequence embedding vector and the method for determining the number of dimensions are not particularly limited, but for example, the number of dimensions may be 512, and the method for determining the number of dimensions may be a manual method. Thus, the larger the number of dimensions, the more information the text word sequence embedding vector contains.

[0038] As an example, the first pre-trained model generation unit 12 generates a pre-trained model for calculating text word sequence embedding vectors by using text word embedding vectors and text word sequence embedding vectors as training data for calculating text word sequence embedding vectors. The first pre-trained model generation unit 12 generates a pre-trained model for calculating text word sequence embedding vectors by using the training data for calculating text word sequence embedding vectors to machine-learn the pre-trained model for calculating text word sequence embedding vectors until the model parameters of the pre-trained model for calculating text word sequence embedding vectors are optimized.

[0039] The first pre-trained model generation unit 12 may generate a pre-trained model for calculating template word embedding vectors, which is used in the calculation of template word embedding vectors by the template word embedding vector calculation unit 133. Here, a template word embedding vector is a vector into which first semantic information indicating the meaning of one or more words in the template is embedded. This first semantic information is used to determine the locations of words in the template that correspond to words in the text, which are the locations to be masked in the template. The number of dimensions of the template word embedding vector can be as high as 512 dimensions, and the number of dimensions can be determined by manual determination, etc. Thus, the larger the number of dimensions, the more information the template word embedding vector contains.

[0040] As an example, the first trained model generation unit 12 generates a trained model for calculating template word embedding vectors using a template and a template word embedding vector in which the first semantic information is embedded in the template as training data for calculating template word embedding vectors. The first trained model generation unit 12 generates a trained model for calculating template word embedding vectors by machine learning the trained model for calculating template word embedding vectors using the training data for calculating template word embedding vectors until the model parameters of the trained model for calculating template word embedding vectors are optimized.

[0041] The first trained model generation unit 12 may generate a trained model for calculating text and template word embedding vectors, which is used in the calculation of text and template embedding vectors by the text and template word embedding vector calculation unit 134. Here, text and template embedding vectors refer to template word embedding vectors into which information of text word sequence embedding vectors is embedded. Any function can be used as a method for embedding information of text word sequence embedding vectors into template word embedding vectors, as long as it uses a conversion function that converts these two vectors into a single vector. Examples of methods that use a function to convert these two vectors into a single vector include using a function that adds these two vectors together, and concatenating the two vectors. In these methods, the number of dimensions of the template word embedding vector and the number of dimensions of the text word sequence embedding vector must be the same.

[0042] As an example, the first pre-trained model generation unit 12 generates a pre-trained model for calculating text and template word embedding vectors by using template word embedding vectors, text word sequence embedding vectors, and text and template embedding vectors as training data for calculating text and template word embedding vectors. The first pre-trained model generation unit 12 generates a pre-trained model for calculating text and template word embedding vectors by using the training data for calculating text and template word embedding vectors to machine-learn the pre-trained model until the model parameters of the pre-trained model for calculating text and template word embedding vectors are optimized.

[0043] The first trained model generation unit 12 may generate a trained model for calculating posterior probabilities used in the calculation of posterior probabilities by the first posterior probability calculation unit 135. Here, the first posterior probability refers to the posterior probability that a mask label is masked for a given word in a template. Examples of the first posterior probability include those in which each mask label can be represented as a vector, and the probability distribution of the posterior probability can be expressed by vector transformation.

[0044] As an example, the first trained model generation unit 12 generates a first trained model for calculating posterior probabilities by using text, template word embedding vectors, and the first posterior probability as training data for calculating the first posterior probability. The first trained model generation unit 12 generates a first trained model for calculating posterior probabilities by using the first training data for calculating posterior probabilities to train the first trained model for calculating posterior probabilities until the model parameters of the first trained model for calculating posterior probabilities are optimized.

[0045] The first trained model generation unit 12 may generate a trained model for mask template generation used by the mask template generation unit 136 to generate a mask template. As an example, the first trained model generation unit 12 generates a trained model for mask template generation using the first posterior probability and mask template as training data for mask template generation. The first trained model generation unit 12 generates a trained model for mask template generation by machine learning the trained model for mask template generation using the training data for mask template generation until the model parameters of the trained model for mask template generation are optimized.

[0046] [1-1-3. Example of the configuration of the mask template generation section] The mask template generation unit 13 generates a mask template based on the text and template received as input by the first input receiving unit 11, using the first trained model generated by the first trained model generation unit 12. In this case, the mask template generation unit 13 inputs the text and template into the first trained model and generates a mask template based on the output information output from the first trained model by probability calculation according to the model parameters of the first trained model. For example, the mask template generation unit 13 generates a mask template with the corresponding number of words in the mask template masked by the mask label, based on the output information indicating which word in the mask template, starting from the first word, is masked by the mask label.

[0047] The mask template generation unit 13 may use comments or templates selected from the comments or templates used in the first training data. The mask template generation unit 13 may use templates selected from the feedback comments used in the second training data for which the second trained model generation unit 22 (described later) is generated. The mask template generation unit 13 may use comments or templates selected from the storage unit 2, which functions as a data pool. Any method can be used to select a template, as long as it extracts one template. The method of selecting a template may be to select the one with the highest similarity between the template and the feedback comments used in the second training data, or it may be selected randomly.

[0048] The following describes in detail an example of the mask template generation unit 13 using mathematical formulas. The mask template generation unit 13 inputs text and a template into a first trained model and generates a mask template based on the output information output from the first trained model by performing probability calculations according to the model parameters of the first trained model.

[0049] For example, the mask template generation unit 13 masks the nth word of the template word sequence Y by using the mask label sequence L output from the first trained model, which masks the nth word of the template word sequence Y, to create the mask template word sequence Y'={y1', ..., y n '} is generated. In this case, the mask template generation unit 13 uses the first trained model to calculate the first posterior probability P(l) of the mask label to be masked for the nth word of the template. n |X, Y, Θ m We obtain the first posterior probability P(l) of the mask label, and determine the mask label ln' that is masked for the nth word by the maximum likelihood criterion according to equation (2) below. n |X, Y, Θ m A detailed explanation of this will be given later.

[0050]

number

[0051] The mask template generation unit 13 generates a mask label sequence L corresponding to the word sequence Y of the template by performing the above-described process for all word sequences included in the template. Here, l n n is an element of a set composed of 0s and 1s. The mask template generation unit 13 determines the value of n corresponding to the location of the word to be masked by the mask label, by assigning 1 to words in the template that are not included in the word sequence of the feedback comment for the text, and 0 to all other words.

[0052] Below, an example of the mask template generation unit 13 will be described in more detail using Figure 3. As an example, the mask template generation unit 13 includes a text word embedding vector calculation unit 131, a text word sequence embedding vector calculation unit 132, a template word embedding vector calculation unit 133, a text and template word embedding vector calculation unit 134, a first posterior probability calculation unit 135, and a mask template generation unit 136.

[0053] The text word embedding vector calculation unit 131 calculates text word embedding vectors using the pre-trained model for text word embedding vector calculation generated by the first pre-trained model generation unit 12. For example, the text word embedding vector calculation unit 131 calculates the text word sequence X = {x1, ..., x m The} is input to a trained model for calculating text word embedding vectors, and the output information from the trained model for calculating text word embedding vectors is used to create the text word embedding vector h m It is calculated as follows. For example, the text word embedding vector calculation unit 131 calculates the optimized model parameter Θ of the pre-trained model for calculating text word embedding vectors. h Based on the transformation function, the text word sequence X is transformed into the text word embedding vector h. m It is calculated as follows.

[0054] In this case, any function capable of converting a variable-length sequence of symbols into a variable-length vector can be used as the conversion function. Examples include functions that construct word frequency vectors in text, recurrent neural networks, bidirectional recurrent neural networks, and self-attention mechanisms such as those described in Reference 3 above.

[0055] The text word sequence embedding vector calculation unit 132 calculates text word sequence embedding vectors using the pre-trained model for calculating text word sequence embedding vectors generated by the first pre-trained model generation unit 12. For example, the text word sequence embedding vector calculation unit 132 calculates the word sequence H={h1, ..., h} of the text word embedding vector calculated by the text word sequence embedding vector calculation unit 131. m The} is input to a trained model for calculating text word sequence embedding vectors, and the output information from the trained model for calculating text word sequence embedding vectors is used to create the text word sequence embedding vector u mIt is calculated as follows. For example, the text word sequence embedding vector calculation unit 132 calculates the optimized model parameter Θ of the trained model for calculating text word sequence embedding vectors. u The word sequence H of the text word embedding vector is transformed into the text word sequence embedding vector u by a transformation function based on this. m It is calculated as follows.

[0056] In this case, any function that converts a variable-length vector sequence into a single vector can be used as the conversion function. Examples include a recurrent neural network and a function that averages each vector in the word sequence of a text word sequence embedding vector. When each vector is averaged, the dimensionality of the text word sequence embedding vector depends on the dimensionality of each text word sequence embedding vector.

[0057] In the example shown in Figure 3 and Figure 4 described later, only the text word sequence embedding vector calculation unit 132 calculates a word sequence embedding vector containing second semantic information from a word embedding vector containing first semantic information. However, if the meaning of the entire text is unclear, or if the word embedding vector is other than a mask template word embedding vector that changes the meaning of the entire text, each component can calculate a word sequence embedding vector from the word embedding vector. For example, the mask template generation unit 13 may further include a template word sequence embedding vector calculation unit and a text and template word sequence embedding vector calculation unit. The template word sequence embedding vector calculation unit calculates a template word sequence embedding vector, and the text and template word embedding vector calculation unit 134 may calculate text and template word sequence embedding vectors.

[0058] The template word embedding vector calculation unit 133 calculates template word embedding vectors using the pre-trained model for template word embedding vector calculation generated by the first pre-trained model generation unit 12. For example, the template word embedding vector calculation unit 133 calculates template word sequence Y={y1, ..., y n The} is input into a trained model for calculating template word embedding vectors, and the output information from the trained model for calculating template word embedding vectors is used to create the template word embedding vector w n It is calculated as follows. For example, the template word embedding vector calculation unit 133 calculates the optimized model parameter Θ of the trained model for template word embedding vector calculation. w The transformation function based on this transforms the template word sequence Y into the template word embedding vector w. n It is calculated as follows. In this case, the conversion function can be the same as the conversion function used in the calculation of the text word embedding vector by the text word embedding vector calculation unit 131 described above.

[0059] The text and template word embedding vector calculation unit 134 calculates text and template word embedding vectors using the pre-trained model for calculating text and template word embedding vectors generated by the first pre-trained model generation unit 12. For example, the text and template word embedding vector calculation unit 134 calculates the text word sequence embedding vector u calculated by the text word sequence embedding vector calculation unit 132. m , and the template word embedding vector w calculated by the template word embedding vector calculation unit 133 n This is input to the trained model for calculating text and template word embedding vectors. Subsequently, the text and template word embedding vector calculation unit 134 takes the output information from the trained model for calculating text and template word embedding vectors and uses it to calculate the text and template word embedding vector v corresponding to the nth word of the template. nIt is calculated as follows. For example, the text and template word embedding vector calculation unit 134 calculates the optimized model parameter Θ of the trained model for calculating text and template word embedding vectors. v Based on the transformation function, the text word sequence embedding vector u m and template word embedding vector w n The converted result is a text and template word embedding vector v corresponding to the nth word of the template. n It is calculated as follows.

[0060] In this case, the conversion function may be the same as the conversion function used in the generation of the pre-trained model for calculating text and template word embedding vectors by the first pre-trained model generation unit 12 described above.

[0061] The first posterior probability calculation unit 135 calculates the first posterior probability using the first trained model for calculating the posterior probability generated by the first trained model generation unit 12. For example, the first posterior probability calculation unit 135 calculates the text and template word embedding vector v corresponding to the nth word of the template, calculated by the text and template word embedding vector calculation unit 134. n The first pre-trained model for calculating posterior probabilities is input, and the output information from the first pre-trained model for calculating posterior probabilities is used to determine the first posterior probability P(l) for which a mask label is assigned to the nth word of the template. n |X, Y, Θ m ) is calculated as follows. For example, the first posterior probability calculation unit 135 calculates the optimized model parameter Θ of the first posterior probability calculation trained model. p Based on the conversion function, the text and template word embedding vector v corresponding to the nth word of the template are obtained. n The transformed result is the first posterior probability P(l) that a mask label is assigned to the nth word of the template. n |X, Y, Θ m It is calculated as follows:

[0062] In this case, any transformation function can be used as long as it transforms the vector into a posterior probability distribution. Examples of transformation functions include a function that performs a transformation using the softmax function, and a function that can transform the sum of the elements of the output vector corresponding to the posterior probability distribution to 1.0.

[0063] The mask template generation unit 136 generates a mask template using the mask template generation trained model generated by the first trained model generation unit 12. The mask template generation unit 136 uses the word sequence Y of the template and the first posterior probability P(1|X, Y, Θ) calculated by the first posterior probability calculation unit 135, which assigns mask labels to the word sequences of the template from the 1st to the nth. m ), ..., P(l n |X, Y, Θ m The output information from the trained model for mask template generation is input to the trained model for mask template generation and calculated as the word sequence Y' of the mask template.

[0064] In this case, the mask template generation unit 136 generates the word sequence Y={y1, ..., y} from the 1st to the nth template. n The first posterior probability P(1|X, Y, Θ) to which a mask label is assigned to} m ), ..., P(l n |X, Y, Θ m Select the one that maximizes ). For example, the mask template generation unit 136 selects the word y of the nth template. n The first posterior probability P(l) corresponds to this. n |X, Y, Θ m ) is selected. Next, the mask template generation unit 136 generates the nth word y of the template. n By assigning a mask label to the corresponding word, that word is replaced with a mask token.

[0065] If consecutive words are replaced by consecutive mask tokens, the mask template generation unit 136 replaces these mask tokens with a single mask token. If none of the words in the template are masked, the mask template generation unit 136 outputs the word sequence of the template as is, as the word sequence of the feedback comment for the text.

[0066] Furthermore, the mask template generation unit 136 may replace consecutive words to be masked with a single mask token using span masking, which allows the feedback comment generation unit 23 (described later) to predict one or more words from a single mask token. Although masked words are replaced with mask tokens, the number of masked words and the number of words to be predicted are not necessarily equal. By using span masking, the mask template generation unit 136 allows the feedback comment generation unit 23 to predict a number of words different from the number of masked words. For details on span masking, please refer to reference 5 below. Reference 5: Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research, pp.1-67, 2020.

[0067] [1-2. Example of the configuration of the second generation unit] The second generation unit 20 generates a second trained model and feedback comments. As shown in Figure 1, the second generation unit 20 includes a second input receiving unit 21, a second trained model generation unit 22, and a feedback comment generation unit 23.

[0068] [1-2-1. Example of the configuration of the second input receiving unit] The second input receiving unit 21 accepts input of text, a mask template, and feedback comments in which the masked portion of the mask template has been modified. The second input receiving unit 21 may accept these inputs from outside the learning device 1, or it may accept inputs selected from those stored in the storage unit 2. The second input receiving unit 21 may also accept input of feedback comments output from the second trained model, i.e., feedback comments generated by the feedback comment generation unit 23.

[0069] [1-2-2. Example of the configuration of the second pre-trained model generation unit] The second trained model generation unit 22 generates a second trained model using the text, mask template, and feedback comments received by the second input receiving unit 21 as second training data. For example, the second trained model generation unit 22 generates a second trained model by using the second training data to machine-learn the second trained model until its model parameters are optimized.

[0070] The second trained model generation unit 22 may use any text as second training data. The second trained model generation unit 22 may use any mask template as second training data, provided that all unmasked words are included in the words of the feedback comments. That is, the second trained model generation unit 22 may generate a second trained model using a mask template as second training data that includes only the words of the feedback comments, excluding the words in the masked areas. For example, the second trained model generation unit 22 may use a mask template as first training data in which the words to be masked have been selected by manual labeling. Alternatively, the second trained model generation unit 22 may use a mask template as second training data in which the words to be masked have been selected by a rule-based method that masks only words of a specific part of speech using part-of-speech information such as verbs and nouns.

[0071] The second trained model generation unit 22 may recursively train the second trained model using the feedback comments output from the second trained model (feedback comments generated by the feedback comment generation unit 23) as second training data. The second trained model generation unit 22 may generate the second trained model using second training data in which the first semantic information is included in at least one of the text, mask template, and feedback comments. The second trained model generation unit 22 may generate the second trained model using second training data in which the second semantic information is included in at least one of the text and feedback comments. The second trained model generation unit 22 may generate the second trained model using second training data in which the feedback comments contain contextual information indicating which word is changed, starting from the first word of the feedback comment, according to the text and mask template. Details of these will be described later.

[0072] The following describes in detail an example of the second pre-trained model generation unit 22 using mathematical formulas. The second pre-trained model generation unit 22 generates a text word sequence X = {x1, ..., x n} and the word sequence of the mask template Y'={y1', ..., y n '} and the word sequence Y''={y1'', ..., y} of the first to n-1 feedback comments already generated by the feedback comment generation unit 235. n-1 ''} and the nth word y in the feedback comment for the text n Set D of pairs with '' g The second training data is input into the second trained model, and the model parameters Θ g The second trained model is then subjected to machine learning until the optimized model parameters Θ are obtained. g The following equation (3) applies: Model parameters Θ generated by the second trained model generation unit 22 g The optimization of the second generation unit 20 is represented by the following equation (3) Θ g The optimized model parameter Θ g It shall be used as such.

[0073]

number

[0074] Below, an example of the second trained model generation unit 22 will be explained in more detail using Figure 4. Figure 4 is a diagram showing an example of the configuration of the second generation unit 20. In the example described above, the second trained model generation unit 22 generates one second trained model using one set of second training data. However, as shown in Figure 4, if the feedback comment generation unit 23 has multiple members, the second trained model generation unit 22 may generate multiple trained models corresponding to each of the multiple members. In this case, the model parameter Θ g is, Θ g ={Θ q , Θ f , Θ s , Θ k It is represented as}.

[0075] For example, the second trained model generation unit 22 may generate a trained model for calculating feedback comment word embedding vectors, which is used in the calculation of feedback comment word embedding vectors by the feedback comment word embedding vector calculation unit 231. A feedback comment word embedding vector is a vector in which first semantic information indicating the meaning of one or more words in a feedback comment is embedded. The second trained model generation unit 22 generates a trained model for calculating feedback comment word embedding vectors using the feedback comment and the feedback comment word embedding vector as training data for calculating feedback comment word embedding vectors. The method for generating the trained model for calculating feedback comment word embedding vectors is the same as the method for generating the trained model for calculating text word embedding vectors, so a detailed explanation is omitted.

[0076] The second pre-trained model generation unit 22 may generate a pre-trained model for calculating text and mask template word embedding vectors, which is used in the calculation of text and mask template word embedding vectors by the text and mask template word embedding vector calculation unit 232. Text and mask template word embedding vectors refer to mask template word embedding vectors into which information of text word sequence embedding vectors is embedded. The second pre-trained model generation unit 22 generates a pre-trained model for calculating text and mask template word embedding vectors using text, mask templates, and text and mask template word embedding vectors as training data for calculating text and mask template word embedding vectors. The method for generating the pre-trained model for calculating text and mask template word embedding vectors is the same as the method for generating the pre-trained model for calculating text word embedding vectors, so a detailed explanation is omitted.

[0077] The second pre-trained model generation unit 22 may generate a pre-trained model for context vector calculation used in the context vector calculation by the context vector calculation unit 233. Here, a context vector is a vector in which context information is embedded, indicating which word in the feedback comment, starting from the first word, is changed, according to the text and mask template. The second pre-trained model generation unit 22 generates a pre-trained model for context vector calculation using the feedback comment word embedding vector, the word embedding vectors of the text and mask template, and the context vector as training data for context vector calculation. The method for generating the pre-trained model for context vector calculation is the same as the method for generating the pre-trained model for text word embedding vector calculation, so a detailed explanation is omitted.

[0078] The second trained model generation unit 22 may generate a trained model for calculating posterior probabilities, which is used by the second posterior probability calculation unit 234 to calculate the second posterior probability. The second posterior probability refers to the posterior probability that a word with a feedback comment corresponds to the text, that is, the posterior probability indicating whether the word at the location where the mask label of the mask template is attached will be replaced with the word corresponding to the text. The second trained model generation unit 22 generates a trained model for calculating the second posterior probability using the context vector and the second posterior probability as training data for calculating the second posterior probability. The method for generating the second trained model for calculating posterior probabilities is the same as the method for generating the first trained model for calculating posterior probabilities, so a detailed explanation is omitted.

[0079] The second trained model generation unit 22 may generate a trained model for generating feedback comments, which will be used by the feedback comment generation unit 235 to generate feedback comments. As an example, the second trained model generation unit 22 generates a trained model for generating feedback comments using the second posterior probability and feedback comments as training data for generating feedback comments. The method for generating the trained model for generating feedback comments is the same as the method for generating the trained model for generating mask templates, so a detailed explanation is omitted.

[0080] [1-2-3. Example of the configuration of the feedback comment generation unit] The feedback comment generation unit 23 generates a feedback comment based on the text and mask template received as input by the second input receiving unit 21, using the second trained model generated by the second trained model generation unit 22. In this case, the feedback comment generation unit 23 inputs the text and mask template into the second trained model and the model parameter Θ of the second trained model. g Based on the output information from the second trained model, a feedback comment is generated by probability calculation according to the formula. For example, the feedback comment generation unit 23 generates a mask template as a feedback comment, which is replaced with the corresponding word, based on the output information indicating which word corresponds to the text, starting from the first word of the feedback comment, that is, which word in the mask template with a mask label is replaced with the corresponding word in the text.

[0081] The feedback comment generation unit 23 may use comments or templates selected from the comments and templates used in the second training data. The feedback comment generation unit 23 may use comments or templates selected from the storage unit 2. The method for selecting templates is the same as described above, so the explanation is omitted.

[0082] The following describes in detail an example of the feedback comment generation unit 23 using mathematical formulas. The feedback comment generation unit 23 inputs text and feedback comments into a second trained model and generates feedback comments based on the output information output from the second trained model by performing probability calculations according to the model parameters of the second trained model.

[0083] For example, the feedback comment generation unit 23 outputs the nth word y from the second trained model.n Replace the ‘’’ with something according to the text to generate the feedback comment Y n ’’. In this case, the feedback comment generation unit 23 uses the second learned model to obtain the second posterior probability P’(y n ’’|X, Y’, Θ g ) of the n-th word of the feedback comment being something according to the text, and determines the n-th word y n ’’ of the feedback comment Y’’ according to the maximum likelihood criterion of the following formula (4). The details of the second posterior probability P’(y n ’’|X, Y’, Θ n ) will be described later. g ) will be described later.

[0084]

Equation

[0085] The feedback comment generation unit 23 recursively performs the above processing to generate the word sequence Y’’ = {y1’’, …, y n ’’} of the feedback comment for the text. The details of the second posterior probability P’(y n ’’|X, Y’, Θ g ) will be described later.

[0086] Hereinafter, an example of the feedback comment generation unit 23 will be described in more detail with reference to FIG. 4. As an example, the feedback comment generation unit 23 includes a feedback comment word embedding vector calculation unit 231, a text and mask template word embedding vector calculation unit 232, a context vector calculation unit 233, a second posterior probability calculation unit 234, and a feedback comment generation unit 235.

[0087] The feedback comment word embedding vector calculation unit 231 calculates a feedback comment word embedding vector using the learned model for calculating the feedback comment word embedding vector generated by the second learned model generation unit 22. For example, the feedback comment word embedding vector calculation unit 231 inputs the word sequence Y’’={y1’’, …, y n-1 ’’} from the first to the (n-1)th word of the feedback comment into the learned model for calculating the feedback comment word embedding vector, and calculates the output information output from the learned model for calculating the feedback comment word embedding vector as the word embedding vector p n of the feedback comment. As an example, the feedback comment word embedding vector calculation unit 231 uses a conversion function based on the optimized model parameters Θ f of the learned model for calculating the feedback comment word embedding vector to calculate the converted word sequence Y’’ from the first to the (n-1)th word of the feedback comment as the word embedding vector p n of the feedback comment. Since the method for calculating the feedback comment word embedding vector is the same as the method for calculating the text word embedding vector, detailed description is omitted.

[0088] The text and mask template word embedding vector calculation unit 232 calculates a text and mask template word embedding vector using the learned model for calculating the text and mask template word embedding vector generated by the second learned model generation unit 22. For example, the text and mask template word embedding vector calculation unit 232 inputs the word sequence X of the text and the word sequence Y’={y1’, …, y k ’} of the mask template into the learned model for calculating the text and mask template word embedding vector, and calculates the output information output from the learned model for calculating the text and mask template word embedding vector as the text and mask template word embedding vector q kIt is calculated as follows. For example, the text and mask template word embedding vector calculation unit 232 calculates the optimized model parameter Θ of the trained model for calculating text and mask template word embedding vectors. q Based on the conversion function, the word sequence X of the text and the word sequence Y' of the mask template are converted to {y1', ..., y k The converted text and mask template word embedding vector q k The calculation is performed as follows. The method for calculating text and mask template word embedding vectors is the same as the method for calculating text word embedding vectors, so a detailed explanation is omitted.

[0089] The context vector calculation unit 233 calculates the context vector using the pre-trained model for context vector calculation generated by the second pre-trained model generation unit 22. For example, the context vector calculation unit 233 calculates the feedback comment word embedding vector p calculated by the feedback comment word embedding vector calculation unit 231. n , and the text and mask template word embedding vector sequence Q = {q1, ..., q} calculated by the text and mask template word embedding vector calculation unit 232. k The} is input to a pre-trained model for context vector calculation, and the output information from the pre-trained model for context vector calculation is used as the context vector s n It is calculated as follows. For example, the context vector calculation unit 233 calculates the optimized model parameter Θ of the pre-trained model for context vector calculation. s Based on the transformation function, the feedback comment word embedding vector p n Furthermore, the text and mask template word embedding vector sequence Q are transformed into context vectors s n It is calculated as follows.

[0090] In this case, any conversion function can be used as long as it converts two variable-length vectors into a single vector. Examples of such conversion functions include a function that expresses the feedback comment word embedding vector and the word embedding vector sequences of the text and mask template as a single context vector by adding an attention mechanism.

[0091] The second posterior probability calculation unit 234 calculates the second posterior probability using the second trained model for calculating posterior probabilities generated by the second trained model generation unit 22. For example, the second posterior probability calculation unit 234 uses the context vector s calculated by the context vector calculation unit 233. n The output information from the second pre-trained model for calculating the posterior probability is used to determine the second posterior probability P'(y) where the nth word of the feedback comment for the text corresponds to the text. n ''|X, Y', Θ g ) is calculated as follows. For example, the second posterior probability calculation unit 234 calculates the optimized model parameter Θ of the second posterior probability calculation trained model. k The transformation function based on this transforms the context vector s n The second posterior probability P'(y) is that the nth word of the feedback comment for the converted text corresponds to the text. n ''|X, Y', Θ g The second posterior probability is calculated as follows. The method for calculating the second posterior probability is the same as the method for calculating the first posterior probability, so a detailed explanation is omitted.

[0092] The feedback comment generation unit 235 generates feedback comments using the trained model for feedback comment generation generated by the second trained model generation unit 22. For example, the feedback comment generation unit 235 calculates the second posterior probability P'(y1''|X, Y', Θ) for the first to nth words of the feedback comment calculated by the second posterior probability calculation unit 234. g ), ..., P'(y n ''|y1', …yn-1 ', X, Y', Θ g The input is then passed to a pre-trained model for generating feedback comments, and the output information from the pre-trained model for generating feedback comments is calculated as the word sequence Y'' of the feedback comments.

[0093] In this case, the feedback comment generation unit 235 calculates the second posterior probability P'(y1''|X, Y', Θ) for the 1st to nth words of the feedback comment. g ), ..., P'(y n ''|y1', …y n-1 ', X, Y', Θ g The one that maximizes ) is selected. As an example, the feedback comment generation unit 235 selects the second posterior probability P'(y) corresponding to the nth word of the feedback comment. n ''|y1', …y n-1 ', X, Y', Θ g Convert the following into the word sequence Y'' of the feedback comment.

[0094] The second posterior probability P'(y n ''|y1', …y n-1 ', X, Y', Θ g As for the method of selecting elements in ), any method that maximizes the second posterior probability can be used, but examples include approximation algorithms such as greedy algorithms. The feedback comment generation unit 235 repeats the above process until all words of the feedback comment are generated. Details of the feedback comment generation process will be described later with reference to Figure 6. In the above example, the feedback comment generation unit 235 generates all words of the feedback comment, but it may also predict only the words in the masked areas, which are masked areas that have been replaced with mask tokens. In this case, the feedback comment generation unit 235 performs a process to generate only the words corresponding to the masked areas and does not need to repeat the above process until all words are generated. In this case as well, the effect of making excessive changes to the template less likely can be obtained, similar to the above example.

[0095] [2. Processing flow by the learning device] The following describes the processing flow of the learning device 1 using Figure 5. Figure 5 is a flowchart showing an example of the processing flow by the learning device 1.

[0096] As shown in Figure 5, the first input receiving unit 11 of the learning device 1 receives input of text, a template for feedback comments on the text, and a mask template in which zero or one or more words of the template are masked (step S1).

[0097] Regarding the mask template, if the number of masked words in the template is one or more (Yes in step S2), the first trained model generation unit 12 of the learning device 1 uses the text, template, and mask template received as input by the first input receiving unit 11 as first training data to generate a first trained model in which the mask template is output when the text and the template are input (step S3). Regarding the mask template, if the number of masked words in the template is not one or more (No in step S2), the learning device 1 terminates processing. Specifically, if the number of masked words in the template is 0, that is, if none of the words in the template are masked, the learning device 1 does not generate a feedback comment through the second generation unit 20, but outputs the template with no masked words as is as the feedback comment.

[0098] The mask template generation unit 13 of the learning device 1 generates a mask template using the first trained model generated by the first trained model generation unit 12 (step S4).

[0099] The second input receiving unit 21 of the learning device 1 receives input of text, a mask template, and feedback comments indicating that the masked portion of the mask template has been changed (step S5).

[0100] The second trained model generation unit 22 of the learning device 1 uses the text, mask template, and feedback comments received as input by the second input receiving unit 21 as second training data to generate a second trained model in which feedback comments are output when text and a mask template are input (step S6).

[0101] The feedback comment generation unit 23 of the learning device 1 generates a feedback comment using the second trained model generated by the second trained model generation unit 22 (step S7).

[0102] In the example described above, the learning device 1 generates the first trained model using the first trained model generation unit 12, and then generates the second trained model using the second trained model generation unit 22. However, the order in which the learning device 1 generates the first trained model and the second trained model is not particularly limited. For example, the learning device 1 may generate the first trained model using the first trained model generation unit 12 and the second trained model using the second trained model generation unit 22 in parallel. Alternatively, the learning device 1 may generate the second trained model using the second trained model generation unit 22, and then generate the first trained model using the first trained model generation unit 12.

[0103] Next, using Figure 6, we will explain in detail an example of the flow of the feedback comment generation process by the learning device 1 in step S7 of Figure 5 described above. Figure 6 is a flowchart showing an example of the flow of the feedback comment generation process by the learning device 1.

[0104] As shown in Figure 6, the feedback comment generation unit 23 of the learning device 1 inputs the word sequence X of the text and the word sequence Y' of the mask template into the second trained model (step S11).

[0105] The feedback comment generation unit 23 generates the first word y1'' of the feedback comment using the second trained model if the word number when counting sequentially from the first word of the feedback comment is 1 or less, i.e., n ≤ 1 (Yes in step S12) (step S13).

[0106] The feedback comment generation unit 23, when n > 1 (No in step S12), uses the second trained model to generate the text word sequence X, the mask template word sequence Y', and the first to n-1 word sequences of the feedback comments Y'' = {y1'', ..., y n-1 The nth word y from the feedback comment ''} n Generate '' (step S14).

[0107] The feedback comment generation unit 23 generates the nth word of the feedback comment and then substitutes n+1 for n (step S15).

[0108] The feedback comment generation unit 23 determines whether the word number n of the feedback comment is equal to the word number N of the last word of the feedback comment. If n=N (Yes in step S16), the feedback comment generation unit 23 terminates the process. If n=N (No in step S16), i.e., if n≠N, the feedback comment generation unit 23 returns to step S12 and repeats the process described above.

[0109] [3. Effects] The learning device 1 according to this embodiment includes a first input receiving unit 11 that accepts input of text, a template for feedback comments on the text, and a mask template in which zero or one or more words of the template are masked, and a first trained model generation unit 12 that uses the text, template, and mask template accepted by the first input receiving unit 11 as first training data to generate a first trained model in which a mask template is output when text and a template are input.

[0110] When text and a template are input, the learning device 1 generates a first trained model that outputs a mask template in which only zero or one or more words that are to be changed from the existing template are masked. Therefore, the learning device 1 can generate a first trained model that outputs a text-specific mask template in which only a portion of the template has been changed. Furthermore, since the learning device 1 generates the first trained model using an existing text-specific template as the first training data, it can prevent the output of a mask template unrelated to the text from the first trained model. As a result, the learning device 1 can generate a mask template that generates feedback text in which only the masked parts have been changed, making it less likely for excessive changes to the template, such as changing parts of the template that correspond to the text, to occur.

[0111] Furthermore, the learning device 1 can generate a first trained model in which a mask template is output that predicts the locations where zero or one or more words are changed as masked locations. Therefore, even if the template is irrelevant and does not correspond to the text at all, the learning device 1 can generate a first trained model in which a mask template is output that is represented only by the masked locations. Thus, if the template corresponds to the text, the learning device 1 can generate a first trained model in which the template is utilized and only the locations that do not correspond to the text are masked. Also, even if the template does not correspond to the text at all, the learning device 1 can flexibly adapt to this case and generate a first trained model in which a mask template is output that is represented only by the masked locations.

[0112] The learning device 1 according to this embodiment further includes a second input receiving unit 21 that receives input of text, a mask template, and feedback comments in which the masked portion of the mask template has been changed, and a second trained model generation unit 22 that uses the text, mask template, and feedback comments received by the second input receiving unit 21 as second training data to generate a second trained model in which feedback comments are output when text and a mask template are input.

[0113] When text and a mask template are input, the learning device 1 generates a second trained model that outputs a feedback comment in which the masked portion of the mask template has been modified. This allows the learning device 1 to generate a second trained model that outputs a text-based feedback comment in which only the masked portion of the mask template has been modified, thus making it less likely for excessive changes to the template to occur.

[0114] In the learning device 1 according to this embodiment, the second trained model generation unit 22 uses the feedback comments output from the second trained model as second training data to train the second trained model. As a result, the learning device 1 can recursively train the second trained model using the feedback comments that reflect the learning results of the second trained model as second training data. Consequently, the learning device 1 can generate a second trained model that outputs feedback comments corresponding to the comments.

[0115] In the learning device 1 according to this embodiment, the second trained model generation unit 22 generates a second trained model using a mask template that includes only the words of the feedback comment, excluding the words in the masked areas, as second training data. As a result, the learning device 1 generates a second trained model using a mask template that is close to the feedback comment corresponding to the comment, excluding the words in the masked areas, as second training data. Consequently, the learning device 1 can generate a second trained model in which the feedback comment is output in which the words in the masked areas of the mask template are replaced with words corresponding to the comment.

[0116] In the learning device 1 according to this embodiment, the first trained model generation unit 12 generates a first trained model using first training data in which at least one of the text and template contains first semantic information indicating the meaning of one or more words and second semantic information indicating the meaning of the entire sentence. The second trained model generation unit 22 generates a second trained model using second training data in which at least one of the text and feedback comments contains first semantic information and at least one of the second semantic information. As a result, the learning device 1 can generate a first trained model that outputs a mask template close to the meaning represented by the input text and template, and a second trained model that outputs a feedback comment close to the meaning represented by the input text.

[0117] In the learning device 1 according to this embodiment, the second trained model generation unit 22 generates a second trained model using second training data that includes contextual information indicating which word is changed in order from the first word of the feedback comment, according to the text and mask template. As a result, the learning device 1 can generate a second trained model that outputs a feedback comment in which the appropriate word in the masked portion of the mask template is changed.

[0118] [4. Program] It is also possible to create a program that describes the processing performed by the learning device 1 according to the above embodiment in a language that can be executed by a computer. In this case, the same effects as in the above embodiment can be obtained by having the computer execute the program. Furthermore, the same processing as in the above embodiment may be realized by recording such a program on a computer-readable recording medium and having the computer read and execute the program recorded on this recording medium.

[0119] Figure 7 shows an example of a computer running a learning program. Computer 1000 includes, for example, memory 1010, CPU 1020, hard disk drive interface 1030, disk drive interface 1040, serial port interface 1050, video adapter 1060, and network interface 1070. These components are connected by bus 1080.

[0120] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM (Random Access Memory) 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, the mouse 1110 and the keyboard 1120. The video adapter 1060 is connected to, for example, the display 1130.

[0121] The hard disk drive 1090 stores, for example, the OS 1091, the application program 1092, the program module 1093, and the program data 1094. That is, the program that defines each process of the learning device 1 is implemented as a program module 1093 in which executable code for the computer is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for performing processes similar to the functional configuration of the learning device 1 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0122] Furthermore, the configuration data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes the processing of the above-described embodiment.

[0123] The program module 1093 and program data 1094 are not limited to being stored on the hard disk drive 1090; for example, they may be stored on a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored on another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read from the other computer by the CPU 1020 via a network interface 1070.

[0124] [5. Addendum] The following additional information is disclosed regarding the embodiments described above.

[0125] (Additional note 1) Memory and At least one processor connected to the memory, Includes, The aforementioned processor, The system accepts input of text, a template for feedback comments on that text, and a mask template in which one or more words in the template are masked. Using the text, template, and mask template received as input as first training data, a first trained model is generated in which the mask template is output when the text and template are input. A learning device characterized by the following features.

[0126] (Additional note 2) A non-temporary storage medium storing a program that can be executed by a computer to perform the learning process using the learning device described in Appendix 1. [Explanation of Symbols]

[0127] 1. Learning device 11. First input reception unit 12 First trained model generation unit 21 Second input reception section 22 Second pre-trained model generation unit

Claims

1. A first input receiving unit that accepts input of text, a template for feedback comments on the text, and a mask template in which zero or one or more words of the template are masked, A first trained model generation unit generates a first trained model in which the mask template is output when the text and template are input, using the text, template and mask template received as input by the first input receiving unit as first training data, A learning device characterized by having the following features.

2. A second input receiving unit that receives input of the aforementioned text, the aforementioned mask template, and feedback comments in which the masked portion of the mask template has been changed, A second trained model generation unit generates a second trained model in which the text, mask template, and feedback comment received as input by the second input receiving unit are used as second training data, and the feedback comment is output when the text and mask template are input. The learning device according to claim 1, further comprising the above.

3. The learning device according to claim 2, characterized in that the second trained model generation unit trains the second trained model using the feedback comments output from the second trained model as second training data.

4. The learning device according to claim 3, characterized in that the second trained model generation unit generates the second trained model using the mask template, which includes only the words of the feedback comments except for the words in the masked areas, as the second training data.

5. The first trained model generation unit generates the first trained model using the first training data, which includes at least one of the first semantic information indicating the meaning of one or more words and the second semantic information indicating the meaning of the entire sentence, in at least one of the text and the template. The learning device according to claim 4, characterized in that the second trained model generation unit generates the second trained model using the second training data, in which at least one of the text and the feedback comments includes at least one of the first semantic information and the second semantic information.

6. The learning device according to any one of claims 2 to 5, characterized in that the second trained model generation unit generates the second trained model using the second training data, which includes contextual information in the feedback comment indicating which word is changed in order from the first word of the feedback comment, according to the text and the mask template.

7. A learning method performed by a learning device, A first input receiving step that accepts input of text, a template for feedback comments on the text, and a mask template in which zero or one or more words of the template are masked, A first trained model generation step generates a first trained model in which the mask template is output when the text and template are input, using the text, template and mask template received in the first input acceptance step as first training data, A learning method characterized by including the following.

8. A first input acceptance step that accepts input of text, a template for feedback comments on the text, and a mask template in which zero or one or more words of the template are masked, A first trained model generation step generates a first trained model in which the mask template is output when the text and template are input, using the text, template and mask template received in the first input acceptance step as first training data, A learning program characterized by having a computer execute it.

Images