Question generation device

Abstract

A question generating device 20 comprises: an analysis unit 22 that analyzes an input text from a user and then extracts, from among multiple predetermined categories relating to the content of the text, one or more first categories that are identified or presumed to be included in the input text; a question text generating unit 23 that, on the basis of the input text and at least one second category remaining after the one or more first categories have been eliminated from among the multiple categories, generates one or more question texts corresponding to the input text and the second categories; and a question text selecting unit 24 that, if multiple question texts have been generated by the question text generating unit 23, selects a question text to be presented to the user from among the multiple question texts, on the basis of the input text and the multiple question texts.

Description

Technical Field

[0001] One aspect of the present invention relates to a question generation device.

Background Art

[0002] A mechanism for automatically generating a response message to a user utterance is known (for example, Patent Document 1). Patent Document 1 discloses the following processing. First, a plurality of candidate question sentences for the user utterance are generated. Then, for each of the plurality of question sentences, the content rate of the feature amounts (each word obtained by morphological analysis) included in the user utterance is calculated, and the question sentence with a high content rate is excluded from the question candidates.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] According to the processing disclosed in Patent Document 1 above, a question sentence with a high content rate (that is, a redundant question regarding the content already mentioned in the user utterance) can be excluded. However, in the above processing, since it is necessary to compare the content rates among all the question sentence candidates after first generating a plurality of question sentence candidates, it is not efficient.

[0005] Therefore, an object of one aspect of the present invention is to provide a question generation device capable of more efficiently generating a question sentence according to a user input sentence.

Means for Solving the Problems

[0006] A question generation device according to one aspect of the present invention comprises: an analysis unit that analyzes a user's input sentence to extract one or more first classifications from a predetermined plurality of classifications relating to the content of the sentence that are identified or estimated to be included in the input sentence; a question sentence generation unit that generates one or more question sentences corresponding to the input sentence and each second classification based on the input sentence and the one or more second classifications remaining after excluding one or more first classifications from the plurality of classifications; and a question sentence selection unit that, when a plurality of question sentences are generated by the question sentence generation unit, selects a question sentence to present to the user from among the plurality of question sentences based on the input sentence and the plurality of question sentences.

[0007] According to one aspect of the present invention, a question generation device generates only question sentences corresponding to the second category, other than the first category which is identified or presumed to be included in the input sentence, from among a predetermined set of classifications. In other words, it is possible to omit the process of generating redundant questions related to content that is highly likely to be included in the input sentence (first category), and to generate only meaningful questions (i.e., questions related to the second category which is unlikely to be included in the input sentence). Furthermore, since the generation of questions related to the first category is avoided and the number of candidate question sentences is narrowed in advance, the processing load on the question sentence selection unit is reduced. Therefore, question sentences corresponding to user input can be generated more efficiently. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to provide a question generation device that can more efficiently generate question sentences in response to user input. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows the overall configuration of the information processing system according to the embodiment. [Figure 2] This figure shows an example of a question generated by a question generation model. [Figure 3] This flowchart shows an example of the processing procedure for a model generation device. [Figure 4] This figure shows a first example of the processing performed by the analysis unit. [Figure 5] This figure shows a second example of the processing performed by the analysis unit. [Figure 6] This figure shows an example of the processing in the question generation unit. [Figure 7] This flowchart shows an example of the processing procedure for a question generation device. [Figure 8] This figure shows the first example of the question generation process. [Figure 9] This figure shows a second example of the question generation process. [Figure 10] This figure shows an example of the hardware configuration of a model generation device and a question generation device. [Modes for carrying out the invention]

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings. In the description of the drawings, the same or equivalent elements will be denoted by the same reference numerals, and redundant descriptions will be omitted.

[0011] Figure 1 shows the overall configuration of an information processing system 1 according to one embodiment. The information processing system 1 includes a model generation device 10, a question generation device 20, and one or more user terminals 30. The model generation device 10 is a device that generates a question generation model M. The question generation model M is a trained model created by machine learning to generate a question sentence in response to an arbitrary sentence as input. The question generation device 20 is a device that generates a question sentence in response to a user's input sentence and presents the generated question sentence to the user. In this embodiment, the question generation device 20 generates a question sentence in response to a user's input sentence by utilizing the question generation model M generated by the model generation device 10. The question generation device 20 is configured to communicate with the user terminal 30. The question generation device 20 is configured to receive an input sentence from the user terminal 30 via user operation through the user terminal 30 and to transmit the generated question sentence to the user terminal 30. The user terminal 30 is any terminal possessed and operated by the user. Specific examples of user terminals 30 include smartphones, tablet devices, desktop PCs, laptop PCs, and wearable devices. The model generation device 10 and the question generation device 20 may each be configured by a single server device or by multiple server devices. Furthermore, the model generation device 10 and the question generation device 20 may be configured by the same server device.

[0012] In this embodiment, as an example, the information processing system 1 is a system that supports the user's language learning (in this embodiment, English learning). Specifically, the user's input sentence is in English, and the question sentences generated by the question generation device 20 are also in English. For example, the question generation device 20 acquires any English sentence input by the user, presents the user with a question sentence corresponding to the content of the English sentence, and supports the user's English learning by prompting the user to answer in English. Here, from the viewpoint of improving the user's learning efficiency and motivation, the generation and presentation of redundant (meaningless) question sentences should be eliminated as much as possible. For example, consider the case where the user's input sentence is "I went to ski last winter." In this case, the input sentence contains information about the time (when) (here, "last winter"). Therefore, in this case, a question sentence such as "When did you go to ski?" which asks about the time included in the input sentence, is a redundant question sentence and should not be presented to the user. The question generation device 20 is configured to avoid generating and presenting such redundant question sentences, and instead generates and presents question sentences with appropriate content corresponding to the input sentence.

[0013] [Configuration of the model generation system] The model generation device 10 includes a pair information acquisition unit 11, a training data generation unit 12, a model generation unit 13, and a model storage unit 14.

[0014] The pair information acquisition unit 11 acquires pair information including a question sentence and an answer sentence for the question sentence. Regarding the pair of a question sentence and its corresponding answer sentence, for example, in commercially available English question collections and the like, there are a very large number of samples (example sentences). The pair information acquisition unit 11 may acquire such samples as pair information, for example. However, the information source of the pair information is not limited to the above example. For example, the pair information acquisition unit 11 may acquire pair information from any information source capable of extracting pairs of questions and answers, such as dialogue logs accumulated in an arbitrary dialogue system that interacts with a user, answer results (Q&A) of web questionnaires, and the like. Examples of pair information include a pair of the question sentence "What animal do you like?" and the answer sentence "I don't really like animals." for the question sentence.

[0015] The teacher data generation unit 12 generates teacher data in which the answer sentence included in the pair information acquired by the pair information acquisition unit 11 is associated with the input data and the question sentence included in the pair information is associated with the output data. In other words, the teacher data generation unit 12 generates teacher data with the temporal order of the question sentence and the answer sentence included in the pair information reversed. More specifically, the teacher data generation unit 12 generates teacher data having a relationship of predicting the question sentence (output data) that occurred earlier in time from the answer sentence (input data) that occurred later in time.

[0016] In this embodiment, the training data generation unit 12 associates the answer sentences included in the pair information with the input data, and generates training data that further associates classification information indicating the classification of the question sentences included in the pair information with the input data. Here, the classification information is a category related to the question content (subject of the question). In this embodiment, as an example, the classification information is information indicating the classification corresponding to English question words. For example, the classification information is information indicating the six classifications known as the 5W1H. That is, the classification information is information indicating the six classifications of "when," "where," "who," "what," "why," and "how."

[0017] For example, the training data generation unit 12 extracts interrogative words contained in the question sentence and uses the extracted interrogative words as classification information. For example, in the pair information example described above, the training data generation unit 12 extracts the interrogative word "what" contained in the question sentence "What animal do you like?" and uses the interrogative word "what" as classification information. In this case, training data is generated in which the interrogative word "what" and the answer sentence "I don't really like animals." correspond to the input data, and the question sentence "What animal do you like?" corresponds to the output data. For example, the training data is represented in the following data format. In this example, the training data consists of a token specifying the classification of interrogative words (5W1H), an answer sentence, and a token indicating the separator between the answer sentence and the question sentence. <sep>, question, token indicating the end of a sentence <eos>It is composed of ". In training data represented in this data format, tokens <sep>The part before that corresponds to the input data, token <sep>The part after that corresponds to the output data. Also, tokens <eos>Because it is possible to recognize the boundaries between training data, if there are multiple training data, training can be performed using a single text data that concatenates multiple training data represented in the following format. (Example of training data) <what>I don’t really like animals. <sep>What animal do you like? <eos>

[0018] In addition, when the question sentence does not include an interrogative word (5W1H) (for example, when the question sentence included in the pair information is a closed question such as "Do you~"), the teacher data generation unit 12 may set information indicating the absence of a specific interrogative word (others) (for example, information indicating null, etc.) in the classification information. By performing learning by the model generation unit 13 described later using such teacher data (that is, teacher data including classification information of "others"), even when the classification information input to the question generation model M is null (that is, when there is no specification of an interrogative word), it is possible to generate a question generation model M capable of generating a question sentence (for example, a question sentence corresponding to a closed question).

[0019] The model generation unit 13 generates a question generation model M that outputs a question sentence (second sentence) for an arbitrary first sentence by performing machine learning using the teacher data generated by the teacher data generation unit 12. As described above, in the present embodiment, the teacher data includes classification information as data corresponding to the input data. Therefore, the question generation model M is configured to input an arbitrary first sentence and arbitrary classification information and output a question sentence (second sentence) corresponding to the classification indicated by the arbitrary classification information (in the present embodiment, any interrogative word of 5W1H or a classification indicating "others").

[0020] FIG. 2 shows examples of question sentences corresponding to each classification information (interrogative word) obtained from a certain input sentence "I studied English yesterday." by the question generation model M created by the present inventor. As described above, by machine learning using the above-described teacher data, by inputting a certain input sentence (first sentence) and arbitrary classification information, it is possible to take into account the content of the input sentence and generate and output a question sentence (second sentence) that questions matters regarding the input classification information, thereby obtaining a question generation model M.

[0021] The model generation unit 13 may generate a question generation model M by performing additional training on a pre-prepared language model. For example, the model generation unit 13 may generate a question generation model M by using a pre-trained large-scale language model such as BERT (Bidirectional Encoder Representations from Transformers) or GPT-2 (Generative Pre-trained Transformer) as a base and performing additional training (e.g., fine-tuning, transfer learning, etc.) using the aforementioned training data. By generating a question generation model M based on such a language model, it becomes possible to obtain a question generation model M capable of generating more natural sentences (second sentences).

[0022] The question generation model M generated by the model generation unit 13 is stored in the model storage unit 14. The question generation model M stored in the model storage unit 14 is used by the question generation device 20 (more specifically, the question sentence generation unit 23, which will be described later).

[0023] Referring to the flowchart shown in Figure 3, an example of the processing procedure of the model generation device 10 (model generation method according to one embodiment) will be described. First, the pair information acquisition unit 11 acquires pair information (pair information acquisition step, step S1). The pair information acquisition unit 11 acquires a sufficient number of pair information to train the question generation model M. Next, the training data generation unit 12 generates training data based on the pair information acquired by the pair information acquisition unit 11 (training data generation step, step S2). Specifically, the pair information acquisition unit 11 generates training data that associates the answer sentences included in the pair information with the input data and the question sentences included in the pair information with the output data. In this embodiment, the training data generation unit 12 further associates classification information indicating the classification of the question sentences included in the pair information (for example, question words included in the question sentences) with the input data. Next, the model generation unit 13 generates a question generation model M by performing machine learning using the training data generated by the training data generation unit 12 (model generation step, step S3). The generated question generation model M is stored in the model storage unit 14.

[0024] [Configuration of the Question Generation Device] The question generation device 20 includes a reception unit 21, an analysis unit 22, a question text generation unit 23, a question text selection unit 24, a question text correction unit 25, and a presentation unit 26.

[0025] The reception unit 21 obtains (receives) the user's input text from the user terminal 30.

[0026] The analysis unit 22 analyzes the user's input sentence obtained by the reception unit 21 and extracts one or more first classifications from a predetermined set of classifications relating to the content of the sentence that are identified or presumed to be included in the input sentence. In this embodiment, for example, the multiple classifications are the same as the classification information described above. That is, the multiple classifications are classifications corresponding to English question words (e.g., 5W1H). The following describes a specific example of the processing that the analysis unit 22 performs to extract the first classifications.

[0027] As a first example, if the input sentence contains a named entity corresponding to a single interrogative word (in this embodiment, any of the six classifications of 5W1H), the parsing unit 22 extracts the classification corresponding to that interrogative word as the first classification. Figure 4 shows a first example of the processing of the parsing unit 22. Figure 4 shows an example where the input sentence is "I went to Kyoto yesterday." First, the parsing unit 22 extracts all named entities contained in the input sentence. Known methods (e.g., Bi-LSTM-CRF) can be used to extract named entities.

[0028] Named entities are defined into several predetermined classes. For example, classes such as personal names, place names, organization names, time, date (including period), amount, geographical features, and facility names are defined. Each class is pre-associated with the aforementioned classification of interrogative words. For example, "personal names" are associated with the interrogative word "who." "Place names," "geographical features," and "facility names" are associated with the interrogative word "where." "Time" and "dates" are associated with the interrogative word "when." Such associations can be arbitrarily set in advance by operators or other means.

[0029] The named entities extracted from the input sentence by the parsing unit 22 belong to one of the above classes. That is, the parsing unit 22 extracts named entities contained in the input sentence and obtains information about the class to which those named entities belong. In the example in Figure 4, the parsing unit 22 extracts from the input sentence the named entity "yesterday," which belongs to the class "date" corresponding to the interrogative word "when," and the named entity "Kyoto," which belongs to the class "place name," corresponding to the interrogative word "where." Therefore, in the example in Figure 4, the parsing unit 22 extracts the interrogative words "when" and "where" as the first classification. In other words, the parsing unit 22 analyzes the input sentence to contain content related to "when" and "where."

[0030] As a second example, the analysis unit 22 may extract the first classification by utilizing a trained model (classification model). Figure 5 shows a second example of the processing performed by the analysis unit 22. Examples of classification models include a single classification model M1 that performs multi-class classification and a classification model M2 that performs binary classification prepared for each question word.

[0031] As shown on the left side of Figure 5, the classification model M1, which performs multi-class classification, is a pre-trained model configured to take an input sentence as input and output the degree of fit (the probability that the input sentence contains content related to each category) between the input sentence and each category (in this case, the six categories of 5W1H and "other," for a total of seven categories). In the example in Figure 5, the input sentence "I went to ski last winter to Nagano with Hanako." was input to the classification model M1, and the probability that the input sentence contains each interrogative word (the probability that it contains content corresponding to each interrogative word) was output. In this example, the input sentence contains the named entity "Nagano," which corresponds to the class "Place Names" corresponding to the interrogative word "where," the named entity "winter," which corresponds to the class "Date" corresponding to the interrogative word "when," and the named entity "Hanako," which corresponds to the class "Person Names" corresponding to the interrogative word "who." Therefore, in the output of the classification model, the probabilities of the interrogative words "where," "when," and "who" are higher than the probabilities of the other interrogative words. For example, the analysis unit 22 may extract classifications (in the example in Figure 5, the interrogative words "where," "when," and "who") corresponding to probability values ​​above a predetermined threshold (here, "0.2" as an example) as the first classification.

[0032] As shown on the right side of Figure 5, a classification model M2 for binary classification is prepared for each classification of interrogative words. For example, the classification model M2 corresponding to the interrogative word "when" is configured to take an input sentence as input and output the probability that the input sentence contains "when" (i.e., the probability that the input sentence contains content corresponding to "when"). Similarly, the classification model M2 corresponding to the interrogative word "how" is configured to take an input sentence as input and output the probability that the input sentence contains "how" (i.e., the probability that the input sentence contains content corresponding to "how"). The same applies to the classification models M2 corresponding to other interrogative words. For example, the analysis unit 22 may refer to the probability values ​​output by the classification model M2 corresponding to each interrogative word and extract the classification corresponding to a probability value above a predetermined threshold as the first classification.

[0033] The classification models M1 and M2 described above can be constructed, for example, by a single-layer or multi-layer neural network. Classification models M1 and M2 are generated, for example, by performing machine learning (e.g., deep learning) using training data consisting of a sentence and information on the question words contained in that sentence (i.e., the correct labels).

[0034] The question generation unit 23 obtains one or more second categories by excluding one or more first categories extracted by the analysis unit 22 from a predetermined set of categories (in this embodiment, six categories of 5W1H). Then, based on the input sentence and the one or more second categories, the question generation unit 23 generates one or more question sentences corresponding to the input sentence and each of the second categories.

[0035] In the example shown in Figure 5, the analysis unit 22 extracts classifications corresponding to the interrogative words "where," "when," and "who" as the first classification. In this case, the question generation unit 23 obtains the remaining three classifications corresponding to the interrogative words "what," "why," and "how" as the second classification. In this case, as shown in Figure 6, the analysis unit 22 generates question sentences corresponding to each pair of input sentence and second classification (in this example, "input sentence + what," "input sentence + why," and "input sentence + how"). As an example, the question generation unit 23 generates question sentences using the question generation model M (stored in the model storage unit 14) generated by the model generation device 10. More specifically, the question generation unit 23 obtains question sentences output from the question generation model M by inputting pairs of input sentence and second classification into the question generation model M. For example, the question generation unit 23 obtains question sentences in a format similar to the part of the training data that corresponds to the input data (for example, " <what>The following sentence is input to the question generation model M: "I went to ski last winter to Nagano with Hanako." As a result, the question generation model M generates and outputs a question sentence corresponding to the specified interrogative word (in this example, "what") and the input sentence.

[0036] In the example in Figure 6, the question generation unit 23 generates the question "What did you do last winter?" as the question corresponding to the "input sentence + what" pair, "Why did you go skiing last winter?" as the question corresponding to the "input sentence + why" pair, and "How did you get there?" as the question corresponding to the "input sentence + how" pair.

[0037] When multiple question sentences are generated by the question sentence generation unit 23, the question sentence selection unit 24 selects a question sentence to present to the user from among the multiple question sentences based on the input sentence and the multiple question sentences. For example, if only one question sentence is to be presented to the user, the question sentence selection unit 24 determines which question sentence to present to the user from among the multiple question sentences generated by the question sentence generation unit 23. If it is possible to present a predetermined number of questions (N questions (where N is an integer greater than or equal to 2)) to the user, the question sentence selection unit 24 may select N question sentences if the number of question sentences generated by the question sentence generation unit 23 is greater than N. Details of the processing of the question sentence selection unit 24 will be described later.

[0038] The question text correction unit 25 corrects the question text generated by the question text generation unit 23 (for example, one question text selected by the question text selection unit 24) into a more appropriate question text that takes into account the content of the input text. However, if there are no parts that need correction, the question text correction unit 25 does not need to correct the question text. Also, the timing of the question text correction unit 25 correcting the question text may be before the processing of the question text selection unit 24. In this case, the question text correction unit 25 only needs to execute the correction process (including the process of determining whether correction is necessary) for each question text generated by the question text generation unit 23.

[0039] For example, the question text correction unit 25 compares the question text generated by the question text generation unit 23 with the input text, and if the first word in the question text and the second word in the input text are found to have the same named entity classification, it replaces the first word in the question text with the second word. Details of the processing of the question text correction unit 25 will be described later.

[0040] The presentation unit 26 presents the question text generated by the question text generation unit 23 (or, if multiple question texts are generated, the question text selected by the question text selection unit 24) to the user. In this embodiment, the presentation unit 26 transmits the question text to the user terminal 30 and presents the question text to the user by displaying it on a display unit such as a display on the user terminal 30.

[0041] An example of the processing procedure of the question generation device 20 will be explained with reference to the flowchart shown in Figure 7.

[0042] First, the reception unit 21 obtains the user's input sentence from the user terminal 30 (step S11). Next, the analysis unit 22 analyzes the user's input sentence obtained by the reception unit 21 and extracts a first classification that is identified or estimated to be included in the input sentence (step S12). For example, as shown in Figure 4, if the analysis unit 22 extracts a named entity belonging to a named entity class corresponding to one of a predetermined number of classifications (in this embodiment, classifications corresponding to each of the 5W1H question words) from the input sentence using the named entity extraction method described above, it may identify the classification (question word) corresponding to that named entity class as the first classification. Alternatively, the analysis unit 22 may use classification models M1 and M2, as shown in Figure 5, to estimate the classification (question word) for which a probability value above a predetermined threshold is output as the first classification. In the example in Figure 4, the analysis unit 22 extracts two classifications corresponding to the question words "when" and "where" as the first classification.

[0043] Next, the question generation unit 23 obtains one or more second categories from among multiple classifications (six classifications of 5W1H) by excluding one or more first categories extracted by the analysis unit 22 (step S13). In the example in Figure 4, this process obtains four classifications corresponding to the question words "who," "what," "why," and "how" as second categories.

[0044] Next, the question generation unit 23 generates a question for each pair of input sentences and second classifications (step S14). As an example, the question generation unit 23 generates question sentences using a question generation model M. Specifically, as shown in Figure 6, the question generation unit 23 inputs pairs of input sentences and second classifications into the question generation model M, and generates question sentences corresponding to each pair from the output results of the question generation model M.

[0045] Next, if the question text selection unit 24 generates multiple question texts by the question text generation unit 23 (i.e., if there are multiple second categories) (step S15: YES), it selects a question text to present to the user from among the multiple question texts (step S16). If only one question text is generated by the question text generation unit 23 (step S15: NO), the processing of the question text selection unit 24 is omitted.

[0046] Next, the question text correction unit 25 corrects the question text generated by the question text generation unit 23 (in this embodiment, one question text selected by the question text selection unit 24) to a more appropriate question text that takes into account the content of the input text (step S18) if the question text generated by the question text generation unit 23 (in this embodiment, one question text selected by the question text selection unit 24) satisfies predetermined correction conditions (step S17: YES). If the predetermined correction conditions are not met (step S17: NO), the processing of the question text correction unit 25 is omitted. Here, the predetermined correction conditions are, for example, that the first and second words described above are extracted from the question text and the input text. That is, if the first and second words described above are extracted from the question text and the input text, in step S18, the first word included in the question text is replaced with the second word included in the input text.

[0047] Next, the presentation unit 26 presents the question generated by the question generation unit 23 (or, if multiple question sentences are generated, the question selected by the question selection unit 24) to the user (step S19).

[0048] (First embodiment) Referring to Figure 8, a first embodiment of the process shown in the flowchart of Figure 7 described above will be explained. Figure 8 is an example of a screen displaying the processing content and processing results of the question generation device 20 using a command-line interface (CUI).

[0049] In the first embodiment, the reception unit 21 obtains the input sentence "I went to ski last winter to Nagano with Hanako." (Step S11). Subsequently, the analysis unit 22 extracts the proper noun "winter" belonging to the class "date" corresponding to the interrogative word "when", the proper noun "Nagano" belonging to the class "place name" corresponding to the interrogative word "where", and the proper noun "Hanako" belonging to the class "person name" corresponding to the interrogative word "who" (Step S12). In other words, "when", "where", and "who" are extracted as the first classification. As a result, the analysis unit 22 extracts the remaining interrogative words "what", "how", and "why" as the second classification (Step S13).

[0050] Next, the question generation unit 23 generates question sentences corresponding to each of the second categories (step S14). In this example, three question sentences are generated: "What did you do last winter?" corresponding to the interrogative word "what", "How did you get there?" corresponding to the interrogative word "how", and "Why did you go skiing last winter?" corresponding to the interrogative word "why".

[0051] Next, the question selection unit 24 selects one question from the three question sentences above to be presented to the user (step S16). For example, the question selection unit 24 may calculate the similarity between each question sentence generated by the question generation unit 23 and the input sentence, and select the question sentence with a lower similarity to the input sentence as the question sentence to be presented to the user with greater priority.

[0052] As an example, the question selection unit 24 creates the input sentences and 2-grams for each question as shown below. In the following, 2-gram(sentence) indicates the 2-gram of the sentence in question.

[0053] 2-gram(input sentence)={'I went','went to','to ski','ski last','last winter','winter to','to Nagano','Nagano with','with Hanako','Hanko .'}

[0054] 2-gram (question sentences using "what") = {'What did', 'did you', 'you do', 'do last', 'last winter', 'winter?'}

[0055] 2-gram (question sentences using "how") = {'How did', 'did you', 'you get', 'get there', 'there?'}

[0056] 2-gram (question sentences using "why") = {'Why did', 'did you', 'you go', 'go skiing', 'skiing last', 'last winter', 'winter?'}

[0057] The question selection unit 24 creates the intersection of the 2-gram of the input sentence and the 2-gram of each question. In this example, the intersection of the 2-gram of the input sentence and the 2-gram of the question "what" is {'last winter'}, the intersection of the 2-gram of the input sentence and the 2-gram of the question "how" is {'last winter'}, and the intersection of the 2-gram of the input sentence and the 2-gram of the question "why" is the empty set {}.

[0058] Here, the more elements a question has in the intersection of the above sets (i.e., the higher the textual match rate with the input text), the higher the degree of similarity (similarity) to the input text. Therefore, the question selection unit 24 may set a higher priority for question sentences with fewer elements in the intersection of the sets (i.e., question sentences with lower similarity). In the above example, the priority of the "why" question sentence (intersection of "0" elements) is set higher than the priority of the "what" and "how" questions sentences (intersection of "1" elements). Through this process, the question selection unit 24 selects the "why" question sentence "Why did you go skiing last winter?" as the question sentence to be presented to the user.

[0059] Next, it is determined whether or not to correct the question text using the question text correction unit 25. However, in the first embodiment, the above-mentioned correction conditions are not met (step S17: NO), so the question text correction process (step S18) is not executed. As a result, the presentation unit 26 presents the user with the "why" question text selected by the question text selection unit 24 (step S19).

[0060] (Second example) Referring to Figure 9, a second embodiment of the process shown in the flowchart of Figure 7 described above will be explained. Figure 9 is an example of a screen displaying the processing content and processing results of the question generation device 20 using a command-line interface (CUI).

[0061] In the second embodiment, the reception unit 21 obtains the input sentence "I like to go to Kyoto." (Step S11). Subsequently, the analysis unit 22 extracts the proper noun "Kyoto," which belongs to the class "Place Name" corresponding to the interrogative word "where" (Step S12). That is, "where" is extracted as the first classification. Here, in the second embodiment, four classifications are predetermined as multiple classifications (candidates for interrogative words): "where," "when," "how," and "why." Therefore, the analysis unit 22 extracts the remaining interrogative words "when," "how," and "why" as the second classification (Step S13).

[0062] Next, the question generation unit 23 generates question sentences corresponding to each of the second categories (step S14). In this example, three question sentences are generated: "When do you like to go to Kyoto?" corresponding to the interrogative word "when", "How would you like to travel, by train or by bus?" corresponding to the interrogative word "how", and "Why are you interested in Japan?" corresponding to the interrogative word "why".

[0063] Next, the question selection unit 24 selects one question from the three question sentences above to present to the user (step S16). For example, the question selection unit 24 may select a question sentence based on 2-grams, similar to the first embodiment described above. In the second embodiment, the intersection of the 2-gram of the input sentence and the 2-gram of the question "when" is {'like to','to go','to Kyoto','go to'}, the intersection of the 2-gram of the input sentence and the 2-gram of the question "how" is {'like to'}, and the intersection of the 2-gram of the input sentence and the 2-gram of the question "why" is the empty set {}.

[0064] Therefore, in the above example, the priority of the "why" question (intersection of 0 elements) is set higher than the priority of the "when" question (intersection of 4 elements) and the "how" question (intersection of 1 element). Through this process, the question selection unit 24 selects the "why" question "Why are you interested in Japan?" as the question to be presented to the user.

[0065] Next, it is determined whether or not to correct the question sentence by the question sentence correction unit 25. In the second embodiment, the input sentence contains the named entity "Kyoto" which belongs to the class "Place Name". Also, the why question sentence contains the named entity "Japan" which belongs to the class "Place Name". In other words, in this example, the first word "Japan" in the question sentence and the second word "Kyoto" in the input sentence, which belong to the same named entity classification, are extracted. Therefore, since the above correction condition is met (step S17: YES), the question sentence correction process (step S18) is executed. That is, the question sentence correction unit 25 replaces the first word "Japan" in the question sentence with the second word "Kyoto". In other words, the question sentence correction unit 25 corrects the question sentence "Why are you interested in Japan?" to "Why are you interested in Kyoto?". Next, the presentation unit 26 presents the corrected question sentence to the user (step S19).

[0066] The model generation device 10 described above comprises the pair information acquisition unit 11, the training data generation unit 12, and the model generation unit 13. With such a model generation device 10, a question generation model M can be obtained that takes an arbitrary first sentence as input and generates a question sentence (second sentence) for that first sentence. In order to generate such a question generation model M, it is natural to use a pair of an arbitrary sentence corresponding to the first sentence and a question sentence for that arbitrary sentence as training data. More specifically, since the first sentence input to the question generation model M exists in time before the second sentence output from the question generation model M, in order to generate the question generation model M, it is natural to use training data in which an arbitrary sentence corresponding to the first sentence is associated with the input data, and a question sentence that occurred in time after that arbitrary sentence (i.e., a question sentence for that arbitrary sentence) is associated with the output data. In contrast, the model generation device 10 uses training data with the temporal order reversed (i.e., training data in which the question sentence that occurred in time earlier corresponds to the output data, and the answer sentence that occurred in time later corresponds to the input data). As shown in the example in Figure 2, according to the inventor's findings, even when training data is used in which the temporal relationship between the question and answer sentences is reversed, a question generation model M can be obtained that outputs a second sentence, which is a question sentence in response to the input first sentence. In other words, a question generation model M can be obtained that can generate a second sentence, which is a question sentence related to the content of the first sentence. Furthermore, as mentioned above, pair information including a question sentence and an answer sentence to that question sentence can be obtained relatively easily from example sentences written in commercially available English workbooks, etc. Therefore, the model generation device 10 can efficiently generate a model that generates question sentences in response to an input sentence.

[0067] In this embodiment, the training data generation unit 12 generates training data that associates answer sentences with input data and further associates classification information indicating the classification of the question sentence (for example, the classification of question words such as 5W1H) with the input data. The model generation unit 13 then generates a question generation model M that takes an input sentence (first sentence) and arbitrary classification information as input and outputs a question sentence (second sentence) corresponding to the classification indicated by the arbitrary classification information. With the above configuration, it is possible to obtain a question generation model M that takes into account the content of the input sentence and can generate a question sentence related to the content indicated by the classification information (see Figure 2).

[0068] In this embodiment, the training data generation unit 12 extracts interrogative words (in this embodiment, interrogative words corresponding to the 5W1H) contained in the question sentence, and uses the extracted interrogative words as classification information. With the above configuration, classification information can be easily and appropriately extracted based on the interrogative words contained in the question sentence of the paired information.

[0069] The question generation device 20 described above comprises an analysis unit 22, a question sentence generation unit 23, and a question sentence selection unit 24. With such a question generation device 20, only question sentences corresponding to the second category, other than the first category which is identified or estimated to be included in the input sentence, are generated from among a predetermined set of classifications. In other words, the process of generating redundant questions related to content that is highly likely to be included in the input sentence (first category) is omitted, and only meaningful questions (i.e., questions related to the second category which is unlikely to be included in the input sentence) can be generated. Furthermore, by avoiding the generation of questions related to the first category in this way and narrowing down the number of candidate question sentences in advance, the processing load of the question sentence selection unit 24 is reduced. Therefore, question sentences corresponding to user input can be generated more efficiently.

[0070] In this embodiment, the analysis unit 22 extracts the classification corresponding to a single interrogative word as the first classification when the input sentence contains a named entity corresponding to that interrogative word. With the above configuration, the extraction of the first classification can be easily and efficiently performed by using a known named entity recognition method.

[0071] In this embodiment, the question selection unit 24 calculates the similarity between each question generated by the question generation unit 23 and the input sentence, and selects question sentences with lower similarity to the input sentence as the question sentences to be presented to the user with greater priority. With the above configuration, it becomes possible to present the user with question sentences that are not superficially or semantically similar to the input sentence (i.e., question sentences with less redundancy).

[0072] In this embodiment, as an example, the number of elements in the intersection of the 2-gram of the input sentence and the 2-gram of each question was used as the indicator of similarity. However, the question selection unit 24 may select the question sentences to present to the user based on similarity criteria other than those described above. For example, the question selection unit 24 may vectorize the input sentence and each question sentence, calculate the cosine similarity between the vectorized input sentence and each vectorized question sentence, and select the question sentence with the smallest cosine similarity to present to the user. Alternatively, the question selection unit 24 may use n-grams (where n is an integer of 3 or more) or 1-grams (bag-of-word, word matching degree) instead of 2-grams.

[0073] Furthermore, the question selection unit 24 may calculate the probability that the input sentence contains content related to each of the second categories (precision rate for each second category), and may prioritize selecting question sentences corresponding to the second category with a lower probability as questions to present to the user. For example, the question selection unit 24 may input the input sentence into the classification model described above (see Figure 5) and calculate the probability that the input sentence contains each category. Then, the question selection unit 24 may extract the second category with the lowest probability from among the multiple second categories, and select the question sentence corresponding to the extracted second category as a question to present to the user. With the above configuration, it is possible to appropriately select question sentences that concern content that is unlikely to be mentioned in the input sentence (in this embodiment, content from the 5W1H that is not included in the input sentence) as questions to present to the user.

[0074] In this embodiment, the question generation device 20 includes a question sentence correction unit 25. With the above configuration, as shown in the example in Figure 9, a more natural question sentence corresponding to the input sentence can be generated by replacing a part of the description contained in the question sentence (e.g., "Japan" in the example in Figure 9) with a named entity belonging to the same class described in the input sentence (e.g., "Kyoto" in the example in Figure 9). In the example in Figure 9, a more natural question sentence corresponding to the input sentence is obtained by converting "Japan" contained in the question sentence to the more specific expression "Kyoto" contained in the input sentence.

[0075] The correction conditions for the question text correction unit 25 are not limited to the conditions described in the above embodiment (that the first and second words belonging to the same named entity class are extracted from the question text and the input text). For example, consider the case where the input text contains one named entity belonging to the class "place name" and the question text contains two named entities belonging to the class "place name". In this case, it may not be possible to determine which of the two named entities in the question text corresponds to the named entity belonging to the class "place name" in the input text. Therefore, the question text correction unit 25 may replace the first word with the second word only if there is one named entity (first word and second word) belonging to a common named entity class in both the question text and the input text.

[0076] As mentioned above, in training the question generation model M, by also using training data based on pair information of question sentences (for example, closed questions such as "Do you~" without question words) and answer sentences when the classification information is empty, the question generation model M can also generate question sentences that correspond to closed questions. For example, if the second classification is not extracted in the processing by the analysis unit 22, the question sentence generation unit 23 may input the input sentence and classification information indicating null to the question generation model M to generate a question sentence that corresponds to a closed question.

[0077] Furthermore, although the above embodiment describes processing targeting English text, texts written in languages ​​other than English may also be processed. For example, the input text to the question generation model M and the question text output from the question generation model M may be texts written in languages ​​other than English. For example, by performing machine learning using texts written in languages ​​other than English as training data, a question generation model M that supports languages ​​other than English can be obtained.

[0078] The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining the above one device or the above multiple devices with software.

[0079] Functions include, but are not limited to, judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, deem, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning.

[0080] For example, the model generation device 10 and the question generation device 20 in one embodiment of the present disclosure may function as a computer that performs the model generation method and the question generation method of the present disclosure. Figure 10 is a diagram showing an example of a hardware configuration common to the model generation device 10 and the question generation device 20 according to one embodiment of the present disclosure. Each of the model generation device 10 and the question generation device 20 may be physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0081] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the model generation device 10 and the question generation device 20 may include one or more of the devices shown in Figure 10, or it may be configured to omit some of the devices.

[0082] Each function in the model generation device 10 and the question generation device 20 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which causes the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of the reading and writing of data in the memory 1002 and storage 1003.

[0083] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control units, arithmetic units, registers, etc.

[0084] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, the model generation unit 13 of the model generation device 10 and the question statement generation unit 23 of the question generation device 20 may be implemented by a control program stored in the memory 1002 and running on the processor 1001, and other functional blocks may be implemented similarly. The above-described various processes have been explained as being executed by one processor 1001, but they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0085] Memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 1002 may also be called a register, cache, main memory, etc. Memory 1002 can store executable programs (program code), software modules, etc., for implementing a communication control method according to one embodiment of the present disclosure.

[0086] Storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. Storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003.

[0087] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also called a network device, network controller, network card, communication module, etc.

[0088] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0089] Furthermore, each device, such as the processor 1001 and memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.

[0090] Furthermore, the model generation device 10 and the question generation device 20 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.

[0091] Although this embodiment has been described in detail above, it will be clear to those skilled in the art that this embodiment is not limited to the embodiments described herein. This embodiment can be implemented as a modified and altered form without departing from the spirit and scope of the invention as defined by the claims. Therefore, the description herein is for illustrative purposes only and is not intended to be restrictive in any way to this embodiment.

[0092] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described herein may be reordered, provided they are consistent with each other. For example, the methods described herein present various step elements in an exemplary order and are not limited to that specific order.

[0093] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be transmitted to other devices.

[0094] The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0095] Each aspect / embodiment described herein may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0096] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.

[0097] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technology (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technology (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0098] The information, signals, etc. described in this disclosure may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0099] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0100] The names used for the parameters described above are not restrictive in any way. Furthermore, the formulas and other expressions using these parameters may differ from those expressly disclosed in this disclosure. Since various information elements can be identified by any suitable name, the various names assigned to these various information elements are not restrictive in any way.

[0101] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0102] Any reference to elements using the designations “first,” “second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the first and second elements do not imply that only two elements may be employed, or that the first element must precede the second element in any way.

[0103] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0104] In this disclosure, if articles are added through translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0105] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different." [Explanation of Symbols]

[0106] 10...Model generation device, 11...Pair information acquisition unit, 12...Training data generation unit, 13...Model generation unit, 20...Question generation device, 21...Reception unit, 22...Analysis unit, 23...Question text generation unit, 24...Question text selection unit, 25...Question text correction unit, 26...Presentation unit, M...Question generation model.< / what> < / eos> < / sep> < / what> < / eos> < / sep> < / sep> < / eos> < / sep>

Claims

1. An analysis unit analyzes the user's input text and extracts one or more first categories from a predetermined set of categories relating to the content of the text that are identified or estimated to be included in the input text. A question generation unit generates one or more question sentences corresponding to each of the above-mentioned second classifications, based on the input sentence and the one or more second classifications remaining after excluding one or more first classifications from the above-mentioned plurality of classifications, When the question generation unit generates multiple question sentences, the question selection unit selects a question sentence to present to the user from among the multiple question sentences based on the input sentence and the multiple question sentences, A question generation device equipped with the following features.

2. The question generation device according to claim 1, wherein the analysis unit extracts a classification corresponding to a single interrogative word as the first classification when the input sentence contains a named entity corresponding to a single interrogative word.

3. The question generation apparatus according to claim 1 or 2, wherein the question selection unit calculates the similarity between each of the question sentences generated by the question generation unit and the input sentence, and selects the question sentences with lower similarity to the input sentence as question sentences to be presented to the user with greater priority.

4. The question generation device according to claim 1 or 2, wherein the question selection unit calculates the probability that the input sentence contains content relating to each of the second categories, and selects the question sentences corresponding to the second categories with a lower probability as questions to be presented to the user with greater priority.

5. The question generation device according to any one of claims 1 to 4, further comprising a question correction unit that compares the question generated by the question generation unit with the input sentence and replaces the first word with the second word when a first word in the question and a second word in the input sentence that are classified as identical named entities are extracted.

Images