Information processing device, information processing method, and information processing program

Abstract

This information processing device is provided with: a storage unit (22) that stores a first learning model (220) that outputs the meaning of a word, and a second learning model (230) that outputs answer information to a question; and a processor (21). The processor (21) performs: a first identification process for identifying the meaning of a word in a question sentence using the first learning model (220); an analysis process for analyzing the meaning of the question sentence on the basis of the identified meaning of the word; and an acquisition process for entering the analysis result of the question sentence into the second learning model (230) to acquire first answer information. The analysis process further includes: outputting inquiry information for inquiring about the meaning of a word that could not be identified; identifying, on the basis of second answer information to the inquiry information, the meaning of the word that could not be identified; and analyzing the meaning of the question sentence on the basis of the first identification process and the meaning of the word that is identified on the basis of the second answer information.

Description

Information processing device, information processing method, and information processing program

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program.

[0002] In recent years, artificial intelligence (also referred to as "AI") has been used to generate content such as text and images in response to prompts. For example, in a technology for generating a sentence from keywords contained in an input sentence, when the input sentence contains an unknown word that is not in a learned dictionary, a technology has been proposed in which the unknown word in the input sentence is replaced with a predicted word to generate a sentence, and the predicted word in the generated sentence is replaced with the unknown word (see, for example, Patent Document 1).

[0003] Japanese Patent Application Publication No. 2019-185400

[0004] When generating answer information to a question from a user, if the question contains an unknown word that is not in the learned dictionary, the accuracy of the answer information to the question may be reduced. For example, even if an unknown word contained in an input sentence is replaced with a predicted word, the predicted word may have a different meaning from the user's intended meaning. In such a case, the accuracy of the generated answer information may also be reduced.

[0005] One aspect of the technology of the present disclosure aims to provide an information processing device, an information processing method, and an information processing program that can improve the accuracy of answer information to input information such as a question sentence, etc. In particular, the present disclosure aims to provide an information processing device, an information processing method, and an information processing program that can output relevant answer information as output to input information based on the meaning and content of the input information.

[0006] One aspect of the technology of the present disclosure is exemplified by the following information processing device. The information processing device of the present disclosure includes a storage unit that stores a first learning model that, when a word is input, outputs text data indicating the meaning of the word, and a second learning model that, when a question is input, outputs answer information to the question, and a processor connected to the storage unit. The processor executes a first identification process that identifies the meaning of each word included in a question input by a user using the first learning model, an analysis process that analyzes the meaning of the question based on the meanings of the words identified in the first identification process, an input process that inputs the analysis result of the question by the analysis process into the second learning model, an acquisition process that acquires first answer information to the question from the second learning model, and an output process that outputs the first answer information. The analysis process further includes outputting inquiry information inquiring about the meaning of the word in at least one of the cases where the meaning cannot be identified in the first identification process, or where the meaning of the word can be identified but the first answer information cannot be output by the output process based on that meaning of the word, accepting second answer information in response to the inquiry information, and, when the meaning of the word that could not be identified by the first identification process is identified based on the second answer information, analyzing the meaning of the question sentence based on the meaning of the word identified in the first identification process and the meaning of the word identified based on the second answer information.

[0007] The information processing device of the present disclosure outputs inquiry information when the question contains a word whose meaning cannot be determined. The information processing device then identifies the meaning of the word whose meaning cannot be determined based on the second answer information to the inquiry information, and analyzes the meaning of the question. By exchanging such inquiry information and second answer information, the information processing device can analyze the meaning of the question from the user with higher accuracy. Consequently, the information processing device can improve the accuracy of answer information for questions containing unknown words.

[0008] The information processing device of the present disclosure may further include the following feature: The processor further executes a learning process to train the first learning model based on the words identified based on the second answer information and the meanings of the words identified based on the second answer information. By providing such a feature, the information processing device can expand the words that can be identified by the first learning model. Consequently, the information processing device can reduce the workload of having the user respond to the inquiry information.

[0009] The information processing device of the present disclosure may further include the following feature: The processor further executes a storage process for storing the second answer information in the storage unit in association with the question sentence and the word whose meaning could not be identified. By including such a feature, the information processing device of the present disclosure can identify the meaning of the word whose meaning could not be identified by the first learning model by referring to the storage unit. Consequently, the information processing device of the present disclosure can reduce the workload of having the user respond to the inquiry information.

[0010] The information processing device of the present disclosure may further include the following feature: Identifying based on the second answer information further includes notifying that an answer is not possible when the meaning of the word whose meaning could not be identified cannot be identified based on the second answer information. By including such a feature, the information processing device of the present disclosure can suppress output of an answer that does not conform to the intent of the question sentence.

[0011] The information processing device of the present disclosure may further include the following feature: The output process further includes a process of including location information indicating a location of a document related to the first answer information in the first answer information. By including such a feature, the information processing device of the present disclosure can present the user with reference materials as answer information to the question sentence.

[0012] The information processing device of the present disclosure may further include the following feature: The processor executes the input process of inputting a first analysis result for a first question sentence into the second learning model, and the acquisition process of acquiring, from the second learning model, the first answer information for a second question sentence whose analysis result has already been input into the second learning model. By including such a feature, the information processing device of the present disclosure can shorten the time it takes to generate final answer information for the question sentence from the user.

[0013] The technology of the present disclosure can also be understood from the aspects of an information processing method and an information processing program.

[0014] According to the technology of the present disclosure, it is possible to improve the accuracy of answer information to a question sentence that includes unknown words.

[0015] FIG. 1 is a diagram illustrating an example of an answer system according to an embodiment. FIG. 2 is a diagram illustrating an example of the hardware configuration of an answer device according to an embodiment. FIG. 3 is a diagram illustrating an example of the hardware configuration of a user terminal according to an embodiment. FIG. 4 is a diagram illustrating an example of a processing block of an answer device according to an embodiment. FIG. 5 is a diagram illustrating an example of an answer model. FIG. 6 is a diagram illustrating an example of a model feature table stored in a management database according to an embodiment. FIG. 7 is a first diagram illustrating an example of a processing flow of an answer device according to an embodiment. FIG. 8 is a second diagram illustrating an example of a processing flow of an answer device according to an embodiment. FIG. 9 is a third diagram illustrating an example of a processing flow of an answer device according to an embodiment. FIG. 10 is a diagram illustrating an example of a processing flow of a sentence analysis process by a first analysis unit of an answer device according to an embodiment. FIG. 11 is a diagram illustrating an example of a processing flow of a query sentence generation process by a confirmation unit of an answer device according to an embodiment. FIG. 12 is a first diagram illustrating an example of a processing flow of a label assignment process by a first analysis unit and a second analysis unit of an answer device according to an embodiment. FIG. 13 is a second diagram illustrating an example of a processing flow of a label assignment process by a first analysis unit and a second analysis unit of an answer device according to an embodiment. FIG. 14 is a first diagram illustrating an example of a processing flow of an answer model selection process by a second analysis unit and a selection unit of an answer device. FIG. 15 is a second diagram showing an example of a processing flow of an answer model selection process by a second analysis unit and a selection unit of the answer device. FIG. 16 is a third diagram showing an example of a processing flow of an answer model selection process by a second analysis unit and a selection unit of the answer device. FIG. 17 is a first diagram showing an example of a user screen displayed on an output unit of a user terminal according to the embodiment. FIG. 18 is a second diagram showing an example of a user screen displayed on an output unit of a user terminal according to the embodiment. FIG. 19 is a third diagram showing an example of a user screen displayed on an output unit of a user terminal according to the embodiment. FIG. 20 is a fourth diagram showing an example of a user screen displayed on an output unit of a user terminal according to the embodiment.

[0016] <Embodiments> Hereinafter, embodiments will be described with reference to the drawings. Fig. 1 is a diagram showing an example of a response system 1 according to an embodiment. The response system 1 includes a response device 2, a user terminal 3, and a network N1. The response device 2 and the user terminal 3 are connected to each other via the network N1 so as to be able to communicate with each other.

[0017] The answering device 2 is an information processing device that, upon receiving a question input to the user terminal 3 by the user U1, transmits an answer to the question to the user terminal 3. The question is an example of input information, and is, for example, a sentence indicating the content of the answer that the user U1 requests from the answering device 2. The question includes one or more words and is transmitted from the user terminal 3 to the answering device 2. Question information including data other than the question and the sentence may be transmitted from the user terminal 3 to the answering device 2. Examples of data other than the sentence included in the answer information include audio data, video data, image data, etc. The user terminal 3 is an information processing device that transmits the question input by the user U1 to the answering device 2 and outputs the answer received from the answering device 2 to a display device such as a display.

[0018] The network N1 connects information processing devices to each other so that they can communicate with each other. The network N1 may be, for example, a local area network (LAN), a mobile communication system, or the Internet. The network N1 may be wired or wireless.

[0019] 2 is a diagram illustrating an example of a hardware configuration of the answering device 2 according to the embodiment. The answering device 2 includes a CPU 21, a main memory unit 22, an auxiliary memory unit 23, a communication unit 24, and a connection bus B2. The CPU 21, the main memory unit 22, the auxiliary memory unit 23, and the communication unit 24 are interconnected by the connection bus B2.

[0020] The CPU 21 is also referred to as a microprocessor unit (MPU) or processor. The CPU 21 is not limited to a single processor and may have a multiprocessor configuration. Furthermore, a single CPU 21 connected via a single socket may have a multi-core configuration. At least a portion of the processing performed by the CPU 21 may be performed by a processor other than the CPU 21, such as a dedicated processor such as a digital signal processor (DSP), a graphics processing unit (GPU), a numerical calculation processor, a vector processor, an image processing processor, or a neural processing unit (NPU). Furthermore, at least a portion of the processing performed by the CPU 21 may be performed by an integrated circuit (IC) or other digital circuit. Furthermore, at least a portion of the CPU 21 may include an analog circuit. The integrated circuit includes a large-scale integrated circuit (LSI), an application-specific integrated circuit (ASIC), and a programmable logic device (PLD). The PLD includes, for example, a field-programmable gate array (FPGA). The CPU 21 may be a combination of a processor and an integrated circuit. This combination is called, for example, a microcontroller unit (MCU), a system-on-a-chip (SoC), a system LSI, or a chipset. In the answering device 2, the CPU 21 expands a program stored in the auxiliary memory unit 23 into a working area in the main memory unit 22 and controls peripheral devices through the execution of the program. This allows the answering device 2 to execute processing consistent with a predetermined purpose. The main memory unit 22 and the auxiliary memory unit 23 are recording media that can be read by the CPU 21 .

[0021] The main storage unit 22 is exemplified as a storage unit that is directly accessed by the CPU 21. The main storage unit 22 includes a random access memory (RAM) and a read only memory (ROM).

[0022] The auxiliary storage unit 23 stores various programs and various data on a readable and writable recording medium. The auxiliary storage unit 23 is also called an external storage device. The auxiliary storage unit 23 stores an operating system (OS), various programs, various tables, etc. The OS includes a communication interface program that exchanges data with external devices connected via the communication unit 24. The external devices include, for example, other information processing devices and external storage devices connected via a computer network, etc. The auxiliary storage unit 23 may be, for example, part of a cloud system, which is a group of computers on a network.

[0023] The auxiliary storage unit 23 is, for example, an erasable programmable ROM (EPROM), a solid state drive (SSD), a hard disk drive (HDD), etc. The auxiliary storage unit 23 is, for example, a compact disc (CD) drive, a digital versatile disc (DVD) drive, a Blu-ray (registered trademark) disc (BD) drive, etc. The auxiliary storage unit 23 may be provided by a network attached storage (NAS) or a storage area network (SAN).

[0024] The communication unit 24 is, for example, an interface with the network N1, and communicates with external devices via the network N1.

[0025] 3 is a diagram illustrating an example of the hardware configuration of a user terminal 3 according to an embodiment. The user terminal 3 includes a CPU 31, a main memory unit 32, an auxiliary memory unit 33, a communication unit 34, an input unit 35, an output unit 36, and a connection bus B3. The CPU 31, the main memory unit 32, the auxiliary memory unit 33, the communication unit 34, and the connection bus B3 have the same configuration as the CPU 21, the main memory unit 22, the auxiliary memory unit 23, the communication unit 24, and the connection bus B2 of the response device 2, and therefore a description thereof will be omitted.

[0026] The input unit 35 accepts input from the user U1. The input unit 35 is, for example, a keyboard, a mouse, a trackball, a touch panel, a voice input device, etc. The output unit 36 ​​outputs data processed by the CPU 31 and data stored in the main memory unit 32. The output unit 36 ​​can be, for example, a display, a printer, or a speaker.

[0027] 4 is a diagram showing an example of a processing block of the answering device 2 according to the embodiment. The answering device 2 includes a login processing unit 200, a first analysis unit 201, a confirmation unit 202, an unknown word specification unit 203, a second analysis unit 204, a selection unit 205, an analysis result input unit 206, an answer acquisition unit 207, an output unit 208, a billing unit 209, a management database 210 (denoted as "management DB210" in the drawing), a word model 220, and an answer model 230. The response device 2 executes processing as each part of the response device 2, such as the login processing unit 200, first analysis unit 201, confirmation unit 202, unknown word identification unit 203, second analysis unit 204, selection unit 205, analysis result input unit 206, answer acquisition unit 207, output unit 208, billing unit 209, management database 210, word model 220 and answer model 230, by having the CPU 21 execute a computer program that is executable and deployed in the main memory unit 22.

[0028] The word model 220 outputs text data indicating the meaning of an input word. The word model 220 is, for example, a learning model constructed by machine learning using words and text data indicating their meanings as training data. The word model 220 may be, for example, a table that associates words with text data indicating their meanings. A word model 220 is prepared for each language, such as Japanese, English, and German. The word model 220 is an example of a "first learning model."

[0029] The answer model 230 generates an answer to an input question. The answer is a sentence indicating an answer to the question. The answer model 230 may generate answer information for an input question, including an answer and data other than the sentence. Examples of data other than the sentence included in the answer information include audio data, video data, and image data. The answer model 230 generates an answer using, for example, a large-scale language model (LLM). The answer generated by the answer model 230 may include information indicating the location of materials related to the answer to the question. The information indicating the location of the materials is, for example, a uniform resource identifier (URI). Furthermore, if the conditions included in the input question are insufficient to output an answer, the answer model 230 may output a message to that effect. The answer model 230 includes, for example, multiple learning models corresponding to the information field of the question. The answer model 230 is, for example, a learning model constructed based on training data. The answer model 230 is an example of a "second learning model." The answer sentence generated by the answer model 230 is an example of "answer information." Information indicating the location of materials is an example of "location information."

[0030] FIG. 5 is a diagram showing an example of the answer model 230. The answer model 230 includes, for example, a first maintenance model 231, a second maintenance model 232, a third maintenance model 233, a first travel model 234, and a second travel model 235. The first maintenance model 231, the second maintenance model 232, and the third maintenance model 233 are types of learning models constructed using information related to vehicle maintenance as training data. The first travel model 234 and the second travel model 235 are types of learning models constructed using information related to travel as training data. In other words, the answer model 230 is composed of multiple types of learning models, and the types include multiple learning models that differ for each information field of the input information (input sentence, question sentence). The learning models in this case may also be referred to as a first type learning model or a second type learning model depending on their type. In the following, a first maintenance model 231, a second maintenance model 232, and a third maintenance model 233 are shown as examples of a first type learning model, and a first travel model 234 and a second travel model 235 are shown as examples of a second type learning model. Examples of other learning models will be described later.

[0031] The first maintenance model 231 is constructed using information related to automobile maintenance as training data. Examples of information related to automobile maintenance include maintenance manuals and official websites published by automobile manufacturers, information related to maintenance work published by automobile repair shops, information related to maintenance published by automobile mechanics, blogs and social networking services (SNS) related to maintenance examples published by automobile mechanics, information published in automobile information magazines, automobile customization examples posted on Internet bulletin boards, etc.

[0032] The second maintenance model 232 is constructed using highly reliable information related to automobile maintenance as training data. Examples of highly reliable information include maintenance manuals and official websites published by automobile manufacturers, information related to maintenance work published by automobile repair shops, and information related to maintenance published by automobile mechanics.

[0033] The third maintenance model 233 is constructed using information related to automobile maintenance as training data. The third maintenance model 233 is constructed with, for example, less training data than the first maintenance model 231, aiming to construct a lightweight model.

[0034] Because the training data used to build the second maintenance model 232 is limited to highly reliable information related to automobile maintenance, the second maintenance model 232 tends to generate answers in a narrower range of fields than the first maintenance model 231. On the other hand, the second maintenance model 232, which is built based on such highly reliable information, tends to generate more reliable answers to input questions related to automobile maintenance than the first maintenance model 231. In other words, although the answers generated by the first maintenance model 231 are less reliable than those generated by the second maintenance model 232, the first maintenance model 231 tends to be able to generate answers to a wider range of questions related to maintenance. Furthermore, because the third maintenance model 233 is a lightweight model, it can generate answers more quickly than the first maintenance model 231.

[0035] The first travel model 234 is constructed using travel-related information as training data, such as tourism information published by prefectural governments, overseas safety information published by the Ministry of Foreign Affairs, tourism information published by ministries and agencies responsible for attracting tourists to each country, websites where travel agencies publish information about tourist destinations, blogs and social media where individuals publish travel records, etc.

[0036] The second travel model 235 is constructed using highly reliable travel-related information as training data, such as tourism information published by prefectural governments, overseas safety information published by the Ministry of Foreign Affairs, and tourism information published by ministries and agencies responsible for attracting tourists to each country.

[0037] The training data used to build the second travel model 235 is limited to highly reliable information related to travel, and therefore the field in which answers can be generated tends to be narrower than that of the first travel model 234. On the other hand, the second travel model 235, which is built based on such highly reliable information, tends to generate more reliable answers to input travel-related questions than the first travel model 234. In other words, although the answers generated by the first travel model 234 are less reliable than those generated by the second travel model 235, they tend to be able to generate answers to a wider range of travel-related questions.

[0038] 5 illustrates the first maintenance model 231 and the second maintenance model 232 related to automobile maintenance, and the first travel model 234 and the second travel model 235 related to travel as examples of learning models included in the answer model 230, but the answer model 230 may further include learning models other than these. Furthermore, the answer model 230 may include other learning models instead of the first maintenance model 231, the second maintenance model 232, the first travel model 234, and the second travel model 235. The answer model 230 may include, for example, a learning model that outputs text data indicating information related to health maintenance, a learning model that outputs text data indicating information related to movies, a learning model that outputs text data indicating information related to music, etc.

[0039] Although details will be described later, instead of using a single answer model for a single piece of input information such as an input sentence or a question, the input information of the input sentence or question may be separated into multiple input contents (in a broad sense, input information) by separators (for example, punctuation marks, line breaks, etc.), and an answer model 230 may be used for each separated input content.

[0040] Returning to FIG. 4 , the management database 210 is a database that manages the features of each model included in the answer model 230. FIG. 6 is a diagram illustrating an example of a model feature table 211 stored in the management database 210 in the embodiment. The model feature table 211 includes the following fields: "Model," "Label," "Reliability," and "Processing Speed." "Model" stores information indicating each model included in the answer model 230. "Label" stores information indicating the field of the training data used to build the model. In other words, "Label" stores the field suitable for the answer provided by the model. Note that "Label" may store labels indicating multiple fields. "Reliability" stores information indicating the reliability of the answer generated by the model. "Processing Speed" stores information indicating the speed from when a question is input until an answer is generated. Note that the information stored in the model feature table 211 is not limited to the information exemplified in FIG. 6 . The model feature table 211 may also include, for example, information indicating the amount of training data used to build the model, the usage fee for the model, etc. The information stored in "label," "reliability," and "processing speed" is an example of "feature information indicating features related to answer accuracy." The "label" in the model feature table 211 is an example of "first field information."

[0041] 4, the login processing unit 200 performs a login process for the user U1. The login processing unit 200 performs the login process using, for example, a user name and a password.

[0042] When the first analysis unit 201 receives an input sentence exemplified by a question sentence from user U1 and an answer sentence from the answer model 230, it generates a thread for the input sentence. The input sentence is an example of input information and includes one or more words. The input sentence is, for example, a sentence input to the first analysis unit 201 and analyzed by the first analysis unit 201. Examples of the input sentence include a question sentence input by user U1, an inquiry sentence output by the confirmation unit 202 described below, and an answer sentence from user U1 to the inquiry sentence. Note that input information including data other than the input sentence and the sentence may also be input to the first analysis unit 201. Examples of data other than the sentence included in the input information include audio data, video data, image data, etc. The thread includes a series of question sentences and answer sentences. The series of question sentences and answer sentences include, for example, a question sentence input by user U1, an inquiry sentence generated by the confirmation unit 202 described below in response to the question sentence, an answer sentence input by user U1 to the inquiry sentence, and an answer sentence acquired by the answer acquisition unit 207 described below. The thread may include an additional question from user U1 in response to the answer acquired by the answer acquisition unit 207. For example, the first analysis unit 201 associates the generated thread with the username of user U1 who has been logged in by the login processing unit 200, and stores the thread in the auxiliary storage unit 23.

[0043] The first analysis unit 201 also identifies the language of the input sentence. For example, the first analysis unit 201 identifies whether the input sentence is in Japanese, English, German, etc. The first analysis unit 201 may include a learning model that, when a sentence is input, outputs text data indicating the language used in the sentence, and may identify the language using the learning model.

[0044] The first analysis unit 201 then identifies the meaning of each word included in the input sentence. The first analysis unit 201 performs morphological analysis on the question sentence to break down the input sentence into morphemes (e.g., words). The first analysis unit 201 may further identify the part of speech of each morpheme through morphological analysis. The first analysis unit 201 inputs each word included in the input sentence into a word model 220 to identify its meaning. Note that the first analysis unit 201 may select a word model 220 to use for identifying the meaning of a word depending on the language of the identified input sentence. For example, if the language of the identified input sentence is Japanese, the first analysis unit 201 identifies the meaning of the word using the Japanese word model 220. The first analysis unit 201 identifies as an unknown word a word whose meaning cannot be identified even when input into the word model 220. The first analysis unit 201 may also identify as an unknown word a word that cannot be successfully clustered by the second analysis unit 204 (described later). The processing by the first analysis unit 201 is an example of a "first identification processing."

[0045] The confirmation unit 202 generates a query sentence inquiring of the user U1 about the meaning of the unknown word identified by the first analysis unit 201. That is, the confirmation unit 202 generates a query sentence inquiring about the meaning of a word that could not be identified by the first analysis unit 201. The confirmation unit 202 may store a fixed phrase such as "What does this word '...' mean?" in advance in the auxiliary storage unit 23 and generate the query sentence by replacing "..." with an unknown word. Furthermore, when an answer sentence indicating missing information is output from the answer model 230, the confirmation unit 202 may use the answer sentence as the query sentence. The confirmation unit 202 outputs the query sentence to the user terminal 3. The processing by the confirmation unit 202 is an example of processing to "output query information." The query sentence is an example of "inquiry information."

[0046] In addition to generating the above query sentences, the confirmation unit 202 can also generate a query sentence to resolve unclear elements (words, context, etc.) when generating an answer sentence, even if the meaning of a word can be identified by the first identification process but the meaning of the word does not allow or makes it difficult to output an answer sentence (first answer information) by the output process described below. The query sentence generated by these generation processes may be at least any of the above query sentences.

[0047] When the unknown word identification unit 203 receives a response sentence input by the user U1 in response to the query sentence output by the confirmation unit 202, it analyzes the received response sentence. For example, the unknown word identification unit 203 causes the first analysis unit 201 to perform a morphological analysis of the received response sentence and identify the meanings of words included in the response sentence. The unknown word identification unit 203 acquires the meaning of the response sentence using the result of the word meaning identification by the first analysis unit 201. The unknown word identification unit 203 identifies the meaning of the unknown word based on the meaning of the acquired response sentence. The response sentence input by the user U1 in response to the query sentence is an example of "second response information."

[0048] The unknown word identification unit 203 may perform additional learning processing on the word model 220 using the correspondence between the meaning of the identified unknown word and the unknown word as training data. Furthermore, the unknown word identification unit 203 may associate the unknown word, the meaning of the identified unknown word, and the input sentence including the unknown word, and store the associations in the auxiliary storage unit 23. Note that here, if the meaning of the unknown word cannot be identified even from the meaning of the answer sentence acquired by the unknown word identification unit 203, the confirmation unit 202 may be caused to output a further inquiry sentence. Furthermore, if the meaning of the unknown word cannot be identified even from the meaning of the answer sentence to the inquiry sentence, the unknown word identification unit 203 may instruct the output unit 208 to output a message indicating that an answer is not possible to the user terminal 3.

[0049] The second analysis unit 204 performs an overview of the input sentence based on morphological analysis, assigns (also referred to as setting) labels to the input sentence, and performs clustering. For example, the second analysis unit 204 obtains an overview of what the input sentence is about based on the results of the morphological analysis by the first analysis unit 201. For example, the second analysis unit 204 can determine whether the input sentence is a question about travel or car maintenance. Furthermore, by performing clustering, it is possible to obtain an overall picture of the content of the input sentence according to, for example, Grice's postulates.

[0050] The second analysis unit 204 analyzes the meaning of the input sentence, for example, using the meanings of the words identified by the first analysis unit 201 and the meanings of the unknown words identified by the unknown word identification unit 203. In the analysis by the second analysis unit 204, for example, labels representing the meanings of these morphemes are assigned to each of the morphemes (words and sentences) resolved by the morphological analysis by the first analysis unit 201. For example, the meanings of the words identified by the word model 220 may be used as the labels.

[0051] The second analysis unit 204 clusters each morpheme included in the input sentence so that morphemes with the same label belong to the same cluster. The second analysis unit 204 acquires the meaning of the entire question sentence based on the clustering results. Examples of the meaning of the entire input sentence analyzed by the second analysis unit 204 include the field to which the input sentence belongs, whether the input sentence has a positive or negative content, whether the question is a question requesting reliable information, or whether the question is a question requesting less reliable but still useful information. The second analysis unit 204 assigns a label to the question sentence based on the acquired meaning of the question sentence. The second analysis unit 204 may also acquire the length of the input sentence. The length of the text may be, for example, the number of characters included in the input sentence. The length of the text may also be the number of words included in the input sentence. Furthermore, clustering by the second analysis unit 204 may not be performed correctly. An example of a case in which clustering is not performed correctly is when there is a cluster into which a small number of morphemes (e.g., one) are classified. The label assigned to the question sentence is an example of "second field information."

[0052] The selection unit 205 refers to the model feature table 211 based on the analysis results by the second analysis unit 204 and selects one or more answer models 230 to be used in answering the question. For example, when the selection unit 205 analyzes that the label assigned to the question by the second analysis unit 204 is "automobile maintenance," the selection unit 205 may select as candidates for the learning model to be used a first maintenance model 231, a second maintenance model 232, and a third maintenance model 233, each of which includes "automobile maintenance" in the "label" field in the model feature table 211. The answer model 230 selected by the selection unit 205 is an example of a "selected learning model."

[0053] Furthermore, when the second analysis unit 204 analyzes that the reliability of the answer sought by the question is high, the selection unit 205 may select the second maintenance model 232, which has the highest reliability, from the first maintenance model 231, the second maintenance model 232, and the third maintenance model 233. Furthermore, when the second analysis unit 204 analyzes that a large amount of useful information is sought, even if the information is low in reliability, the selection unit 205 may select all of the first maintenance model 231, the second maintenance model 232, and the third maintenance model 233, which have labels including "automobile maintenance," as learning models to use in answering the question. The process of selecting one or more answer models 230 by the selection unit 205 is an example of a "selection process."

[0054] Here, when selecting the answer model 230 to be used to answer the question sentence, the selection unit 205 may calculate the relevance between the label assigned to the question sentence by the second analysis unit 204 and the "label" in the model feature table 211, and prioritize the selection of the answer model 230 with the highest calculated relevance. For example, the selection unit 205 may select the answer model 230 with the highest calculated relevance as the answer model 230 to be used to answer the question sentence. Furthermore, the selection unit 205 may select one or more answer models 230 to be used to answer the question sentence in descending order of calculated relevance. There is no limitation on the method for calculating the relevance. For example, the selection unit 205 may calculate the similarity between the label of the question sentence assigned by the second analysis unit 204 and the word (or sentence) indicating the label stored in the "label" in the model feature table 211, and use the calculated similarity as the relevance. Various known techniques can be applied to the method of calculating the similarity between words (or sentences). The relevance is an example of "domain relevance."

[0055] The analysis result input unit 206 inputs the analysis result by the second analysis unit 204 to the answer model 230 selected by the selection unit 205. When the analysis result input unit 206 receives question information from the user terminal 3, it may also input data other than the sentences included in the question information to the answer model 230. For example, the analysis result input unit 206 may process the analysis result by the second analysis unit 204 to make it easier to obtain an answer from the answer model 230, and input the processed analysis result to the answer model 230. In the processing by the analysis result input unit 206, for example, the analysis result is processed so that instructions for the answer model 230 are placed at the beginning and supplementary information is placed thereafter. When multiple answer models 230 are selected by the selection unit 205, the analysis result input unit 206 inputs the analysis result by the second analysis unit 204 to each of the selected multiple answer models 230. In addition, when the analysis result input unit 206 inputs the analysis results by the second analysis unit 204 to multiple answer models 230, the analysis result input unit 206 may input the analysis results by the second analysis unit 204 in order, for example, from the answer model 230 with the highest relevance calculated by the selection unit 205 to the answer model 230 with the lowest relevance.

[0056] The answer acquisition unit 207 acquires the answer sentence generated by the answer model 230. When multiple answer models 230 are selected by the selection unit 205, the answer acquisition unit 207 acquires answer sentences from the selected multiple answer models 230. The answer acquisition unit 207 may process the answer sentence from the acquired answer model 230. When answer sentences are acquired from multiple answer models 230, examples of processing by the answer acquisition unit 207 include synthesis, summarization, etc. of these multiple answer sentences.

[0057] Examples of the synthesis of answer sentences by the answer acquisition unit 207 include a method of directly concatenating answer sentences from multiple answer models 230, and a method of synthesizing answer sentences by prioritizing answer sentences from highly related answer models 230. Examples of the method of prioritizing answer sentences from highly related answer models 230 include a method of placing answer sentences from highly related answer models 230 at the beginning of answer sentences to be output by the output unit 208, and a method of setting answer sentences from highly related answer models 230 to be larger in size than answer sentences from less related answer models 230. Furthermore, when the answer acquisition unit 207 acquires answer sentences from multiple answer models 230, it may summarize these multiple answer sentences. Upon completing processing of the answer sentences, the answer acquisition unit 207 instructs the output unit 208 to output the processed answer sentences. In addition, the response acquisition unit 207 adds the inquiry statement output by the confirmation unit 202, the response statement from user U1 to the inquiry statement, and the processed response statement to the thread associated with the username of user U1.

[0058] Furthermore, the answer acquiring unit 207 may assign a label to each of the answer sentences from the multiple answer models 230 by performing analysis using the first analysis unit 201 and the second analysis unit 204. Then, for each of the answer sentences, the answer acquiring unit 207 may calculate the relevance (answer relevance) between the label assigned to the question sentence and the label assigned to the answer sentence, and arrange and combine the answer sentences so that answer sentences with higher calculated answer relevance are prioritized. Furthermore, the answer acquiring unit 207 may suppress output by the output unit 208 of answer sentences whose answer relevance is less than a predetermined threshold.

[0059] When the answer acquisition unit 207 acquires an answer sentence, the selection unit 205 evaluates the answer model 230 that generated the answer sentence. The answer acquisition unit 207 assigns a label to the answer sentence generated using the second analysis unit 204. The answer acquisition unit 207 compares the label assigned to the answer sentence with the label assigned to the question sentence to calculate an evaluation value for the answer model 230. For example, the answer acquisition unit 207 calculates a higher evaluation value the more the labels assigned to the answer sentence and the labels assigned to the question sentence match. The number of matching labels is also referred to as the degree of match. The selection unit 205 may update the "reliability" field in the model feature table 211 based on the calculated evaluation value. Furthermore, if the evaluation value of the answer model 230 that generated the answer sentence is low, the answer acquisition unit 207 may select another answer model 230 and input the analysis result to the analysis result input unit 206.

[0060] The output unit 208 transmits the answer sentence acquired by the answer acquisition unit 207 to the user terminal 3. When answer information is acquired by the answer acquisition unit 207, the output unit 208 may transmit the acquired answer information to the user terminal 3. When the answer sentence acquired by the answer acquisition unit 207 includes information indicating the location of materials related to the answer sentence, the output unit 208 may also output information indicating the location of the materials. The billing unit 209 calculates a usage fee for the answer system 1 based on the amount of text in the question sentence analyzed by the analysis result input unit 206, and charges the user U1.

[0061] 7 to 9 are diagrams showing an example of a processing flow of the response device 2 according to the embodiment. Hereinafter, an example of the processing flow of the response device 2 will be described with reference to FIGS. 7 to 9.

[0062] 7, a login process is performed in step S1. The login processing unit 200 executes the login process for the user U1 based on, for example, the user name and password transmitted from the user terminal 3.

[0063] In step S2, the first analysis unit 201 determines whether or not an input sentence has been received, either a question sentence from the user terminal 3 or an answer sentence from the answer model 230. If an input sentence has been received (YES in step S2), the process proceeds to step S3. If an input sentence has not been received (NO in step S2), the process of step S2 is repeated.

[0064] In step S3, a history registration and inquiry process is performed. In this example, in step S3, the first analysis unit 201 generates a thread including the input sentence received in step S2. The first analysis unit 201 associates the generated thread with the user name that performed the login process in step S1, and stores the association in the auxiliary storage unit 23. In step S4, a sentence analysis process is performed. In this example, in step S4, the first analysis unit 201 identifies the language of the input sentence received in step S2.

[0065] In step S5, the first analysis unit 201 performs morphological analysis of the input sentence received in step S2 based on the language identified in step S4. The first analysis unit 201 further inputs each word included in the input sentence received in step S2 into the word model 220 selected based on the language identified in step S4, to identify the meaning of the word.

[0066] In step S6, a labeling process is performed. In this example, in step S6, the second analysis unit 204 assigns a label representing the meaning of each morpheme resolved by the morphological analysis in step S5. The second analysis unit 204 clusters the morphemes included in the input sentence so that morphemes with the same label belong to the same cluster.

[0067] In step S7, the confirmation unit 202 generates a query statement if there is an inquiry item. For example, if there is a word whose meaning could not be identified in step S5, the confirmation unit 202 generates a query statement about the meaning of the word. Furthermore, if there is a word whose clustering could not be performed successfully in step S6, the confirmation unit 202 generates a query statement about the word. If a query statement has been generated (YES in step S7), the process proceeds to step S2, and the processes from step S2 onwards are executed for the generated query statement. If a query statement has not been generated (NO in step S7), the process proceeds to step S8.

[0068] 8, the answer acquisition unit 207 determines whether the input sentence received in step S2 is an answer sentence from the answer model 230. If it is an answer sentence from the answer model 230 (YES in step S8), the process proceeds to step S9. If it is not an answer sentence from the answer model 230 (NO in step S8), the process proceeds to step S14.

[0069] In step S9, an answer processing process is performed. In this example, in step S9, the answer acquisition unit 207 processes the answer sentence from the answer model 230 accepted in step S2. Examples of processing of the answer sentence by the answer acquisition unit 207 include synthesizing and summarizing answer sentences from multiple answer models 230. When the answer acquisition unit 207 has completed processing of the answer sentence, it instructs the output unit 208 to output the processed answer sentence.

[0070] In step S10, an answer model evaluation process is performed. In this example, in step S10, the selection unit 205 evaluates each of the answer models 230 that output the answer sentences received in step S2.

[0071] In step S11, the output unit 208 determines whether or not an instruction to output a response sentence has been received from the response acquisition unit 207 in step S9. If an output instruction has been received (YES in step S11), the process proceeds to step S12. If an output instruction has not been received (NO in step S11), the process proceeds to step S14.

[0072] In step S12, an answer output process is performed. In this example, in step S12, the output unit 208 outputs the answer sentence processed in step S9 to the user terminal 3. When the user terminal 3 receives the answer sentence processed in step S9 from the answering device 2, it outputs the received answer sentence to the output unit 36.

[0073] In step S13, a history registration process is performed. In this example, in step S13, the response acquisition unit 207 adds the inquiry statement output by the confirmation unit 202, the response statement from user U1 to the inquiry statement, and the processed response statement to the thread associated with the username of user U1.

[0074] 9, a question generation process is performed in step S14. In this example, in step S14, the analysis result input unit 206 processes the input sentence received in step S2 into a format that makes it easier to obtain an answer from the answer model 230.

[0075] In step S15, a relevance calculation process is performed. In this example, in step S15, the selection unit 205 calculates the relevance between the label assigned to the input sentence accepted in step S2 and the “label” in the model feature table 211.

[0076] In step S16, an answer model selection process is performed. In this example, in step S16, the selection unit 205 selects one or more answer models 230 based on the relevance calculated in step S15.

[0077] In step S17, an answer request process is performed. In this example, in step S17, the analysis result input unit 206 inputs the input sentence processed in step S14 to the answer model 230 selected in step S16.

[0078] In step S18, the answer acquisition unit 207 determines whether or not answer sentences have been acquired from all answer models 230 selected in step S16. If answer sentences have been acquired (YES in step S18), the process proceeds to step S4 in Fig. 7, and the processes from step S4 onwards are executed for the acquired answer sentences. If there is an answer model 230 from which an answer sentence has not been acquired (NO in step S18), the process proceeds to step S17.

[0079] Fig. 10 is a diagram showing an example of a processing flow of the sentence analysis processing by the first analysis unit 201 of the response device 2 according to the embodiment. The processing flow illustrated in Fig. 10 corresponds to the processing from step S3 to step S4 in Fig. 7. Hereinafter, an example of the processing flow of the sentence analysis processing by the first analysis unit 201 will be described with reference to Fig. 10.

[0080] In step S21, a thread is formed and registered. In this example, in step S21, the first analysis unit 201 receives an input sentence exemplified by a question sentence from user U1 and an answer sentence from the answer model 230, and generates a thread for the input sentence. The first analysis unit 201 associates the generated thread with the username of user U1 and stores it in the auxiliary storage unit 23.

[0081] In step S22, a language identification process is performed. In this example, in step S22, the first analysis unit 201 identifies the language of the question sentence for which the thread was generated in step S21. For example, the first analysis unit 201 may store information indicating the identified language in the auxiliary storage unit 23 in association with the username of user U1 and the thread generated in step S21.

[0082] Fig. 11 is a diagram showing an example of a processing flow of the query statement generation processing by the confirmation unit 202 of the answering device 2 according to the embodiment. The processing flow illustrated in Fig. 11 corresponds to the processing of step S7 in Fig. 7. Hereinafter, an example of the processing flow of the query statement generation processing by the confirmation unit 202 will be described with reference to Fig. 11.

[0083] In step S31, the confirmation unit 202 determines whether or not the clustering by the second analysis unit 204 was successful. If the clustering was successful (YES in step S31), the process ends. If the clustering was not successful (NO in step S31), the process proceeds to step S32.

[0084] In step S32, a query statement generation process is performed. In this example, in step S32, the confirmation unit 202 generates a query statement regarding the meaning of the word that could not be clustered successfully in step S31. In step S33, a query statement storage process is performed. In this example, in step S33, the confirmation unit 202 associates the query statement generated in step S32 with the word that could not be clustered successfully in step S31, and stores the query statement in the auxiliary storage unit 23.

[0085] 12 and 13 are diagrams showing an example of a processing flow of label assignment by the first analysis unit 201 and the second analysis unit 204 of the answering device 2 according to the embodiment. The processing flows illustrated in Fig. 12 and 13 correspond to the processing of step S6 in Fig. 7. Hereinafter, an example of a processing flow of label assignment by the first analysis unit 201 and the second analysis unit 204 will be described with reference to Fig. 12.

[0086] 12, a summary analysis process is performed in step S41. In this example, in step S41, the second analysis unit 204 grasps the summary of the input sentence based on the result of the morphological analysis by the first analysis unit 201.

[0087] In step S42, a process of identifying the meaning of a word is performed. In this example, in step S42, the first analysis unit 201 inputs each of the words included in the input sentence into the word model 220 and identifies the meaning of each word.

[0088] In step S43, a labeling process is performed. In this example, in step S43, the second analysis unit 204 uses the meaning of the word identified in step S42 to assign labels representing the meanings of the morphemes to each of the morphemes resolved by the morphological analysis performed by the first analysis unit 201.

[0089] In step S44, the second analysis unit 204 determines whether the thread includes an input sentence other than the current input sentence that is the target of analysis by the first analysis unit 201 and the second analysis unit 204. Examples of the other input sentence include a query sentence from the confirmation unit 202 and a response sentence to the query sentence. If there is another input sentence (YES in step S44), the process proceeds to step S45. If there is no other input sentence (NO in step S44), the process proceeds to step S46.

[0090] In step S45, the second analysis unit 204 determines whether the current input sentence and another input sentence have positive or negative content. In step S46, the second analysis unit 204 determines whether the current input sentence has positive or negative content.

[0091] In step S47, the second analysis unit 204 assigns a label to the current input sentence indicating whether the content is positive or negative, as determined in step S45 or step S46.

[0092] In step S48 shown in Fig. 13, a clustering process is performed. In this example, in step S48, the second analysis unit 204 performs clustering of each morpheme based on the labels assigned in steps S43 and S47. If the clustering is completed successfully (YES in step S49), the process proceeds to step S51. If the clustering is not completed successfully (NO in step S49), the process ends.

[0093] In step S51, a label assignment process indicating the attributes of the user is performed. In this example, in step S51, the second analysis unit 204 assigns a label indicating the attributes of user U1, whose login process was accepted in step S1 of FIG. 7, to the current input sentence. Examples of the attributes of user U1 include the age, gender, nationality, and place of residence of user U1. The attributes of user U1 are stored in the auxiliary storage unit 23 in association with the user name when the user is registered in the response device 2, for example.

[0094] In step S52, the input sentence, the words, and the labels are stored. In this example, in step S52, the second analysis unit 204 associates the current input sentence, the words whose meanings have been identified, and the assigned labels, and stores them in the auxiliary storage unit 23.

[0095] 14 to 16 are diagrams showing an example of a processing flow of the selection process of the answer model 230 by the second analysis unit 204 and the selection unit 205 of the response device 2. The processing in Fig. 14 to 16 corresponds to the processing in step S16 in Fig. 9. Hereinafter, an example of the processing flow of the selection process of the answer model 230 by the second analysis unit 204 and the selection unit 205 will be described with reference to Fig. 14 to 16.

[0096] 14, the selection unit 205 determines whether or not the input is to the answer model 230. If the input is to the answer model 230 (YES in step S61), the process proceeds to step S62. If the input is not to the answer model 230 (NO in step S61), the process proceeds to step S68.

[0097] In step S62, the selection unit 205 extracts candidates for the answer model 230 to be used for the answer, based on the result of the clustering by the second analysis unit 204 and the assigned labels.

[0098] In step S63, the selection unit 205 selects an answer model 230 to be used for the answer from the candidates extracted in step S62. If multiple answer models 230 are selected (YES in step S64), the process proceeds to step S65. If multiple answer models 230 are not selected (NO in step S64), the process proceeds to step S66.

[0099] In step S65, a process for determining the intention of the input sentence is performed. In this example, in step S65, the second analysis unit 204 determines the intention of the answer sought by the question sentence based on the assigned labels and the clustering results. Examples of the intention of the answer sought by the question sentence include wanting multiple helpful answers, wanting a more accurate answer, etc.

[0100] In step S66, the selection unit 205 associates the input sentence accepted in step S2 of FIG. 7 with the answer model 230 selected in step S63, and stores them in the auxiliary storage unit 23.

[0101] In step S67, a relevance calculation process is performed. In this example, in step S67, the selection unit 205 calculates the relevance between the answer model 230 selected in step S63 and the input sentence accepted in step S2 of FIG.

[0102] 15, the selection unit 205 determines whether or not the answer model 230 is being switched. If the answer model 230 is being switched (YES in step S68), the process proceeds to step S69. If the answer model 230 is not being switched (NO in step S68), the process proceeds to step S61.

[0103] In step S69, a determination process is performed as to whether or not there is an answer model that has not been requested. In this example, in step S69, the selection unit 205 determines whether or not there is an answer model 230 for which the analysis result has not been input by the analysis result input unit 206. If there is an answer model 230 for which the analysis result has not been input (YES in step S69), the process proceeds to step S70. If there is not an answer model 230 for which the analysis result has not been input (NO in step S69), the process ends.

[0104] In step S70, an answer model selection process is performed. In this example, in step S70, the selection unit 205 selects an answer model 230 to which the analysis result is to be input by the analysis result input unit 206 from among the answer models 230 selected in step S63 and to which the analysis result has not yet been input, in accordance with the relevance calculated in step S67.

[0105] Fig. 16 is a diagram showing an example of a processing flow of the evaluation process of the answer model 230 by the selection unit 205 of the answer device 2 according to the embodiment. The processing in Fig. 16 corresponds to the processing of step S10 in Fig. 8. Hereinafter, an example of the processing flow of the evaluation process of the answer model 230 by the selection unit 205 will be described with reference to Fig. 16.

[0106] In step S81, an input sentence and a label are read in. In this example, in step S81, the selection unit 205 acquires the input sentence stored in the auxiliary storage unit 23 and the label assigned to the input sentence.

[0107] In step S82, a process of reading an answer sentence and a label from the answer model is performed. In this example, in step S82, the selection unit 205 acquires an answer sentence and a label assigned to the answer from the answer model 230 stored in the auxiliary storage unit 23.

[0108] In step S83, an evaluation value calculation process is performed. In this example, in step S83, the selection unit 205 calculates an evaluation value of the answer model 230 that generated the answer sentence obtained in step S82, based on the degree of coincidence (also referred to as "relevance") between the label obtained in step S81 and the label obtained in step S82. If the calculated evaluation value is equal to or greater than the threshold (YES in step S84), the process proceeds to step S85. If the calculated evaluation value is less than the threshold (NO in step S84), the process proceeds to step S86.

[0109] In step S85, an output setting process is performed. In this example, in step S85, the selection unit 205 instructs the output unit 208 to output the answer sentence output by the answer model 230. In step S86, the selection unit 205 switches the answer model 230 used for the answer to another answer model 230 selected in step S63 of FIG. 14 .

[0110] <Example> A more specific example of the embodiment described above will be described with reference to Fig. 17 to Fig. 20. In this example, an example will be described in which a question about travel is received and the answering device 2 answers the question.

[0111] FIG. 17 is a diagram illustrating an example of a user screen 301 displayed on the output unit 36 ​​of the user terminal 3 according to the embodiment. The user screen 301 is used for inputting a question by the user U1 and outputting an inquiry and an answer from the user terminal 3. The user screen 301 includes a history area 302 and a question and answer area 303. Furthermore, on the user screen 301, a user icon 304 indicating that the input sentence from the user U1 is ascribed to the sentence input from the user U1. Furthermore, on the user screen 301, a reply device icon 305 (see FIG. 18 ) indicating that the output sentence from the reply device 2 is ascribed to the sentence output from the reply device 2. The history area 302 displays a history of questions that have been asked so far. The question and answer area 303 is used for inputting a question and for displaying an inquiry and an answer from the user terminal 3. The example of FIG. 17 illustrates a state in which a question is input from the user U1, such as, "I'd like to go on a hot spring trip in January of next year. Where do you recommend?" The input question is sent from the user terminal 3 to the answering device 2 via the network N1.

[0112] The first analysis unit 201 generates a thread for the received question. The first analysis unit 201 identifies the language of the received question, for example, as described in step S4 of FIG. 7 (step S4 of FIG. 7). The first analysis unit 201 also performs morphological analysis on the question. Table 1 below illustrates the results of the morphological analysis. In Table 1, "writing form" stores words extracted from the question by the morphological analysis. "lexeme" stores a notation that summarizes the writing form of a word regardless of its inflected form. "lexical reading" stores a katakana notation indicating the reading of the lexeme. "part of speech" stores information indicating the part of speech of the morpheme. The first analysis unit 201 may store the results of the morphological analysis in the auxiliary storage unit 23, associating them with "writing form," "lexeme," "lexical reading," and "part of speech" as shown in Table 1.

[0113] Table 1 Morphological analysis results

[0114]

[0115] The second analysis unit 204 assigns a label to each morpheme in the question sentence. For example, the second analysis unit 204 extracts lexemes whose parts of speech are noun, verb, sentence-final particle, or auxiliary symbol from the results of the morphological analysis. In this example, the following lexemes are extracted: "next year," "one," "month," "hot spring," "travel," "go," "recommend," "ka," and "?". The second analysis unit 204 may, for example, instruct the first analysis unit 201 to input each extracted morpheme into the word model 220 and obtain the meaning of each morpheme.

[0116] Furthermore, the second analysis unit 204 may recognize that the question is "January 2024" by concatenating "next year," "one," and "month." Based on the meanings and auxiliary symbols acquired from the word model 220, the second analysis unit 204 assigns the label "hot spring" to "hot spring," the label "travel" to "travel," the labels "January 2024" to "next year" and "January," the labels "hope" to "go," "want," and "i," the labels "recommended" to "o" and "susume," and the labels "interrogative" to "desu," "ka," and "?". Based on these assigned labels, the second analysis unit 204 then assigns the labels "travel plan search" and "travel plan suggestion" to the entire question sentence.

[0117] As a result of the analysis by the second analysis unit 204, the selection unit 205 recognizes that the question is related to a travel plan search and a travel plan suggestion. The selection unit 205 calculates the degree of association between the question and the labels assigned to each morpheme contained in the question and the labels associated with each answer model 230 in the model feature table 211, and selects an answer model 230 to use for generating an answer to the question based on the calculated degree of association. When the model feature table 211 is in the state illustrated in Fig. 6 , the first travel model 234, which matches the two labels "travel" and "hot spring", is selected from the answer models 230.

[0118] When the analysis result input unit 206 inputs the analysis results from the second analysis unit 204 into the first travel model 234, a response statement is output from the first travel model 234. Here, if there are a large number of applicable travel plans (above a predetermined threshold), the first travel model 234 outputs a response statement indicating that the conditions are insufficient. Here, it is assumed that the response statement obtained from the first travel model 234 is "Your desired area, budget, and other conditions are insufficient. There are many applicable plans."

[0119] Here, the confirmation unit 202 outputs, for example, an answer sentence obtained from the first travel model 234 to the user terminal 3 as an inquiry sentence for the user U1. The user terminal 3 displays the inquiry sentence received from the answering device 2 on the user screen 301. FIG. 18 is a diagram illustrating a state in which the inquiry sentence received from the answering device 2 is displayed on the user screen 301. In the example of FIG. 18, the inquiry sentence displayed is, "Your desired area, budget, and other conditions are insufficient. There are many plans that meet this requirement." By displaying the question sentence from the answering device 2 on the user screen 301, the answering device 2 can prompt the user U1 to supplement the missing information in the question sentence.

[0120] 19 is a diagram illustrating a user screen 301 on which a reply to an inquiry from the reply device 2 is input by the user U1. In the example of Fig. 19, the reply is "I'm thinking of staying in the Kanto area for about two nights and three days. I'd like to stay at a hot spring with good quality water." The input reply is sent from the user terminal 3 to the reply device 2 via the network N1.

[0121] The answering device 2 adds the received answer sentence to the thread. Then, in the answering device 2, the meaning of the answer sentence is understood for the received question sentence, for example, by processing by the first analysis unit 201 and the second analysis unit 204. Then, in the answering device 2, the selection unit 205 selects an answer model 230 based on the question sentence and answer sentence included in the thread, and outputs an answer sentence. FIG. 20 is a diagram showing an example of a state in which an answer sentence from the answering device 2 is displayed on a user screen 301. In the example of FIG. 20, the answer sentence that reads, "If you are looking for a hot spring with good water quality near the Kanto region, the following areas are recommended. 1. Hakone: famous for its sulfur springs..." is output.

[0122] In this way, the answer sentence to the question sentence received from the answering device 2 is input by the user U1 and transmitted to the answering device 2. Then, by repeating such an exchange, the answering device 2 can output an answer that meets the request of the user U1 with higher accuracy.

[0123] <Effects of the embodiment> The answering device 2 according to the present embodiment outputs a query sentence when a question sentence received from the user terminal 3 contains an unknown word whose meaning cannot be identified or when clustering cannot be performed normally. Then, the answering device 2 understands the meaning of the question sentence based on the reply sentence from the user terminal 3 to the query sentence. By exchanging such a query sentence and reply sentence, the answering device 2 can understand the meaning of the question sentence from the user U1 with higher accuracy. Consequently, the answering device 2 can improve the accuracy of reply sentences to question sentences containing unknown words.

[0124] When the meaning of an unknown word is identified, the answering device 2 according to this embodiment uses the meaning of the identified unknown word as training data and performs additional learning processing using the word model 220. Therefore, the answering device 2 can expand words that can be identified by the word model 220. Consequently, the answering device 2 can reduce the workload of having the user U1 answer a query sentence.

[0125] The answering device 2 according to the present embodiment associates an unknown word, the meaning of the identified unknown word, and an input sentence including the unknown word, and stores the associations in the auxiliary storage unit 23. Therefore, even if an additional learning process for the unknown word is not performed on the word model 220, the answering device 2 can refer to the auxiliary storage unit 23 and identify the meaning of the unknown word that could not be identified by the word model 220. Consequently, the answering device 2 can reduce the workload of the user U1 in responding to a query sentence.

[0126] The answering device 2 according to the present embodiment may notify the user terminal 3 that an answer is not possible when the meaning of the unknown word cannot be identified even by the unknown word identifying unit 203. Therefore, the answering device 2 can suppress output of an answer that does not conform to the intention of the question sentence.

[0127] The answering device 2 according to the present embodiment also outputs information indicating the location of materials related to the answer sentence if such materials exist, so that the answering device 2 can present to the user U1 not only the answer sentence but also materials that can be used as a reference for answering the question sentence.

[0128] In this embodiment, the features of each model in the answer model 230 are managed by the model feature table 211. Then, the answer device 2 selects an answer model 230 to which the analysis result by the second analysis unit 204 is to be input, based on the label assigned to the question sentence by the second analysis unit 204 and the label associated with each model in the model feature table 211. The label assigned to the question sentence indicates the field of the question sentence, and the label associated with each model indicates the field of the training data for that model. Therefore, the answer device 2 can select an answer model 230 that is preferable for generating an answer sentence to a question sentence.

[0129] In this embodiment, the answer device 2 calculates the relevance between the label assigned to the question sentence and the label associated with each model in the model feature table 211, and preferentially selects an answer model 230 with a high calculated relevance. Therefore, the answer device 2 can select an answer model 230 that is more relevant to the question sentence as the answer model 230 that will generate an answer sentence. Ultimately, the answer device 2 can improve the accuracy of the answer to the question sentence. Furthermore, when multiple answer models 230 are selected to generate an answer sentence, the answer device 2 inputs the analysis results in order from the answer model 230 with the highest relevance. Therefore, the answer device 2 can acquire an answer sentence with higher accuracy at an earlier stage.

[0130] In this embodiment, the answer device 2 calculates the answer relevance between the question sentence and the answer sentence obtained from the answer model 230, and preferentially outputs the answer sentence with a high calculated answer relevance. Furthermore, if the calculated answer relevance is less than a predetermined threshold, the answer device 2 suppresses output of the answer sentence. Therefore, the answer device 2 can present the user U1 with an answer sentence that is more relevant to the question sentence.

[0131] In this embodiment, the reply device 2 synthesizes the plurality of reply sentences when acquiring reply sentences from a plurality of reply models 230. Therefore, the reply device 2 can make the user U1 view the reply sentences acquired from the plurality of reply models 230.

[0132] In this embodiment, the answering device 2 calculates the usage fee for the answering system 1 based on the amount of text in the question text and charges the user U1. Therefore, the answering device 2 can charge the fee according to the usage mode (amount of usage) of the answering system 1 by the user U1.

[0133] <Modification> In the embodiment described above, the process of inputting the analysis result to the answer model 230 by the analysis result input unit 206 and the process of acquiring the answer sentence from the answer model 230 by the answer acquisition unit 207 are performed consecutively. However, the process of inputting the analysis result (an example of a “first analysis result”) of the question sentence (an example of a “first question sentence”) by the analysis result input unit 206 to the answer model 230 and the process of acquiring the answer sentence (an example of “first answer information”) from the answer model 230 regarding the analysis result of the question sentence (an example of a “second question sentence”) already input to the answer model 230 may be performed in parallel. By performing these processes in parallel, the answer device 2 can shorten the time it takes to generate a final answer sentence to the question sentence from the user U1. This parallel execution is also referred to as multitasking. Parallel execution refers to a state in which the input process and the acquisition process are performed simultaneously, even if only temporarily. This does not mean that the two processes are started simultaneously, although the two processes may be started simultaneously.

[0134] In the embodiment described above, the response device 2 and the user terminal 3 are separate devices, but this is not limiting, and the response device 2 and the user terminal 3 may be realized by a single information processing device.

[0135] The embodiments and modifications disclosed above can be combined with each other.

[0136] <Computer-readable recording medium> An information processing program that causes a computer or other machine or device (hereinafter referred to as a computer, etc.) to realize any of the above functions can be recorded on a recording medium that can be read by a computer, etc. Then, by having the computer, etc. read and execute the program from this recording medium, the function can be provided.

[0137] Here, a computer-readable recording medium refers to a recording medium that stores information such as data and programs electrically, magnetically, optically, mechanically, or chemically and that can be read by a computer, etc. Among such recording media, those that are removable from a computer, etc. include, for example, flexible disks, magneto-optical disks, Compact Disc Read Only Memory (CD-ROM), Compact Disc-Recordable (CD-R), Compact Disc-Rewritable (CD-RW), Digital Versatile Disc (DVD), Blu-ray Disc (BD), Digital Audio Tape (DAT), 8mm tape, flash memory, external hard disk drives, and Solid State Drives (SSD). Furthermore, examples of recording media fixed to a computer or the like include built-in hard disk drives, SSDs, ROMs, and the like.

[0138] The above-mentioned learning refers to, for example, a method in which data patterns are input into a computer based on training data, which is a set of information and correct judgments, and the computer then finds new features and associations based on these patterns. Another example of the above-mentioned learning refers to a method in which a computer finds commonalities and characteristic data in the input data. Another example of the above-mentioned learning refers to a method known as deep learning, in which a computer builds a neural network to analyze, extract, and output local or characteristic data. Of course, this learning also includes preprocessing, such as supplementing missing data, eliminating unnecessary data, and standardizing data formats, as well as postprocessing, in which the learning results are stored so that they can be used for other learning.

[0139] The terms "unit," "means," "device," and "system" used in the above-described embodiments and claims do not simply mean physical means, but also include cases where the functions they possess are realized by software or software services. Furthermore, the functions of a single "unit," "means," "device," or "system" may be realized not only by a single physical means, software, software module, or device, but also by multiple physical means, software, software modules, devices, or a combination of these.

[0140] The terms used in the above embodiments and claims should be interpreted as open-ended terms. For example, the term "including" should be interpreted as "not limited to what is described as including." The term "containing" should be interpreted as "not limited to what is described as containing." The term "comprising" should be interpreted as "not limited to what is described as comprising." The term "having" should be interpreted as "not limited to what is described as having." The term "comprising" should be interpreted as "not limited to what is described as comprising."

[0141] Supplementary Note 1: A system includes a storage unit that stores a first learning model that, when a word is input, outputs text data indicating the meaning of the word, and a second learning model that, when a question is input, outputs answer information to the question, and a processor connected to the storage unit, wherein the processor performs a first identification process that identifies the meaning of each word included in a question sentence input by a user using the first learning model, an analysis process that analyzes the meaning of the question sentence based on the meanings of the words identified in the first identification process, an input process that inputs the analysis result of the question sentence by the analysis process into the second learning model, an acquisition process that acquires first answer information to the question sentence from the second learning model, and and an output process for outputting answer information, wherein the analysis process further includes outputting inquiry information inquiring about the meaning of the word when the meaning cannot be identified in the first identification process or when the meaning of the word can be identified but the output process does not allow the output of the first answer information based on that meaning, accepting second answer information to the inquiry information, and, when the meaning of the word whose meaning could not be identified in the first identification process is identified based on the second answer information, analyzing the meaning of the question sentence based on the meaning of the word identified in the first identification process and the meaning of the word identified based on the second answer information. <Supplementary Note 2> The information processing device according to Supplementary Note 1, wherein the processor further executes a learning process for training the first learning model based on the words identified based on the second answer information and the meaning of the words identified based on the second answer information. <Supplementary Note 3> The information processing device according to Supplementary Note 1 or Supplementary Note 2, wherein the processor further executes a storage process for storing the second answer information in the storage unit in association with the question sentence and the words whose meaning could not be identified. <Supplementary Note 4> The information processing device according to any one of Supplementary Note 1 to Supplementary Note 3, wherein identifying based on the second answer information further includes notifying that an answer is not possible when the meaning of the word whose meaning could not be identified cannot be identified based on the second answer information.<Supplementary Note 5> The information processing device according to any one of Supplements 1 to 4, wherein the output process further includes a process of including, in the first answer information, location information indicating the location of materials related to the first answer information. <Supplementary Note 6> The information processing device according to any one of Supplements 1 to 5, wherein the processor executes the input process of inputting a first analysis result for a first question sentence into the second learning model, and the acquisition process of acquiring, from the second learning model, the first answer information for a second question sentence whose analysis result has already been input to the second learning model. <Supplementary Note 7> A computer having a memory unit storing a first learning model that outputs text data indicating the meaning of a word when a word is input, and a second learning model that outputs answer information to a question when a question is input, performs a first identification process that identifies the meaning of each word included in a question input by a user using the first learning model, an analysis process that analyzes the meaning of the question based on the meanings of the words identified in the first identification process, an input process that inputs the analysis result of the question by the analysis process into the second learning model, an acquisition process that acquires first answer information to the question from the second learning model, and an output process that outputs the first answer information. wherein the analysis process further includes outputting inquiry information inquiring about the meaning of the word when the meaning cannot be identified in the first identification process, or when the meaning of the word can be identified but the first answer information cannot be output by the output process based on the meaning of the word; receiving second answer information in response to the inquiry information; and when the meaning of the word whose meaning could not be identified by the first identification process is identified based on the second answer information, analyzing the meaning of the question sentence based on the meaning of the word identified in the first identification process and the meaning of the word identified based on the second answer information.<Supplementary Note 8> A computer including a storage unit storing a first learning model that outputs text data indicating the meaning of a word when the word is input, and a second learning model that outputs answer information to a question when the question is input, is provided with a first identification process that identifies the meaning of each word included in a question input by a user using the first learning model, an analysis process that analyzes the meaning of the question based on the meanings of the words identified in the first identification process, an input process that inputs the analysis result of the question by the analysis process into the second learning model, an acquisition process that acquires first answer information to the question from the second learning model, and an output process that outputs the first answer information. and wherein the analysis processing further includes outputting inquiry information inquiring about the meaning of the word in at least one of a case where the meaning cannot be identified in the first identification processing, or a case where the meaning of the word can be identified but the first answer information cannot be output by the output processing based on the meaning of the word; receiving second answer information in response to the inquiry information, identifying the meaning of the word whose meaning could not be identified by the first identification processing based on the second answer information; and analyzing the meaning of the question sentence based on the meaning of the word identified in the first identification processing and the meaning of the word identified based on the second answer information.

[0142] <Supplementary Note 11> An information processing device comprising: a memory unit that stores a plurality of learning models that generate an answer sentence to a sentence when an analysis result of the sentence is input; and feature information that indicates features related to the accuracy of the answer to the analysis result for each of the plurality of learning models; and a processor connected to the memory unit, wherein, when the processor receives the analysis result of a question sentence input by a user, it executes an input process that inputs the analysis result to one or more selected learning models selected from the plurality of learning models; and an output process that outputs the answer sentence from the selected learning model to which the analysis result has been input, and the input process further includes a selection process that selects one or more selected learning models from the plurality of learning models to which the analysis result is to be input by referring to the feature information. <Supplementary Note 12> The information processing device according to Supplementary Note 11, wherein the feature information includes first domain information indicating the domain to which the learning model corresponds, the analysis result includes second domain information indicating the domain of the question sentence, and the selection process includes a process of calculating a domain relevance indicating the degree of relevance between the first domain information and the second domain information, and preferentially selecting the learning model with a higher calculated domain relevance as the selected learning model. <Supplementary Note 13> The information processing device according to Supplementary Note 12, wherein the input process, when multiple selected learning models are selected, inputs the analysis results to the selected learning models in order according to the degree of the field relevance. <Supplementary Note 14> The information processing device according to Supplementary Note 12 or Supplementary Note 13, wherein the output process further includes a process of calculating an answer relevance indicating the degree of relevance between the answer sentence and the second domain information, and preferentially outputting the answer sentence with a higher answer relevance. <Supplementary Note 15> The information processing device according to any one of Supplementary Notes 12 to 14, wherein the output process further includes a process of calculating an answer relevance indicating a degree of relevance between the answer sentence and the second field information, and suppressing output of the answer sentence having the answer relevance lower than a threshold. <Supplementary Note 16> The information processing device according to any one of Supplementary Notes 11 to 15, wherein the output process includes a process of synthesizing the answer sentences from the plurality of selected learning models when a plurality of the selected learning models are selected in the selection process.and an output process for outputting the answer sentence from the selected learning model to which the analysis result has been input, wherein the input process further includes a selection process for selecting, from the selected learning model, one or more of the selected learning models to which the analysis result has been input, with reference to the feature information. <Supplementary Note 19> An information processing program that causes a computer having a memory unit that stores a plurality of learning models that generate an answer sentence to a sentence when an analysis result of the sentence is input, and feature information that indicates, for each of the plurality of learning models, features related to the accuracy of the answer to the analysis result, to execute, upon receiving an analysis result of a question sentence input by a user, an input process that inputs the analysis result to one or more selected learning models selected from the plurality of learning models, and an output process that outputs the answer sentence from the selected learning model to which the analysis result has been input, wherein the input process further includes a selection process that selects, from the plurality of learning models, one or more selected learning models to which the analysis result is to be input, by referring to the feature information.

[0143] This application is based on Japanese Patent Application No. 2023-223636 filed on December 28, 2023, the contents of which are incorporated herein by reference.

Claims

1. A storage unit that stores a first learning model that outputs text data indicating the meaning of a word when the word is input, and a second learning model that outputs answer information for the question when the question is input; and a processor connected to the storage unit, wherein the processor performs: a first specifying process of specifying the meaning of each word included in the question sentence input by the user using the first learning model; an analyzing process of analyzing the meaning of the question sentence based on the meaning of the word specified in the first specifying process; an inputting process of inputting the analysis result of the question sentence by the analyzing process into the second learning model; an acquiring process of acquiring first answer information for the question sentence from the second learning model; and an outputting process of outputting the first answer information, and the analyzing process outputs inquiry information for inquiring about the meaning of the word when at least one of the following cases occurs: when the meaning cannot be specified in the first specifying process, or when the meaning of the word can be specified but the first answer information cannot be output by the outputting process based on the meaning of the word, receives second answer information for the inquiry information, and when the meaning of the word that could not be specified in the first specifying process is specified based on the second answer information by the first specifying process, analyzes the meaning of the question sentence based on the meaning of the word specified in the first specifying process and the meaning of the word specified based on the second answer information. An information processing apparatus further including the above.

2. The processor of claim 1 further performs a learning process of training the first learning model based on the word specified based on the second answer information and the meaning of the word specified based on the second answer information.

3. The processor of claim 1 further performs a storage process of storing the second answer information in the storage unit in association with the question sentence and the word for which the meaning could not be specified.

4. The information processing apparatus of claim 1, wherein specifying based on the second answer information further includes notifying that no answer is available when the meaning of the word for which the meaning could not be specified cannot be specified based on the second answer information.

5. The information processing apparatus according to claim 1, wherein the output processing further includes a process of including location information indicating the location of materials related to the first response information in the first response information.

6. The information processing apparatus according to any one of claims 1 to 5, wherein the processor executes the input processing of inputting a first analysis result for a first question sentence into the second learning model, and the acquisition processing of acquiring the first response information for a second question sentence whose analysis result has already been input into the second learning model from the second learning model.

7. A computer including a storage unit that stores a first learning model that outputs text data indicating the meaning of a word when the word is input, and a second learning model that outputs response information for a question when the question is input, performs a first specifying process of specifying the meaning of each word included in a question sentence input by a user using the first learning model, an analysis process of analyzing the meaning of the question sentence based on the meaning of the word specified in the first specifying process, an input process of inputting the analysis result of the question sentence by the analysis process into the second learning model, an acquisition process of acquiring first response information for the question sentence from the second learning model, and an output process of outputting the first response information, and the analysis process further includes outputting inquiry information for inquiring about the meaning of the word when at least one of the following cases occurs: when the meaning cannot be specified in the first specifying process, or when the meaning of the word can be specified but the first response information cannot be output by the output process based on the meaning of the word; receiving second response information for the inquiry information, and when the meaning of the word that could not be specified in the first specifying process is specified based on the second response information by the first specifying process, analyzing the meaning of the question sentence based on the meaning of the word specified in the first specifying process and the meaning of the word specified based on the second response information.

8. A computer comprising a storage unit that stores a first learning model that outputs text data indicating the meaning of a word when the word is input, and a second learning model that outputs response information for a question when the question is input, performs: a first specifying process of specifying the meaning of each word included in a question sentence input by a user using the first learning model; an analyzing process of analyzing the meaning of the question sentence based on the meaning of the word specified in the first specifying process; an inputting process of inputting the analysis result of the question sentence by the analyzing process into the second learning model; an acquiring process of acquiring first response information for the question sentence from the second learning model; and an outputting process of outputting the first response information. The analyzing process further includes: outputting inquiry information for inquiring about the meaning of the word when at least one of the following cases occurs: when the meaning cannot be specified in the first specifying process, or when the meaning of the word can be specified but the first response information cannot be output by the outputting process based on the meaning of the word; receiving second response information for the inquiry information, specifying the meaning of the word for which the meaning could not be specified in the first specifying process based on the second response information by the first specifying process, and analyzing the meaning of the question sentence based on the meaning of the word specified in the first specifying process and the meaning of the word specified based on the second response information. An information processing program.

Images