Information processing device, information processing method, and information processing program

Abstract

It generates highly accurate answers to questions in various fields. [Solution] This information processing device includes a plurality of learning models that generate response sentences to a given text when the analysis results of the text are input, a storage unit that stores feature information indicating the characteristics related to the accuracy of the response to the analysis results for each of the plurality of learning models, and a processor connected to the storage unit. When the processor receives the analysis results of a question text input by a user, it performs an input process that inputs the analysis results to one or more selected learning models selected from the plurality of learning models, and an output process that outputs response sentences from the selected learning models that have received the analysis results. The input process further includes a selection process that, by referring to the feature information, selects one or more selected learning models from the plurality of learning models to input the analysis results.

Description

[[Technical Field]]

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program. [[Background Art]]

[0002] In recent years, artificial intelligence (also referred to as "AI") that generates content such as text and images in response to prompts has been used. For example, in the technology of generating a sentence from keywords included in an input sentence, when the input sentence contains an unknown word that is not in a trained dictionary, there has been proposed a technology of replacing the unknown word in the input sentence with a predicted word to generate a sentence, and then replacing the predicted word with the unknown word in the generated sentence (see, for example, Patent Document 1). [[Prior Art Literature]] [[Patent Literature]]

[0003] [[Patent Document 1]] Japanese Unexamined Patent Publication No. 2019-185400 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0004] For generative AI, the field of question sentences for which answer sentences can be generated with high accuracy differs depending on the explanatory variables and objective variables used for learning. Therefore, it is difficult for generative AI to generate highly accurate answer sentences for question sentences in various fields.

[0005] One aspect of the disclosed technology aims to provide an information processing apparatus, an information processing method, and an information processing program capable of generating highly accurate answer sentences for question sentences in various fields. [[Means for Solving the Problem]]

[0006] One aspect of the disclosed technology is exemplified by the following information processing device. This information processing device includes a storage unit that stores a plurality of learning models that generate response sentences to a given text when the result of text analysis is input, and feature information that indicates the characteristics related to the accuracy of the response to the analysis result for each of the plurality of learning models, and a processor connected to the storage unit. When the processor receives the result of question text input by a user, it performs an input process that inputs the analysis result to one or more selected learning models selected from the plurality of learning models, and an output process that outputs a response sentence from the selected learning model that has received the analysis result. The input process further includes a selection process that, by referring to the feature information, selects one or more selected learning models from the plurality of learning models to input the analysis result.

[0007] In this information processing device, feature information indicating characteristics related to the accuracy of responses to the analysis results is stored in the memory unit for each of the above-mentioned learning models. Then, the information processing device refers to the above-mentioned feature information and selects one or more of the above-mentioned learning models to be input with the analysis results from among the above-mentioned learning models. Therefore, the information processing device can select a learning model that is preferable for generating response sentences to question sentences.

[0008] The information processing device may further have the following features: The feature information includes first field information indicating the field to which the learning model corresponds; the analysis result includes second field information indicating the field of the question; and the selection process includes calculating a field relevance score indicating the degree of relevance between the first field information and the second field information, and selecting the learning model with a high calculated field relevance score as the selected learning model. By having these features, the information processing device can select the learning model that is more relevant to the question as the selected learning model. You can choose.

[0009] The information processing device may further have the following features: If multiple selected learning models are selected, the input processing inputs the analysis results to the selected learning models in an order corresponding to the degree of relevance to the field. By inputting the analysis results to the selected learning models in an order corresponding to the degree of relevance to the field, the information processing device can obtain more accurate response sentences at an earlier stage.

[0010] The information processing device may further include the following features: The output processing further includes a process that calculates an answer relevance score indicating the degree of relevance between the answer sentence and the second field information, and prioritizes outputting the answer sentences with high answer relevance scores. By having such features, the information processing device can present the user with answer sentences that are more relevant to the question sentence.

[0011] The information processing device may further include the following features: The output processing further includes a process that calculates an answer relevance score indicating the degree of relevance between the answer sentence and the second field information, and suppresses the output of the answer sentence whose answer relevance score is lower than a threshold. By having such features, the information processing device can suppress the output of answer sentences that are undesirable as answers to the above-mentioned question sentence.

[0012] The information processing device may further have the following features: The output processing includes a process for synthesizing the response sentences from the multiple selected learning models when multiple selected learning models are selected in the selection processing. By having such features, the information processing device can display a list of the response sentences obtained from the selected learning models to the user.

[0013] The information processing device may further have the following features: The analysis result includes text volume information relating to the length of the text, and the processor further performs a calculation process to calculate a usage fee based on the text volume information. By having these features, the information processing device can bill the user for the usage fee of the information processing device.

[0014] The disclosed technique can also be understood from the aspects of an information processing method and an information processing program. Effects of the Invention

[0015] The disclosed technique can generate highly accurate answer sentences for question sentences in various fields. Brief Description of the Drawings

[0016] [Figure 1] Figure 1 is a diagram illustrating an example of an answer system according to an embodiment. [Figure 2] Figure 2 is a diagram illustrating an example of the hardware configuration of an answer device according to an embodiment. [Figure 3] Figure 3 is a diagram illustrating an example of the hardware configuration of a user terminal according to an embodiment. [Figure 4] Figure 4 is a diagram illustrating an example of a processing block of an answer device according to an embodiment. [Figure 5] Figure 5 is a diagram illustrating an example of an answer model. [Figure 6] Figure 6 is a diagram illustrating an example of a model feature table stored in a management database in an embodiment. [Figure 7] Figure 7 is a first diagram illustrating an example of a processing flow of an answer device according to an embodiment. [Figure 8] Figure 8 is a second diagram illustrating an example of a processing flow of an answer device according to an embodiment. [Figure 9] Figure 9 is a third diagram illustrating an example of a processing flow of an answer device according to an embodiment. [Figure 10] Figure 10 is a diagram illustrating an example of a processing flow of sentence analysis processing by a first analysis unit of an answer device according to an embodiment. [Figure 11] Figure 11 is a diagram illustrating an example of a processing flow of inquiry sentence generation processing by a confirmation unit of an answer device according to an embodiment. [Figure 12] Figure 12 is a first diagram illustrating an example of a processing flow of labeling by a first analysis unit and a second analysis unit of an answer device according to an embodiment. [Figure 13]FIG. 13 is a second diagram showing an example of a labeling processing flow by the first analysis unit and the second analysis unit of the answering device according to the embodiment. [Figure 14] FIG. 14 is a first diagram showing an example of a processing flow of answer model selection processing by the second analysis unit and the selection unit of the answering device. [Figure 15] FIG. 15 is a second diagram showing an example of a processing flow of answer model selection processing by the second analysis unit and the selection unit of the answering device. [Figure 16] FIG. 16 is a third diagram showing an example of a processing flow of answer model selection processing by the second analysis unit and the selection unit of the answering device. [Figure 17] FIG. 17 is a first diagram showing an example of a user screen displayed on the output unit of the user terminal according to the example. [Figure 18] FIG. 18 is a second diagram showing an example of a user screen displayed on the output unit of the user terminal according to the example. [Figure 19] FIG. 19 is a third diagram showing an example of a user screen displayed on the output unit of the user terminal according to the example. [Figure 20] FIG. 20 is a fourth diagram showing an example of a user screen displayed on the output unit of the user terminal according to the example. MODE FOR CARRYING OUT THE INVENTION

[0017] <Embodiment> Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is a diagram showing an example of an answer system 1 according to an embodiment. The answer system 1 includes an answering device 2, a user terminal 3, and a network N1. The answering device 2 and the user terminal 3 are communicatively connected to each other via the network N1.

[0018] The answering device 2 is an information processing device that, upon receiving a question sentence input by user U1 into user terminal 3, transmits an answer sentence to user terminal 3. The question sentence is an example of input information, and is, for example, a sentence indicating what user U1 is seeking an answer from answering device 2, and contains one or more words, and is transmitted from user terminal 3 to answering device 2. The answering device 2 may also receive question information from user terminal 3 that includes the question sentence and data other than text. Examples of data other than text included in the answer information include sound data, video data, image data, etc. User terminal 3 is an information processing device that transmits the question sentence input by user U1 to answering device 2 and outputs the answer sentence received from answering device 2 to a display device such as a display.

[0019] Network N1 connects information processing devices in a way that allows them to communicate with each other. Network N1 can be, for example, a Local Area Network (LAN), a mobile communication system, or the Internet. Network N1 may be wired or wireless.

[0020] Figure 2 shows an example of the hardware configuration of the answering device 2 according to the embodiment. The answering device 2 comprises a CPU 21, a main memory unit 22, an auxiliary memory unit 23, a communication unit 24, and a connection bus B2. The CPU 21, the main memory unit 22, the auxiliary memory unit 23, and the communication unit 24 are interconnected by the connection bus B2.

[0021] The CPU21 is also called a microprocessor unit (MPU) or processor. The CPU 21 is not limited to a single processor and may be a multi-processor configuration. Furthermore, a single CPU 21 connected via a single socket may have a multi-core configuration. At least a portion of the processing performed by the CPU 21 may be performed by other processors, such as dedicated processors like Digital Signal Processors (DSPs), Graphics Processing Units (GPUs), numerical processors, vector processors, image processing processors, and Neural Processing Units (NPUs). Also, at least a portion of the processing performed by the CPU 21 may be performed by integrated circuits (ICs) and other digital circuits. Furthermore, at least a portion of the CPU 21 may include analog circuits. Integrated circuits include Large Scale Integrated Circuits (LSIs), Application Specific Integrated Circuits (ASICs), and Programmable Logic Devices (PLDs). PLDs include, for example, Field-Programmable Gate Arrays (FPGAs). The CPU 21 may be a combination of a processor and an integrated circuit. Such combinations are called, for example, microcontroller units (MCUs), System-on-a-chip (SoCs), system LSIs, or chipsets. In the answering device 2, the CPU 21 loads the program stored in the auxiliary storage unit 23 into the work area of ​​the main memory unit 22 and controls peripheral devices through program execution. This allows the answering device 2 to perform processing that matches a predetermined purpose. The main memory unit 22 and the auxiliary storage unit 23 are recording media that can be read by the CPU 21.

[0022] The main memory unit 22 is exemplified as a memory unit that is directly accessed by the CPU 21. The main memory unit 22 includes Random Access Memory (RAM) and Read Only Memory (ROM).

[0023] The auxiliary storage unit 23 stores various programs and data on a recording medium in a read-write manner. The auxiliary storage unit 23 is also called an external storage device. The auxiliary storage unit 23 stores the operating system (OS), various programs, various tables, etc. The OS includes a communication interface program that exchanges data with external devices connected via the communication unit 24. External devices include, for example, other information processing devices and external storage devices connected by a computer network. The auxiliary storage unit 23 may also be, for example, part of a cloud system, which is a group of computers on a network.

[0024] The auxiliary storage unit 23 may be, for example, an Erasable Programmable ROM (EPROM), a Solid State Drive (SSD), or a Hard Disk Drive (HDD). Alternatively, the auxiliary storage unit 23 may be a Compact Disc (CD) drive, a Digital Versatile Disc (DVD) drive, or a Blu-ray® Disc (BD) drive. Furthermore, the auxiliary storage unit 23 may be provided by a Network Attached Storage (NAS) or Storage Area Network (SAN).

[0025] The communication unit 24 is, for example, an interface with the network N1. The communication unit 24 communicates with external devices via the network N1.

[0026] Figure 3 is a diagram showing an example of the hardware configuration of a user terminal 3 according to the embodiment. The user terminal 3 comprises a CPU 31, a main memory unit 32, an auxiliary memory unit 33, a communication unit 34, an input unit 35, an output unit 36, and a connection bus B3. The CPU 31, main memory unit 32, auxiliary memory unit 33, communication unit 34, and connection bus B3 are the same as the CPU 21, main memory unit 22, and auxiliary memory unit 23 of the answering device 2. Since it has the same configuration as communication unit 24 and connection bus B2, the explanation will be omitted.

[0027] The input unit 35 receives input from user U1. The input unit 35 can be, for example, a keyboard, mouse, trackball, touch panel, or voice input device. The output unit 36 ​​outputs data processed by the CPU 31 or data stored in the main memory unit 32. The output unit 36 ​​can output, for example, a display, printer, or speaker.

[0028] <Processing block of response device 2> Figure 4 shows an example of a processing block of the answering device 2 according to the embodiment. The answering device 2 comprises a login processing unit 200, a first analysis unit 201, a confirmation unit 202, an unknown word identification unit 203, a second analysis unit 204, a selection unit 205, an analysis result input unit 206, an answer acquisition unit 207, an output unit 208, a billing unit 209, a management database 210 (labeled "Management DB210" in the figure), a word model 220, and an answer model 230. The answering device 2 performs processing as each of its respective units, such as the login processing unit 200, the first analysis unit 201, the confirmation unit 202, the unknown word identification unit 203, the second analysis unit 204, the selection unit 205, the analysis result input unit 206, the answer acquisition unit 207, the output unit 208, the billing unit 209, the management database 210, the word model 220, and the answer model 230, by having the CPU 21 execute a computer program that has been expanded in executable form in the main memory unit 22.

[0029] The word model 220 outputs text data that shows the meaning of the input word. The word model 220 is a learning model constructed by machine learning using text data showing words and their meanings as training data. The word model 220 may also be, for example, a table that associates words with text data showing their meanings. The word model 220 is prepared for each language, for example, Japanese, English, and German. The word model 220 is an example of the "first learning model".

[0030] The response model 230 generates a response to an input question. The response is a sentence that indicates the answer to the question. The response model 230 may generate response information that includes the response and data other than text for the input question. Examples of data other than text included in the response information include sound data, video data, image data, etc. The response model 230 generates the response using, for example, a large-scale language model (LLM). The response generated by the response model 230 may include information indicating the location of materials related to the response to the question. Information indicating the location of materials is, for example, a Uniform Resource Identifier (URI). The response model 230 may also output a message indicating that the conditions included in the input question are insufficient to produce the response. The response model 230 includes, for example, multiple learning models corresponding to the information domain of the question. The response model 230 is, for example, a learning model constructed based on training data. The response model 230 is an example of a "second learning model". The response text generated by response model 230 is an example of "response information." Information indicating the location of the document is an example of "location information."

[0031] Figure 5 shows an example of the response model 230. The response model 230 includes, for example, a first maintenance model 231, a second maintenance model 232, a third maintenance model 233, a first travel model 234, and a second travel model 235. The first maintenance model 231, the second maintenance model 232, and the third maintenance model 233 are types of learning models constructed using information related to vehicle maintenance as training data. In other words, the first maintenance model 231, the second maintenance model 232, and the third maintenance model 233 can be said to be response models 230 specialized for vehicle maintenance. The first travel model 234 and the second travel model 235 are types of learning models constructed using information related to travel as training data. In other words, the first travel model 234 and the second travel model 235 can be said to be response models 230 specialized for travel. That is, the response model 230 is composed of multiple types of learning models, and the types are based on the input information (input sentences, question sentences). This includes multiple learning models, each different for each information field. These learning models may also be referred to as the first type of learning model, the second type of learning model, or the first type of learning model, depending on their type. Furthermore, in constructing the answer models 230 specialized for each field, data collected by the company operating the answer device 2 (hereinafter also referred to as "our company") may be used, or data collected by our company and data provided by business partners may be used. Below, the first maintenance model 231, the second maintenance model 232, and the third maintenance model 233 are shown as examples of the first type of learning models, and the first travel model 234 and the second travel model 235 are shown as examples of the second type of learning models. Examples of other learning models will be described later.

[0032] The first maintenance model 231 is constructed using information related to automobile maintenance as training data. Examples of information related to automobile maintenance include maintenance manuals and official websites published by automobile manufacturers, information related to maintenance work published by automobile repair shops, information related to maintenance published by automobile mechanics, blogs and social networking services (SNS) related to maintenance cases published by automobile mechanics, information published in automobile magazines, and examples of automobile customization posted on internet bulletin boards, etc.

[0033] The second maintenance model 232 is constructed using highly reliable information related to automobile maintenance as training data. Examples of highly reliable information include maintenance manuals and official websites published by automobile manufacturers, information related to maintenance work published by automobile repair shops, and information related to maintenance published by automobile mechanics.

[0034] The third maintenance model 233 is constructed using information related to automobile maintenance as training data. The third maintenance model 233 aims to create a lightweight model, and for example, it is constructed with less training data than the first maintenance model 231.

[0035] The training data used to construct the second maintenance model 232 is limited to highly reliable information related to automobile maintenance, and therefore tends to have a narrower range of areas for which it can generate answers compared to the first maintenance model 231. On the other hand, the second maintenance model 232, constructed based on such highly reliable information, tends to generate more reliable answers to input questions related to automobile maintenance than the first maintenance model 231. In other words, although the reliability of the answers generated by the first maintenance model 231 is lower than that of the second maintenance model 232, it tends to be able to generate answers to a wider range of maintenance-related questions. Furthermore, the third maintenance model 233 is a lightweight model and can generate answers faster than the first maintenance model 231.

[0036] The first travel model 234 is constructed using travel-related information as training data. This travel-related information includes tourism information published by prefectural governments, overseas safety information published by the Ministry of Foreign Affairs, tourism information published by ministries responsible for attracting tourists in each country, websites where travel companies publish information about tourist destinations, and blogs and social media accounts run by individuals that publish travel records.

[0037] The second travel model 235 is constructed using highly reliable travel-related information as training data. Examples of highly reliable information include travel-related information.

[0038] The training data used to construct the second travel model 235 is limited to highly reliable travel-related information, and therefore tends to have a narrower range of areas for which it can generate answers compared to the first travel model 234. On the other hand, the second travel model 235, constructed based on such highly reliable information, tends to generate more reliable answers to input travel-related questions than the first travel model 234. In other words, although the reliability of the answers generated by the first travel model 234 is lower than that of the second travel model 235, it can generate answers to a wider range of travel-related questions. There is a tendency to do so.

[0039] In Figure 5, the first maintenance model 231 and the second maintenance model 232 related to automobile maintenance, and the first travel model 234 and the second travel model 235 related to travel are shown as examples of learning models possessed by the response model 230. However, the response model 230 may also have other learning models besides these. Furthermore, the response model 230 may have other learning models instead of the first maintenance model 231, the second maintenance model 232, the first travel model 234, and the second travel model 235. For example, the response model 230 may have a learning model that outputs text data showing information related to health maintenance, a learning model that outputs text data showing information related to movies, a learning model that outputs text data showing information related to music, and so on.

[0040] As will be explained in more detail later, instead of using a single response model for a single input piece such as an input sentence or question, the input piece or question may be divided into multiple input contents (in a broad sense, input information) based on delimiters (e.g., punctuation marks, line breaks, etc.), and a separate response model 230 may be used for each separated input content.

[0041] Returning to Figure 4, the management database 210 is a database that manages the characteristics of each model possessed by the response model 230. Figure 6 shows an example of a model feature table 211 stored in the management database 210 in an embodiment. The model feature table 211 includes the items "Model," "Label," "Reliability," and "Processing Speed." "Model" stores information indicating each model possessed by the response model 230. "Label" stores information indicating the field of the training data used to build the model. In other words, "Label" can be said to store fields suitable for answers provided by the model. Note that "Label" may store labels indicating multiple fields. "Reliability" stores information indicating the reliability of the answers generated by the model. "Processing Speed" stores information indicating the speed from when a question is entered until an answer is generated. Note that the information stored in the model feature table 211 is not limited to the information exemplified in Figure 6. The model feature table 211 may also include, for example, the amount of training data used to build the model, information indicating the usage fee for the model, etc. The information stored in "Label," "Reliability," and "Processing Speed" is an example of "feature information indicating characteristics related to response accuracy." The "Label" in Model Feature Table 211 is an example of "first domain information."

[0042] Returning to Figure 4, the login processing unit 200 performs the login process for user U1. The login processing unit 200 performs the login process using, for example, a username and password.

[0043] The first analysis unit 201 receives input text exemplified by a question from user U1 and an answer from answer model 230, and generates an input text thread. An input text is an example of input information, containing one or more words, and is, for example, a sentence that is input to the first analysis unit 201 and is the subject of analysis by the first analysis unit 201. Examples of input text include a question entered by user U1, an inquiry output by the confirmation unit 202 (described later), and an answer from user U1 to the inquiry. The first analysis unit 201 may also receive input information that includes data other than text. Examples of data other than text included in the input information include sound data, video data, image data, etc. A thread includes a series of question texts and answer texts. A series of question texts and answer texts includes, for example, a question entered by user U1, an inquiry generated by the confirmation unit 202 (described later) in response to the question text, an answer entered by user U1 in response to the inquiry text, and an answer obtained by the answer acquisition unit 207 (described later). The thread may include additional questions from user U1 regarding the answer obtained by the answer acquisition unit 207. The first analysis unit 201 stores the generated thread in the auxiliary storage unit 23, for example, associating it with the username of user U1 who has been logged in by the login processing unit 200. To make someone do it.

[0044] Furthermore, the first analysis unit 201 identifies the language of the input sentence. For example, the first analysis unit 201 identifies whether the input sentence is in Japanese, English, German, etc. The first analysis unit 201 may include, for example, a learning model that outputs text data indicating the language used in a sentence when a sentence is input, and may use this learning model to identify the language.

[0045] The first analysis unit 201 then identifies the meaning of each word contained in the input sentence. The first analysis unit 201 decomposes the input sentence into morphemes (e.g., words) by performing morphological analysis on the question sentence. The first analysis unit 201 may further identify the part of speech of each morpheme by morphological analysis. The first analysis unit 201 inputs each word contained in the input sentence into the word model 220 and identifies its meaning. The first analysis unit 201 should select the word model 220 to use for identifying the meaning of words according to the language of the identified input sentence. For example, if the language of the identified input sentence is Japanese, the first analysis unit 201 will use the Japanese word model 220 to identify the meaning of the words. The first analysis unit 201 identifies words whose meaning cannot be identified even when input into the word model 220 as unknown words. The first analysis unit 201 may also identify words that cannot be properly clustered by the second analysis unit 204, described later, as unknown words. The processing performed by the first analysis unit 201 is an example of the "first specific processing".

[0046] The verification unit 202 generates a query statement that inquires with user U1 about the meaning of the unknown word identified by the first analysis unit 201. That is, the verification unit 202 generates a query statement that inquires about the meaning of a word that could not be identified by the first analysis unit 201. The verification unit 202 may, for example, store a standard phrase such as "What does this word '●●' mean?" in the auxiliary storage unit 23 in advance, and generate a query statement by replacing "●●" with the unknown word. Alternatively, if the answer model 230 outputs an answer statement indicating missing information, the verification unit 202 may use that answer statement as the query statement. The verification unit 202 outputs the query statement to user terminal 3. The processing by the verification unit 202 is an example of the process of "outputting query information". The query statement is an example of "query information".

[0047] Furthermore, in addition to generating the above-mentioned query sentences, the verification unit 202 can also generate query sentences to resolve unknown elements (words, context, etc.) in generating the answer sentence (first answer information) by the output process described later, even if the meaning of a word has been identified by the first identification process, but the meaning of that word does not allow for or makes it difficult to output the answer sentence (first answer information). At least one of these query sentences is sufficient to be generated by these generation processes.

[0048] When the Unknown Word Identification Unit 203 receives the response text entered by user U1 in response to the inquiry text output by the Confirmation Unit 202, it analyzes the received response text. For example, the Unknown Word Identification Unit 203 causes the First Analysis Unit 201 to perform morphological analysis of the received response text and identify the meanings of the words contained in the response text. The Unknown Word Identification Unit 203 obtains the meaning of the response text using the results of the word meaning identification by the First Analysis Unit 201. Based on the obtained meaning of the response text, the Unknown Word Identification Unit 203 identifies the meaning of the unknown word. The response text entered by user U1 in response to the inquiry text is an example of "second response information".

[0049] The unknown word identification unit 203 may perform additional learning processing on the word model 220 using the correspondence between the meaning of the identified unknown word and the unknown word itself as training data. Alternatively, the unknown word identification unit 203 may store the unknown word, the meaning of the identified unknown word, and the input sentence containing the unknown word in the auxiliary storage unit 23, associating them with each other. If the meaning of the unknown word cannot be identified even with the meaning of the response sentence obtained by the unknown word identification unit 203, the output of a further inquiry sentence may be confirmed. The recognition unit 202 may perform the operation. Furthermore, if the meaning of the unknown word cannot be determined by the meaning of the response to the inquiry, the unknown word identification unit 203 may instruct the output unit 208 to output "cannot respond" to the user terminal 3.

[0050] The second analysis unit 204 grasps the general outline of the input text based on morphological analysis, assigns labels to the input text (also called setting labels), and performs clustering. For example, the second analysis unit 204 grasps the general outline of what the input text is about based on the results of morphological analysis by the first analysis unit 201. For example, the second analysis unit 204 can determine whether the input text is a question about travel or a question about car maintenance. Furthermore, by performing clustering, it is possible to grasp the overall picture of the content of the input text, for example, according to Grice's postulate.

[0051] The second analysis unit 204 analyzes the meaning of the input sentence, for example, using the meanings of words identified by the first analysis unit 201 and the meanings of unknown words identified by the unknown word identification unit 203. In the analysis by the second analysis unit 204, for example, labels representing the meaning of each morpheme (word or sentence) decomposed by morphological analysis by the first analysis unit 201 are assigned. For example, the meanings of words identified by the word model 220 may be used as labels.

[0052] The second analysis unit 204 clusters each morpheme in the input sentence so that morphemes with the same label belong to the same cluster. Based on the clustering results, the second analysis unit 204 obtains the meaning of the entire question sentence. The meaning of the entire input sentence analyzed by the second analysis unit 204 may include, for example, the field the input sentence is in, whether the input sentence is positive or negative, whether the question sentence seeks highly reliable information, or whether it seeks information that is useful even if it is not highly reliable. Based on the obtained meaning of the question sentence, the second analysis unit 204 assigns a label to the question sentence. The second analysis unit 204 may also obtain the length of the input sentence. The length of the sentence may be, for example, the number of characters in the input sentence. Alternatively, the length of the sentence may be the number of words in the input sentence. Furthermore, there may be cases where clustering by the second analysis unit 204 is not performed correctly. An example of a case where clustering is not performed correctly is when a cluster is created in which only a small number of morphemes (for example, one) are classified. The labels assigned to the question sentence are an example of "second field information".

[0053] The selection unit 205 refers to the model feature table 211 based on the analysis results by the second analysis unit 204 and selects one or more answer models 230 to be used to answer the question. For example, if the second analysis unit 204 analyzes that the label assigned to the question is "automobile maintenance", the selection unit 205 can select the first maintenance model 231, the second maintenance model 232, and the third maintenance model 233, which include "automobile maintenance" in the "label" in the model feature table 211, as candidates for learning models. The answer models 230 selected by the selection unit 205 are examples of "selected learning models".

[0054] Furthermore, if the second analysis unit 204 analyzes that the reliability of the answer requested by the question is high, the selection unit 205 may select the second maintenance model 232 from the first maintenance model 231, the second maintenance model 232, and the third maintenance model 233, as it is the most reliable. Alternatively, if the second analysis unit 204 analyzes that the question is seeking a large amount of useful information, even if it is not highly reliable, the selection unit 205 may select all three of the first maintenance model 231, the second maintenance model 232, and the third maintenance model 233, all of which have "automobile maintenance" in their labels, as the learning models to use in answering the question. The process of the selection unit 205 selecting one or more answer models 230 is an example of a "selection process".

[0055] Here, the selection unit 205 may calculate the degree of association between the label assigned to the question by the second analysis unit 204 and the "label" in the model feature table 211, and select the answer model 230 to be used to answer the question by prioritizing the answer model 230 with the highest calculated degree of association. For example, the selection unit 205 may select the answer model 230 with the highest calculated degree of association as the answer model 230 to be used to answer the question. Alternatively, the selection unit 205 may select one or more answer models 230 to be used to answer the question in order of the highest calculated degree of association. There are no limitations on the method of calculating the degree of association. For example, the selection unit 205 may calculate the similarity between the label of the question assigned by the second analysis unit 204 and the word (or sentence) that represents the label stored in the "label" in the model feature table 211, and use the calculated similarity as the degree of association. Various known techniques can be applied to calculate the similarity of words (or sentences). The degree of association is an example of "field association".

[0056] The analysis result input unit 206 inputs the analysis results from the second analysis unit 204 into the answer model 230 selected by the selection unit 205. When the analysis result input unit 206 receives question information from the user terminal 3, it may also input data other than text included in the question information into the answer model 230. The analysis result input unit 206 may, for example, process the analysis results from the second analysis unit 204 to make it easier to obtain an answer from the answer model 230, and input the processed analysis results into the answer model 230. In the processing by the analysis result input unit 206, for example, the analysis results are processed so that instructions for the answer model 230 are placed at the beginning, followed by supplementary information. When multiple answer models 230 are selected by the selection unit 205, the analysis result input unit 206 inputs the analysis results from the second analysis unit 204 into each of the selected multiple answer models 230. Furthermore, when inputting the analysis results from the second analysis unit 204 into multiple response models 230, the analysis result input unit 206 may, for example, input the analysis results from the second analysis unit 204 in order from the response model 230 with the highest relevance calculated by the selection unit 205 to the response model 230 with the lowest relevance.

[0057] The response acquisition unit 207 acquires the response text generated by the response model 230. If multiple response models 230 are selected by the selection unit 205, the response acquisition unit 207 acquires the response text from the selected multiple response models 230. The response acquisition unit 207 may process the response text from the acquired response models 230. For example, processing by the response acquisition unit 207 may involve combining or summarizing the multiple response texts obtained from multiple response models 230.

[0058] Examples of how the response acquisition unit 207 synthesizes response texts include a method of directly concatenating response texts from multiple response models 230, and a method of arranging and synthesizing response texts from response models 230 with a higher degree of relevance to be prioritized. Examples of methods for prioritizing response texts from response models 230 with a higher degree of relevance include a method of placing the response text from the response model 230 with a higher degree of relevance at the beginning of the response text to be output by the output unit 208, and a method of setting the size of the response text from the response model 230 with a higher degree of relevance to be larger than the response text from the response model 230 with a lower degree of relevance. In addition, if the response acquisition unit 207 has acquired response texts from multiple response models 230, it may summarize these multiple response texts. Once the response acquisition unit 207 has finished processing the response texts, it instructs the output unit 208 to output the processed response texts. Furthermore, the response acquisition unit 207 adds the query output by the confirmation unit 202, the response from user U1 to the query, and the processed response to a thread associated with user U1's username.

[0059] Furthermore, the response acquisition unit 207 may assign labels to each of the response sentences from multiple response models 230 by performing analysis using the first analysis unit 201 and the second analysis unit 204. Then, for each response sentence, the response acquisition unit 207 calculates the degree of association (response relevance) between the label assigned to the question sentence and the label assigned to the response sentence, and the response with the highest calculated response relevance The text may be arranged and combined in a way that prioritizes sentences. Furthermore, the response acquisition unit 207 may suppress the output by the output unit 208 for response sentences whose response relevance is below a predetermined threshold.

[0060] When the response text is obtained by the response acquisition unit 207, the selection unit 205 evaluates the response model 230 that generated the response text. The selection unit 205 assigns a label to the response text generated using the second analysis unit 204. The generated response text may also be assigned a specific label (hereinafter also referred to as a "priority label") that indicates the specialized content (specialized content) or specialized items (specialized items) (for example, various fields, information types, etc.) of the response model 230 used to generate the response text. The priority label may be, for example, information indicating a field that is an example of the specialized content of the response model 230 (for example, the automobile maintenance field, the travel field, etc.), or information that the response model 230 requires for the generation of the response text (for example, in the case of the automobile maintenance field, information indicating the vehicle type to be serviced, the parts to be serviced, etc.). The priority labels are stored in advance in the auxiliary storage unit 23, for example, associated with each of the response models 230. There may be one or more priority labels associated with each of the response models 230. The selection unit 205 then assigns priority labels to the answer text, for example, by assigning priority labels associated with the answer model 230 that generated the answer text to the answer text. The selection unit 205 compares the labels assigned to the answer text with the labels assigned to the question text and calculates an evaluation value for the answer model 230. For example, the selection unit 205 calculates a higher evaluation value the more labels that match between the labels assigned to the answer text and the labels assigned to the question text. The number of matching labels is also called the degree of agreement. Other examples of when the selection unit 205 calculates a high evaluation value include when the percentage of labels that match between the labels assigned to the answer text and the labels assigned to the question text is high, and when the priority labels assigned to the answer text match between the labels assigned to the question text. For example, the selection unit 205 may calculate a higher evaluation value the higher the percentage of labels that match between the labels assigned to the answer text and the labels assigned to the question text. Furthermore, the selection unit 205 may, for example, calculate the highest evaluation value if the priority label assigned to the answer statement matches the label assigned to the question statement.The highest evaluation value is calculated for the answer model 230 whose question label matches the priority label. This makes it easier for the selection unit 205 to select an answer from an answer model 230 that matches the field of the question, thereby improving the accuracy of the answers to the question. The selection unit 205 may update the "reliability" item in the model feature table 211 based on the calculated evaluation value. If the evaluation value of the answer model 230 that generated the answer is low, the selection unit 205 may select another answer model 230 and input the analysis result into the analysis result input unit 206.

[0061] The output unit 208 transmits the response text obtained by the response acquisition unit 207 to the user terminal 3. If response information has been obtained by the response acquisition unit 207, the output unit 208 may also transmit the obtained response information to the user terminal 3. If the response text obtained by the response acquisition unit 207 includes information indicating the location of materials related to the response text, the output unit 208 may also output information indicating the location of such materials. The billing unit 209 calculates the usage fee for the response system 1 based on the amount of text in the question text analyzed by the analysis result input unit 206 and bills user U1. The amount of text in the question text is an example of "text length information".

[0062] <Processing flow of response system 1> Figures 7 to 9 show an example of the processing flow of the answer device 2 according to the embodiment. The following description of the example of the processing flow of the answer device 2 will refer to Figures 7 to 9.

[0063] In step S1, the login process is performed. The login processing unit 200 executes the login process for user U1 based on the username and password sent from user terminal 3, for example.

[0064] In step S2, the first analysis unit 201 determines whether it has received an input sentence, either a question sentence from the user terminal 3 or an answer sentence from the answer model 230. If an input sentence has been received (YES in step S2), the process proceeds to step S3. If an input sentence has not been received (NO in step S2), the process in step S2 is repeated.

[0065] In step S3, the first analysis unit 201 generates a thread containing the input text received in step S2. The first analysis unit 201 associates the generated thread with the username used for login in step S1 and stores it in the auxiliary storage unit 23. In step S4, the first analysis unit 201 identifies the language of the input text received in step S2.

[0066] In step S5, the first analysis unit 201 performs morphological analysis of the input sentence received in step S2 based on the language identified in step S4. The first analysis unit 201 further inputs each word contained in the input sentence received in step S2 into the word model 220 selected based on the language identified in step S4, and identifies the meaning of the word.

[0067] In step S6, the second analysis unit 204 assigns a label representing the meaning of each morpheme decomposed by the morphological analysis in step S5. The second analysis unit 204 then clusters the morphemes in the input sentence so that morphemes with the same label belong to the same cluster.

[0068] In step S7, the verification unit 202 generates a query statement if there are any questions. For example, if there is a word whose meaning could not be determined in step S5, the verification unit 202 generates a query statement about the meaning of that word. Also, if there is a word that could not be clustered successfully in step S6, the verification unit 202 generates a query statement about that word. If a query statement is generated (YES in step S7), the process proceeds to step S2, and the processing from step S2 onward is executed for the generated query statement. If no query statement is generated (NO in step S7), the process proceeds to step S8.

[0069] In step S8, the response acquisition unit 207 determines whether the input sentence received in step S2 is a response sentence from the response model 230. If it is a response sentence from the response model 230 (YES in step S8), the process proceeds to step S9. If it is not a response sentence from the response model 230 (NO in step S8), the process proceeds to step S14.

[0070] In step S9, the response acquisition unit 207 processes the response text received from the response model 230 in step S2. Processing of the response text by the response acquisition unit 207 can include combining and summarizing response texts from multiple response models 230. Once the response acquisition unit 207 has completed processing the response text, it instructs the output unit 208 to output the processed response text.

[0071] In step S10, the selection unit 205 evaluates each of the response models 230 that output the response sentences received in step S2.

[0072] In step S11, the output unit 208 determines whether or not it has received an output instruction for the response text from the response acquisition unit 207 in step S9. If it has received an output instruction (YES in step S11), the process proceeds to step S12. If it has not received an output instruction (NO in step S11), the process proceeds to step S14.

[0073] In step S12, the output unit 208 outputs the response text processed in step S9 to the user terminal 3. The user terminal 3 then outputs the response text processed in step S9 to the response device 2. Upon receiving the request, the received response is output to the output unit 36.

[0074] In step S13, the response acquisition unit 207 adds the query output by the confirmation unit 202, the response from user U1 to the query, and the processed response to the thread associated with user U1's username.

[0075] In step S14, the analysis result input unit 206 processes the input sentence received in step S2 into a format that makes it easier to obtain a response from the response model 230.

[0076] In step S15, the selection unit 205 calculates the degree of association between the label assigned to the input sentence received in step S2 and the "label" in the model feature table 211.

[0077] In step S16, the selection unit 205 selects one or more response models 230 based on the relevance calculated in step S15.

[0078] In step S17, the analysis result input unit 206 inputs the input sentence processed in step S14 into the response model 230 selected in step S16.

[0079] In step S18, the response acquisition unit 207 determines whether or not it has acquired response sentences from all response models 230 selected in step S16. If it has acquired them (YES in step S18), the process proceeds to step S4, and the processing from step S4 onward is executed on the acquired response sentences. If there are response models 230 that have not been acquired (NO in step S18), the process proceeds to step S17.

[0080] Figure 10 shows an example of the processing flow of the text analysis process performed by the first analysis unit 201 of the answering device 2 according to the embodiment. The processing flow exemplified in Figure 10 corresponds to the processing in steps S3 to S4 of Figure 7. Hereinafter, an example of the processing flow of the text analysis process performed by the first analysis unit 201 will be described with reference to Figure 10.

[0081] In step S21, the first analysis unit 201 receives input sentences exemplified by the question sentence from user U1 and the answer sentence from the answer model 230, and generates threads for the input sentences. The first analysis unit 201 stores the generated threads in the auxiliary storage unit 23, associating them with the username of user U1.

[0082] In step S22, the first analysis unit 201 identifies the language of the question statement that generated the thread in step S21. The first analysis unit 201 may, for example, associate the identified language with the username of user U1 and the thread generated in step S21 and store this information in the auxiliary storage unit 23.

[0083] Figure 11 shows an example of the processing flow of the query statement generation process by the verification unit 202 of the answering device 2 according to the embodiment. The processing flow exemplified in Figure 11 corresponds to the process of step S7 in Figure 7. Hereinafter, an example of the processing flow of the query statement generation process by the verification unit 202 will be described with reference to Figure 11.

[0084] In step S31, the verification unit 202 determines whether the clustering by the second analysis unit 204 was successful. If the clustering was successful (YES in step S31), the process is terminated. If the clustering was not successful (NO in step S31), the process proceeds to step S32.

[0085] In step S32, the verification unit 202 determines that clustering could not be performed successfully in step S31. The system generates a query about the meaning of the word. In step S33, the verification unit 202 associates the query generated in step S32 with the words that could not be successfully clustered in step S31 and stores them in the auxiliary storage unit 23.

[0086] Figures 12 and 13 show an example of the labeling process flow by the first analysis unit 201 and the second analysis unit 204 of the answering device 2 according to the embodiment. The process flow illustrated in Figures 12 and 13 corresponds to the process in step S6 of Figure 7. Hereinafter, an example of the labeling process flow by the first analysis unit 201 and the second analysis unit 204 will be described with reference to Figure 12.

[0087] In step S41, the second analysis unit 204 grasps the general outline of the input sentence based on the results of morphological analysis performed by the first analysis unit 201.

[0088] In step S42, the first analysis unit 201 inputs each word in the input sentence into the word model 220 to determine its meaning.

[0089] In step S43, the second analysis unit 204 uses the meaning of the word identified in step S42 to assign a label representing the meaning of each morpheme decomposed by the morphological analysis performed by the first analysis unit 201.

[0090] In step S44, the second analysis unit 204 determines whether the thread contains an input statement other than the current input statement which is the subject of analysis by the first analysis unit 201 and the second analysis unit 204. Examples of other input statements include the query statement from the confirmation unit 202 and the response statement to that query statement. If there is another input statement (YES in step S44), the process proceeds to step S45. If there is no other input statement (NO in step S44), the process proceeds to step S46.

[0091] In step S45, the second analysis unit 204 determines whether the current input sentence and other input sentences are positive or negative. In step S46, the second analysis unit 204 determines whether the current input sentence is positive or negative.

[0092] In step S47, the second analysis unit 204 assigns a label to the input sentence indicating whether it is positive or negative, as determined in step S45 or step S46.

[0093] In step S48, the second analysis unit 204 performs clustering of each morpheme based on the labels assigned in steps S43 and S47. If clustering is completed successfully (YES in step S49), the process proceeds to step S51. If clustering is not completed successfully (NO in step S49), the process is terminated.

[0094] In step S51, the second analysis unit 204 assigns a label to the input text indicating the attributes of user U1, who was accepted for login in step S1 of Figure 7. These attributes may include user U1's age, gender, nationality, and place of residence. User U1's attributes are stored in the auxiliary storage unit 23, for example, in association with the username, when the user is registered with the response device 2.

[0095] In step S52, the second analysis unit 204 stores the input sentence, the words whose meanings have been identified, and the assigned labels in the auxiliary storage unit 23, associating them with each other.

[0096] Figures 14 to 16 show an example of the processing flow for the selection of the answer model 230 by the second analysis unit 204 and the selection unit 205 of the answer device 2. The processing in Figures 14 to 16 corresponds to the processing in step S16 of Figure 9. Hereinafter, an example of the processing flow for the selection of the answer model 230 by the second analysis unit 204 and the selection unit 205 will be described with reference to Figures 14 to 16.

[0097] In step S61, the selection unit 205 determines whether or not it is an input to the answer model 230. If it is an input to the answer model 230 (YES in step S61), the process proceeds to step S62. If it is not an input to the answer model 230 (NO in step S61), the process proceeds to step S68.

[0098] In step S62, the selection unit 205 extracts candidate response models 230 to be used for the response based on the clustering results and assigned labels from the second analysis unit 204.

[0099] In step S63, the selection unit 205 selects an answer model 230 to be used for the answer from the candidates extracted in step S62. If there are multiple selected answer models 230 (YES in step S64), the process proceeds to step S65. If there are not multiple selected answer models 230 (NO in step S65), the process proceeds to step S67.

[0100] In step S65, the second analysis unit 204 determines the answer requested by the question based on the assigned labels and clustering results. Examples of answers requested by the question include wanting multiple helpful answers, wanting a more accurate answer, etc.

[0101] In step S66, the selection unit 205 associates the input sentence received in step S2 in Figure 7 with the answer model 230 selected in step S63 and stores them in the auxiliary storage unit 23.

[0102] In step S67, the selection unit 205 calculates the degree of association between the response model 230 selected in step S63 and the input sentence received in step S2 of Figure 7.

[0103] In step S68, the selection unit 205 determines whether or not to switch the answer model 230. If a switch is made (YES in step S68), the process proceeds to step S69. If no switch is made (NO in step S68), the process proceeds to step S61.

[0104] In step S69, the selection unit 205 determines whether or not there is an answer model 230 for which the analysis result input unit 206 has not yet entered the analysis result. If such a model exists (YES in step S69), the process proceeds to step S70. If such a model does not exist (NO in step S69), the process ends.

[0105] In step S70, the selection unit 205 selects, for example, a response model 230 from among the response models 230 selected in step S63 that have not yet had analysis results entered, according to the degree of relevance calculated in step S67, and prompts the analysis result input unit 206 to input the analysis results.

[0106] Figure 16 is a diagram showing an example of the processing flow for evaluating the answer model 230 by the selection unit 205 of the answer device 2 according to the embodiment. The processing in Figure 16 corresponds to the processing in step S10 of Figure 8. Hereinafter, an example of the processing flow for evaluating the answer model 230 by the selection unit 205 will be described with reference to Figure 16.

[0107] In step S81, the selection unit 205 selects the input statement stored in the auxiliary storage unit 23 and the input Retrieve the labels assigned to the force statement.

[0108] In step S82, the selection unit 205 retrieves the answer text and the label assigned to the answer from the answer model 230 stored in the auxiliary storage unit 23.

[0109] In step S83, the selection unit 205 compares the labels obtained in step S81 with the labels obtained in step S82 to calculate an evaluation value for the response model 230 that generated the response sentence obtained in step S82. The selection unit 205 may calculate a higher evaluation value if, for example, there are more matches between the labels assigned to the response sentence and the labels assigned to the question sentence. Alternatively, the selection unit 205 may calculate a higher evaluation value if, for example, the proportion of matches between the labels assigned to the response sentence and the labels assigned to the question sentence is high. The selection unit 205 may also calculate a higher evaluation value if, for example, the priority label assigned to the response sentence matches the label assigned to the question sentence. If the calculated evaluation value is above the threshold (YES in step S84), the process proceeds to step S85. If the calculated evaluation value is below the threshold (NO in step S84), the process proceeds to step S86.

[0110] In step S85, the selection unit 205 instructs the output unit 208 to output the response text output by the response model 230. In step S86, the selection unit 205 switches the response model 230 used for the response to another response model 230 selected in step S63 of Figure 14.

[0111] <Examples> The embodiments described above will now be explained based on more specific examples. In this embodiment, we will describe an example in which a question about travel is received and the answering device 2 answers the question.

[0112] Figure 17 shows an example of a user screen 301 displayed on the output unit 36 ​​of the user terminal 3 according to the embodiment. The user screen 301 is used for inputting a question by user U1 and outputting an inquiry and answer from the user terminal 3. The user screen 301 includes a history area 302 and a question / answer area 303. In addition, the user screen 301 displays a user icon 304 to indicate that the input from user U1 is an input from user U1. In addition, the user screen 301 displays an answer device icon 305 (see Figure 18) to indicate that the output from the answer device 2 is an output from the answer device 2. The history area 302 displays the history of items that have been asked so far. The question / answer area 303 is used for inputting a question and displaying an inquiry and answer from the user terminal 3. In the example in Figure 17, the state in which the question "I would like to go on a hot spring trip in January next year, where do you recommend?" has been input by user U1 is illustrated. The entered question text is sent from the user terminal 3 to the answering device 2 via network N1.

[0113] The first analysis unit 201 generates a thread for the received question text. The first analysis unit 201 also identifies the language of the received question text, for example, as explained in step S4 of Figure 7 (step S4 of Figure 7). The first analysis unit 201 also performs morphological analysis on the question text. Table 1 below is an example of the results of morphological analysis. In Table 1, "Written form" stores the words extracted from the question text by morphological analysis. "Lexime" stores the notation that groups the written form of the word regardless of differences in inflection. "Lexime reading" stores the katakana notation that indicates the reading of the lexime. "Part of speech" stores information indicating the part of speech of the morpheme. The first analysis unit 201 may store the results of morphological analysis in the auxiliary storage unit 23, associating them with "Written form," "Lexime," "Lexime reading," and "Part of speech" as shown in Table 1. [Table 1]

[0114] The second analysis unit 204 assigns a label to each morpheme of the question sentence. The second analysis unit 204 extracts lexemes from the morphological analysis results, for example, morphemes whose part of speech is a noun, verb, sentence-ending particle, or auxiliary symbol. In this embodiment, "next year," "one," "month," "hot spring," "travel," "go," "recommend," "or," and "?" are extracted. The second analysis unit 204 may, for example, instruct the first analysis unit 201 to input each extracted morpheme into the word model 220 and obtain the meaning of each morpheme.

[0115] Furthermore, the second analysis unit 204 may recognize that "next year," "one," and "month" are linked together to form "January 2024." Then, based on the meanings obtained from the word model 220 and the meanings of the auxiliary symbols, the second analysis unit 204 assigns the label "hot spring" to "hot spring," the label "travel" to "travel," the labels "next year" and "January" to "January 2024," the labels "hope" to "go," "want," and "want," the labels "recommend" to "o" and "susume," and the labels "question form" to "desu," "ka," and "?". Then, based on these assigned labels, the second analysis unit 204 assigns the labels "travel plan search" and "travel plan suggestion" to the entire question sentence.

[0116] The selection unit 205 recognizes, based on the analysis by the second analysis unit 204, that the question relates to travel plan search and travel plan suggestion. The selection unit 205 then considers the question and the questions contained within the question. The degree of association between the label assigned to each morpheme and the label associated with each of the response models 230 in the model feature table 211 is calculated, and the response model 230 to be used to generate the response to the question is selected based on the calculated degree of association. If the model feature table 211 is in the state shown in Figure 6, then the first travel model 234, which has matching labels "travel" and "hot springs," is selected from among the response models 230.

[0117] When the analysis results from the second analysis unit 204 are input to the first travel model 234 via the analysis result input unit 206, the first travel model 234 outputs a response message. Here, if there are many matching travel plans (above a predetermined threshold), the first travel model 234 outputs a response message indicating that the conditions are insufficient. In this case, it is assumed that the response message obtained from the first travel model 234 is, "The conditions such as desired area and budget are insufficient. There are many matching plans."

[0118] Here, the verification unit 202 outputs, for example, the response text obtained from the first travel model 234 as an inquiry text to user U1 to user terminal 3. User terminal 3 displays the inquiry text received from the response device 2 on user screen 301. Figure 18 is an example of the state in which the inquiry text received from the response device 2 is displayed on user screen 301. In the example of Figure 18, the inquiry text displayed is, "Your desired area, budget, and other conditions are insufficient. There are many matching plans." By displaying the inquiry text from the response device 2 on user screen 301, the response device 2 can prompt user U1 to supplement the missing information in the inquiry text.

[0119] Figure 19 illustrates a user screen 301 in which user U1 has entered a response to an inquiry from response device 2. In the example in Figure 19, the response entered is, "I'm thinking of a 2-night, 3-day trip to the Kanto area. I'm hoping for a hot spring with good water quality." The entered response is sent from user terminal 3 to response device 2 via network N1.

[0120] The answering device 2 adds the received answer to the thread. The answering device 2 then understands the meaning of the answer to the received question through processing by, for example, the first analysis unit 201 and the second analysis unit 204. Based on the question and answer contained in the thread, the answering device 2 selects an answer model 230 by the selection unit 205 and outputs the answer. Figure 20 is an example of the state in which the answer from the answering device 2 is displayed on the user screen 301. In the example of Figure 20, the answer output is, "If you are looking for a hot spring with good water quality near the Kanto region, the following areas are recommended: 1. Hakone: Famous for its sulfur springs..."

[0121] In this way, the user U1 inputs the answer to the question received from the answering device 2 and sends it back to the answering device 2. By repeating this exchange, the answering device 2 can output an answer that more accurately meets the user U1's requirements.

[0122] <Effects of the Embodiment> In this embodiment, the answering device 2 outputs a query when the question received from the user terminal 3 contains an unknown word whose meaning cannot be determined, or when clustering cannot be performed properly. The answering device 2 then understands the meaning of the question based on the answer from the user terminal 3 to the query. Through this exchange of query and answer statements, the answering device 2 can understand the meaning of the question from user U1 with higher accuracy. Consequently, the answering device 2 can improve the accuracy of the answer to a question that contains an unknown word.

[0123] In this embodiment, when the meaning of an unknown word is identified, the answering device 2 uses the identified unknown word and its meaning as training data to perform additional learning processing on the word model 220. Furthermore, the response device 2 can expand the range of words that can be identified by the word model 220. Consequently, the response device 2 can reduce the workload of user U1 in providing answers to the inquiry sentence.

[0124] In this embodiment, the answering device 2 stores the unknown word, the meaning of the identified unknown word, and the input sentence containing the unknown word in association with each other in the auxiliary storage unit 23. Therefore, even if no additional learning process is performed on the unknown word for the word model 220, the meaning of the unknown word that could not be identified by the word model 220 can be identified by referring to the auxiliary storage unit 23. Consequently, the answering device 2 can reduce the workload of having the user U1 provide an answer to the inquiry sentence.

[0125] In this embodiment, if the meaning of an unknown word cannot be determined even by the unknown word identification unit 203, the answering device 2 may notify the user terminal 3 that it cannot answer. Therefore, the answering device 2 can prevent outputting an answer that does not conform to the intent of the question.

[0126] In this embodiment, the answering device 2 also outputs information indicating the location of any materials related to the answer, if such materials exist. Therefore, in addition to the answer, the answering device 2 can present to user U1 any materials that may be helpful as an answer to the question.

[0127] In this embodiment, the characteristics of each model in the answer model 230 are managed by the model feature table 211. The answer device 2 then selects an answer model 230 to input the analysis results from the second analysis unit 204, based on the labels assigned to the question by the second analysis unit 204 and the labels associated with each model in the model feature table 211. The labels assigned to the question indicate the field of the question, and the labels associated with each model indicate the field of the training data for that model. Therefore, the answer device 2 can select an answer model 230 that is preferable for generating an answer to a question.

[0128] In this embodiment, the answering device 2 calculates the degree of relevance between the labels assigned to the question and the labels associated with each model in the model feature table 211, and prioritizes selecting the answering model 230 with the highest calculated relevance. Therefore, the answering device 2 can select the answering model 230 that is more relevant to the question as the answering model 230 for generating the answer. Consequently, the answering device 2 can improve the accuracy of the answer to the question. Furthermore, if multiple answering models 230 are selected for generating the answer, the answering device 2 inputs the analysis results in order from the answering model 230 with the highest relevance. Therefore, the answering device 2 can obtain an answer with higher accuracy at an earlier stage.

[0129] In this embodiment, the answering device 2 calculates the degree of relevance between the question and the answer obtained from the answer model 230, and prioritizes outputting answer sentences with a high calculated relevance. Furthermore, if the calculated relevance is below a predetermined threshold, the answering device 2 suppresses the output of that answer sentence. As a result, the answering device 2 can present the user U1 with answer sentences that are more relevant to the question.

[0130] In this embodiment, when the answering device 2 obtains answer sentences from multiple answer models 230, it synthesizes the multiple answer sentences. Therefore, the answering device 2 can display a list of the answer sentences obtained from the multiple answer models 230 to the user U1.

[0131] In this embodiment, the answering device 2 calculates the usage fee for the answering system 1 based on the length of the question text and charges user U1. Therefore, the answering device 2 can charge user U1 according to how the answering system 1 is used (usage amount).

[0132] <Variation> In the embodiments described above, the analysis result input unit 206 inputs the analysis result to the answer model 230, and the answer acquisition unit 207 acquires the answer text from the answer model 230. The processing is performed sequentially. However, the process of inputting the analysis result (an example of the "first analysis result") of the question (an example of the "first question") by the analysis result input unit 206 into the answer model 230, and the process of obtaining the answer (an example of the "first answer information") from the answer model 230 for the analysis result of the question (an example of the "second question") that has already been input into the answer model 230, may be executed in parallel. By executing in parallel, the answer device 2 can shorten the time it takes to generate the final answer to the question from user U1. Parallel execution in this context is also called multitasking, and refers to a state where the input process and the acquisition process are performed simultaneously, even if only temporarily. It does not mean that the two processes started at the same time, but they may start at the same time.

[0133] In the embodiments described above, the response device 2 and the user terminal 3 were separate devices, but the response device 2 and the user terminal 3 may be implemented by a single information processing device.

[0134] The embodiments and variations disclosed above can be combined in any way.

[0135] <Computer-readable recording medium> An information processing program that enables a computer or other machine or device (hereinafter referred to as "computer, etc.") to perform any of the above functions can be recorded on a recording medium that the computer, etc. can read. By having the computer, etc. read and execute the program on this recording medium, it can be made to provide that function.

[0136] Here, a recording medium that can be read by a computer refers to a recording medium that stores information such as data and programs through electrical, magnetic, optical, mechanical, or chemical means and can be read by a computer. Examples of such recording media that can be removed from a computer include flexible disks, magneto-optical disks, Compact Disc Read Only Memory (CD-ROM), Compact Disc-Recordable (CD-R), Compact Disc-ReWriterable (CD-RW), Digital Versatile Disc (DVD), Blu-ray Disc (BD), Digital Audio Tape (DAT), 8mm tape, flash memory, external hard disk drives, and Solid State Drives (SSDs). In addition, recording media that are fixed to a computer include internal hard disk drives, SSDs, and ROMs.

[0137] The learning described above involves inputting data patterns into a computer based on training data, which consists of information paired with correct judgments. The computer then uses these patterns to discover new features and relationships. Another example is the computer's ability to identify commonalities and characteristic data within the input data. In addition, learning can be exemplified by methods such as deep learning, in which a computer constructs a neural network to analyze, extract, and output local or characteristic data. Of course, this learning also includes preprocessing, which consists of processes such as supplementing missing data, removing unnecessary data, and standardizing data formats, as well as postprocessing, which stores the learned results so that they can be used for other learning.

[0138] As used in the embodiments and claims described above, the terms “part,” “means,” “apparatus,” and “system” do not merely refer to physical means, but also include cases where the functions of these are realized by software or software services. Furthermore, the functions of a single "part," "means," "apparatus," or "system" may not only be realized by a single physical means, software, software module, or apparatus, but may also be realized by multiple physical means, software, software modules, apparatus, or combinations thereof.

[0139] The terms used in the embodiments and claims described above should be interpreted as non-limiting terms. For example, the term “includes” should be interpreted as “not limited to those described as including.” The term “contains” should be interpreted as “not limited to those described as containing.” The term “equips” should be interpreted as “not limited to those described as equipped.” The term “possesses” should be interpreted as “not limited to those described as possessing.” The term “complements” should be interpreted as “not limited to those described as possessing.”

[0140] <Note 1> A storage unit that stores multiple learning models that generate response sentences to a given text when the results of text analysis are input, and feature information that indicates the characteristics related to the accuracy of the response to the analysis results for each of the multiple learning models, A processor connected to the aforementioned storage unit, The aforementioned processor, Upon receiving the analysis results of the question text entered by the user, the system performs an input process in which the analysis results are input to one or more selected learning models chosen from the multiple learning models, The following steps are performed: output processing to output the response sentence from the selected learning model into which the analysis results have been input, The input process further includes a selection process that, by referring to the feature information, selects one or more selected learning models from the plurality of learning models to input the analysis results. Information processing device. <Note 2> The feature information includes first domain information indicating the field to which the learning model corresponds, The analysis results include second field information indicating the field of the question statement, The selection process includes calculating a field relevance score indicating the degree of relevance between the first field information and the second field information, and prioritizing the selection of the learning model with a high calculated field relevance score as the selected learning model. The information processing device described in Appendix 1. <Note 3> If multiple selected learning models are selected, the input process inputs the analysis results to the selected learning models in an order corresponding to the degree of relevance to the field. The information processing device described in Appendix 2. <Note 4> The aforementioned output processing is performed as follows: A response relevance score is calculated to show the degree of relevance between the aforementioned response and the second field information. The process further includes prioritizing the output of the aforementioned response sentences that have a high degree of relevance to the aforementioned response, The information processing device described in Appendix 2. <Note 5> The aforementioned output processing is performed as follows: A response relevance score is calculated to show the degree of relevance between the aforementioned response and the second field information. The process further includes suppressing the output of response sentences whose relevance to the response is lower than a threshold. The information processing device described in Appendix 2. <Note 6> The output process includes, when multiple selected learning models are selected in the selection process, a process for synthesizing the response sentences from the multiple selected learning models. The information processing apparatus according to claim 1. <Note 7> The analysis results include text volume information relating to the amount of text in the document, The processor further performs a calculation process to calculate the usage fee based on the text volume information. An information processing device as described in any one of the items 1 to 6 of the appendix. <Note 8> A computer comprising a storage unit that stores multiple learning models that generate response sentences to a given text when the results of text analysis are input, and feature information that indicates the characteristics related to the accuracy of the response to the analysis results for each of the multiple learning models, Upon receiving the analysis results of the question text entered by the user, the system performs an input process in which the analysis results are input to one or more selected learning models chosen from the multiple learning models, The following steps are performed: output processing to output the response sentence from the selected learning model into which the analysis results have been input, The input process further includes a selection process that, by referring to the feature information, selects one or more selected learning models from the plurality of learning models to input the analysis results. Information processing methods. <Note 9> A computer having a storage unit that stores multiple learning models that generate response sentences to a given text when the results of text analysis are input, and feature information that indicates the characteristics related to the accuracy of the response to the analysis results for each of the multiple learning models, Upon receiving the analysis results of the question text entered by the user, the system performs an input process in which the analysis results are input to one or more selected learning models chosen from the multiple learning models, The system then performs an output process that outputs a response sentence from the selected learning model into which the analysis results have been input. The input process further includes a selection process that, by referring to the feature information, selects one or more selected learning models from the plurality of learning models to input the analysis results. Information processing program. <Note 11> A storage unit that stores a first learning model that outputs text data indicating the meaning of a word when a word is input, and a second learning model that outputs answer information to a question when a question is input. A processor connected to the aforementioned storage unit, The aforementioned processor, A first identification process that identifies the meaning of each word in the question sentence entered by the user using the first learning model, Based on the meaning of the word identified in the first identification process described above, an analysis process is performed to analyze the meaning of the question sentence, Input processing involves inputting the analysis results of the question sentence obtained by the analysis process into the second learning model, An acquisition process for obtaining first answer information to the aforementioned question from the second learning model, The output process that outputs the first response information is executed, The aforementioned analysis process is In at least one of the following cases, where the first identification process cannot identify the word, or where the meaning of the word can be identified but the output process cannot output the first response information based on that meaning, query information is output to inquire about the meaning of the word. The process further includes receiving a second response to the aforementioned inquiry information, determining the meaning of the word whose meaning could not be determined by the first identification process based on the second response information, and then analyzing the meaning of the question sentence based on the meaning of the word determined by the first identification process and the meaning of the word determined based on the second response information. Information processing device. <Note 12> The aforementioned processor, Based on the word identified based on the second response information and the meaning of the word identified based on the second response information, a learning process is further executed to train the first learning model. The information processing device described in Appendix 11. <Note 13> The aforementioned processor, Further, a storage process is performed to store the second answer information in the storage unit, corresponding to the aforementioned question and the aforementioned word whose meaning could not be identified. The information processing device described in Appendix 11. <Note 14> Identifying based on the second response information further includes notifying that an answer is unavailable if the meaning of the word whose meaning could not be identified cannot be identified based on the second response information. The information processing device described in Appendix 11. <Note 15> The output process further includes a process of including location information indicating the location of materials related to the first response information into the first response information. The information processing device described in Appendix 11. <Note 16> The aforementioned processor, The system performs the following steps: inputting the first analysis result for the first question sentence into the second learning model, and obtaining the first answer information for the second question sentence, for which the analysis result has already been input into the second learning model, from the second learning model. An information processing device as described in any one of the appendices 11 to 15. <Note 17> A computer having a memory unit that stores a first learning model that outputs text data indicating the meaning of a word when a word is input, and a second learning model that outputs answer information to a question when a question is input, A first identification process that identifies the meaning of each word in the question sentence entered by the user using the first learning model, Based on the meaning of the word identified in the first identification process described above, an analysis process is performed to analyze the meaning of the question sentence, Input processing involves inputting the analysis results of the question sentence obtained by the analysis process into the second learning model, An acquisition process for obtaining first answer information to the aforementioned question from the second learning model, The output process that outputs the first response information is executed, The aforementioned analysis process is In at least one of the following cases, where the meaning cannot be identified in the first specific processing, or where the meaning of the word can be identified but the output of the first answer information by the output processing cannot be performed with that meaning, query information to inquire about the meaning of the word is output. The process further includes receiving a second response to the aforementioned inquiry information, determining the meaning of the word whose meaning could not be determined by the first identification process based on the second response information, and then analyzing the meaning of the question sentence based on the meaning of the word determined by the first identification process and the meaning of the word determined based on the second response information. Information processing methods. <Note 18> The system includes a first learning model that outputs text data indicating the meaning of a word when a word is input, and a second learning model that outputs answer information to a question when a question is input. A computer equipped with a memory unit, A first identification process that identifies the meaning of each word in the question sentence entered by the user using the first learning model, Based on the meaning of the word identified in the first identification process described above, an analysis process is performed to analyze the meaning of the question sentence, Input processing involves inputting the analysis results of the question sentence obtained by the analysis process into the second learning model, An acquisition process for obtaining first answer information to the aforementioned question from the second learning model, The output process that outputs the first response information is executed, The aforementioned analysis process is In at least one of the following cases, where the meaning cannot be identified in the first specific processing, or where the meaning of the word can be identified but the output of the first answer information by the output processing cannot be performed with that meaning, query information to inquire about the meaning of the word is output. The process further includes receiving a second response to the aforementioned inquiry information, determining the meaning of the word whose meaning could not be determined by the first identification process based on the second response information, and analyzing the meaning of the question sentence based on the meaning of the word determined by the first identification process and the meaning of the word determined based on the second response information. Information processing program. [Explanation of Symbols]

[0141] 1. Answer System 2··Answer device 3. User terminal 21. CPU 22...Main memory 23...Auxiliary storage section 24. Communications Department 31. CPU 32...Main memory 33...Auxiliary storage section 34. Communications Department 35. Input section Output section of 36·· 200 · Login Processing Unit 201...1st Analysis Department 202. Verification Section 203...Unknown word identification section 204...Second Analysis Department 205...Selection section 206 ··Analysis Result Input Section 207...Response acquisition part 208 Output section 209 ··Billing Section 210. Management Database 211 Model Features Table 220-word model 230 ··Answer Model 231 ··First maintenance model 232 ··Second maintenance model 233 ··Third maintenance model 234 ··First Travel Model 235 ··2nd Travel Model 301. User Screen 302 · ·History area 303...Question / Answer area 304 ··User Icon 305 ··Answer device icon B2 connecting bus B3 connecting bus N1 Network U1 ··User

Claims

1. A storage unit that stores multiple learning models that generate response sentences to a given text when the results of text analysis are input, and feature information that indicates the characteristics related to the accuracy of the response to the analysis results for each of the multiple learning models, A processor connected to the aforementioned storage unit, The aforementioned processor, Upon receiving the analysis results of the question text entered by the user, the system performs an input process in which the analysis results are input to one or more selected learning models chosen from the multiple learning models, The following steps are performed: output processing to output the response sentence from the selected learning model into which the analysis results have been input, The input process further includes a selection process that, by referring to the feature information, selects one or more selected learning models from the plurality of learning models to input the analysis results. Information processing device.

2. The feature information includes first domain information indicating the field to which the learning model corresponds, The analysis results include second field information indicating the field of the question statement, The selection process includes calculating a field relevance score indicating the degree of relevance between the first field information and the second field information, and prioritizing the selection of the learning model with a high calculated field relevance score as the selected learning model. The information processing apparatus according to claim 1.

3. If multiple selected learning models are selected, the input process inputs the analysis results to the selected learning models in an order corresponding to the degree of relevance to the field. The information processing apparatus according to claim 2.

4. The output processing described above is: A response relevance score is calculated to show the degree of relevance between the aforementioned response and the second field information. The process further includes prioritizing the output of the aforementioned response sentences that have a high degree of relevance to the aforementioned response, The information processing apparatus according to claim 2.

5. The output processing described above is: A response relevance score is calculated to show the degree of relevance between the aforementioned response and the second field information. The process further includes suppressing the output of response sentences whose relevance to the response is lower than a threshold. The information processing apparatus according to claim 2.

6. The output process includes, when multiple selected learning models are selected in the selection process, a process for synthesizing the response sentences from the multiple selected learning models. The information processing apparatus according to claim 1.

7. The analysis results include text volume information relating to the amount of text in the document, The processor further performs a calculation process to calculate the usage fee based on the text volume information. The information processing apparatus according to any one of claims 1 to 6.

8. A computer comprising a storage unit that stores multiple learning models that generate response sentences to a given text when the results of text analysis are input, and feature information that indicates the characteristics related to the accuracy of the response to the analysis results for each of the multiple learning models, Upon receiving the analysis results of the question text entered by the user, the system performs an input process in which the analysis results are input to one or more selected learning models chosen from the multiple learning models, The following steps are performed: output processing to output the response sentence from the selected learning model into which the analysis results have been input, The input process further includes a selection process that, by referring to the feature information, selects one or more selected learning models from the plurality of learning models to input the analysis results. Information processing methods.

9. A computer having a storage unit that stores multiple learning models that generate response sentences to a given text when the results of text analysis are input, and feature information that indicates the characteristics related to the accuracy of the response to the analysis results for each of the multiple learning models, Upon receiving the analysis results of the question text entered by the user, the system performs an input process in which the analysis results are input to one or more selected learning models chosen from the multiple learning models, The system executes an output process that outputs a response sentence from the selected learning model into which the analysis results have been input. The input process further includes a selection process that, by referring to the feature information, selects one or more selected learning models from the plurality of learning models to input the analysis results. Information processing program.

Images