Information processing system, information processing method, and program

Abstract

The purpose of the present invention is to provide an answer to a user's inquiry. This information processing system comprises an input reception means and a determination means. The input reception means receives an input of a first inquiry from a user. The determination means compares a plurality of answers generated on the basis of the first inquiry by a plurality of answer generation models. Then, the determination means outputs, to different output destinations, an answer or a plurality of answers, to the first inquiry, selected from the plurality of answers on the basis of the comparison result.

Description

Information processing system, information processing method and program

[0001] The present disclosure relates to an information processing system, an information processing method, and a program.

[0002] In recent years, services that allow administrative procedures to be performed over a network have become widespread (see, for example, Patent Document 1). For example, the provision of services that allow applications to be submitted to government agencies has progressed. However, the specifications of the systems that provide these services vary from system to system.

[0003] However, when the system providers and system purposes are different, the procedure procedures, formats to be used, and system operation methods may differ. For this reason, when a user tries to perform a procedure through a system, they may contact the help desk to find out, for example, which system to access and how to perform the procedure.

[0004] When a help desk receives an inquiry, an operator generally prepares a response and provides it to the user. However, preparing a response to a user's inquiry about a system or program requires a lot of manpower and effort.

[0005] Japanese Patent Application Laid-Open No. 2023-113928

[0006] Therefore, efforts are being made to reduce the manpower and workload required at help desks by using artificial intelligence (AI) to generate answers to inquiries, but it is unclear whether the answers generated by AI are relevant to the inquiries.

[0007] The information processing system according to the present disclosure includes an input receiving means for receiving input of a first inquiry from a user, and a determination means for comparing a plurality of answers generated based on the first inquiry using a plurality of answer generation models, and for outputting an answer to the first inquiry selected from the plurality of answers or the plurality of answers to different output destinations based on the comparison results.

[0008] The information processing method disclosed herein receives input of a first inquiry from a user, compares multiple answers generated based on the first inquiry using multiple answer generation models, and based on the comparison results, outputs an answer to the first inquiry selected from the multiple answers or the multiple answers to different output destinations.

[0009] The program disclosed herein causes a computer to perform the following processes: accepting input of a first inquiry from a user; comparing multiple answers generated based on the first inquiry using multiple answer generation models; and, based on the comparison results, outputting an answer to the first inquiry selected from the multiple answers or the multiple answers to different output destinations.

[0010] According to the present disclosure, it is possible to provide answers to user inquiries.

[0011] FIG. 1 is a diagram schematically illustrating a configuration of an information processing system according to an embodiment. FIG. 2 is a diagram illustrating an example of information and prompts handled in the information processing system. FIG. 3 is a diagram illustrating an overview of answer selection in an output destination determination unit. FIG. 4 is a flowchart illustrating the operation of an information processing system according to an embodiment. FIG. 5 is a diagram schematically illustrating a configuration of an information processing system according to an embodiment. FIG. 6 is a flowchart illustrating the operation of an information processing system according to an embodiment. FIG. 7 is a diagram illustrating an example configuration of a computer for realizing an information processing system.

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same elements are designated by the same reference numerals, and redundant explanations will be omitted as necessary.

[0013] When referring to one embodiment below, it means that the present invention can be applied to any one of the embodiments described below or a combination of two or more embodiments, and is not limited to a specific embodiment.

[0014] First Embodiment An information processing system according to a first embodiment will be described. FIG. 1 is a diagram schematically illustrating the configuration of an information processing system according to one embodiment. The information processing system 100 is configured to be accessible by a user 1000 via a terminal external to the information processing system 100. The user 1000 can input an inquiry Q1 to the information processing system 100, such as how to use a service provided by a system external to the information processing system 100, or which service should be used based on the user 1000's needs. The information processing system 100 outputs an answer ANS to the inquiry Q1 from the user 1000 to the user 1000.

[0015] Here, as an example, assume a case in which user 1000 uses a public service such as an administrative service. For example, when user 1000 applies to a public corporation such as a national or local government through a system that provides public services, if user 1000 does not know how to apply or which public service to use, user 1000 inputs inquiry Q1 into information processing system 100. In this case, information processing system 100 outputs a response ANS to encourage appropriate use of the public service, such as application procedures and which public service to use, depending on the content of inquiry Q1, for example, depending on the application that user 1000 is about to make.

[0016] The information processing system 100 includes an input receiving unit 10 and an output destination determining unit 20 .

[0017] The input receiving unit 10 receives a query Q1 from a user 1000. The input receiving unit 10 generates a prompt P, which is information including the query Q1 and user-related information acquired based on the query Q1.

[0018] 2 is a diagram showing an example of information and prompts handled by the information processing system. In this embodiment, the inquiry Q1 includes user identification information for individually identifying the user 1000 and the inquiry content. In this example, the inquiry content is, "I submitted the documents, but the orderer pointed out that there are some deficiencies in the documents. I'm in trouble and would like to resolve this immediately."

[0019] The user identification information may be an ID number or the like that has been assigned in advance to the user 1000. In this case, the input receiving unit 10 may identify the user 1000 by referring to a user database that has been constructed in advance based on the user identification information such as the ID number.

[0020] Based on the identification result of the user 1000, the input receiving unit 10 may acquire the attribute ATT of the user 1000 as user-related information from an external database or the like. The attributes of each user may include the name, user ID, office, user category, user characteristics, etc. of the user 1000. For example, the input receiving unit 10 may acquire information such as "Name: XXXX, User ID: 123456, Office: XX River Office, Category: Contractor, Characteristics: Veteran, Caution" as the attribute ATT of the user 1000.

[0021] Furthermore, based on the identification result of the user 1000, the input receiving unit 10 may acquire the user's 1000 past system usage status ST as user-related information from an external database or the like. The system usage status ST of each user may include information on sites accessed in the past, the system type, the function type, the procedure status, etc. The site information may be, for example, a national institution or a local government. The system type may be, for example, "information sharing" or "application reception." The function type may be, for example, "document operation" or "deliverable product registration operation." The procedure status may be, for example, "before application for use," "in use," "in approval," "approval completed," etc.

[0022] The input receiving unit 10 combines the query Q1 input by the user 1000 with the acquired attributes ATT and system usage status ST to create a prompt P. In this example, the query Q1 and the acquired attributes ATT and system usage status ST are input to a prompt generation model MA, whereby the prompt P is generated.

[0023] In this example, the prompt P includes usage information, response level, inquiry content, and urgency related to the inquiry Q1 from the user 1000. The usage information is information indicating basic items of the current inquiry Q1, and includes, for example, the site, system, function, and procedure status used by the user 1000.

[0024] The response level is information that specifies the required quality of the response, such as how detailed the response needs to be depending on the attributes of the user 1000. The response level may be determined depending on the past usage record of the user 1000 that can be understood from the system usage status ST or the characteristics of the user 1000 that are included in the attribute ATT.

[0025] The inquiry content is transcribed or converted from inquiry Q1. At this time, the input receiving unit 10 may use various general natural language processing means to convert the sentences constituting inquiry Q1 into simple and concise sentences without changing the gist of the sentences, in order to check for grammatical errors or redundancies. In this case, the inquiry content of prompt P is a conversion of the gist of the inquiry contained in inquiry Q1, "I submitted a document, but the client pointed out a defect in the document. I'm in trouble and would like to resolve this immediately," into the simple sentence, "Please tell me how to correct the document I submitted."

[0026] The urgency is information indicating how quickly the user 1000 wants to solve the problem related to the inquiry Q1. The urgency may be derived from a sentence indicating the content of the inquiry Q1.

[0027] The input receiving unit 10 outputs the generated prompt P to a plurality of answer generation models M1 to Mn that generate an answer to the inquiry Q1.

[0028] Each of the answer generation models M1 to Mn is configured to generate an answer A1 to An to the inquiry data in accordance with the user-related information of the input prompt P. Each of the answer generation models M1 to Mn can use, for example, a large-scale language model, and can be constructed by inputting learning data stored in an external database or the like that indicates correspondences between prompts, user attributes, and model answers into the answer generation models M1 to Mn in advance and performing machine learning.

[0029] For example, if the information processing system 100 accepts inquiries regarding public services provided by government agencies, each of the answer generation models M1 to Mn may learn documents stored in an external database that contribute to formulating answers to inquiries to government agencies. Documents that contribute to formulating answers to inquiries to government agencies may include, for example, various documents related to the provision of public services, such as procedural guidelines that describe procedures for applications, forms to be used, and documents to be submitted, as well as rules and notices, and system instruction manuals and specifications.

[0030] In this example, the answer generation models M1 to Mn will be described as not being included in the information processing system 100. Large-scale language models generally require high computational resources and a large amount of learning processing, so it may be difficult to build and maintain a large-scale language model in-house. Therefore, by using an answer generation model that is separate from the information processing system 100 and that implements an external large-scale language model, the load on the information processing system 100 can be reduced and the information processing system 100 can be built more easily.

[0031] Generally, in a language processing service using a large-scale language model that is provided for a fee, a fee is charged each time an API (Application Programming Interface) is called to provide a prompt to the large-scale language model. Therefore, compared to a case in which some or all of the answer generation models M1 to Mn in which a large-scale language model is implemented are incorporated into the information processing system 100, the cost incurred can be limited to when the answer generation models are used, and therefore, it is expected that the operating costs of the information processing system 100 can be reduced.

[0032] However, this is merely an example, and does not preclude the incorporation of some or all of the answer generation models M1 to Mn into the information processing system 100.

[0033] The answer generation models M1 to Mn generate answers A1 to An by inputting the received prompt P, and output the generated answers A1 to An to the output destination determination unit 20.

[0034] The output destination determination unit 20 selects an answer ANS from the answers A1 to An as an appropriate answer to the inquiry Q1 according to the similarity of the answers A1 to An, and outputs the selected answer ANS.

[0035] At this time, if there are answers A1 to An that are similar to a predetermined degree, the output destination determination unit 20 selects an answer ANS from the similar answers, and outputs the selected answer ANS to the user 1000.

[0036] Furthermore, if the answers A1 to An are not similar to one another, the output destination determination unit 20 does not select an answer ANS, but leaves the selection of the answer ANS to an external determination means. In this example, if the answers A1 to An are not similar to one another, the output destination determination unit 20 outputs the answers A1 to An to the external answer review unit 4. The answer review unit 4 is configured to manually select an answer ANS that is suitable for the inquiry Q1 from the answers A1 to An. In the answer review unit 4, for example, an operator may compare the answers A1 to An displayed on the screen and output the answer that is determined to be more suitable for the inquiry Q1 to the user 1000 as the answer ANS.

[0037] 3 is a diagram showing an overview of answer selection in the output destination determination unit. For simplicity, this example shows an example in which answers A1 and A2 are output from two answer generation models M1 and M2, respectively.

[0038] The content of response A1 is, "If the submitter's next approver has not yet approved the document, you will need to reject and return the document and then resubmit it." The content of response A2 is, "If other users have not yet approved the document, you will need to cancel the submission and delete the submitted document before submitting a new document." RES also shows an example of the comparison results between responses A1 and A2.

[0039] 3, the output destination determination unit 20 analyzes the similarity of the answer A2 with respect to the answer A1. If the similarity between the answers A1 and A2 falls within a predetermined range, i.e., if they are similar, the output destination determination unit 20 analyzes the suitability for the query Q1.

[0040] In this example, the correlation coefficient of answer A2 with answer A1 is used as the similarity. If the correlation coefficient is 0.8 or higher, it is determined that the reference answer and the comparison target are similar. In this example, the correlation coefficient of answer A2 with answer A1 is 0.9, so the output destination determination unit 20 determines that answers A1 and A2 are similar.

[0041] The output destination determination unit 20 selects the answer A1 or A2 that is most suitable for the query Q1 as the answer ANS. In this example, the output destination determination unit 20 determines the superiority based on the suitability of the answer to the user attributes and the suitability of the answer to the urgency of the query. In Figure 3, a large integer is assigned as the suitability when the answer has a high superiority or is more closely matched to the urgency.

[0042] The degree of suitability may be determined by analyzing the semantic content of the query and the answer, or by taking into account user attributes such as whether the user is a beginner or an expert, or whether the answer requires special attention.

[0043] In this example, although the answers A1 and A2 have the same suitability for the urgency of the inquiry, the answer A2 has a higher suitability for the user attributes. Therefore, the output destination determination unit 20 determines that the answer A2 is more suitable for the inquiry Q1 than the answer A1, and outputs the answer A2 as the answer ANS.

[0044] On the other hand, if the similarity between answers A1 and A2 is outside a predetermined range, i.e., if they are not similar, the output destination determination unit 20 determines that the answers A1 and A2 require consideration of their suitability to the query Q1, and outputs them to the answer review unit 4 provided separately from the information processing system 100. In this case, in the answer review unit 4, for example, an operator considers which of answers A1 and A2 is more suitable for the query Q1. Then, the answer that the operator determines to be more suitable for the query Q1 is output to the user 1000 as the answer ANS.

[0045] Next, the operation of the information processing system 100 will be described. Fig. 4 is a flowchart showing the operation of the information processing system according to one embodiment. The operation of the information processing system 100 is composed of the following steps S11 to S12. Note that step SA in Fig. 4 is an operation performed by answer generation models M1 to Mn external to the information processing system 100, and is therefore not included in the operation of the information processing system 100.

[0046] Step S11: The input accepting unit 10 accepts a query Q1 from the user 1000. Then, the input accepting unit 10 acquires the attribute ATT and system usage status ST of the user 1000 based on the user identification information included in the query Q1. The input accepting unit 10 combines the query Q1, the attribute ATT and system usage status ST of the user 1000, and generates a prompt P. The input accepting unit 10 outputs the prompt P to the answer generation models M1 to Mn.

[0047] Step SA: The answer generation models M1 to Mn generate answers A1 to An, respectively, by inputting the prompt P. Then, the answer generation models M1 to Mn output the generated answers A1 to An to the output destination determination unit 20.

[0048] In step S12, the output destination determination unit 20 determines whether any of the answers A1 to An are similar. If any of the answers A1 to An are similar, the one that best suits the query Q1 is selected as the answer ANS. In this case, the output destination determination unit 20 outputs the selected answer ANS to the user 1000. On the other hand, if the answers A1 to An are not similar to one another, the output destination determination unit 20 outputs the answers A1 to An to the answer examination unit 4, as they require examination of their suitability to the query Q1.

[0049] In the answer review unit 4, for example, an operator selects an answer ANS from among the answer generation models M1 to Mn that is suitable for the inquiry Q1. The answer review unit 4 outputs the selected answer ANS to the user 1000.

[0050] As described above, the information processing system 100 can provide a user with an answer selected from a plurality of answers as being appropriate for the inquiry, based on the inquiry from the user and user-related information.

[0051] Therefore, it is expected that the answer will adequately resolve the doubts regarding the user's inquiry, and the user will be able to refer to the answer to resolve the doubts regarding the inquiry and complete the desired procedure.

[0052] This also reduces the manpower and man-hours required to provide a response ANS to a user who has made an inquiry, resulting in an information system that can provide responses to users more efficiently and at lower cost than conventional methods.

[0053] Second Embodiment In this embodiment, the information processing device according to the first embodiment will be described in more detail. Fig. 5 is a diagram schematically illustrating a configuration of an information processing system according to one embodiment. The information processing system 200 in Fig. 5 is a specific example of the information processing system 100 according to the first embodiment. The information processing system 200 in Fig. 5 is a specific example of the information processing system 100 according to the first embodiment.

[0054] In this example, a user 1000 inputs information to an information processing system 200 via a user terminal 2000 and receives information from the information processing system 200 .

[0055] In the information processing system 200 , the input receiving unit 10 includes an information acquiring unit 11 and a prompt generating unit 12 .

[0056] The information acquisition unit 11 accepts an inquiry Q1 input by the user 1000 to the user terminal 2000. Based on the identification result of the user 1000, the information acquisition unit 11 may acquire the attribute ATT of the user 1000 as user-related information, for example, from a pre-constructed user attribute database (hereinafter also referred to as a user attribute DB) 1. The information acquisition unit 11 outputs the inquiry Q1 and the user-related information acquired based on the inquiry Q1 to the prompt generation unit 12. Furthermore, based on the identification result of the user 1000, the information acquisition unit 11 may acquire the past system usage status ST of the user 1000 from a pre-constructed usage status database (hereinafter also referred to as a usage status DB) 2.

[0057] The prompt generation unit 12 combines the query Q1 input by the user 1000 with the acquired attributes ATT and system usage status ST to create a prompt P. At this time, the prompt generation unit 12 may convert the sentences constituting the query Q1 into simple and concise sentences using various general natural language processing means, with respect to grammatical errors and redundancies, as long as the gist of the sentences is not changed.

[0058] The prompt generation unit 12 has a prompt generation model MA that outputs a prompt by inputting a query Q1, an attribute ATT, and a system usage status ST. The prompt generation model MA can be constructed, for example, by inputting learning data stored in a material database (hereinafter also referred to as a material DB) 3, which indicates correspondence between the query content, usage status, user attributes, and prompts, into the prompt generation model in advance and performing machine learning.

[0059] The prompt generator 12 outputs the generated prompt P to a plurality of answer generation models M1 to Mn that generate answers to the inquiry Q1.

[0060] Each of the answer generation models M1 to Mn may use, for example, a large-scale language model, and may be constructed by performing machine learning by inputting learning data stored in the material DB 3, which data indicates correspondences between prompts, user attributes, and model answers, into the answer generation models M1 to Mn in advance. In this case, the material DB 3 may store documents that contribute to the construction of answers to inquiries to government agencies, as described above.

[0061] The output destination determination unit 20 has an answer selection model MB that selects an answer ANS from the answers A1 to An as an appropriate answer to the inquiry Q1 according to the similarity of the answers A1 to An and outputs the answer ANS. For example, the output destination determination unit 20 inputs the answers A1 to An and user-related information into the answer selection model MB, and selects an answer ANS according to the similarity of the answers A1 to An. The user-related information input into the answer selection model MB may be both the attribute ATT and the system usage status ST, or either one of them. Here, the description will be given assuming that the attribute ATT is input into the model MB.

[0062] The answer selection model MB can be constructed by inputting learning data indicating the correspondence between a prompt and a model answer that matches the prompt into the answer selection model MB and performing machine learning.

[0063] At this time, the result of the process in which the answer selection model MB of the output destination determination unit 20 selects the answer ANS from the answers A1 to An is the same as in the first embodiment, so a duplicated explanation will be omitted.

[0064] Next, the operation of the information processing system 200 will be described. Fig. 6 is a flowchart showing the operation of the information processing system according to one embodiment. The operation of the information processing system 200 is composed of the following steps S21 to S26. Note that steps SA and SB in Fig. 6 are operations performed by answer generation models M1 to Mn and answer review unit 4 external to the information processing system 200, and are therefore not included in the operation of the information processing system 200.

[0065] In the following, steps S21 to S23 correspond to step S11 in Fig. 4. Steps S24 to S26 correspond to step S12 in Fig. 4.

[0066] Step S21: The information acquisition unit 11 receives an inquiry Q1 from the user terminal 2000.

[0067] Step S22: The information acquisition unit 11 acquires the attribute ATT and the system usage status ST of the user 1000 based on the user identification information included in the inquiry Q1.

[0068] In step S23, the prompt generator 12 inputs the query Q1, the attribute ATT of the user 1000, and the system usage status ST into the prompt generation model MA to generate a prompt P. The prompt generator 12 outputs the prompt P to the answer generation models M1 to Mn.

[0069] Step SA: The answer generation models M1 to Mn generate answers A1 to An, respectively, by inputting the prompt P. Then, the answer generation models M1 to Mn output the generated answers A1 to An to the output destination determination unit 20.

[0070] Step S24: The output destination determination unit 20 inputs the answers A1 to An into the answer selection model MB, and determines whether any of the answers A1 to An are similar.

[0071] In step S25, if there are similar answers among the answers A1 to An, the output destination determination unit 20 selects the answer that best suits the inquiry Q1 from among the similar answers, and outputs the selected answer ANS to the user terminal 2000.

[0072] Step S26: If the answers A1 to An are not similar to one another, the output destination determination unit 20 outputs the answers A1 to An to the answer examination unit 4 as answers that require examination of their suitability to the inquiry Q1.

[0073] Step SB: The answer examination unit 4 selects the answer generation model M1 to Mn that is suitable for the inquiry Q1 as the answer ANS. The answer examination unit 4 outputs the answer ANS to the user 1000.

[0074] As described above, according to the information processing system 200, similar to embodiment 1, an answer selected from a plurality of answers as being appropriate to the inquiry from the user can be provided to the user in accordance with the inquiry and user-related information from the user.

[0075] Therefore, by expecting that the answer will satisfactorily resolve the user's question, the user can complete the desired procedure. This also reduces the manpower and labor required to provide answers to users who have made inquiries, making it possible to realize an information system that can provide answers to users more efficiently and at lower cost than conventional methods.

[0076] In the above-described embodiment, an information processing system has been described that selects an answer ANS to be provided to the user 1000 from answers generated by a plurality of answer generation models in response to an inquiry Q1. As a result, the user 1000 can access a service providing system that provides the desired service based on the answer ANS in order to receive the desired service.

[0077] However, depending on the degree of learning of the answer selection model MB of the output destination determination unit 20 in the information processing systems 100 and 200 according to the above-described embodiments, it is possible that an answer ANS that is inappropriate for the query Q1 may be selected. For example, in a situation where a user 1000 is inquiring about a specific procedure, it is conceivable that the answer selection model MB may respond by instructing the user 1000 to access a system that provides a service that does not match the procedure the user 1000 is attempting to perform. In this case, the user 1000 will access an inappropriate system that is not appropriate for performing the intended procedure. As a result, even if the user 1000 accesses the system indicated by the answer ANS, there is a risk that the user 1000 will not be able to perform the intended procedure.

[0078] Furthermore, depending on the attributes of the user 1000, the output destination determination unit 20 may select an insufficient answer to the inquiry Q1. For example, if the user 1000 has little or no experience with network procedures, if the output destination determination unit 20 selects an answer that includes a brief explanation of the procedure, the user 1000 may not fully understand the procedure. In this case, the user 1000 may not be able to access a system that allows them to perform the intended procedure. Furthermore, even if the user 1000 is able to access an appropriate system, they may not be able to perform the intended procedure due to a lack of understanding of the procedure.

[0079] Therefore, in this embodiment, an information processing system will be described in which an evaluation of the answer ANS based on the behavior of the user 1000 after outputting the answer ANS is fed back to the output destination determination unit 20.

[0080] 7 is a diagram schematically illustrating a configuration of an information processing system according to an embodiment. Compared to the information processing systems 100 and 200, the information processing system 300 further includes an answer history database (hereinafter also referred to as an answer history DB) 30.

[0081] In the information processing system 300, the output destination determination unit 20 records an answer log indicating the relationship between the inquiry Q1 and the answer ANS in the answer history DB 30. This answer log includes answers A1 to An that were compared by the output destination determination unit 20 and information indicating which of the answers A1 to An was selected as the answer ANS.

[0082] As explained in the first embodiment, even when the output destination determination unit 20 provides the answer ANS to the user 1000 via the user terminal 2000, there may be cases where the user 1000 is unable to solve the problem based on the answer ANS. In this case, the user 1000 may make an inquiry Q2 to a help desk operator via a communication tool such as telephone, email, or chat about a solution to the problem that could not be solved by the answer ANS. The operator then informs the user 1000 of the solution to the problem that could not be solved by the answer ANS. In this embodiment, the help desk or the help desk operator is represented as the answer evaluation unit 5.

[0083] In this embodiment, the answer evaluation unit 5 acquires an answer log corresponding to the answer ANS from the answer history DB 30 in response to an inquiry Q2 that arises as a result of providing the answer ANS to the user 1000. If the answer ANS is selected by the output destination determination unit 20, the operator evaluates whether or not the selection of the answer ANS from the answers A1 to An was appropriate, based on the inquiry Q2 and the answer ANS. The operator then feeds back to the output destination determination unit 20 an evaluation E1 indicating the evaluation result as to whether or not the selection of the answer ANS from the answers A1 to An was appropriate.

[0084] As a result, the output destination determination unit 20 can improve answer selection performance by inputting the prompt P, answers A1 to An, and evaluation E1 as input data into the answer selection model MB and performing additional machine learning.

[0085] The answer evaluation unit 5 also generates a more appropriate model answer for the answer ANS. The answer evaluation unit 5 then additionally records an evaluation E2 that associates at least the inquiry Q1 with the generated model answer in the material DB 3. The material DB 3 additionally stores the inquiry Q1 and the model answer included in the evaluation E2. This allows the material DB 3 to be updated based on the results of actual operation of the information processing system 300.

[0086] The answer generation models M1 to Mn can increase the probability of generating an answer that matches the inquiry by additionally learning the correspondence between inquiries and answers by machine learning based on the information stored in the updated material DB3.

[0087] Next, a description will be given of the operation of the information processing system 300. Fig. 8 is a flowchart showing the operation of the information processing system according to one embodiment.

[0088] Step S31: The output destination determination unit 20 records, in the answer history DB 30, an answer log indicating the relationship between the inquiry Q1 and the answer ANS.

[0089] Step S32: The answer evaluation unit 5 acquires an answer log corresponding to the answer ANS from the answer history DB 30 in response to the inquiry Q2 that arises as a result of providing the answer ANS to the user 1000.

[0090] In step S33, the answer evaluation unit 5 evaluates whether the selection of the answer ANS from the answers A1 to An was appropriate based on the inquiry Q2 and the answer ANS. Then, the answer evaluation unit 5 outputs an evaluation E1 to the output destination determination unit 20.

[0091] In step S34, the output destination determination unit 20 inputs the prompt P, the answers A1 to An, and the evaluation E1 as input data into the answer selection model MB, and performs additional machine learning. This improves the answer selection performance of the output destination determination unit 20.

[0092] In step S35, the answer evaluation unit 5 generates a model answer that is more appropriate for the answer ANS. The answer evaluation unit 5 then outputs an evaluation E2 that associates at least the inquiry Q1 with the generated model answer to the reference material DB 3. The reference material DB 3 additionally stores the inquiry Q1 and the model answer included in the evaluation E2. This updates the reference material DB 3 based on the results of actual operation of the information processing system 300. This allows some or all of the answer generation models M1 to Mn to additionally learn the correspondence between inquiries and answers through machine learning based on the information stored in the updated reference material DB 3, thereby increasing the probability of generating an answer that matches the inquiry.

[0093] As described above, the information processing system 300 can evaluate the appropriateness of the selection of an answer ANS from the answers A1 to An based on repeated inquiries Q2 from users who have referenced the answer ANS. The evaluation results can then be fed back to the answer generation models M1 to Mn and the answer selection model MB. This allows additional learning of the answer generation models M1 to Mn based on the evaluation, thereby efficiently and continuously improving the probability of generating an appropriate answer to an inquiry.

[0094] Other Embodiments The present disclosure has been described above with reference to the embodiments, but the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0095] Some or all of the user attribute DB1, the usage status DB2, and the material DB3 may be stored in a storage device provided in a system separate from the information processing system. Also, some or all of the user attribute DB1, the usage status DB2, and the material DB3 may be stored in a storage device (not shown) provided in the information processing system.

[0096] The answer history DB 30 may be stored in a storage device provided in a system separate from the information processing system 300 .

[0097] In the above-described embodiment, the information processing system according to the present disclosure has been described primarily as a hardware configuration, but is not limited to this. The information processing system according to the present disclosure can also be realized by having a computer execute a computer program to perform any process. These processes may be realized by having a computer including at least one processor (e.g., a microprocessor, a CPU, a GPU, an MPU, or a DSP (Digital Signal Processor)) execute the program. Specifically, one or more programs including instructions for causing a computer to perform these algorithms related to transmission signal processing or reception signal processing may be created, and the programs may be supplied to the computer.

[0098] A computer program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may be provided to the computer by various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transient computer-readable media can provide the program to the computer via a wired communication path such as an electrical wire or optical fiber, or via a wireless communication path.

[0099] An example of the configuration of a computer for realizing the information processing system according to the above-described embodiment is shown below. FIG. 7 is a diagram showing an example of the configuration of a computer for realizing the information processing system. The information processing system can be realized by a computer 9000, such as a dedicated computer or a personal computer (PC). However, the computer does not need to be physically single; multiple computers may be used when performing distributed processing. As shown in FIG. 7, the computer 9000 includes, for example, a processor 9001, a ROM (Read Only Memory) 9002, a RAM (Random Access Memory) 9003, a storage unit 9004, a communication interface 9005, and a user interface 9006.

[0100] The processor 9001, ROM 9002, RAM 9003, storage unit 9004, communication interface 9005, and user interface 9006 are connected to each other so as to be able to communicate with each other via a bus 9007. Note that although explanation of the OS software for operating the computer is omitted, it is also installed in the computer 9000 as appropriate.

[0101] The ROM 9002 is configured by, for example, a nonvolatile semiconductor memory device, etc. The ROM 9002 stores information such as various programs used by the computer 9000.

[0102] The storage unit 9004 is configured with various storage devices such as a hard disk, a solid state disk, etc. Furthermore, the storage unit 9004 is not limited to a storage device installed in the computer 9000, but may be a storage device external to the computer 9000. The external storage device may be a cloud storage device connected to the computer 9000 via various communication means, for example, a network. The storage unit 9004 stores information such as various programs and data used by the computer 9000.

[0103] The RAM 9003 is configured by a volatile semiconductor memory device, etc. Programs, data, and other information used by the processor 9001 are loaded into the RAM 9003 from one or both of the ROM 9002 and the storage unit 9004 as appropriate.

[0104] The processor 9001 may be configured with, for example, a CPU (Central Processing Unit). Furthermore, the processor 9001 may include not only a CPU but also a GPU (Graphics Processing Unit). A GPU is suitable for performing routine processing in parallel, and by applying it to neural network processing, for example, it is possible to improve processing speed compared to a CPU. The processor 9001 executes various processes based on various programs stored in the ROM 9002 or various programs and data held in the RAM 9003, as appropriate. Furthermore, the processor 9001 may store data generated by the processing in the RAM 9003 or the storage unit 9004, as appropriate.

[0105] The communication interface 9005 is an interface that connects the computer 9000 to a communication network such as the Internet or an intranet via various wired communication means or wireless communication means, etc. This allows the computer 9000 to communicate with other devices, systems, sensors, etc. that are connected to the communication network.

[0106] The user interface 9006 includes, for example, a display unit that provides information so that the user can recognize it using a display device or the like, and an audio output unit that outputs audio. The user interface 9006 also includes an input unit that allows the user to input information to the computer 9000 by operating it, such as a keyboard, a mouse, or a touch panel. The user interface 9006 may also include devices such as sensors that obtain information useful to the user.

[0107] Although the computer 9000 has been described as a single device here, this is merely an example. The computer 9000 may be composed of multiple physically separated devices. Some of the multiple devices may be portable devices, and the other devices may be stationary devices.

[0108] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0109] Each drawing is merely an example for describing one or more embodiments. Each drawing may not relate to only one particular embodiment, but may also relate to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0110] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0111] (Supplementary Note 1) An information processing system comprising: an input receiving means for receiving input of a first inquiry from a user; and a determination means for comparing a plurality of answers generated based on the first inquiry using a plurality of answer generation models, and outputting an answer to the first inquiry selected from the plurality of answers or the plurality of answers to different output destinations based on the comparison results.

[0112] (Supplementary Note 2) The information processing system according to Supplementary Note 1, wherein the determining means determines whether to output the answer to the first inquiry or the plurality of answers based on a similarity between the plurality of answers.

[0113] (Supplementary Note 3) The information processing system according to Supplementary Note 1 or 2, wherein the determining means, if there are similar answers among the plurality of answers, selects an answer to the first inquiry from the similar answers among the plurality of answers, and, if there are no similar answers among the plurality of answers, outputs the plurality of answers.

[0114] (Appendix 4) The information processing system described in Appendix 3, wherein the determination means, if there are similar answers among the plurality of answers, outputs an answer that suits the user based on information about the user as an answer to the first inquiry.

[0115] (Supplementary Note 5) The information processing system according to Supplementary Note 4, wherein the determining means outputs an answer that matches the user's level of understanding based on the information about the user as the answer to the first inquiry.

[0116] (Supplementary Note 6) The information processing system according to Supplementary Note 5, wherein the determining means outputs a response that matches the urgency of the first inquiry as the response to the first inquiry.

[0117] (Supplementary Note 7) The information processing system according to any one of Supplementary Notes 3 to 6, wherein the determining means, if there is a similar answer among the plurality of answers, outputs the answer to the first inquiry to a first output destination for providing the answer to the user.

[0118] (Appendix 8) The information processing system described in Appendix 7, wherein the determination means has an answer selection model that selects an answer to the first inquiry from the plurality of answers based on a comparison result of the plurality of answers, receives an evaluation of the selection of the answer to the first inquiry from the plurality of answers generated in response to a second inquiry regarding the answer to the first inquiry from the first output destination, and performs additional learning of the answer selection model based on the evaluation.

[0119] (Appendix 9) The information processing system described in Appendix 7, wherein the plurality of answer generation models are provided with model answers that are more suitable for the first inquiry than the answer to the first inquiry, and that are generated in response to a second inquiry regarding the answer to the first inquiry from the first output destination, and the plurality of answer generation models additionally learn the correspondence between the first inquiry and the model answers.

[0120] (Appendix 10) An information processing system as described in Appendix 7, wherein if there are no similar answers among the plurality of answers, an answer to the first inquiry is manually output from the plurality of answers to a second output destination that specifies the answer, and the answer to the first inquiry specified at the second output destination is output to the first output destination.

[0121] (Appendix 11) An information processing method that accepts input of a first inquiry from a user, compares multiple answers generated based on the first inquiry using multiple answer generation models, and based on the comparison results, outputs an answer to the first inquiry selected from the multiple answers or the multiple answers to different output destinations.

[0122] (Appendix 12) A program that causes a computer to execute the following processes: accepting input of a first inquiry from a user; comparing multiple answers generated based on the first inquiry using multiple answer generation models; and, based on the comparison results, outputting an answer to the first inquiry selected from the multiple answers or the multiple answers to different output destinations.

[0123] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 10 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Notes 11 and 12 in the same dependency relationship as Supplementary Notes 2 to 10. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods.

[0124] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the invention.

[0125] This application claims priority based on Japanese Patent Application No. 2024-123051, filed on July 30, 2024, the disclosure of which is incorporated herein in its entirety by reference.

[0126] 1 User attribute DB 2 Usage status DB 3 Document DB 4 Answer examination section 5 Answer evaluation section 10 Input reception section 11 Information acquisition section 12 Prompt generation section 20 Output destination determination section 30 Answer history DB 100, 200, 300 Information processing system 1000 User 2000 User terminal 9000 Computer 9001 Processor 9002 ROM 9003 RAM 9004 Storage section 9005 Communication interface 9006 User interface 9007 Bus A1 to An Answer ANS Answer ATT Attribute E1 Evaluation E2 Evaluation M1 to Mn Answer generation model MA Prompt generation model MB Answer selection model P Prompt RES Comparison result ST System usage status

Claims

1. An information processing system comprising: an input receiving means for receiving input of a first inquiry from a user; and a determination means for comparing multiple answers generated based on the first inquiry using multiple answer generation models, and based on the comparison result, selecting an answer to the first inquiry from the multiple answers or the multiple answers to output to different output destinations.

2. The information processing system according to claim 1, wherein said determining means determines whether to output the answer to said first inquiry or one of said plurality of answers based on the similarity of said plurality of answers.

3. The information processing system of claim 1 or 2, wherein the determination means, if there are similar answers among the plurality of answers, selects an answer to the first inquiry from the similar answers among the plurality of answers, and if there are no similar answers among the plurality of answers, outputs the plurality of answers.

4. The information processing system of claim 3, wherein, if there are similar answers among the plurality of answers, the determining means outputs an answer that suits the user based on the user's information as the answer to the first inquiry.

5. The information processing system according to claim 4, wherein said determining means outputs an answer to said first inquiry that matches the user's level of understanding based on the information about said user.

6. The information processing system according to claim 5, wherein said determining means outputs an answer that matches the urgency of said first inquiry as an answer to said first inquiry.

7. The information processing system according to claim 3, wherein the determining means, if there is a similar answer among the plurality of answers, outputs the answer to the first inquiry to a first output destination for providing the answer to the user.

8. The information processing system of claim 7, wherein the determination means has an answer selection model that selects an answer to the first inquiry from the plurality of answers based on a comparison result of the plurality of answers, receives an evaluation of the selection of the answer to the first inquiry from the plurality of answers generated in response to a second inquiry regarding the answer to the first inquiry from the first output destination, and performs additional learning of the answer selection model based on the evaluation.

9. The information processing system of claim 7, wherein the plurality of answer generation models are provided with model answers that are more suitable for the first query than the answer to the first query, and that are generated in response to a second query related to the answer to the first query from the first output destination, and additionally learn the correspondence between the first query and the model answers.

10. An information processing system as described in claim 7, wherein if there are no similar answers among the plurality of answers, an answer to the first inquiry is manually selected from the plurality of answers and output to a second output destination that specifies the answer, and the answer to the first inquiry specified at the second output destination is output to the first output destination.

11. An information processing method comprising: receiving input of a first inquiry from a user; comparing multiple answers generated based on the first inquiry using multiple answer generation models; and, based on the comparison results, outputting an answer to the first inquiry selected from the multiple answers or the multiple answers to different output destinations.

12. A program that causes a computer to perform the following processes: accepting input of a first inquiry from a user; comparing multiple answers generated based on the first inquiry using multiple answer generation models; and, based on the comparison results, selecting an answer to the first inquiry from the multiple answers or outputting the multiple answers to different output destinations.

Images