Information processing system

Abstract

Provided is an information processing system that has a different configuration from conventional configurations. An information processing system (1) comprises: an interface unit (11) that receives input from a user (U); a local language model (LM1); a remote language model (LM2) that is different from the local language model (LM1); and a selection unit (12) that selects, from among at least one of the local language model (LM1) and the remote language model (LM2), a language model that should be used for generating a response to a question that the user (U) has inputted to the interface unit (11).

Description

Information Processing Systems

[0001] The following disclosure relates to an information processing system with a language model.

[0002] In recent years, various proposals have been made regarding information processing techniques using language models. For example, Patent Document 1 listed below discloses an example configuration of an information processing system that causes a language model to generate a summary sentence.

[0003] Japanese Patent Application Publication No. 2023-73095

[0004] An object of one aspect of the present disclosure is to provide an information processing system with a different configuration from conventional systems.

[0005] An information processing system according to one aspect of the present disclosure includes an interface unit that accepts input from a user, a local language model, a remote language model that is different from the local language model, and a selection unit that selects a language model to be used to generate a response to a question input by the user into the interface unit from at least one of the local language model and the remote language model.

[0006] According to one aspect of the present disclosure, it is possible to provide an information processing system having a configuration different from conventional ones.

[0007] 1 shows an example of the configuration of an information processing system in embodiment 1. FIG. 2 shows an example of the configuration of an information processing system in embodiment 2. FIG. 3 shows an example of the configuration of an information processing system in embodiment 3. FIG. 4 shows an example of the configuration of an information processing system in embodiment 4. FIG. 5 shows an example of the configuration of an information processing system in embodiment 4. FIG. 6 shows an example of the configuration of an information processing system in embodiment 4. FIG. 7 shows an example of the configuration of an information processing system in embodiment 4. FIG. 8 shows an example of the configuration of an information processing system in embodiment 4. FIG. 9 shows an example of the configuration of an information processing system in embodiment 4. FIG. 10 shows an example of the configuration of an information processing system in embodiment 4. FIG. 11 shows an example of the configuration of an information processing system in embodiment 4.

[0008] [Embodiment 1] Embodiment 1 will be described below. For convenience of explanation, components having the same functions as those described in embodiment 1 will be denoted by the same reference numerals in the following embodiments, and their description will not be repeated. For simplicity, descriptions of well-known technical matters will also be omitted as appropriate. Unless otherwise specified, the components and numerical values ​​described in this disclosure are merely examples. Therefore, for example, unless otherwise specified, the positional relationship and connection relationship of the components are not limited to the examples in the figures.

[0009] 1 shows an example of the configuration of an information processing system 1 according to the first embodiment. The information processing system 1 includes an interface unit 11 and a selection unit 12. The information processing system 1 also includes a plurality of language models.

[0010] In the example of Fig. 1, the information processing system 1 includes a local language model LM1 and a remote language model LM2. The remote language model LM2 is a language model different from the local language model LM1. The local language model LM1 and the remote language model LM2 may be referred to as a first language model and a second language model, respectively. Both the local language model LM1 and the remote language model LM2 may be publicly known trained models.

[0011] The interface unit 11 accepts input from the user U. In this specification, the input from the user U is referred to as a user input. The interface unit 11 supplies the accepted user input to the selection unit 12. In the first embodiment, a case where the user input is text data indicating a question from the user U to the language model is mainly illustrated.

[0012] As another example, the user input may be voice data. The voice data may be, for example, speech data of a user U. Alternatively, the user input may be image data. The user input may be any data that can be accepted by a language model according to one aspect of the present disclosure.

[0013] In the first embodiment, a response to a user input is generated by a language model. In the first embodiment, a case where the language model generates text data as a response is mainly illustrated. However, the language model according to one aspect of the present disclosure may also generate voice data or image data as a response.

[0014] The interface unit 11 acquires the response generated by the language model via the selection unit 12. The interface unit 11 presents the acquired response to the user U. For this reason, the interface unit 11 may also be referred to as an input / output unit.

[0015] The interface unit 11 may accept a user input in any style. The interface unit 11 may present a response acquired via the selection unit 12 to the user U in any style. As an example, the interface unit 11 may accept a user input and present a response to the user U in an interactive format.

[0016] As shown in FIG. 1 , the information processing system 1 may include a terminal device 10 and a server 90. The terminal device 10 may be referred to as an edge device. As an example, the terminal device 10 may be a device owned by a user U. The server 90 may be, for example, a cloud server. In the information processing system 1, it is sufficient that the terminal device 10 and the server 90 are connected so as to be able to communicate with each other.

[0017] In the example of Fig. 1, the terminal device 10 includes an interface unit 11, a selection unit 12, and a local language model LM1. The local language model LM1 in the first embodiment is a smaller language model than the remote language model LM2. Therefore, the local language model LM1 can generate a response faster than the remote language model LM2. In this specification, the response of the local language model LM1 is referred to as a local response.

[0018] In the example of Fig. 1, the server 90 includes a remote language model LM2. The remote language model LM2 in the first embodiment is a language model with a larger scale than the local language model LM1. Therefore, the remote language model LM2 can generate a more reliable response than the remote language model LM2. In this specification, the response of the remote language model LM2 is referred to as a remote response.

[0019] For example, the administrator of the information processing system 1 (or the administrator of the server 90) may update the remote language model LM2 by re-learning the remote language model LM2 as needed, thereby enabling more reliable remote responses to be presented to the user U.

[0020] In the information processing system 1, a response to a user input may be generated using at least one of the local language model LM1 and the remote language model LM2. Therefore, the information processing system 1 is provided with a selection unit 12.

[0021] The selection unit 12 selects a language model to be used to generate a response to a user input from at least one of the local language model LM1 and the remote language model LM2. The selection unit 12 makes it possible to generate a response to a user input by selectively using the local language model LM1 and the remote language model LM2.

[0022] 2 is a flowchart showing an example of a processing flow in the information processing system 1. First, in step S0, the interface unit 11 accepts a user input. The interface unit 11 supplies the user input to the selection unit 12. In step S1, the selection unit 12 outputs a response generation request to the local language model LM1 based on the user input. In this way, the selection unit 12 may first select the local language model LM1 as the language model to be used to generate a response to the user input.

[0023] In step S2, the local language model LM1 generates a local response corresponding to the acquired response generation request. As an example, the local language model LM1 generates a local response corresponding to a question included in the response generation request. The local language model LM1 supplies the generated local response to the selection unit 12.

[0024] In step S3 , the selection unit 12 inputs the acquired local response to the interface unit 11 .

[0025] However, if a local response is not supplied to the selection unit 12 even after a first predetermined time (e.g., 10 seconds) has elapsed since the selection unit 12 outputted a response generation request to the local language model LM1 in step S1, the selection unit 12 may skip the processing of step S3. If the processing of step S3 is skipped, the local response is not input to the interface unit 11. The example of the first predetermined time, 10 seconds, is assumed to be the time from when the user U asks a question to the information processing system 1 until the user U begins to feel anxious that the information processing system 1 has not accepted the user U's question.

[0026] The selection unit 12 may further select the remote language model LM2 as a language model to be used to generate a response to the user input. For example, in step S5 following step S2, the selection unit 12 may output a response generation request to the remote language model LM2 based on the user input. In this case, the selection unit 12 may generate the response generation request to the remote language model LM2 as information including the user input and the local response.

[0027] In step S6, the remote language model LM2 generates a remote response corresponding to the acquired response generation request. By supplying the response generation request including the local response to the remote language model LM2, the remote language model LM2 can generate a remote response based on the local response. The remote language model LM2 supplies the generated remote response to the selection unit 12.

[0028] In step S7, the selection unit 12 inputs the acquired remote response to the interface unit 11.

[0029] However, if a remote response is not supplied to the selection unit 12 even after a second predetermined time (e.g., 30 seconds) has elapsed since the selection unit 12 outputted a response generation request to the remote language model LM2 in step S5, the selection unit 12 may skip the process of step S7. If the process of step S7 is skipped, the remote response is not input to the interface unit 11.

[0030] In general, it is considered that the time required to obtain a remote response from the remote language model LM2 is longer than the time required to obtain a local response from the local language model LM1. For this reason, it is considered that a longer waiting time when using the remote language model LM2 than when using the local language model LM1 is acceptable to the user U. Therefore, the second predetermined time (e.g., 30 seconds) may be set to be longer than the first predetermined time (e.g., 10 seconds).

[0031] In step S9, the interface unit 11 may determine which response to present to the user U based on whether a local response and / or a remote response has been input to the interface unit 11 after a predetermined time has elapsed. The start point of the predetermined time may be, for example, the time when step S0, step S1, or step S5 is executed.

[0032] For example, if only the local response is input to the interface unit 11, the process proceeds to step S8-a, in which the interface unit 11 presents the local response to the user U.

[0033] If the local response and the remote response are input to the interface unit 11, the process proceeds to step S8-b. In step S8-b, the interface unit 11 first presents the local response to the user U. Then, after presenting the local response, the interface unit 11 further presents the remote response to the user U.

[0034] If only the remote response is input to the interface unit 11, the process proceeds to step S8-c, where the interface unit 11 presents the remote response to the user U.

[0035] If neither the remote response nor the remote response has been input to the interface unit 11, the process proceeds to step S8-d. In the example of FIG. 2, in step S8-d, the interface unit 11 does not present any response to the user U. As another example, in step S8-d, the interface unit 11 may present text such as "No response was received" to the user U.

[0036] As another example, the determination process of step S9 may be executed by the selection unit 12. In this case, the selection unit 12 does not perform the processes of steps S3 and S7, and after the determination process of step S9, supplies the response that has been determined to be presented to the user to the interface unit 11.

[0037] 2, if the process proceeds to step S8-d as a result of the determination process of step S9, no response is presented to the user U by the interface unit 11. Therefore, in order to avoid proceeding to step S8-d as much as possible, the selection unit 12 may wait in step S3 until a local response is provided from the local language model LM1. The selection unit 12 may wait in step S7 until a remote response is provided from the remote language model LM2.

[0038] 3 shows an example of the configuration of an information processing system 2 according to embodiment 2. Unlike the information processing system 1, the information processing system 2 further includes an attribute acquisition unit 21. In the example of FIG. 3, the terminal device 10 includes the attribute acquisition unit 21.

[0039] The attribute acquiring unit 21 may acquire question attributes, which are attributes of the question asked by the user U. As an example, the attribute acquiring unit 21 may derive the question attributes by analyzing a user input.

[0040] The question attributes may include at least one of the question content, the question format, and the question language. The question content refers to the type of question, the question topic, the user's request, etc. The question format refers to, for example, the data format of the user input. The question language refers to the type of language in which the text indicating the question is written. The question type may be information indicating which of the predetermined question types the question falls into. The question topic may be information indicating which of the predetermined topics the question falls into. The user's request may be information indicating whether a detailed answer is desired.

[0041] The attribute acquisition unit 21 may acquire system attributes, which are attributes of the information processing system 2. The system attributes may include at least one of a range of capabilities of the information processing system 2, constraints of the information processing system 2, and quick responsiveness of the information processing system 2.

[0042] The range of capabilities of the information processing system 2 refers to the range of topics and knowledge that can be handled by the information processing system 2. The constraints of the information processing system 2 refer to the maximum input size that can be processed by the information processing system 2 and constraints on the response time of the information processing system 2. The quick response of the information processing system 2 refers to the range in which the information processing system 2 can use standard phrases that do not require the use of a language model, for example.

[0043] The attribute acquisition unit 21 may acquire local attributes, which are attributes of the local language model LM1. The local attributes may include at least one of training data of the local language model LM1, a response time of the local language model LM1, accuracy of the local language model LM1, and constraints of the local language model LM1.

[0044] The attribute acquisition unit 21 may acquire remote attributes, which are attributes of the remote language model LM2. The remote attributes may include at least one of training data of the remote language model LM2, a response time of the remote language model LM2, accuracy of the remote language model LM2, and constraints of the remote language model LM2.

[0045] The training data of a language model refers to the type and amount of data that the language model has learned, etc. The constraints of a language model refer to constraints on the maximum input size that the model can process, constraints on the response time of the model, etc.

[0046] The response time of a language model refers to the time it takes for the language model to generate a response, and is affected by, for example, the amount of input information (e.g., the number of text characters in the question).

[0047] Therefore, for example, the relationship between the number of input characters of a predetermined template and the response time may be measured in advance, and based on this relationship, it is possible to predict an approximate value of the response time when the language model receives an actual question.

[0048] The response time of a language model is also affected by the knowledge domain used to train the language model. Therefore, for example, response times to questions in various knowledge domains (e.g., entertainment and sports) may be measured in advance. Then, the approximate response time may be corrected according to the response time for each knowledge domain.

[0049] The accuracy of a language model is information indicating the degree of accuracy with which the language model can output an answer to a certain question. The accuracy of a language model may be determined in advance for each field to which the question input to the language model belongs.

[0050] The accuracy of a language model is affected by the knowledge domains used to train the language model. If the relationship between knowledge domains and accuracy is known (e.g., if the relationship is publicly available), a similarity between each knowledge domain and a question input to the language model may be calculated. For example, the similarity may be normalized to identify the knowledge domain to which the question input to the language model belongs, and the accuracy corresponding to the identified knowledge domain may be derived.

[0051] If the relationship between knowledge domain and accuracy is unknown (e.g., the relationship is not publicly available), answers to the questions can be obtained by repeating various questions in a given knowledge domain, and the reliability of the answers can be predicted based on the statistical results of the various answers obtained.

[0052] As described above, the attribute acquisition unit 21 may acquire at least one of the question attribute, the system attribute, the local attribute, and the remote attribute. Therefore, in the second embodiment, the selection unit 12 may select the language model to be used to generate a response to a user input from at least one of the local language model LM1 and the remote language model LM2, based on at least one of the question attribute, the system attribute, the local attribute, and the remote attribute. According to the second embodiment, it is possible to more effectively use the local language model LM1 and the remote language model LM2.

[0053] 4 and 5 are flowcharts showing an example of the flow of processing in the information processing system 2. Explanation of the steps described in the first embodiment will be omitted. First, refer to FIG.

[0054] After step S0, the attribute acquisition unit 21 acquires a user input from the selection unit 12. Then, in step S10, the attribute acquisition unit 21 derives question attributes by analyzing the user input. In the example of FIG. 4 , the attribute acquisition unit 21 derives, as one of the question attributes, a user question attribute indicating whether the question by the user U requests detailed information. In this specification, the fact that a question requests detailed information is referred to as “details.” Whether a question requests detailed information may be determined based on (1) whether the content of the question contains specific phrases used to request detailed information, such as “in detail,” “in detail,” or “specifically,” or (2) whether the number of characters or data size of the question is equal to or greater than a predetermined value. The attribute acquisition unit 21 supplies the derived question attribute to the selection unit 12.

[0055] In the information processing system according to an aspect of the present disclosure, for example, a function of deriving question attributes may be provided to the selection unit 12. That is, the selection unit and the attribute acquisition unit according to an aspect of the present disclosure may be integrated into a single functional unit.

[0056] After step S1, in step S11, the selection unit 12 outputs a predicted value of the reliability of the local response based on the user input and the question attributes. The reliability prediction method is the same as the method for measuring the "accuracy of the language model" described above. Because a response from a language model with high answer accuracy has a high reliability, "accuracy" and "reliability" are conceptually distinguished, but the "accuracy" and "reliability" can be used interchangeably to perform the determination process described below.

[0057] As described above, the question attributes are derived by analyzing the user input. Therefore, in step S11, the selection unit 12 can be said to output a predicted value of the reliability of the local response based on the user input.

[0058] In step S12, the selection unit 12 determines whether a local response has been acquired. If the result of step S12 is YES, i.e., if a local response has been acquired, the process proceeds to step S13. On the other hand, if the result of step S12 is NO, i.e., if a local response has not been acquired, the process proceeds to step S14.

[0059] In step S12, the determination by the selection unit 12 may be made, for example, after a third predetermined time (e.g., 10 seconds) has elapsed since the selection unit 12 outputted a response generation request to the local language model LM1 in step S1.

[0060] In step S13, the selector 12 outputs the reliability of the acquired local response, and then the process proceeds to step S14. The reliability is calculated as described above.

[0061] In step S14, the selector 12 determines whether the reliability of the local response is low. As an example, the selector 12 may determine whether the reliability of the local response is low based on the predicted value of the reliability of the local response obtained in step S11.

[0062] As another example, in step S14, the selector 12 may determine whether the reliability of the local response is low based on the reliability of the local response obtained in step S13.

[0063] In this way, the predicted value of the reliability of the local response can be used in the determination process in the same way as the reliability of the local response. In other words, the predicted value of the reliability of the local response is a type of reliability of the local response.

[0064] If the answer is YES in step S14, i.e., if the reliability of the local response is low, proceed to step S5 in Fig. 5 (see symbol "B" in Figs. 4 and 5). On the other hand, if the answer is NO in step S14, i.e., if the reliability of the local response is not low, proceed to step S15.

[0065] In step S15, the selection unit 12 determines whether the user question attribute is "details." If the result in step S15 is YES, i.e., if the user question attribute is "details," the process proceeds to step S45 in Fig. 5 (see symbol "C" in Figs. 4 and 5). On the other hand, if the result in step S15 is NO, i.e., if the user question attribute is not "details," the process proceeds to step S3 in Fig. 5 (see symbol "D" in Figs. 4 and 5).

[0066] Next, refer to Fig. 5. After step S6 corresponding to symbol "B" in Fig. 5, in step S16, the selection unit 12 determines whether the response time of the remote language model LM2 (i.e., the time it takes for the remote language model LM2 to output a remote response) is within a fourth predetermined time (e.g., 30 seconds). As an example, the start point of the response time may be the time when the selection unit 12 issues a response generation request to the remote language model LM2 in step S5 in Fig. 5.

[0067] If the answer is YES in step S16, i.e., if the response time of the remote language model LM2 is within the fourth predetermined time, the process proceeds to step S7. On the other hand, if the answer is NO in step S16, i.e., if the response time of the remote language model LM2 exceeds the fourth predetermined time, the process proceeds to step S3.

[0068] The process of step S45, which corresponds to the symbol "C" in Fig. 5, is the same as the process of step S5 described above. The process of step S46 following step S45 is the same as the process of step S6 described above. After step S46, in step S17, the selection unit 12 determines whether the response time of the remote language model LM2 is within a fifth predetermined time (e.g., 30 seconds). The process of step S17 is the same as step S16.

[0069] If the answer is YES in step S17, i.e., if the response time of the remote language model LM2 is within the fifth predetermined time, proceed to step S18. On the other hand, if the answer is NO in step S17, i.e., if the response time of the remote language model LM2 exceeds the fifth predetermined time, proceed to step S3.

[0070] In step S18, the selection unit 12 integrates the local response and the remote response to generate an integrated response. Then, the selection unit 12 provides the generated integrated response to the interface unit 11. Therefore, in step S19, the interface unit 11 presents the integrated response provided by the selection unit 12 in step S18 to the user U.

[0071] In the process flow corresponding to symbol "B" in Figures 4 and 5, there is a concern that the reliability of the local response is low. Therefore, in this process flow, a remote response that is expected to be more reliable than the local response is also generated. Therefore, in this process flow, in step S19, the interface unit 11 presents the remote response provided by the selection unit 12 in step S7 to the user U.

[0072] However, as shown in Fig. 5, if a remote response is not received within a predetermined time, the interface unit 11 presents the local response supplied from the selection unit 12 in step S3 to the user U. This also applies to the processing flow corresponding to symbol "C" in Figs. 4 and 5 described below.

[0073] In the process flow corresponding to symbol "C" in Figures 4 and 5, the reliability of the local response is expected to be high. In addition, it is considered that user U requests a detailed response. Therefore, in this process flow, the selection unit 12 generates the above-mentioned integrated response in step S18. The integrated response can be more detailed than the remote response. In this way, by integrating the local response and the remote response, a detailed response that meets user U's wishes can be presented to user U.

[0074] In the process flow corresponding to symbol "D" in Figures 4 and 5, the reliability of the local response is also expected to be high. However, it is considered that user U does not request a detailed response. Therefore, in this process flow, the process of generating a remote response is not executed. Therefore, in this process flow, in step S19, the interface unit 11 presents to user U the local response provided by the selection unit 12 in step S3.

[0075] 6 shows an example of the configuration of an information processing system 3 according to embodiment 3. In the information processing system 3, the selection unit 12 in the information processing system 1 is replaced with a selection unit 32.

[0076] The selection unit 32 may obtain a local response to a user U's question by providing the local language model LM1 with the user input, which includes the question. The selection unit 32 may then determine, based on the local response, whether to further provide the user input to the remote language model LM2. That is, the selection unit 32 may determine, based on the local response, whether to further provide the question to the remote language model LM2.

[0077] 7 is a flowchart showing an example of the flow of processing in the information processing system 3. After step S2, in step S40, the selection unit 32 analyzes the local response to output local language model response attributes.

[0078] The local language model response attributes include at least one of the accuracy of the local response, the confidence of the local response, and the specificity of the local response. In the example of Figure 7, the local language model response attributes include the confidence and specificity of the local response.

[0079] The accuracy of the local response refers to, for example, how close the local response is to the facts. The accuracy of the local response may be determined separately by the selection unit 32. Alternatively, the accuracy of the local response may be determined by a functional unit other than the selection unit 32. The reliability of the local response refers to, for example, how reproducible the response of the local language model LM1 to the same question is. The detail of the local response refers to, for example, the amount of information in the local response (number of characters, data size, etc.).

[0080] After step S40, in step S20, the selection unit 32 performs an additional determination based on the reliability and detail of the local response. In step S20, the selection unit 32 determines whether the reliability of the local response is equal to or greater than a predetermined reliability threshold. The selection unit 32 also determines whether the detail of the local response is equal to or greater than a predetermined detail threshold. In step S20, branching processing is performed according to the following branches A to C.

[0081] In step S10, if a "detailed" user question attribute is obtained, the selection unit 32 may increase the detail threshold from the initial setting value. By increasing the detail threshold from the initial setting value, it is possible to improve the possibility that a more detailed response will be presented to the user U.

[0082] Branch A: If the reliability of the local response is equal to or greater than the reliability threshold and the detail level of the local response is equal to or greater than the detail level threshold, it is considered that both the reliability and detail level of the local response are sufficient. Therefore, in this case, proceed to step S3. This is because if the reliability and detail level of the local response are both sufficient, it is considered that there is no problem in presenting the local response to user U as it is.

[0083] Branch B: If the confidence of the local response is equal to or greater than the confidence threshold but the detail of the local response is less than the detail threshold, it is considered that the confidence of the local response is sufficient but the detail of the local response is insufficient. Therefore, in this case, return to step S1. This is because if the detail of the local response is insufficient, it is considered preferable to request a more detailed local response from the local language model LM1. It is considered that the detail of the local response is relatively likely to improve by repeatedly querying the local language model LM1.

[0084] The detail threshold may be set to, for example, the number of characters, "100." According to the detail threshold, if the reliability of the local response is equal to or greater than the reliability threshold and the number of characters in the local response is less than 100, branch B is selected as the branch destination. If the number of characters in the local response is less than 100, the selection unit 32 may add a condition to the local language model LM1, for example, "Please respond with 100 characters or more," and request another local response.

[0085] Branch C: If the confidence of the local response is below the confidence threshold, the confidence of the local response is considered to be insufficient. It is considered that the confidence of the local response is not likely to improve even if the local language model LM1 is repeatedly queried. Therefore, in this case, proceed to step S5, because it is considered that the remote language model LM2 should be used to obtain a more reliable response.

[0086] After step S11, in step S21, the selector 32 determines whether the reliability of the local response is low. As an example, the selector 32 may determine whether the reliability of the local response is low based on the predicted value of the reliability of the local response obtained in step S11.

[0087] If the answer is YES in step S21, i.e., if the confidence level of the local response is low, the process proceeds to step S5, since it is considered that the remote language model LM2 should be used to obtain a more reliable response.

[0088] On the other hand, if the answer to step S21 is NO, that is, if the reliability of the local response is not low, the process proceeds to step S22, where the process in this branch route is stopped.

[0089] If the reliability of the local response is not low, branch A or branch B is selected by the determination process in step S20 described above. Therefore, if the reliability of the local response is not low, the local response is presented to user U. For this reason, if the reliability of the local response is not low, there is no problem in halting the processing in the above branch route in step S22.

[0090] In the case of branch A, in step S8, the interface unit 11 presents the local response supplied from the selection unit 32 in step S3 to the user U. On the other hand, in the case of branch C, in step S8, the interface unit 11 presents the remote response supplied from the selection unit 32 in step S7 to the user U.

[0091] (Additional Information Regarding Branch B) (1) If the level of detail of the local response does not reach the level of detail threshold even after a predetermined number of queries to the local language model LM1, the additional determination in step S20 may be terminated. In this case, the process proceeds to step S3 after the additional determination is completed. This is because even if the level of detail of the local response does not reach the level of detail threshold, the reliability of the local response is high, and there is no problem in presenting the local response to the user U.

[0092] (2) The processing of step S40 in FIG. 7 does not need to be implemented. In this case, in step S20, the selection unit 32 may perform additional determination based on the predicted value of the reliability of the local response obtained in step S11. Because the predicted value of the reliability of the local response is a value derived without using the actual local response, the value does not change even if questions are repeatedly posed to the local language model LM1. For this reason, the predicted value of the reliability of the local response cannot be used to determine whether the actual reliability of the local response has increased. For this reason, in step S20, the additional determination based on the predicted value of the reliability of the local response may be performed only once.

[0093] (3) Generally, when the local language model LM1 is re-asked for the first time, the level of detail of the local response tends to improve. On the other hand, when the local language model LM1 is re-asked for the second or subsequent times, the level of detail of the local response does not improve so much. For this reason, the number of re-asks to the local language model LM1 may be limited to one.

[0094] (Another Example of Processing Flow in Embodiment 3) In step S20, the selection unit 32 may determine whether or not a response generation request to the remote language model LM2 is necessary, based on the local language model response attributes obtained in step S40.

[0095] As a first example, the selection unit 32 may determine whether or not a response generation request to the remote language model LM2 is necessary based on the accuracy of the local response. The selection unit 32 may determine that a response generation request to the remote language model LM2 is necessary when the accuracy is low (e.g., when the accuracy is less than a predetermined accuracy threshold). On the other hand, the selection unit 32 may determine that a response generation request to the remote language model LM2 is unnecessary when the accuracy is equal to or greater than the accuracy threshold.

[0096] For example, the perplexity of the local response may be used as an accuracy index. Alternatively, an output value from a functional unit that evaluates the accuracy of the local response may be used as the accuracy index. A machine learning model constructed by learning to evaluate the accuracy of an input sentence may be used as the functional unit. The machine learning model may be, for example, a known neural network.

[0097] As a second example, the selection unit 32 may determine whether a response generation request to the remote language model LM2 is necessary based on the reliability of the local response. If the reliability is low (e.g., if the reliability is less than a reliability threshold), the selection unit 32 may determine that a response generation request to the remote language model LM2 is necessary. On the other hand, if the reliability is equal to or greater than the reliability threshold, the selection unit 32 may determine that a response generation request to the remote language model LM2 is unnecessary.

[0098] In this case, the output value from a function that evaluates the confidence of the local response may be used as an indicator of accuracy. The function may be a machine learning model that is constructed by learning to evaluate the confidence of an input sentence.

[0099] As a third example, the selection unit 32 may determine whether a response generation request to the remote language model LM2 is necessary based on the level of detail of the local response. If the level of detail is low (e.g., if the level of detail is less than the level of detail threshold), the selection unit 32 may determine that a response generation request to the remote language model LM2 is necessary. On the other hand, if the level of detail is equal to or greater than the level of detail threshold, the selection unit 32 may determine that a response generation request to the remote language model LM2 is not necessary.

[0100] In this case, for example, the length of the local response (e.g., the number of characters in the local response) may be used as an indicator of specificity. Generally, the more characters in the local response, the higher the specificity assigned.

[0101] In step S20, the selection unit 32 may determine whether or not a response generation request to the remote language model LM2 is necessary based on the user question attributes obtained in step S10 and the local language model response attributes obtained in step S40.

[0102] Therefore, as a fourth example, the selection unit 32 may determine whether or not a response generation request to the remote language model LM2 is necessary based on the level of detail of the local response and the user question attribute. For example, if the user question attribute is "detailed," the selection unit 32 may increase the level of detail threshold from an initial setting value. Then, if the level of detail is less than the increased level of detail threshold, the selection unit 32 may determine whether or not a response generation request to the remote language model LM2 is necessary. On the other hand, if the level of detail is equal to or greater than the increased level of detail threshold, the selection unit 32 may determine whether or not a response generation request to the remote language model LM2 is necessary.

[0103] In any of the first to fourth examples described above, if it is determined that a response generation request to the remote language model LM2 is necessary, branch C is selected and the process proceeds to step S5. On the other hand, if it is determined that a response generation request to the remote language model LM2 is not necessary, branch A is selected and the process proceeds to step S3.

[0104] (Variations of a response generation request to the remote language model LM2) Next, variations of a response generation request to the remote language model LM2 when it is determined that the response generation request is necessary will be described. In this specification, four variations are exemplified.

[0105] (1) The selection unit 32 may supply a response generation request including context information of the user input to the remote language model LM2. For example, the history of past user inputs and responses from the information processing system 3 may be used as the context information. This allows the remote language model LM2 to generate a remote response taking into account, for example, the tendency of past questions asked by the user U. As a result, a more appropriate remote response can be generated by the user U. The range of the history may be set as appropriate.

[0106] (2) The selection unit 32 may provide the remote language model LM2 with a response generation request including information about the reliability of the local response. This allows the remote language model LM2 to generate a remote response taking the reliability of the local response into consideration. As a result, a more appropriate remote response can be generated.

[0107] The selector 32 may provide the remote language model LM2 with an answer generation request that further includes information on the accuracy of the local answer in addition to information on the reliability of the local answer.The selector 32 may provide the remote language model LM2 with an answer generation request that further includes information on the detail of the local answer in addition to information on the reliability of the local answer.

[0108] (3) The selection unit 32 may provide the remote language model LM2 with a response generation request including information about the user U's request. The information about the user U's request may be information that is not included in the user input. The information about the user U's request may be, for example, information indicating a specific topic that the user U has specified in advance prior to the user input. This allows a more appropriate remote response to be generated in line with the topic that the user U has specified in advance.

[0109] As another example, the information about the user U's request may be information indicating an upper limit on the number of characters in a response sentence that the user U preset prior to the user's input. This prevents the remote language model LM2 from generating a lengthy remote response that exceeds the upper limit. As a result, a concise remote response that meets the user U's intentions can be generated.

[0110] When the selection unit 12 generates an integrated response by integrating the local response and the remote response, the selection unit 12 may generate the integrated response so that the number of characters in the integrated response does not exceed the above upper limit.

[0111] (4) The selection unit 32 may provide the remote language model LM2 with a response generation request that includes information indicating how the local response should be used to generate a remote response. The selection unit 32 may provide the remote language model LM2 with a response generation request that includes information indicating that the remote response is to be later integrated with the local response. By having the remote language model LM2 generate a remote response based on such a response generation request, the selection unit 32 can obtain a more appropriate integrated response.

[0112] After generating the integrated response, the selection unit 32 may further supply a response generation request including an instruction to generate a summary sentence for the integrated response to the remote language model LM2.

[0113] [Modification of the Third Embodiment] In the flowchart of the third embodiment shown in Fig. 7, after the process of step S11 is executed, the process of step S10 and the process of step S21 are executed in parallel. However, the flowchart of Fig. 7 may be modified as follows, for example.

[0114] As an example, after the process of step S11 is executed, only the process of step S21 may be executed. If the result of step S21 is NO, the process proceeds to step S10 instead of step S22. Even with such a change, the information processing system 3 functions in the same way as when the process according to the flowchart of FIG. 7 is executed.

[0115] In the modified flowchart, if step S21 is YES, steps S10, S1, S2, S40, S20, and S3 are not executed, so if step S21 is YES, the response generation request to the local language model LM1 (see step S1) is not output.

[0116] 8 shows an example of the configuration of an information processing system 4 according to a fourth embodiment. Unlike the information processing system 1, the information processing system 4 further includes a first attribute management unit 41 and a second attribute management unit 42. In the example of FIG. 8, the terminal device 10 includes the first attribute management unit 41, and the server 90 includes the second attribute management unit 42.

[0117] In the fourth embodiment, the first attribute management unit 41 manages the above-described local attributes and remote attributes. The first attribute management unit 41 records the local attributes. The first attribute management unit 41 may or may not record the remote attributes. Not having the first attribute management unit 41 record the remote attributes can simplify the operation of the first attribute management unit 41. It should be noted that in this specification, "management" means handling certain information, and does not necessarily mean recording the information.

[0118] The second attribute management unit 42 records remote attributes, unlike the first attribute management unit 41. The second attribute management unit 42 may periodically update the remote attributes. If the first attribute management unit 41 does not record the remote attributes, the first attribute management unit 41 may acquire the remote attributes from the second attribute management unit 42.

[0119] The information processing system 4 may further include another server different from the server 90. In this case, the other server may have the second attribute management unit 42. That is, in the information processing system 4, a single server does not need to have both the remote language model LM2 and the second attribute management unit 42.

[0120] 9 and 10 are flowcharts showing an example of the flow of processing in the information processing system 4. FIG. 11 shows an example of a table used in some of the determination processing in the information processing system 4.

[0121] In this specification, the process of generating a standardized rule-based response to a question from a user U without using a language model is referred to as "quick response processing." The quick response processing allows a response to be presented to the user U more quickly than when a response is generated by a language model. The quick response processing allows a response to be presented to the user U even for a question that includes content that is not suitable for learning using a language model.

[0122] 9. After step S10, in step S23, the selection unit 12 executes a first quick response process. In this specification, the quick response process that handles content that can be responded to once the user U's question is confirmed is referred to as the first quick response process. On the other hand, the quick response process that generates a response according to the status of the information processing system 4 is referred to as the second quick response process.

[0123] In this specification, the response generated by the first quick response process is referred to as a “first quick response.” As an example, the selection unit 12 may generate a standard greeting (e.g., “Hello”) or a reply (e.g., “Yes”) as the first quick response.

[0124] As another example, the selection unit 12 may generate a response to a standard question (e.g., a response to a question asking what the current date is) as the first quick response. For example, the selection unit 12 can identify what the current date is by referring to calendar data pre-recorded in the terminal device 10.

[0125] As another example, the selection unit 12 may generate a response indicating that the question of the user U is unclear as the first quick response, thereby prompting the user U to ask a more clear question. Whether the question of the user U is unclear may be determined based on a known syntax analysis method.

[0126] The selection unit 12 inputs the generated first quick response to the interface unit 11. In this case, the interface unit 11 presents the first quick response acquired from the selection unit 12 to the user U in step S4.

[0127] After step S11, in step S24, the selection unit 12 outputs the degree of detail of the local response. As described above, as an example, the selection unit 12 may output the number of characters in the local response as the degree of detail of the local response.

[0128] After step S24, in step S25, the selection unit 12 executes a first inquiry determination based on the predicted value of the reliability of the local response obtained in step S11 and the level of detail of the local response obtained in step S24.

[0129] In the example of the fourth embodiment, a first prediction threshold and a second prediction threshold greater than the first prediction threshold are set for the predicted value of the reliability of the local response, and the selector 12 compares the predicted value with the first prediction threshold and the second prediction threshold.

[0130] The selector 12 determines that the reliability of the local response is "low" if the predicted value of the reliability of the local response is less than a first prediction threshold. The selector 12 determines that the reliability of the local response is "medium" if the predicted value is equal to or greater than the first prediction threshold and less than a second prediction threshold. The selector 12 determines that the reliability of the local response is "high" if the predicted value is equal to or greater than the second prediction threshold.

[0131] In the example of the fourth embodiment, it is also assumed that a detail level threshold is set for the detail level of the local response. If the detail level of the local response is less than the detail level threshold, the selection unit 12 determines that the detail level is "insufficient." If the detail level of the local response is equal to or greater than the detail level threshold, the selection unit 12 determines that the detail level is "sufficient."

[0132] The selection unit 12 then determines the branch destination from step S25 by comparing the result of the determination process with a first table TB1 shown in Fig. 11. As shown in the first table TB1, if the reliability of the local response is "low", the selection unit 12 selects branch 1 as the branch destination, regardless of the level of detail of the local response. This is because if the reliability of the local response is "low", it is considered preferable to further use the remote language model LM2.

[0133] If branch 1 is selected, the process proceeds to step S37. In step S37, the selection unit 12 generates a response generation request for the remote language model LM2 by combining the local response and the user input. The selection unit 12 provides the generated response generation request to the remote language model LM2.

[0134] 11, if the reliability of the local response is "high" and the level of detail of the local response is "sufficient," the selector 12 selects branch 3 as the branch destination. This is because in this case, it is considered that there is little need to further use the remote language model LM2. If branch 3 is selected, the process proceeds to step S3.

[0135] If the reliability of the local response is "medium," the selection unit 12 selects branch 2 as the branch destination, regardless of the level of detail of the local response. If the reliability of the local response is "high" and the level of detail of the local response is "insufficient," the selection unit 12 also selects branch 2. This is because, in these cases, it is considered preferable to ask the user U for instructions on whether or not to further use the remote language model LM2.

[0136] If Branch 2 is selected, the process proceeds to step S26. In step S26, the selection unit 12 asks a question to seek an instruction from the user U. For example, the selection unit 12 may present to the user U, via the interface unit 11, a fixed phrase asking the user U, "Do you wish to have the remote language model LM2 generate a remote response even if it requires waiting time?" The selection unit 12 acquires the user U's answer to the question via the interface unit 11.

[0137] In step S27, the selection unit 12 analyzes the answer obtained from the user U and acquires user answer attributes, which are attributes of the answer.

[0138] In step S28, the selection unit 12 acquires remote attributes from the first attribute management unit 41. Upon receiving a request from the selection unit 12, the first attribute management unit 41 supplies the remote attributes to the selection unit 12. If the first attribute management unit 41 has recorded a remote attribute, the first attribute management unit 41 supplies the remote attribute to the selection unit 12. In the example of the fourth embodiment, the remote attributes include a predicted time for a remote response.

[0139] If the first attribute management unit 41 does not record the remote attribute, the first attribute management unit 41 acquires the remote attribute from the second attribute management unit 42. Upon receiving a request from the first attribute management unit 41, the second attribute management unit 42 supplies the remote attribute to the first attribute management unit 41. The first attribute management unit 41 supplies the remote attribute acquired from the second attribute management unit 42 to the selection unit 12.

[0140] In step S29, the selection unit 12 performs a second inquiry determination based on the predicted value of the reliability of the local response obtained in step S11, the level of detail of the local response obtained in step S24, the user response attributes obtained in step S27, and the predicted time of the remote response obtained in step S28.

[0141] In the example of the fourth embodiment, a first time threshold and a second time threshold greater than the first time threshold are set for the predicted time of the remote response, and the selector 12 compares the predicted time with the first and second prediction thresholds.

[0142] The selection unit 12 determines that the predicted time of the remote response is "short" when the predicted time of the remote response is less than the first time threshold. The selection unit 12 determines that the predicted time of the remote response is "long" when the predicted time of the remote response is equal to or greater than the first time threshold and less than the second time threshold. The selection unit 12 determines that the predicted time of the remote response is "very long" when the predicted time of the remote response is equal to or greater than the second time threshold. The first time threshold and the second time threshold are set based on the magnitude relationship of "short" < "long" < "very long" regarding the predicted time of the remote response.

[0143] The selection unit 12 determines whether the user answer attributes include the attributes of “time priority” and “detail.” “Time priority” means that the user U desires to receive a response from the language model as quickly as possible.

[0144] The selector 12 then compares the results of the determination process with the second table TB2 shown in FIG. 11 to determine the branch destination from step S29. As an example, consider a case where the reliability of the local response is "high" and the level of detail of the local response is "insufficient." The determination result in this case does not depend on the user response attributes.

[0145] As shown in the first and second items from the top of the second table TB2, if the predicted time of the remote response is "very long" or "long," the selector 12 selects branch 5 as the branch destination. This is because in these cases, it is considered that a considerable amount of time is required to obtain a remote response from the remote language model LM2. If branch 5 is selected, the process proceeds to step S3. If branch 5 is selected, the remote language model LM2 is not used.

[0146] On the other hand, as shown in the third item from the top of the second table TB2, if the predicted time for the remote response is "short," the selector 12 selects branch 4 as the branch destination. This is because in this case, it is considered that a remote response can be obtained from the remote language model LM2 in a relatively short time. If branch 4 is selected, the process proceeds to step S37. If branch 4 is selected, the remote language model LM2 is used.

[0147] Next, we will describe the case where the level of detail of the local response is "medium." In this case, the determination result does not depend on the level of detail of the local response. As an example, if the user response attribute is "detailed" and the predicted time of the remote response is "long," the selection unit 12 selects branch 6 as the branch destination. If branch 6 is selected, the process proceeds to step S30.

[0148] If the user response attribute is "detailed" and the predicted time of the remote response is "long," it is considered that it will take a considerable amount of time for the remote language model LM2 to generate a detailed remote response that meets the intention of the user U. Therefore, in step S30, the selection unit 12 executes the second quick response process.

[0149] The second quick response process may be an interrupt process that occurs depending on the status of the information processing system 4. In this specification, a response generated by the second quick response process is referred to as a second quick response. By presenting the second quick response to the user U who is waiting for the generation of a remote response, the psychological burden on the user U can be reduced.

[0150] As an example, the selection unit 12 may output a standard phrase indicating that a remote response is being prepared as the second quick response. By presenting the second quick response to the user U, the user U can recognize that the information processing system 4 has accepted the user U's question. This can give the user U a sense of security. In this case, the selection unit 12 selects branch 7 as the branch destination. If branch 7 is selected, the process proceeds to step S37.

[0151] As another example, the selection unit 12 may output, as the second quick response, a question asking the user U whether or not to wait any longer for the generation of the remote response, because in some cases, it may take an extremely long time that is not practically reasonable for the remote language model LM2 to generate the remote response.

[0152] The selection unit 12 acquires the user U's answer to the question via the interface unit 11. If the selection unit 12 receives an answer from the user U indicating that they will continue to wait, the selection unit 12 selects branch 7.

[0153] On the other hand, if the selection unit 12 receives a response from the user U indicating that it will not continue waiting, the selection unit 12 selects branch 8. In this case, as an example, the selection unit 12 may generate a warning message indicating that, although the reliability of the local response is not high, the local response will be presented in accordance with the user U's wishes, and present the warning message to the user U via the interface unit 11.

[0154] Next, refer to Fig. 10. As indicated by the symbol "F" in Fig. 9 and Fig. 10, after step S37, the process proceeds to step S31. In step S31, the selection unit 12 performs a determination based on the reliability of the local response obtained in step S11 and the detail level of the local response obtained in step S24.

[0155] As an example, if the selection unit 12 determines that the reliability of the local response is "low," the process proceeds to step S35. In this case, it is considered that there is little need to present the local response to the user U. Therefore, in step S35, the selection unit 12 supplies only the remote response to the interface unit 11. In this case, in step S34, the interface unit 11 presents only the remote response acquired from the selection unit 12 to the user U.

[0156] As another example, if the reliability of the local response is "high" and the level of detail of the local response is "sufficient," the process proceeds to step S32. In step S32, the selection unit 12 integrates the local response and the remote response to generate an integrated response. In step S33, the selection unit 12 provides the generated integrated response to the interface unit 11. In this case, in step S34, the interface unit 11 presents the integrated response obtained from the selection unit 12 to the user U.

[0157] How the local response and the remote response should be integrated may vary depending on the relationship between the local response and the remote response. Therefore, in step S32, the selection unit 12 may derive a similarity between the local response and the remote response. Then, the selection unit 12 may select a method for integrating the local response and the remote response based on the similarity.

[0158] For example, the selection unit 12 may compare the similarity with a predetermined similarity threshold. If the similarity is equal to or greater than the similarity threshold, the selection unit 12 may generate a summary of the local response. Then, the selection unit 12 may combine the summary with the remote response to generate an integrated response. This is because, if the similarity is high, it is considered unnecessary to present the entire local response together with the remote response to the user U.

[0159] On the other hand, if the similarity is less than the similarity threshold, the selector 12 may generate an integrated response by combining the local response and the remote response, because if the similarity is low, it is considered preferable to present the entire local response together with the remote response to the user U.

[0160] [Supplementary Notes on Each Embodiment] The processes described in the above-mentioned embodiments 1 to 4 include common elements. The following provides additional information on such processes.

[0161] (Supplementary Note 1) A process in which the selection unit supplies a user's question to a remote language model based on some condition. Specifically, this process is the process leading to step S5 in embodiment 2, the process leading to step S45 in embodiment 2, the process leading to step S5 in embodiment 3, and the process leading to step S37 in embodiment 4. As described above, the conditions leading to these processes are any of the following 1 to 3: (1) when the confidence level of the local response is less than a predetermined value; (2) when the detail level of the local response is less than a predetermined value; (3) when the question attribute includes "details."

[0162] (Supplementary Note 2) A process in which the selection unit supplies the question of user U to the local language model multiple times based on some condition. Specifically, this process is a process that proceeds to branch B as a result of the determination process in step S20 in embodiment 3. As described above, the condition for proceeding to branch B is when the reliability of the local response is equal to or greater than a predetermined value and the detail level of the local response is less than a predetermined value.

[0163] (Supplementary Note 3) A process in which the selection unit presents only local responses to the user based on some condition. Specifically, this process is the process leading to step S3 in the second embodiment, and the process proceeding to branch A as a result of the determination process in step S20 in the third embodiment. As described above, the condition leading to these processes is when the reliability of the local response is equal to or greater than a predetermined value.

[0164] (Supplementary Note 4) A process in which the selection unit presents only remote responses to the user based on some condition. Specifically, this process is the process leading to step S7 in embodiment 2, the process leading to step S7 in embodiment 3, and the process leading to step S35 in embodiment 4. As described above, the condition leading to these processes is when the reliability of the local response is less than a predetermined value.

[0165] (Supplementary Note 5) A process in which the selection unit presents the user with an integrated response of the local response and the remote response based on some condition. Specifically, this process is the process leading to step S18 in embodiment 2 and the process leading to step S32 in embodiment 4. As described above, the condition for leading to these processes is that the selection unit has acquired both the local response and the remote response, and the reliability of the local response is equal to or greater than a predetermined value.

[0166] [Example of implementation using software] The functions of information processing systems 1 to 4 (hereinafter referred to as "devices" for convenience) can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device.

[0167] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The functions described in each of the above embodiments are realized by executing the program using the control device and storage device.

[0168] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0169] In addition, some or all of the functions of each of the control blocks can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of one aspect of the present disclosure. In addition, the functions of each of the control blocks can also be realized by, for example, a quantum computer.

[0170] As described above, each process described in each of the above embodiments can be executed by AI (Artificial Intelligence). The AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0171] [Summary] The information processing system according to aspect 1 of the present disclosure includes an interface unit that accepts input from a user, a local language model, a remote language model that is different from the local language model, and a selection unit that selects from at least one of the local language model and the remote language model a language model to be used to generate a response to a question input into the interface unit by the user.

[0172] In an information processing system according to aspect 2 of the present disclosure, in aspect 1, the selection unit may select the language model to be used based on at least one of question attributes, which are attributes of the question, system attributes, which are attributes of the information processing system, local attributes, which are attributes of the local language model, and remote attributes, which are attributes of the remote language model.

[0173] In an information processing system according to aspect 3 of the present disclosure, in aspect 2, the system attributes may include at least one of the range of capabilities of the information processing system, the constraints of the information processing system, and the quick response of the information processing system.

[0174] In the information processing system according to aspect 4 of the present disclosure, in aspects 2 or 3, the question attributes may include at least one of (i) the content of the question, including at least one of the type of the question, the topic of the question, and the user's request regarding the question, (ii) the format of the question, and (iii) the language of the question.

[0175] In an information processing system according to aspect 5 of the present disclosure, in any one of aspects 2 to 4, the local attributes may include at least one of training data of the local language model, response time of the local language model, accuracy of the local language model, and constraints of the local language model.

[0176] In an information processing system according to aspect 6 of the present disclosure, in any one of aspects 2 to 5, the remote attributes may include at least one of training data of the remote language model, response time of the remote language model, accuracy of the remote language model, and constraints of the remote language model.

[0177] In an information processing system according to aspect 7 of the present disclosure, in any one of aspects 1 to 6, the selection unit may obtain a local response, which is a response to the question by the local language model, by supplying the question to the local language model, and may determine whether to further supply the question to the remote language model based on the local response.

[0178] The information processing system according to aspect 8 of the present disclosure may be any one of aspects 2 to 7, further comprising a first attribute management unit that manages the local attributes and the remote attributes.

[0179] An information processing system according to aspect 9 of the present disclosure may further include a second attribute management unit different from the first attribute management unit in aspect 8, and if the first attribute management unit does not record the remote attribute, the first attribute management unit may acquire the remote attribute from the second attribute management unit that records the remote attribute.

[0180] In an information processing system according to aspect 10 of the present disclosure, in any one of aspects 1 to 9, the selection unit may supply the question to the local language model and the remote language model, and may present to the user through the interface unit the response obtained after a predetermined time has elapsed, from among a local response, which is a response to the question from the local language model, and a remote response, which is a response to the question from the remote language model.

[0181] In an information processing system according to aspect 11 of the present disclosure, in any one of aspects 1 to 10, the selection unit may derive the reliability and level of detail of a local response, which is a response to the question by the local language model, and may derive question attributes, which are attributes of the question, by analyzing the question, and may supply the question to the remote language model if any of the following conditions 1 to 3 is satisfied: (1) the reliability of the local response is less than a predetermined value; (2) the level of detail of the local response is less than a predetermined value; or (3) the question attribute includes "details".

[0182] In an information processing system according to aspect 12 of the present disclosure, in any one of aspects 1 to 11, the selection unit may supply the question to the local language model, derive the reliability and detail of a local response that is a response to the question by the local language model, and if the reliability of the local response is greater than or equal to a predetermined value and the detail of the local response is less than the predetermined value, supply the question to the local language model again.

[0183] In an information processing system according to aspect 13 of the present disclosure, in any one of aspects 1 to 12, the selection unit may supply the question to the local language model, derive a reliability of a local response that is a response to the question by the local language model, and if the reliability of the local response is greater than or equal to a predetermined value, present the local response to the user through the interface unit.

[0184] In an information processing system relating to aspect 14 of the present disclosure, in any one of aspects 1 to 13, the selection unit may obtain a remote response, which is a response to the question by the remote language model, by supplying the question to the remote language model, may derive a reliability of a local response, which is a response to the question by the local language model, and if the reliability of the local response is less than a predetermined value, may present the remote response to the user through the interface unit.

[0185] In an information processing system according to aspect 15 of the present disclosure, in aspect 14, the selection unit may obtain the local response by supplying the question to the local language model, and may derive a reliability of the local response based on the local response.

[0186] In the information processing system according to aspect 16 of the present disclosure, in aspect 14, the selection unit may derive the reliability of the local response by analyzing the question.

[0187] In an information processing system according to aspect 17 of the present disclosure, in any one of aspects 1 to 16, the selection unit may supply the question to the local language model and the remote language model, thereby obtaining a local response which is a response to the question from the local language model, and a remote response which is a response to the question from the remote language model, and may derive a reliability of the local response, and if the reliability of the local response is greater than or equal to a predetermined value, may integrate the local response and the remote response to derive an integrated response, and may present the integrated response to the user through the interface unit.

[0188] [Additional Notes] One aspect of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of one aspect of the present disclosure. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.

[0189] (Cross-reference to related applications) This application claims the benefit of priority to Japanese Patent Application No. 2023-190993, filed on November 8, 2023, the entire contents of which are incorporated herein by reference.

[0190] 1, 2, 3, 4 Information processing system 10 Terminal device 11 Interface unit 12, 32 Selection unit 21 Attribute acquisition unit 41 First attribute management unit 42 Second attribute management unit 90 Server LM1 Local language model LM2 Remote language model U User

Claims

1. An information processing system comprising: an interface unit that accepts input from a user; a local language model; a remote language model different from the local language model; and a selection unit that selects a language model to be used from at least one of the local language model and the remote language model to generate a response to a question input by the user to the interface unit.

2. The information processing system of claim 1, wherein the selection unit selects the language model to be used based on at least one of question attributes which are attributes of the question, system attributes which are attributes of the information processing system, local attributes which are attributes of the local language model, and remote attributes which are attributes of the remote language model.

3. The information processing system according to claim 2, wherein the system attributes include at least one of: a range of capabilities of the information processing system; constraints of the information processing system; and quick responsiveness of the information processing system.

4. The information processing system of claim 2, wherein the question attributes include at least one of: the content of the question, including at least one of the type of the question, the topic of the question, and the user's request content regarding the question; the format of the question; and the language of the question.

5. The information processing system of claim 2, wherein the local attributes include at least one of: training data of the local language model; response time of the local language model; accuracy of the local language model; and constraints of the local language model.

6. The information processing system of claim 2, wherein the remote attributes include at least one of: training data of the remote language model; response time of the remote language model; accuracy of the remote language model; and constraints of the remote language model.

7. An information processing system as described in any one of claims 1 to 6, wherein the selection unit obtains a local response, which is a response to the question by the local language model, by supplying the question to the local language model, and determines whether or not to further supply the question to the remote language model based on the local response.

8. An information processing system according to any one of claims 2 to 6, further comprising a first attribute management unit that manages the local attributes and the remote attributes.

9. An information processing system as described in claim 8, further comprising a second attribute management unit different from the first attribute management unit, wherein when the first attribute management unit does not record the remote attribute, the first attribute management unit obtains the remote attribute from the second attribute management unit which records the remote attribute.

10. An information processing system as described in any one of claims 1 to 6, wherein the selection unit supplies the question to the local language model and the remote language model, and presents to the user via the interface unit a response obtained at a predetermined time point from among a local response to the question from the local language model and a remote response to the question from the remote language model.

11. An information processing system as described in any one of claims 1 to 6, wherein the selection unit derives the reliability and detail of a local response, which is a response to the question by the local language model, derives question attributes, which are attributes of the question, by analyzing the question, and supplies the question to the remote language model when any of the following conditions 1 to 3 is satisfied: (1) if the reliability of the local response is less than a predetermined value; (2) if the detail of the local response is less than a predetermined value; or (3) if the question attribute includes "details".

12. An information processing system as described in any one of claims 1 to 6, wherein the selection unit supplies the question to the local language model, derives a confidence and a level of detail of a local response which is a response to the question by the local language model, and if the confidence of the local response is greater than or equal to a predetermined value and the level of detail of the local response is less than the predetermined value, supplies the question again to the local language model.

13. An information processing system as described in any one of claims 1 to 6, wherein the selection unit supplies the question to the local language model, derives a reliability of a local response which is a response to the question by the local language model, and if the reliability of the local response is equal to or greater than a predetermined value, presents the local response to the user via the interface unit.

14. An information processing system as described in any one of claims 1 to 6, wherein the selection unit: supplies the question to the remote language model, thereby obtaining a remote response which is a response to the question by the remote language model; derives a reliability of a local response which is a response to the question by the local language model; and if the reliability of the local response is less than a predetermined value, presents the remote response to the user via the interface unit.

15. The information processing system according to claim 14, wherein the selection unit obtains the local response by supplying the question to the local language model, and derives a confidence level of the local response based on the local response.

16. The information processing system according to claim 14, wherein the selection unit derives the confidence of the local response by analyzing the question.

17. An information processing system as described in any one of claims 1 to 6, wherein the selection unit: supplies the question to the local language model and the remote language model to obtain a local response which is a response to the question from the local language model, and a remote response which is a response to the question from the remote language model; derives a reliability of the local response; and, if the reliability of the local response is equal to or greater than a predetermined value, integrates the local response and the remote response to derive an integrated response; and presents the integrated response to the user via the interface unit.

Images