Information processing system

Abstract

Provided is an information processing system that differs from conventional information processing systems. An information processing system (1) includes an interface unit (11) that receives input from a user (U), a local language model (LM1), a remote language model (LM2) that differs from the local language model (LM1), and a selection unit (12) that selects, from at least one of the local language model (LM1) and the remote language model (LM2), a language model that should be used to generate a response to a question input by the user (U) to the interface unit (11).

Description

Technical Field

[0001] The following disclosure relates to an information processing system with a language model. Background Technology

[0002] In recent years, various solutions have been proposed for information processing technologies that utilize language models. For example, Patent Document 1 below discloses an example of the structure of an information processing system that enables a language model to generate abstract text.

[0003] Existing technical documents Patent documents Patent Document 1: Japanese Patent Application Publication No. 2023-73095 Summary of the Invention The problem the invention aims to solve The purpose of this disclosure is to provide an information processing system that is different from the past.

[0004] Solution for solving the problem The information processing system disclosed herein includes: an interface unit for receiving input from a user; a local language model; a remote language model different from the local language model; and a selection unit that selects from at least one of the local language model and the remote language model a language model to be used for generating a response to a question input by the user to the interface unit.

[0005] Invention Effects According to one aspect of this disclosure, it is possible to provide an information processing system that differs from the conventional one. Attached Figure Description

[0006] Figure 1 An example of the configuration of the information processing system in Implementation 1 is shown.

[0007] Figure 2 This is a flowchart illustrating the processing flow in Implementation Method 1.

[0008] Figure 3 An example of the configuration of the information processing system in Embodiment 2 is shown.

[0009] Figure 4 This is a flowchart illustrating the processing flow in Implementation Method 2.

[0010] Figure 5 This is a flowchart illustrating the processing flow in Implementation Method 2.

[0011] Figure 6 An example of the configuration of the information processing system in Embodiment 3 is shown.

[0012] Figure 7 This is a flowchart illustrating the processing flow in Implementation Method 3.

[0013] Figure 8 An example of the configuration of the information processing system in Embodiment 4 is shown.

[0014] Figure 9 This is a flowchart illustrating the processing flow in Implementation Method 4.

[0015] Figure 10 This is a flowchart illustrating the processing flow in Implementation Method 4.

[0016] Figure 11 An example of a table used for a portion of the determination process in Implementation 4 is shown. Detailed Implementation

[0017] [Implementation Method 1] The following describes Embodiment 1. For ease of explanation, components having the same function as those described in Embodiment 1 will be labeled with the same reference numerals in the following embodiments and will not be described again. For simplicity, well-known technical matters are also appropriately omitted. The components and values ​​described in this disclosure are merely illustrative unless there is a particular contradiction. Therefore, for example, unless there is a particular contradiction, the positional and connection relationships of the components are not limited to the examples in the figures.

[0018] Figure 1 An example configuration of the information processing system 1 in Embodiment 1 is shown. The information processing system 1 includes an interface unit 11 and a selection unit 12. Furthermore, the information processing system 1 includes multiple language models.

[0019] exist Figure 1 In the example, information processing system 1 has a local language model LM1 and a remote language model LM2. The remote language model LM2 is a different language model from the local language model LM2. The local language model LM1 and the remote language model LM2 can also be referred to as the first language model and the second language model, respectively. Both the local language model LM1 and the remote language model LM2 can be well-known learned models.

[0020] Interface unit 11 receives input from user U. In this specification, the input from user U is referred to as user input. Interface unit 11 supplies the received user input to selection unit 12. In Embodiment 1, the case where the user input is text data representing a question posed by user U to the language model is mainly illustrated.

[0021] As another example, user input can also be voice data. Voice data could be, for example, the speech data of user U. Alternatively, user input can also be image data. User input can be any data that the language model involved in this disclosure can receive.

[0022] In Implementation 1, the response to user input is generated by a language model. Implementation 1 primarily illustrates the case where the language model generates text data as a response. However, the language model disclosed in this embodiment can also generate speech data or image data as a response.

[0023] The interface unit 11 acquires the response generated by the language model via the selection unit 12. The interface unit 11 then presents the acquired response to the user U. Therefore, the interface unit 11 can also be referred to as an input / output unit.

[0024] Interface unit 11 can receive user input in any manner. Interface unit 11 can also prompt the user U with the response obtained via selection unit 12 in any manner. As an example, interface unit 11 can receive user input in a dialog form and prompt the user U with the response.

[0025] like Figure 1 As shown, the information processing system 1 may include a terminal device 10 and a server 90. The terminal device 10 may also be referred to as an edge device. As an example, the terminal device 10 may be a device owned by user U. The server 90 may be, for example, a cloud server. In the information processing system 1, it is sufficient that the terminal device 10 and the server 90 are connected in a communicative manner.

[0026] exist Figure 1 In the example, the terminal device 10 includes an interface unit 11, a selection unit 12, and a local language model LM1. In Embodiment 1, the local language model LM1 is a language model smaller than the remote language model LM2. Therefore, based on the local language model LM1, responses can be generated more quickly than those generated by the remote language model LM2. In this specification, the response of the local language model LM1 is referred to as a local response.

[0027] exist Figure 1 In the example, server 90 has a remote language model LM2. In Implementation 1, the remote language model LM2 is a language model larger than the local language model LM1. Therefore, based on the remote language model LM2, responses with higher credibility than those based on the local language model LM2 can be generated. In this specification, responses based on the remote language model LM2 are referred to as remote responses.

[0028] As an example, the administrator of information processing system 1 (or the administrator of server 90) can relearn the remote language model LM2 as needed, thereby updating LM2. This allows for the provision of more reliable remote responses to user U.

[0029] In information processing system 1, responses to user input can be generated using at least one of a local language model LM1 and a remote language model LM2. Therefore, information processing system 1 includes a selection unit 12.

[0030] The selection unit 12 selects the language model to be used for generating a response to user input from at least one of the local language model LM1 and the remote language model LM2. According to the selection unit 12, it is possible to distinguish between using the local language model LM1 and the remote language model LM2 to generate a response to user input.

[0031] Figure 2 This is a flowchart illustrating an example of the processing flow in information processing system 1. First, in step S0, interface unit 11 receives user input. Interface unit 11 supplies the user input to selection unit 12. In step S1, selection unit 12 outputs a response generation request to local language model LM1 based on the user input. Thus, selection unit 12 can first select local language model LM1 as the language model to be used to generate a response to the user input.

[0032] In step S2, the local language model LM1 generates a local response corresponding to the acquired response generation request. For example, the local language model LM1 generates a local response corresponding to the question included in the response generation request. The local language model LM1 then supplies the generated local response to the selection unit 12.

[0033] In step S3, the selection unit 12 inputs the acquired local response to the interface unit 11.

[0034] However, if the selection unit 12 does not receive a local response after the selection unit 12 outputs a response generation request to the local language model LM1 in step S1, the selection unit 12 may skip the processing in step S3. If the processing in step S3 is skipped, the local response will not be input to the interface unit 11. The 10 seconds, as an example of the first predetermined time, is assumed to be the time after user U asks a question to information processing system 1 and begins to worry whether information processing system 1 has received user U's question.

[0035] The selection unit 12 can also select a remote language model LM2 as the language model to be used for generating a response to user input. For example, in step S5 after step S2, the selection unit 12 can output a response generation request to the remote language model LM2 based on user input. In this case, the selection unit 12 can generate the response generation request to the remote language model LM2 as information including user input and local response.

[0036] In step S6, the remote language model LM2 generates a remote response corresponding to the acquired response generation request. By supplying the response generation request containing the local response to the remote language model LM2, the remote language model LM2 can generate a remote response based on the local response. The remote language model LM2 then supplies the generated remote response to the selection unit 12.

[0037] In step S7, the selection unit 12 inputs the acquired remote response to the interface unit 11.

[0038] However, if the selection unit 12 does not receive a remote response after the selection unit 12 outputs a response generation request to the remote language model LM2 in step S5, the selection unit 12 may skip the processing in step S7. If the processing in step S7 is skipped, the remote response will not be input to the interface unit 11.

[0039] It is generally believed that the time required to obtain a remote response from the remote language model LM2 is longer than the time required to obtain a local response from the local language model LM1. Therefore, it is assumed that user U can tolerate a longer waiting time when using the remote language model LM2 compared to when using the local language model LM1. Thus, the second specified time (e.g., 30 seconds) can be set to be longer than the first specified time (e.g., 10 seconds).

[0040] In step S9, after a specified time has elapsed, the interface unit 11 may determine which response to prompt to the user U based on whether a local response and / or a remote response have been input to the interface unit 11. The starting point of the specified time may be, for example, any one of the execution times of step S0, step S1, or step S5.

[0041] For example, if only a local response is input to the interface unit 11, the process proceeds to step S8-a. In step S8-a, the interface unit 11 prompts the user U with a local response.

[0042] When both local and remote responses are input to interface unit 11, the process proceeds to step S8-b. In step S8-b, interface unit 11 first prompts user U with a local response. Furthermore, after prompting with the local response, interface unit 11 further prompts user U with a remote response.

[0043] If only a remote response is input to the interface unit 11, proceed to step S8-c. In step S8-c, the interface unit 11 prompts the user U for a remote response.

[0044] If no remote response or any response from the remote response is input to interface unit 11, proceed to step S8-d. Figure 2In the example, in step S8-d, the interface unit 11 does not prompt any response to the user U. As another example, in step S8-d, the interface unit 11 may also prompt the user U with text such as "failed to obtain a response".

[0045] As another example, the determination process in step S9 can also be performed by the selection unit 12. In this case, the selection unit 12 does not perform the processes in steps S3 and S7, and after the determination process in step S9, it provides the response that should be prompted to the user to the interface unit 11.

[0046] exist Figure 2 In the example, if the decision process of step S9 results in step S8-d, the interface unit 11 does not prompt the user U with any response. Therefore, to avoid entering step S8-d as much as possible, the selection unit 12 can standby in step S3 until a local response is provided from the local language model LM1. The selection unit 12 can also standby in step S7 until a remote response is provided from the remote language model LM2.

[0047] [Implementation Method 2] Figure 3 An example configuration of the information processing system 2 in Embodiment 2 is shown. Unlike information processing system 1, information processing system 2 also includes an attribute acquisition unit 21. Figure 3 In the example, the terminal device 10 has an attribute acquisition unit 21.

[0048] The attribute acquisition unit 21 can acquire the attributes of the user U's question, i.e., the question attributes. As an example, the attribute acquisition unit 21 can derive the question attributes by parsing the user input.

[0049] A question's attributes can include at least one of the following: question content, question format, and question language. Question content refers to the type of question, the topic of the question, and the user's requirements. Question format refers to, for example, the format of the data entered by the user. Question language refers to the type of language used to record the text expressing the question. The question type can indicate which of the predefined question types it belongs to. The question topic can indicate which of the predefined topics it belongs to. The user's requirements can indicate whether a detailed answer is requested.

[0050] The attribute acquisition unit 21 can also acquire system attributes, which are attributes of the information processing system 2. The system attributes may include at least one of the following: the scope of the information processing system 2, the limitations of the information processing system 2, and the rapid responsiveness of the information processing system 2.

[0051] The scope of information processing system 2 refers to the range of topics and knowledge that information processing system 2 can handle. The limitations of information processing system 2 refer to the maximum input size that information processing system 2 can process and the limitations of its response time. The rapid responsiveness of information processing system 2 refers, for example, to the range of standardized text that can be used within information processing system 2 without the need for a language model.

[0052] The attribute acquisition unit 21 can also acquire the attributes of the local language model LM1, i.e., local attributes. The local attributes may include at least one of the following: the learning data of the local language model LM1, the response time of the local language model LM1, the accuracy of the local language model LM1, and the limitations of the local language model LM1.

[0053] The attribute acquisition unit 21 can also acquire the attributes of the remote language model LM2, i.e., the remote attributes. The remote attributes may include at least one of the following: the learning data of the remote language model LM2, the response time of the remote language model LM2, the accuracy of the remote language model LM2, and the limitations of the remote language model LM2.

[0054] The learning data for a language model refers to the types and amounts of data that the language model learns from. Limitations of a language model refer to limitations on the maximum input size it can process and limitations on its response time.

[0055] The response time of a language model refers to the time required for the language model to generate a response. The response time of a language model is affected by factors such as the amount of input information (e.g., the number of characters in the question text).

[0056] Therefore, for example, the relationship between the number of characters in a predefined input text and the response time can be determined in advance. Based on this relationship, it is possible to predict the approximate response time of the language model when it receives an actual question.

[0057] Furthermore, the response time of the language model is also influenced by the knowledge domain used for learning the language model. Therefore, for example, the response time for questions in various knowledge domains (e.g., entertainment and sports) can be predetermined. And, the approximate value of the response time can be corrected based on the response time for each knowledge domain.

[0058] The precision of a language model refers to information indicating the level of precision with which the language model can output an answer to a given question. The precision of a language model can also be predefined for each domain to which the input question belongs.

[0059] The accuracy of a language model is influenced by the knowledge domain used for its learning. When the relationship between knowledge domain and accuracy is known (e.g., when the relationship is publicly available), the similarity between each knowledge domain and the question input into the language model can be calculated. For example, by normalizing the similarity, it is possible to determine which knowledge domain the question input into the language model belongs to, and derive the accuracy corresponding to that determined knowledge domain.

[0060] When the relationship between the knowledge domain and precision is unknown (e.g., when the relationship is not publicly disclosed), responses to the questions can be obtained by repeatedly defining various questions within the knowledge domain. In this case, the credibility of the responses can be predicted based on the statistical results of the obtained responses.

[0061] As described above, the attribute acquisition unit 21 can acquire at least one of the following: query attribute, system attribute, local attribute, and remote attribute. Therefore, in Embodiment 2, the selection unit 12 can select, based on at least one of the query attribute, system attribute, local attribute, and remote attribute, the language model to be used for generating a response to user input from at least one of the local language model LM1 and the remote language model LM2. According to Embodiment 2, it is possible to more effectively distinguish between using the local language model LM1 and the remote language model LM2.

[0062] Figures 4-5 This is a flowchart illustrating an example of the processing flow in information processing system 2. The steps described in embodiment 1 are omitted from the explanation. First, refer to... Figure 4 .

[0063] After step S0, the attribute acquisition unit 21 acquires user input from the selection unit 12. Furthermore, in step S10, the attribute acquisition unit 21 derives the query attributes by parsing the user input. Figure 4 In the example, the attribute acquisition unit 21 exports a user question attribute, which is one of the question attributes and indicates whether user U's question requires detailed information. In this specification, the requirement for detailed information is referred to as "detailed". Whether a question requires detailed information can be determined based on (1) whether the question content contains specific phrases used when requesting detailed information, such as "detailed", "specifically", or "more specifically", or based on (2) whether the number of characters or the data size of the question is above a specified value. The attribute acquisition unit 21 supplies the exported question attribute to the selection unit 12.

[0064] In the information processing system disclosed herein, for example, the function of exporting query attributes can also be assigned to the selection unit 12. That is, the selection unit and the attribute acquisition unit in this disclosure can also be integrated as a single functional unit.

[0065] Following step S1, in step S11, the selection unit 12 outputs a predicted value of the credibility of the local response based on the user input and question attributes. The method for predicting credibility is the same as the method for measuring the "accuracy of the language model" described above. Since the credibility of a response obtained from a language model with high response accuracy is high, "accuracy" and "credibility" are conceptually distinguished, but "accuracy" and "credibility" can be used equally to perform the judgment process described later.

[0066] Furthermore, as mentioned above, the question attributes are derived by parsing user input. Therefore, in step S11, it can also be said that the selection unit 12 outputs a confidence prediction value of the local response based on user input.

[0067] In step S12, the selection unit 12 determines whether a local response has been obtained. If the result in step S12 is "yes," meaning a local response has been obtained, the process proceeds to step S13. Conversely, if the result in step S12 is "no," meaning a local response has not been obtained, the process proceeds to step S14.

[0068] In step S12, for example, after the selection unit 12 outputs a response generation request to the local language model LM1 in step S1, a third predetermined time (e.g., 10 seconds) may pass before the selection unit 12 performs a determination.

[0069] In step S13, the selection unit 12 outputs the credibility of the acquired local response. Then, it proceeds to step S14. The credibility calculation method is as described above.

[0070] In step S14, the selection unit 12 determines whether the credibility of the local response is low. As an example, the selection unit 12 may determine whether the credibility of the local response is low based on the predicted credibility value of the local response obtained in step S11.

[0071] As another example, in step S14, the selection unit 12 can determine whether the credibility of the local response is low based on the credibility of the local response obtained in step S13.

[0072] In this way, the predicted confidence value of the local response can be used for decision processing in the same way as the confidence value of the local response. In other words, the predicted confidence value of the local response is a kind of confidence value of the local response.

[0073] If step S14 is "Yes", meaning the reliability of the local response is low, then proceed to... Figure 5 Step S5 (refer to) Figures 4-5 (Referring to the attached figure "B"). On the other hand, if the answer in step S14 is "No", that is, if the credibility of the local response is not low, proceed to step S15.

[0074] In step S15, the selection unit 12 determines whether the user's question attribute is "detailed". If the result in step S15 is "yes", meaning the user's question attribute is "detailed", then proceeds to... Figure 5 Step S45 (refer to) Figures 4-5 (See attached figure numeral "C"). On the other hand, if step S15 is "No", that is, if the user's question attribute is not "detailed", then proceed to... Figure 5 Step S3 (refer to) Figures 4-5 (The attached figure is labeled "D").

[0075] Next, refer to Figure 5 .correspond Figure 5 Following step S6 (marked by reference numeral "B"), in step 16, the selection unit 12 determines whether the response time of the remote language model LM2 (i.e., the time when the remote language model LM2 outputs a remote response) is within a fourth predetermined time (e.g., 30 seconds). As an example, the starting point of the response time could be... Figure 5 In step S5, select the moment when part 12 sends a response generation request to the remote language model LM2.

[0076] If step S16 is "Yes", meaning the response time of the remote language model LM2 is within the fourth predetermined time, proceed to step S7. On the other hand, if step S16 is "No", meaning the response time of the remote language model LM2 exceeds the fourth predetermined time, proceed to step S3.

[0077] correspond Figure 5 The processing of step S45, indicated by reference numeral "C" in the attached figure, is the same as the processing of step S5 described above. The processing of step S46, following step S45, is the same as the processing of step S6 described above. After step S46, in step S17, the selection unit 12 determines whether the response time of the remote language model LM2 is within a fifth predetermined time (e.g., 30 seconds). The processing of step S17 is the same as that of step S16.

[0078] If step S17 is "Yes", meaning the response time of the remote language model LM2 is within the fifth predetermined time, proceed to step S18. On the other hand, if step S17 is "No", meaning the response time of the remote language model LM2 exceeds the fifth predetermined time, proceed to step S3.

[0079] In step S18, the selection unit 12 integrates the local response and the remote response to generate an integrated response. Then, the selection unit 12 supplies the generated integrated response to the interface unit 11. Therefore, in step S19, the interface unit 11 prompts the user U with the integrated response supplied from the selection unit 12 in step S18.

[0080] In the corresponding Figures 4-5In the processing flow indicated by reference numeral "B" in the attached diagram, there is concern about the low reliability of the local response. Therefore, in this processing flow, a remote response, which is expected to have higher reliability than the local response, is further generated. Consequently, in step S19 of this processing flow, the interface unit 11 prompts the user U with the remote response provided from the selection unit 12 in step S7.

[0081] However, as Figure 5 As shown, if a remote response is not received within the specified time, the interface unit 11 prompts the user U with the local response provided from the selection unit 12 in step S3. This applies to the following correspondence. Figures 4-5 The processing procedure for the attached figure mark "C" is the same.

[0082] In the corresponding Figures 4-5 In the processing flow indicated by the reference numeral "C", the reliability of the local response is expected to be not low. Furthermore, it is assumed that user U requests a detailed response. Therefore, in the processing flow, selection unit 12 generates the aforementioned integrated response in step S18. The integrated response can be a more detailed response than the remote response. Thus, by integrating the local response and the remote response, a detailed response that meets user U's expectations can be provided to user U.

[0083] In the corresponding Figures 4-5 In the processing flow indicated by the reference numeral "D" in the attached diagram, a high degree of reliability of the local response is expected. However, it is assumed that user U does not require a detailed response. Therefore, the process of generating a remote response is not performed in the processing flow. Consequently, in the processing flow, in step S19, the interface unit 11 prompts user U for the local response provided from the selection unit 12 in step S3.

[0084] [Implementation Method 3] Figure 6 An example configuration of the information processing system 3 in Embodiment 3 is shown. In the information processing system 3, the selection unit 12 in the information processing system 1 is replaced by the selection unit 32.

[0085] Selection unit 32 can obtain a local response to a question by feeding user input containing a question from user U to local language model LM1. Then, selection unit 32 can decide whether to further feed the user input to remote language model LM2 based on the local response. That is, selection unit 32 can decide whether to further feed the question to remote language model LM2 based on the local response.

[0086] Figure 7 This is a flowchart illustrating an example of the processing flow in information processing system 3. After step S2, in step S40, selection unit 32 outputs local language model response attributes by parsing the local response.

[0087] The local language model response attributes include at least one of the following: accuracy of the local response, credibility of the local response, and detail of the local response. Figure 7 The example local language model response attributes include the credibility and detail of the local response.

[0088] The accuracy of a local response refers, for example, to the degree to which the local response closely approximates the facts. The accuracy of a local response can be determined separately by the selection unit 32. Alternatively, the accuracy of a local response can be determined by a functional unit different from the selection unit 32. The credibility of a local response refers, for example, to the degree of reproducibility of the response from the local language model LM1 to the same question. The detail of a local response refers, for example, to the amount of information (number of words, data size, etc.) in the local response.

[0089] After step S40, in step S20, the selection unit 32 performs additional determination based on the credibility and detail of the local response. In step S20, the selection unit 32 determines whether the credibility of the local response is above a predetermined credibility threshold. Furthermore, the selection unit 32 determines whether the detail of the local response is above a predetermined detail threshold. In step S20, branching processing is performed as described below for branches A to C.

[0090] Furthermore, in step S10, if the user's question attribute is found to be "detailed," the selection unit 32 can increase the detail threshold from the initial setting. By increasing the detail threshold from the initial setting, the likelihood of providing the user U with a more detailed response can be increased.

[0091] Branch A: If the credibility of the local response is above the credibility threshold and the detail of the local response is above the detail threshold, then the credibility and detail of the local response are considered sufficient. Therefore, in this case, proceed to step S3. This is because if the credibility and detail of the local response are both sufficient, there is no problem in directly prompting the user U with the local response.

[0092] Branch B: If the credibility of the local response is above the credibility threshold, but the detail of the local response is below the detail threshold, the credibility of the local response is considered sufficient, but the detail of the local response is insufficient. Therefore, in this case, return to step S1. This is because if the detail of the local response is insufficient, it is considered preferable to request a more detailed local response from the local language model LM1. Regarding the detail of the local response, it is considered highly likely to be improved by repeatedly asking the local language model LM1.

[0093] The detail threshold can be set to, for example, 100 characters. Based on this detail threshold, if the credibility of the local response is above the credibility threshold and the number of characters in the local response is less than 100, branch B is selected as the branch destination. If the number of characters in the local response is less than 100, the selection unit 32 can add conditions such as "please respond with 100 characters or more" to the local language model LM1, requesting another local response.

[0094] Branch C: If the credibility of the local response is less than the credibility threshold, the credibility of the local response is considered insufficient. Regarding the credibility of the local response, it is believed that even if the question is repeatedly asked to the local language model LM1, the likelihood of improvement is not very high. Therefore, in this case, proceed to step S5. This is because, to obtain a response with higher credibility, it is believed that the remote language model LM2 should be used.

[0095] After step S11, in step S21, the selection unit 32 determines whether the credibility of the local response is low. As an example, the selection unit 32 may determine whether the credibility of the local response is low based on the predicted credibility value of the local response obtained in step S11.

[0096] If "yes" is selected in step S21, meaning the credibility of the local response is low, proceed to step S5. This is because, to obtain a response with higher credibility, it is considered that the remote language model LM2 should be used.

[0097] On the other hand, if the answer in step S21 is "No," meaning the reliability of the local response is not low, then proceed to step S22. In step S22, the processing in this branch path is terminated.

[0098] If the reliability of the local response is not low, the decision process in step S20 above selects either branch A or branch B. Therefore, if the reliability of the local response is not low, the local response is displayed to user U. Thus, even if the processing in the branch path is terminated in step S22, it is not a problem if the reliability of the local response is not low.

[0099] In the case of branch A, in step S8, the interface unit 11 prompts the user U with the local response provided from the selection unit 32 in step S3. On the other hand, in the case of branch C, in step S8, the interface unit 11 prompts the user U with the remote response provided from the selection unit 32 in step S7.

[0100] (Supplement regarding branch B) (1) If the detail of the local response does not reach the detail threshold even after repeatedly asking the local language model LM1 a specified number of times, the additional determination in step S20 can be terminated. In this case, after the additional determination is completed, step S3 is entered. This is because even if the detail of the local response does not reach the detail threshold, the credibility of the local response is high, so there is no problem in prompting the user U with the local response.

[0101] (2) Figure 7 The processing in step S40 can also be omitted. In this case, in step S20, the selection unit 32 can perform an additional determination based on the credibility prediction value of the local response obtained in step S11. The credibility prediction value of the local response is a value derived without using the actual local response, so the value will not change even if the question is repeatedly asked to the local language model LM1. Therefore, it is impossible to determine whether the actual credibility of the local response has improved using the credibility prediction value of the local response. Thus, in step S20, the additional determination based on the credibility prediction value of the local response can be performed only once.

[0102] (3) Generally speaking, the level of detail in the local response tends to increase when the local language model LM1 is asked a question for the first time. On the other hand, even if the local language model LM1 is asked a question a second or subsequent time, the level of detail in the local response does not increase significantly. Therefore, the number of times the local language model LM1 is asked a question can be limited to once.

[0103] (Another example of the processing flow in Implementation Method 3) In step S20, the selection unit 32 may determine whether it is necessary to send a response generation request to the remote language model LM2 based on the local language model response attributes obtained in step S40.

[0104] As a first example, the selection unit 32 can determine whether a response generation request needs to be sent to the remote language model LM2 based on the accuracy of the local response. If the accuracy is low (e.g., the accuracy is less than a predetermined accuracy threshold), the selection unit 32 can determine that a response generation request needs to be sent to the remote language model LM2. On the other hand, if the accuracy is above the accuracy threshold, the selection unit 32 can determine that a response generation request does not need to be sent to the remote language model LM2.

[0105] As an indicator of accuracy, the perplexity of the local response can be used, for example. Alternatively, the value output from the function that evaluates the accuracy of the local response can be used as an indicator of accuracy. This function can use a machine learning model built by learning to evaluate the accuracy of the input text. The machine learning model can be, for example, a well-known neural network.

[0106] As a second example, the selection unit 32 can determine whether a response generation request needs to be sent to the remote language model LM2 based on the credibility of the local response. If the credibility is low (e.g., the credibility is less than a credibility threshold), the selection unit 32 can determine that a response generation request needs to be sent to the remote language model LM2. Conversely, if the credibility is above the credibility threshold, the selection unit 32 can determine that a response generation request does not need to be sent to the remote language model LM2.

[0107] In this case, the value output from the function unit that evaluates the credibility of the local response can be used as an indicator of accuracy. This function unit can utilize a machine learning model built by learning to evaluate the credibility of the input text.

[0108] As a third example, the selection unit 32 can determine whether a response generation request needs to be sent to the remote language model LM2 based on the level of detail of the local response. If the level of detail is low (e.g., the level of detail is less than a level of detail threshold), the selection unit 32 can determine that a response generation request needs to be sent to the remote language model LM2. On the other hand, if the level of detail is above the level of detail threshold, the selection unit 32 can determine that a response generation request does not need to be sent to the remote language model LM2.

[0109] In this case, the length of the local response (e.g., the number of characters in the local response) can be used as an indicator of detail. Generally speaking, the more characters a local response has, the higher the level of detail it is assigned.

[0110] In step S20, the selection unit 32 may determine whether it is necessary to send a response generation request to the remote language model LM2 based on the user question attributes obtained in step S10 and the local language model response attributes obtained in step S40.

[0111] Therefore, as a fourth example, the selection unit 32 can determine whether a response generation request needs to be sent to the remote language model LM2 based on the detail of the local response and the user's question attributes. For example, if the user's question attribute is "detailed," the selection unit 32 can increase the detail threshold from the initial setting. Furthermore, if the detail is less than the increased detail threshold, the selection unit 32 can determine whether a response generation request needs to be sent to the remote language model LM2. On the other hand, if the detail is greater than or equal to the increased detail threshold, the selection unit 32 can determine whether a response generation request needs to be sent to the remote language model LM2.

[0112] In any of the first to fourth examples above, if it is determined that a response generation request needs to be sent to the remote language model LM2, branch C is selected, and the process proceeds to step S5. On the other hand, if it is determined that a response generation request does not need to be sent to the remote language model LM2, branch A is selected, and the process proceeds to step S3.

[0113] (A variation of the response generation request to the remote language model LM2) Next, variations of the response generation request in cases where it is determined that a response generation request needs to be sent to the remote language model LM2 will be described. Four variations are illustrated in this specification.

[0114] (1) The selection unit 32 can supply a response generation request containing contextual information of user input to the remote language model LM2. The contextual information can be, for example, the history of past user input and responses from the information processing system 3. Thus, for example, the past questioning tendencies of user U can be considered, allowing the remote language model LM2 to generate a remote response. As a result, a remote response more suitable for user U can be generated. The range of the history can be appropriately set.

[0115] (2) The selection unit 32 can supply a response generation request containing information about the credibility of the local response to the remote language model LM2. As a result, the credibility of the local response can be taken into account, and the remote language model LM2 can generate a remote response. As a result, a more suitable remote response can be generated.

[0116] The selection unit 32 may provide a response generation request to the remote language model LM2, which may include information about the accuracy of the local response in addition to information about the reliability of the local response. The selection unit 32 may also provide a response generation request to the remote language model LM2, which may include information about the detail of the local response in addition to information about the reliability of the local response.

[0117] (3) The selection unit 32 can supply a response generation request containing information about the user U's request to the remote language model LM2. The information about the user U's request may be information not included in the user input. For example, the information about the user U's request may be information representing a specific topic that the user U has specified in advance before the user inputs. Thus, a more suitable remote response that conforms to the topic specified in advance by the user U can be generated.

[0118] As another example, information about user U's request could be, for instance, information indicating a pre-defined upper limit for the number of characters in the response text before user input. This prevents the remote language model LM2 from generating redundant remote responses exceeding the upper limit. Consequently, it enables the generation of concise remote responses that meet user U's expectations.

[0119] When the selection unit 12 integrates local and remote responses to generate an integrated response, the selection unit 12 can generate the integrated response in a manner that ensures the number of characters in the integrated response does not exceed the upper limit.

[0120] (4) The selection unit 32 may supply a response generation request containing information indicating how to generate a remote response using the local response. The selection unit 32 may also supply a response generation request containing information indicating that the remote response is planned to be integrated with the local response. By having the remote language model LM2 generate a remote response based on such a response generation request, the selection unit 32 can obtain a more suitable integrated response.

[0121] The selection unit 32 can also supply a response generation request, which includes an instruction to generate a summary text for the integrated response after it has been generated, to the remote language model LM2.

[0122] [Modification of Implementation Method 3] exist Figure 7 In the flowchart of the illustrated implementation method 3, after the processing of step S11, the processing of step S10 and the processing of step S21 are executed in parallel. However, Figure 7 The flowchart can be modified as follows, for example.

[0123] As an example, after executing step S11, it is also possible to execute only step S21. Furthermore, if the result in step S21 is "No," the process proceeds to step S10 instead of step S22. Even with this change, the information processing system 3 remains consistent with the execution based on... Figure 7 The same action is taken in the case of processing the flowchart.

[0124] In the flowchart for making the aforementioned change, if step S21 is "Yes", steps S10, S1, S2, S40, S20, and S3 are not executed. Therefore, if step S21 is "Yes", the output of the response generation request to the local language model LM1 is not executed (refer to step S1).

[0125] [Implementation Method 4] Figure 8 An example configuration of the information processing system 4 in embodiment 4 is shown. Unlike information processing system 1, information processing system 4 further includes a first attribute management unit 41 and a second attribute management unit 42. Figure 8 In the example, the terminal device 10 has a first attribute management unit 41, and the server 90 has a second attribute management unit 42.

[0126] In embodiment 4, the first attribute management unit 41 manages the aforementioned local attributes and remote attributes. The first attribute management unit 41 records local attributes. The first attribute management unit 41 may or may not record remote attributes. Not recording remote attributes in the first attribute management unit 41 simplifies the use of the first attribute management unit 41. It should be noted that "management" in this specification refers to processing certain information and does not necessarily mean recording the information.

[0127] Unlike the first attribute management unit 41, the second attribute management unit 42 records remote attributes. The second attribute management unit 42 can periodically update remote attributes. If the first attribute management unit 41 does not record remote attributes, it can obtain them from the second attribute management unit 42.

[0128] Information processing system 4 may also have another server, different from server 90. In this case, the other server may have a second attribute management unit 42. That is, in information processing system 4, a single server does not necessarily have both a remote language model LM2 and a second attribute management unit 42.

[0129] Figure 9 and Figure 10 This is a flowchart illustrating an example of the processing flow in information processing system 4. Figure 11 An example of a table used for a portion of the decision processing in information processing system 4 is shown.

[0130] In this specification, the process of generating rule-based, formulaic responses to user U's questions without using a language model is referred to as "fast response processing." According to fast response processing, responses can be suggested to user U more quickly than when a language model generates the response. Fast response processing can also suggest responses to user U even for questions containing content that is unsuitable for learning through a language model.

[0131] First, refer to Figure 9 After step S10, in step S23, the selection unit 12 performs the first fast response process. In this specification, the process of handling the content that can be answered when the user U's question is determined in the fast response process is called the first fast response process. On the other hand, the process of generating a response corresponding to the status of the information processing system 4 in the fast response process is called the second fast response process.

[0132] In this specification, the response generated by the first quick response processing is referred to as the first quick response. As an example, the selection unit 12 may generate a standard greeting (e.g., "Hello") or a response (e.g., "Yes") as the first quick response.

[0133] As another example, the selection unit 12 can generate a response to a typed question (e.g., "response to a question asking about today's date") as a first quick response. For example, the selection unit 12 can determine today's date by referring to calendar data pre-recorded in the terminal device 10.

[0134] As another example, selection unit 12 can generate a first quick response indicating that user U's question is ambiguous. This encourages user U to ask a more specific question. The determination of whether user U's question is ambiguous can be based on well-known syntax parsing methods.

[0135] The selection unit 12 inputs the generated first quick response to the interface unit 11. In this case, in step S4, the interface unit 11 prompts the user U with the first quick response obtained from the selection unit 12.

[0136] Following step S11, in step S24, the selection unit 12 outputs the level of detail of the local response. As described above, for example, the selection unit 12 can output the number of characters in the local response as the level of detail of the local response.

[0137] After step S24, in step S25, the selection unit 12 performs a first query determination based on the confidence prediction value of the local response obtained in step S11 and the detail of the local response obtained in step S24.

[0138] In the example of Implementation 4, a first prediction threshold and a second prediction threshold greater than the first prediction threshold are set for the predicted confidence value of the local response. The selection unit 12 compares the predicted value with the first prediction threshold and the second prediction threshold.

[0139] If the predicted confidence value of the local response is less than a first prediction threshold, the selection unit 12 determines that the confidence level of the local response is "low". If the predicted value is above the first prediction threshold but less than a second prediction threshold, the selection unit 12 determines that the confidence level of the local response is "medium". If the predicted value is above the second prediction threshold, the selection unit 12 determines that the confidence level of the local response is "high".

[0140] In the example of Embodiment 4, a detail threshold is also set for the detail of the local response. If the detail of the local response is less than the detail threshold, the selection unit 12 determines that the detail is "insufficient". If the detail of the local response is greater than or equal to the detail threshold, the selection unit 12 determines that the detail is "sufficient".

[0141] Furthermore, the selection unit 12 will determine the result of the processing and... Figure 11 Referring to the first table TB1 shown, the branch destination after step S25 is determined. As shown in the first table TB1, if the confidence level of the local response is "low", the selection unit 12 selects branch 1 as the branch destination regardless of the detail level of the local response. This is because if the confidence level of the local response is "low", it is considered preferable to further use the remote language model LM2.

[0142] If branch 1 is selected, proceed to step S37. In step S37, the selection unit 12 generates a response generation request to the remote language model LM2 by combining the local response with the user input. The selection unit 12 then supplies the generated response generation request to the remote language model LM2.

[0143] like Figure 11 As shown in the first table TB1, if the credibility of the local response is "high" and the detail of the local response is "sufficient", the selection unit 12 selects branch 3 as the branch destination. This is because, in this case, the necessity of further using the remote language model LM2 is considered low. If branch 3 is selected, proceed to step S3.

[0144] When the confidence level of the local response is "medium", the selection unit 12 selects branch 2 as the branch destination regardless of the level of detail in the local response. When the confidence level of the local response is "high" and the level of detail in the local response is "insufficient", the selection unit 12 also selects branch 2. This is because, in these cases, it is considered preferable to solicit instructions from user U on whether to further use the remote language model LM2.

[0145] If branch 2 is selected, proceed to step S26. In step S26, the selection unit 12 asks a question to solicit instructions from user U. For example, the selection unit 12 may prompt user U with a template via the interface unit 11, asking, "Even if there is a waiting time, do you want the remote language model LM2 to generate a remote response?" The selection unit 12 obtains user U's answer to the question via the interface unit 11.

[0146] In step S27, the selection unit 12 parses the answer obtained from user U and obtains the attributes of the answer, namely the user answer attributes.

[0147] In step S28, the selection unit 12 obtains remote attributes from the first attribute management unit 41. The first attribute management unit 41 receives a request from the selection unit 12 and supplies the remote attributes to the selection unit 12. If the first attribute management unit 41 has recorded remote attributes, it supplies the remote attributes to the selection unit 12. In the example of Embodiment 4, the remote attributes include the predicted time of a remote response.

[0148] If the first attribute management unit 41 does not record a remote attribute, it obtains the remote attribute from the second attribute management unit 42. The second attribute management unit 42 receives a request from the first attribute management unit 41 and supplies the remote attribute to the first attribute management unit 41. The first attribute management unit 41 then supplies the remote attribute obtained from the second attribute management unit 42 to the selection unit 12.

[0149] In step S29, the selection unit 12 performs a second query determination based on the credibility prediction value of the local response obtained in step S11, the detail of the local response obtained in step S24, the user answer attribute obtained in step S27, and the predicted time of the remote response obtained in step S28.

[0150] In the example of Embodiment 4, a first time threshold and a second time threshold greater than the first time threshold are set for the predicted time of remote response. The selection unit 12 compares the predicted time with the first prediction threshold and the second prediction threshold.

[0151] If the predicted remote response time is less than a first time threshold, the selection unit 12 determines the predicted time to be "short". If the predicted remote response time is above the first time threshold but less than a second time threshold, the selection unit 12 determines the predicted time to be "long". If the predicted remote response time is above the second time threshold, the selection unit 12 determines the predicted time to be "very long". The first time threshold and the second time threshold are set based on the relationship that the predicted remote response time is "short" < "long" < "very long".

[0152] The selection unit 12 determines whether the user's answer attributes include "time priority" and "details". Time priority means that user U wants to get a response from the language model as soon as possible.

[0153] Furthermore, the selection unit 12 will determine the result of the processing and... Figure 11 Referring to the second table TB2 shown, the branch direction after step S29 is determined. As an example, the case where the credibility of the local response is "high" and the detail of the local response is "insufficient" is explained. In this case, the judgment result does not depend on the user's answer attributes.

[0154] As shown in the first and second items from the top of Table TB2, when the predicted remote response time is "extremely long" or "long", selection unit 12 selects branch 5 as the branch destination. This is because, in these cases, it is assumed that obtaining the remote response from the remote language model LM2 will take a considerable amount of time. If branch 5 is selected, proceed to step S3. If branch 5 is selected, the remote language model LM2 is not used.

[0155] On the other hand, as shown in the third item from the top of Table TB2 in the second table, if the predicted time for the remote response is "short", the selection unit 12 selects branch 4 as the branch destination. This is because, in this case, it is assumed that the remote response can be obtained from the remote language model LM2 in a relatively short time. If branch 4 is selected, proceed to step S37. If branch 4 is selected, the remote language model LM2 is used.

[0156] Next, the case where the detail level of the local response is "medium" will be explained. In this case, the determination result does not depend on the detail level of the local response. As an example, if the user's response attribute is "detailed" and the predicted time of the remote response is "extremely long", the selection unit 12 selects branch 6 as the branch destination. If branch 6 is selected, the process proceeds to step S30.

[0157] If the user's response attribute is "detailed" and the predicted time for the remote response is "extremely long," it is assumed that the remote language model LM2 will require a considerable amount of time to generate a detailed remote response that meets the user U's expectations. Therefore, in step S30, the selection unit 12 performs the second fast response processing.

[0158] The second rapid response processing can be a process generated in an interrupt manner based on the status of the information processing system 4. In this specification, the response generated through the second rapid response processing is referred to as the second rapid response. By prompting the user U, who is waiting for the remote response to be generated, with the second rapid response, the psychological burden on the user U can be reduced.

[0159] As an example, selection unit 12 can output a pre-defined message indicating that a remote response is being prepared as a second quick response. By prompting user U with the second quick response, user U can be made aware that user U's question has been received by information processing system 4. This provides user U with a sense of reassurance. In this case, selection unit 12 selects branch 7 as the branch destination. If branch 7 is selected, the process proceeds to step S37.

[0160] As another example, selection unit 12 can output a question asking user U whether to continue waiting for the remote response to be generated as a second quick response. This is because sometimes the remote language model LM2 may require an inappropriately long time to generate a remote response in practical applications.

[0161] Selection unit 12 obtains the user U's answer to the question via interface unit 11. If selection unit 12 receives the answer "continue waiting" from user U, it selects branch 7.

[0162] On the other hand, if the selection unit 12 receives a "do not continue waiting" response from the user U, it selects branch 8. In this case, as an example, the selection unit 12 may generate a notice stating "The credibility of the local response is not high, but the local response is suggested according to your intention" and suggest the notice to the user U via the interface unit 11.

[0163] Next, refer to Figure 10 .like Figure 9 and Figure 10As indicated by the reference numeral "F" in the attached drawing, after step S37, the process proceeds to step S31. In step S31, the selection unit 12 performs a determination based on the credibility of the local response obtained in step S11 and the detail of the local response obtained in step S24.

[0164] As an example, if the selection unit 12 determines that the reliability of the local response is "low," it proceeds to step S35. In this case, it is considered that the necessity of prompting the user U with the local response is low. Therefore, in step S35, the selection unit 12 only supplies the remote response to the interface unit 11. In this case, in step S34, the interface unit 11 only prompts the user U with the remote response obtained from the selection unit 12.

[0165] As another example, if the credibility of the local response is "high" and the detail of the local response is "sufficient," then proceed to step S32. In step S32, the selection unit 12 integrates the local response and the remote response to generate an integrated response. In step S33, the selection unit 12 supplies the generated integrated response to the interface unit 11. In this case, in step S34, the interface unit 11 prompts the user U with the integrated response obtained from the selection unit 12.

[0166] It should be noted that the method for integrating local and remote responses can vary depending on the relationship between them. Therefore, in step S32, the selection unit 12 can derive the similarity between the local and remote responses. Furthermore, the selection unit 12 can select a method for integrating the local and remote responses based on the similarity.

[0167] As an example, the selection unit 12 can compare the similarity with a predetermined similarity threshold. If the similarity is above the threshold, the selection unit 12 can generate a summary of the local response. Furthermore, the selection unit 12 can generate an integrated response by combining the summary with the remote response. This is because, in the case of high similarity, it is deemed unnecessary to present the entire local response along with the remote response to the user U.

[0168] On the other hand, if the similarity is less than the similarity threshold, the selection unit 12 can generate an integrated response by combining the local response and the remote response. This is because, in the case of low similarity, it is considered more preferable to present the local response as a whole along with the remote response to the user U.

[0169] [Supplementary information regarding each implementation method] The processes described in embodiments 1 to 4 above contain common elements. Further details regarding these processes are provided below.

[0170] (Supplement 1) The selection unit provides the user's question to the remote language model based on certain conditions. This process is specifically the process of reaching step S5 in Embodiment 2, the process of reaching step S45 in Embodiment 2, the process of reaching step S5 in Embodiment 3, and the process of reaching step S37 in Embodiment 4. As described above, the conditions for reaching these processes are any one of the following 1 to 3.

[0171] (1) When the credibility of the local response is less than the specified value; (2) The detail of the local response is less than the specified value; (3) Cases where the question attribute includes "details".

[0172] (Supplement 2) The selection unit repeatedly provides the local language model with the user U's questions based on certain conditions. Specifically, this process is the process of entering branch B as a result of the determination process in step S20 of implementation method 3. As described above, the condition for entering branch B is that the credibility of the local response is above a predetermined value and the detail of the local response is less than a predetermined value.

[0173] (Supplement 3) The selection unit prompts the user only with the local response based on certain conditions. This process specifically includes the process of reaching step S3 in Embodiment 2 and the process of entering branch A as a result of the determination process in step S20 of Embodiment 3. As described above, the condition for reaching these processes is that the reliability of the local response is above a predetermined value.

[0174] (Supplement 4) The selection unit prompts the user only for remote response based on certain conditions. This process specifically includes the process reaching step S7 in Embodiment 2, the process reaching step S7 in Embodiment 3, and the process reaching step S35 in Embodiment 4. As described above, the condition for reaching these processes is that the reliability of the local response is less than a predetermined value.

[0175] (Supplement 5) The selection unit prompts the user with an integrated response of local and remote responses based on certain conditions. This process specifically refers to the process reaching step S18 in Embodiment 2 and the process reaching step S32 in Embodiment 4. As described above, the conditions for reaching these processes are that the selection unit has obtained both the local and remote responses, and the reliability of the local response is above a predetermined value.

[0176] [Software Implementation Example] The functions of the information processing systems 1 to 4 (hereinafter referred to as "devices") can be implemented by programs that enable the computer to operate as the device, or by programs that enable the computer to operate as each control block of the device.

[0177] In this case, the device includes a computer, which serves as hardware for executing the program and has at least one control device (e.g., a processor) and at least one storage device (e.g., a memory). The program is executed by the control device and the storage device to achieve the functions described in the above embodiments.

[0178] The program can be recorded on one or more non-transitory, computer-readable recording media. This recording medium may or may not be present in the device. In the latter case, the program can be supplied to the device via any wired or wireless transmission medium.

[0179] Furthermore, some or all of the functions of the aforementioned control blocks can also be implemented using logic circuits. For example, an integrated circuit having logic circuits for implementing the functions of the aforementioned control modules is also within the protection scope of this embodiment. Alternatively, the functions of the aforementioned control blocks can also be implemented using a quantum computer, for example.

[0180] As described above, the processes described in the various embodiments can be performed by AI (Artificial Intelligence). The AI ​​can operate on the control device described above, or it can operate on other devices (such as edge computers or cloud servers).

[0181] 〔Summarize〕 The information processing system disclosed in Method 1 includes: an interface unit for receiving input from a user; A local language model; a remote language model different from the local language model; and a selection unit that selects from at least one of the local language model and the remote language model the language model to be used for generating a response to a question input by the user to the interface unit.

[0182] In the information processing system disclosed in Method 2, based on Method 1, the selection unit can select the language model to be used based on at least one of the attributes of the question (question attributes), the attributes of the information processing system (system attributes), the attributes of the local language model (local attributes), and the attributes of the remote language model (remote attributes).

[0183] In the information processing system disclosed in Method 3, based on Method 2, the system attributes may include at least one of the following: the scope of the information processing system, the limitations of the information processing system, and the rapid responsiveness of the information processing system.

[0184] In the information processing system disclosed in Method 4, based on Method 2 or 3, the question attribute may include at least one of the following: (i) the content of the question, including the type of the question, the topic of the question, and the user's request content regarding the question; (ii) the form of the question; and (iii) the language of the question.

[0185] In the information processing system disclosed in Method 5, based on any one of Methods 2 to 4, the local attribute may include at least one of the following: the learning data of the local language model, the response time of the local language model, the accuracy of the local language model, and the limitations of the local language model.

[0186] In the information processing system disclosed in Method 6, based on any one of Methods 2 to 5, the remote attribute may include at least one of the following: learning data of the remote language model, response time of the remote language model, accuracy of the remote language model, and limitations of the remote language model.

[0187] In the information processing system disclosed in Method 7, based on any one of Methods 1 to 6, the selection unit can obtain the local language model's response to the question (i.e., the local response) by supplying the question to the local language model, and decide whether to further supply the question to the remote language model based on the local response.

[0188] The information processing system according to Embodiment 8 of this disclosure, based on any one of Embodiments 2 to 7, may include a first attribute management unit for managing the local attributes and the remote attributes.

[0189] The information processing system disclosed in Method 9 may further include a second attribute management unit, which is different from the first attribute management unit, in addition to Method 8. If the first attribute management unit does not record the remote attribute, the remote attribute can be obtained from the second attribute management unit that records the remote attribute.

[0190] In the information processing system disclosed in Method 10, based on any one of Methods 1 to 9, the selection unit can supply the question to the local language model and the remote language model, and from the local language model's response to the question (i.e., the local response) and the remote language model's response to the question (i.e., the remote response), the interface unit will prompt the user with the response that has been obtained after a predetermined time has elapsed.

[0191] In the information processing system disclosed in Method 11, based on any one of Methods 1 to 10, the selection unit can derive the credibility and detail of the local language model's response to the question, i.e., the local response's reliability and detail. It derives the question's attributes, i.e., the question attributes, by parsing the question. If any one of the following conditions 1 to 3 is satisfied, the question is supplied to the remote language model. Conditions 1 to 3 are as follows: (1) The credibility of the local response is less than the specified value; (2) The detail of the local response is less than the specified value; (3) The question attribute includes "details".

[0192] In the information processing system disclosed in Method 12, based on any one of Methods 1 to 11, the selection unit can supply the question to the local language model, derive the credibility and detail of the local language model's response to the question, i.e., the local response, and if the credibility of the local response is above a predetermined value and the detail of the local response is below a predetermined value, the question is supplied to the local language model again.

[0193] In the information processing system disclosed in Method 13, based on any one of Methods 1 to 12, the selection unit can supply the question to the local language model, derive the credibility of the local language model's response to the question (i.e., the local response), and if the credibility of the local response is above a predetermined value, the interface unit will prompt the user with the local response.

[0194] In the information processing system disclosed in Method 14, based on any one of Methods 1 to 13, the selection unit can obtain a remote response as the response of the remote language model to the question by supplying the question to the remote language model, and derive the credibility of the local response as the response of the local language model to the question. If the credibility of the local response is less than a predetermined value, the selection unit will prompt the user with the remote response via the interface unit.

[0195] In the information processing system disclosed in Method 15, based on Method 14, the selection unit can obtain the local response by supplying the question to the local language model, and derive the credibility of the local response based on the local response.

[0196] In the information processing system disclosed in Method 16, based on Method 14, the selection unit can derive the credibility of the local response by parsing the question.

[0197] In the information processing system disclosed in Method 17, based on any one of Methods 1 to 16, the selection unit can obtain the local response (i.e., the local response) and the remote response (i.e., the remote response) of the local language model by supplying the question to the local language model and the remote language model, derive the credibility of the local response, and if the credibility of the local response is above a predetermined value, integrate the local response and the remote response to derive an integrated response, and prompt the integrated response to the user via the interface unit.

[0198] [Additional Notes] This disclosure is not limited to the embodiments described above, and various modifications can be made within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included within the technical scope of this disclosure. Furthermore, by combining the technical means disclosed in each embodiment, new technical features can be formed.

[0199] (Cross-reference to related applications) This application claims priority to Japanese Patent Application No. 2023-190993, filed on November 8, 2023, the entire contents of which are incorporated herein by reference.

[0200] Explanation of reference numerals in the attached figures 1, 2, 3, 4 Information Processing System 10 terminal devices 11 Interface Section 12, 32 Selection Section 21 Attribute Acquisition Department 41 First Attribute Management Department 42 Second Attribute Management Department 90 server LM1 local language model LM2 remote language model U user.

Claims

1. An information processing system, comprising: The interface section for receiving input from the user; Local language model; A remote language model that is different from the local language model; and The selection unit selects, from at least one of the local language model and the remote language model, the language model to be used for generating a response to a question input by the user to the interface unit.

2. The information processing system according to claim 1, characterized in that, The selection unit selects the language model to be used based on at least one of the following: the attributes of the question (question attributes), the attributes of the information processing system (system attributes), the attributes of the local language model (local attributes), and the attributes of the remote language model (remote attributes).

3. The information processing system according to claim 2, characterized in that, The system attributes include at least one of the following: the scope of the information processing system, the limitations of the information processing system, and the rapid responsiveness of the information processing system.

4. The information processing system according to claim 2, characterized in that, The question attributes include: The content of the question includes at least one of the following: the type of question, the topic of the question, and the user's request regarding the question; The form of the question; and At least one of the languages ​​used in the question.

5. The information processing system according to claim 2, characterized in that, The local attributes include at least one of the following: the learning data of the local language model, the response time of the local language model, the accuracy of the local language model, and the limitations of the local language model.

6. The information processing system according to claim 2, characterized in that, The remote attribute includes at least one of the following: the learning data of the remote language model, the response time of the remote language model, the accuracy of the remote language model, and the limitations of the remote language model.

7. The information processing system according to any one of claims 1 to 6, characterized in that, The selection unit provides the question to the local language model, obtains the local language model's response to the question (i.e., the local response), and decides whether to further provide the question to the remote language model based on the local response.

8. The information processing system according to any one of claims 2 to 6, characterized in that, A first attribute management unit is provided to manage the local attributes and the remote attributes.

9. The information processing system according to claim 8, characterized in that, It also has a second attribute management department, which is different from the first attribute management department. If the first attribute management unit does not record the remote attribute, the remote attribute is obtained from the second attribute management unit, which records the remote attribute.

10. The information processing system according to any one of claims 1 to 6, characterized in that, The selection unit provides the question to the local language model and the remote language model, and from the local language model's response to the question (local response) and the remote language model's response to the question (remote response), the interface unit prompts the user with the response that has been obtained after a specified time has elapsed.

11. The information processing system according to any one of claims 1 to 6, characterized in that, The selection unit derives the credibility and detail of the local language model's response to the question, i.e., the local response's answer. The question's attributes, or question attributes, are derived by parsing the question. The query is provided to the remote language model if any one of the following conditions 1 to 3 is met: (1) The credibility of the local response is less than the specified value; (2) The detail of the local response is less than the specified value; (3) The question attribute includes "details".

12. The information processing system according to any one of claims 1 to 6, characterized in that, The selection unit provides the question to the local language model. Export the local language model's response to the question, i.e., the credibility and detail of the local response. If the credibility of the local response is above a specified value and the detail of the local response is below a specified value, the question will be fed back to the local language model.

13. The information processing system according to any one of claims 1 to 6, characterized in that, The selection unit provides the question to the local language model. Derive the credibility of the local language model's response to the question, i.e., the credibility of the local response. If the credibility of the local response is above a specified value, the local response is displayed to the user via the interface unit.

14. The information processing system according to any one of claims 1 to 6, characterized in that, The selection unit provides the question to the remote language model and obtains the remote language model's response to the question, i.e., the remote response. Derive the credibility of the local language model's response to the question, i.e., the credibility of the local response. If the reliability of the local response is less than a specified value, the remote response will be prompted to the user via the interface.

15. The information processing system according to claim 14, characterized in that, The selection unit provides the question to the local language model, obtains the local response, and derives the credibility of the local response based on the local response.

16. The information processing system according to claim 14, characterized in that, The selection unit derives the credibility of the local response by parsing the question.

17. The information processing system according to any one of claims 1 to 6, characterized in that, The selection unit provides the question to the local language model and the remote language model, and obtains the local response (local response) and the remote response (remote response) from the local language model. Export the credibility of the local response. If the credibility of the local response is above a specified value, the local response and the remote response are integrated to derive an integrated response, which is then presented to the user via the interface unit.

Images