Information processing device and program

Abstract

An information processing device and a program are provided that can control the generation of answers to questions from users so as to be efficient in accordance with the load situation. [Solution] An information processing device of an embodiment comprises a first acquisition means for acquiring load information representing the load status of the device itself, an input means for inputting a prompt including an inquiry sentence entered by a user to a generation AI that is functionally configured to generate an answer sentence to the prompt, a second acquisition means for acquiring the answer sentence generated by the generation AI, a provision means for providing the answer sentence acquired by the second acquisition means to the user, and a determination means for determining the number of generation AIs to which the prompt will be input or the number of times the prompt will be input to the generation AI depending on the load status, and the input means inputs the prompt based on the number or number of times determined by the determination means.

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device and a program. [Background technology]

[0002] In recent years, natural language processing systems that utilize generative AI (Artificial Intelligence) such as large language models (LLMs) that can generate natural-sounding sentences have emerged.

[0003] The large-scale language model described above is equipped with a function for generating and outputting answers to questions from users. Since a machine equipped with such a large-scale language model requires resources to generate answers, there is a problem that when the machine is under heavy load, the speed and efficiency of generating answers decreases due to a lack of resources. Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to provide an information processing device and a program that can control the generation of answers to questions from users so as to be efficient in accordance with the load situation. [Means for solving the problem]

[0005] An information processing device of an embodiment comprises a first acquisition means for acquiring load information representing the load status of the device itself, an input means for inputting a prompt including an inquiry sentence entered by a user to a generation AI that is functionally configured to generate an answer sentence to the prompt, a second acquisition means for acquiring the answer sentence generated by the generation AI, a provision means for providing the answer sentence acquired by the second acquisition means to the user, and a determination means for determining the number of generation AIs to which the prompt will be input or the number of times the prompt will be input to the generation AI depending on the load status, and the input means inputs the prompt based on the number or number of times determined by the determination means. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a schematic diagram showing an example of a schematic configuration of an information processing system according to the first embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the edge server according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a data configuration of a load information table according to the first embodiment. [Figure 4] FIG. 4 is a block diagram illustrating an example of a hardware configuration of the edge device according to the first embodiment. [Figure 5] FIG. 5 is a block diagram illustrating an example of the functional configuration of the edge server and the edge device according to the first embodiment. [Figure 6] FIG. 6 is a flowchart illustrating an example of processing performed by each of the edge server and the edge device according to the first embodiment. [Figure 7] FIG. 7 is a flowchart illustrating an example of processing performed by each of the edge server and the edge device according to the first modification of the first embodiment. [Figure 8] FIG. 8 is a block diagram illustrating an example of a hardware configuration of an edge server according to the second embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a data configuration of a user master according to the second embodiment. [Figure 10] FIG. 10 is a block diagram illustrating an example of the functional configuration of the edge server and the edge device according to the second embodiment. [Figure 11] FIG. 11 is a flowchart illustrating an example of processing performed by each of the edge server and the edge device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, an embodiment of an information processing device and a program will be described with reference to the drawings. Note that the present invention is not limited to the following embodiment.

[0008] (First embodiment) FIG. 1 is a schematic diagram showing an example of the overall configuration of an information processing system 1 according to the first embodiment. As shown in FIG. 1, the information processing system 1 includes an edge server 10 and an edge device 20. The information processing system 1 is installed in a store such as a supermarket or a department store (hereinafter simply referred to as a store), and provides functions for responding to product inquiries within the store. The edge server 10 and the edge device 20 are connected to each other via wire or wirelessly so that they can communicate with each other.

[0009] The edge server 10 is an example of an information processing device. The edge server 10 is, for example, a server device installed in a local environment connected to the edge device 20 by wire or in a cloud environment such as the Internet. The edge server 10 generates a response sentence based on a query (hereinafter also referred to as an inquiry sentence) received from the edge device 20. Here, the query is a text including various inquiry contents input by a user. The response sentence is a text including a response to the items (inquiry contents, etc.) described in the query.

[0010] Specifically, when the edge server 10 receives a query from the edge device 20, it acquires load information that indicates the load status of its own device, and determines the number of generation AIs to use to generate a reply sentence according to the acquired load information. Then, the edge server 10 transmits the reply sentence generated using the determined number of generation AIs to the edge device 20. The load information will be described in detail later.

[0011] The edge device 20 is, for example, a terminal device such as a PC used by a user (hereinafter also referred to as a user) of the information processing system 1. The edge device 20 exchanges various information with the edge server 10. Note that the edge device 20 may also be a mobile terminal such as a smartphone, a tablet terminal, or the like.

[0012] Specifically, the edge device 20 transmits a query input by a user operation to the edge server 10. Then, upon receiving a reply sentence from the edge server 10, the edge device 20 presents the received reply sentence to the user.

[0013] In this embodiment, the edge server 10 is described as being realized by a single device, but may be realized by multiple devices. Also, the edge server 10 and the edge device 20 may be integrated into one device.

[0014] Next, a description will be given of the hardware configuration of the edge server 10. Fig. 2 is a block diagram showing an example of the hardware configuration of the edge server 10 according to the first embodiment.

[0015] As shown in FIG. 2, the edge server 10 includes a CPU (Central Processing Unit) 101, which is an example of a processor, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a memory unit 104, a communication unit 105, and the like.

[0016] The CPU 101 is an example of a processor and performs overall control of each unit of the edge server 10. The ROM 102 stores various programs. The RAM 103 is a workspace where programs and various data are expanded.

[0017] The memory unit 104 is a non-volatile memory such as a hard disk drive (HDD) or flash memory that retains stored information even when the power is turned off. The memory unit 104 includes a control program 121, a first LLM 122, a second LLM 123, and a load information table 124. The number of LLMs included in the memory unit 104 is not limited to the illustrated example.

[0018] The control program 121 stores a control program for controlling the edge server 10. The CPU 101, ROM 102, RAM 103, and memory unit 104 are connected to one another via a bus 106. The CPU 101, ROM 102, and RAM 103 constitute a control unit 100 having a computer configuration. That is, the control unit 100 executes a control process for the edge server 10, which will be described later, by the CPU 101 operating in accordance with the control program 121 stored in the ROM 102 or memory unit 104 and loaded into the RAM 103.

[0019] The first LLM 122 and the second LLM 123 are generation AIs that generate text, such as large language models (LLMs). The first LLM 122 and the second LLM 123 receive input of text (also called a prompt) generated based on a query from a user, and generate a response sentence in response to the query. Note that in this embodiment, an LLM is used as the generation AI, but the generation AI is not limited to an LLM as long as it is capable of generating text. Hereinafter, an LLM will also be simply referred to as a model.

[0020] The first LLM 122 and the second LLM 123 are constructed, for example, using known deep learning techniques, and are configured to generate and output a response sentence based on the conditions in response to the input of a prompt that describes the inquiry content and other conditions contained in the query.

[0021] A known generation AI (large-scale language model) can be applied to the first LLM 122 and the second LLM 123. The first LLM 122 and the second LLM 123 may be the same type and version of generation AI, or may be different types or versions of generation AI. Furthermore, the first LLM 122 and the second LLM 123 may be types or versions of generation AI with different accuracy in generating answer sentences. In this embodiment, the first LLM 122 and the second LLM 123 will be described as being the same type of generation AI unless otherwise specified.

[0022] Furthermore, the first LLM 122 and the second LLM 123 may use a known natural language processing technique such as RAG (Retrieval-Augmented Generation) to accept input of a prompt generated based on a query and supplemental information acquired from an external storage device, etc., and generate a response sentence according to the query and supplemental information. In this case, the external storage device may store information about each product sold in the store.

[0023] Furthermore, the first LLM 122 and the second LLM 123 may be fine-tuned to be specific to the intended use of the information processing system 1. The fine tuning may change the content of the response to the input prompt, or may change the wording of the sentence to be output. For example, the first LLM 122 and the second LLM 123 may have learned specific wording, such as writing style and endings, aimed at the user of the information processing system 1.

[0024] The load information table 124 is a data table or database for managing the load information of the edge server 10 in association with the number of first LLMs 122 or second LLMs 123 used to generate response sentences. Fig. 3 is a diagram showing an example of the data configuration of the load information table 124 according to the first embodiment. As shown in Fig. 3, the load information table 124 stores information such as the number of models used in association with the load information of the edge server 10.

[0025] The load information is information that represents various indicators that define the load state of the edge server 10. For example, the load information includes the usage rate of the CPU 101 of the edge server 10, the number of users currently using the edge server 10, the number of prompts waiting for the generation of answer sentences, etc.

[0026] The number of models in use is the number of first LLMs 122 or second LLMs 123 used to generate a response sentence. For example, when the CPU utilization rate, which is an example of load information, is 50% or more, in other words, when the edge server 10 is in a high-load state, the corresponding number of models in use is "1" in FIG. 3, so the edge server 10 inputs a prompt to either the first LLM 122 or the second LLM 123. Also, when the CPU utilization rate is less than 50%, in other words, when the edge server 10 is in a low-load state, the corresponding number of models in use is "2" in FIG. 3, so the edge server 10 inputs a prompt to the first LLM 122 and the second LLM 123. The number of models in use is not limited to the example shown in the figure, and can be set to the number of LLMs provided in the edge server 10.

[0027] Returning to FIG. 2, the control unit 100 is connected to the communication unit 105 via a bus 106. The communication unit 105 is a communication interface such as a LAN I / F, and is connected to a network Na. The communication unit 105 transmits and receives various information to and from, for example, the edge device 20 via the network Na. Furthermore, under the control of the control unit 100, the communication unit 105 can be connected to a network such as the Internet or other information processing devices.

[0028] Next, a description will be given of the hardware configuration of the edge device 20. Fig. 4 is a block diagram showing an example of the hardware configuration of the edge device 20 according to the first embodiment.

[0029] As shown in FIG. 4, the edge device 20 includes a CPU 201, which is an example of a processor, a ROM 202, a RAM 203, a memory unit 204, a communication unit 205, a display unit 206, an operation unit 207, and the like.

[0030] The CPU 201 comprehensively controls each unit of the edge device 20. The ROM 202 stores various programs. The RAM 203 is a workspace where programs and various data are expanded.

[0031] The memory unit 204 is a non-volatile memory such as a HDD or flash memory that retains stored information even when the power is turned off. The memory unit 204 includes a control program 221.

[0032] The control program 221 stores a control program for controlling the edge device 20. The CPU 201, ROM 202, RAM 203, and memory unit 204 are connected to one another via a bus 208. The CPU 201, ROM 202, and RAM 203 constitute a control unit 200 having a computer configuration. That is, the control unit 200 executes a control process for the edge device 20, which will be described later, by the CPU 201 operating in accordance with the control program 221 stored in the ROM 202 or memory unit 204 and loaded into the RAM 203.

[0033] The control unit 200 is connected to a communication unit 205 , a display unit 206 , and an operation unit 207 via a bus 208 .

[0034] The communication unit 205 is a communication interface such as a LAN I / F, and is connected to the network Na. The communication unit 205 transmits and receives various information to and from, for example, the edge server 10, via the network Na. Under the control of the control unit 200, the communication unit 205 can also connect to a network such as the Internet or other information processing devices.

[0035] The display unit 206 is a display device such as an LCD (Liquid Crystal Display), etc. The display unit 206 displays various types of information under the control of the CPU 201.

[0036] The operation unit 207 receives various inputs from the user. The operation unit 207 includes, for example, a keyboard, a pointing device, etc. The operation unit 207 may be a touch panel provided on the display screen of the display unit 206.

[0037] Next, a description will be given of the functional configuration of the edge server 10 and the edge device 20. Fig. 5 is a block diagram showing an example of the functional configuration of the edge server 10 and the edge device 20 according to the first embodiment.

[0038] As shown in FIG. 5, the control unit 100 of the edge server 10 includes a query receiving unit 1001, a prompt generating unit 1002, a machine load calculating unit 1003, a text generation control unit 1004, and an answer providing unit 1005 as functional components.

[0039] Specifically, the control unit 100 (CPU 101) of the edge server 10 realizes the above-mentioned functional configuration by executing the control program 121 stored in the memory unit 104. Note that in this embodiment, the above-mentioned functional configuration is a software configuration realized by cooperation between the processor and program of the edge server 10, but this is not limiting, and a hardware configuration in which part or all of the functional configuration is realized by a dedicated circuit or the like may also be used. Furthermore, the functional configuration of the edge server 10 is not limited to this.

[0040] The query receiving unit 1001 receives information from the edge device 20. Specifically, the query receiving unit 1001 receives a query transmitted from the edge device 20.

[0041] The prompt generator 1002 generates a prompt. Specifically, the prompt generator 1002 generates a prompt based on the query received by the query receiver 1001.

[0042] Here, the prompt includes an instruction specifying the format and details of the answer to be generated. For example, the prompt preferably includes a specific instruction such as "output an answer that is in line with the gist of the query." Furthermore, the prompt may be configured to switch from multiple types of templates depending on the language of the query, etc.

[0043] The machine load calculation unit 1003 is an example of a first acquisition means. The machine load calculation unit 1003 acquires load information related to the edge server 10. Here, the load information acquired by the machine load calculation unit 1003 is, for example, an index value such as a CPU usage rate defined in the load information table 124.

[0044] Specifically, the machine load calculation unit 1003 acquires, as the load information, the utilization rate of the CPU 101. Note that, in addition to the utilization rate of the CPU 101, the machine load calculation unit 1003 may acquire, as the load information, the number of users currently using the edge server 10 and the number of prompts waiting for the generation of answer sentences.

[0045] The machine load calculation unit 1003 may acquire the load information constantly or at predetermined time intervals (e.g., every 10 seconds). Furthermore, the machine load calculation unit 1003 may acquire the load information when it receives an instruction from the text generation control unit 1004, by cooperating with the text generation control unit 1004, which will be described later.

[0046] The text generation control unit 1004 is an example of an input unit, a second acquisition unit, and a determination unit. The text generation control unit 1004 determines the number of LLMs to use according to the load information acquired by the machine load calculation unit 1003. The text generation control unit 1004 also generates a response sentence to a query (prompt) using the determined number of LLMs. Note that the term "determine" in this specification is a concept that also encompasses "adjust" and "control."

[0047] Specifically, the text generation control unit 1004 refers to the load information table 124 and acquires the number of models in use corresponding to the load information acquired by the machine load calculation unit 1003. If the number of models in use corresponding to the load information of the own device is "1", the text generation control unit 1004 inputs a prompt to either the first LLM 122 or the second LLM 123. The text generation control unit 1004 then acquires the answer sentence output by the first LLM 122 or the second LLM 123 to which the prompt was input.

[0048] On the other hand, if the number of models in use corresponding to the load information of the own device is "2", the text generation control unit 1004 inputs a prompt to the first LLM 122 and the second LLM 123. Then, the text generation control unit 1004 acquires the first answer sentence and the second answer sentence output respectively by the first LLM 122 and the second LLM 123 to which the prompt was input.

[0049] Here, the text generation control unit 1004 may employ any method as a model selection method when the number of models used is "1." For example, the text generation control unit 1004 may select the first LLM 122 and the second LLM 123 randomly, in a round-robin manner, or may select a preset model. Furthermore, when the performance of the first LLM 122 and the second LLM 123 models differs, the text generation control unit 1004 may select a model with a lower processing load, in other words, a lower accuracy.

[0050] As described above, by changing the number of models used according to the load information that indicates the load state, it is possible to prevent the edge server 10 from being constantly under high load and to prevent a decrease in the speed and efficiency of generating answer sentences due to a lack of resources. In other words, it is possible to control the generation of answers to user questions so as to be efficient according to the load state.

[0051] The timing for checking the load information may be the timing for receiving a query transmitted from the edge device 20. Furthermore, if the timing for acquiring the load information is controlled by the text generation control unit 1004, the text generation control unit 1004 may transmit an instruction to the machine load calculation unit 1003 at the timing for receiving a query transmitted from the edge device 20, causing the machine load calculation unit 1003 to acquire the load information.

[0052] The answer providing unit 1005 is an example of a providing means. The answer providing unit 1005 provides the user with an answer sentence generated in response to a query. Specifically, the answer providing unit 1005 transmits the answer sentence generated by the text generation control unit 1004 to the edge device 20.

[0053] In addition, if there are multiple answer sentences obtained by the text generation control unit 1004, for example, if answer sentences are generated using multiple models for one query (prompt), the answer providing unit 1005 may provide the multiple answer sentences as they are, or may merge the multiple answer sentences and then provide the merged answer sentence to the user.

[0054] Here, the case where there are multiple answer sentences means that the text generation control unit 1004 inputs prompts to the first LLM 122 and the second LLM 123, thereby obtaining answer sentences from each of the first LLM 122 and the second LLM 123. Furthermore, merging answer sentences (text) means, for example, combining multiple answer sentences into one answer sentence. There is no particular restriction on the merging method, and various methods can be adopted.

[0055] For example, the text generation control unit 1004 may convert multiple answer sentences into one answer sentence by formatting them, such as by deleting overlapping parts. Alternatively, the text generation control unit 1004 may count words that appear in common in multiple answer sentences and extract parts that include the commonly appearing words to convert them into one answer sentence.

[0056] On the other hand, the control unit 200 of the edge device 20 includes a query acquisition unit 2001, an information transmission / reception unit 2002, and a display control unit 2003 as functional components.

[0057] Specifically, the control unit 200 (CPU 201) of the edge device 20 realizes the above-mentioned functional configuration by executing the control program 221 stored in the memory unit 204. Note that, in this embodiment, the above-mentioned functional configuration is a software configuration realized by cooperation between the processor of the edge device 20 and the program, but this is not limiting, and a hardware configuration in which part or all of the functional configuration is realized by a dedicated circuit or the like may also be used. Furthermore, the functional configuration of the edge device 20 is not limited to this.

[0058] The query acquisition unit 2001 acquires information input from a user to the edge device 20. In other words, the query acquisition unit 2001 accepts information input from a user. For example, the query acquisition unit 2001 acquires a query input by a user operation via the operation unit 207.

[0059] The information transmitting / receiving unit 2002 transmits and receives various information to and from the edge server 10 via the communication unit 205. For example, the information transmitting / receiving unit 2002 transmits a query acquired by the query acquisition unit 2001 to the edge server 10. The information transmitting / receiving unit 2002 also receives an answer sentence, which is an example of text, from the edge server 10.

[0060] The display control unit 2003 displays various types of information on the display unit 206. For example, the display control unit 2003 displays a screen for assisting in inputting a query on the display unit 206. The display control unit 2003 also displays on the display unit 206 a response sentence acquired by the information transmitting / receiving unit 2002, that is, a response sentence to a query.

[0061] Next, the control processing of the edge server 10 and the edge device 20 will be described with reference to FIG.

[0062] 6 is a flowchart showing an example of processing performed by each of the edge server 10 and the edge device 20 according to the first embodiment. The flowchart shown in FIG. 6 shows an example of processing when the edge server 10 generates a response sentence to a query acquired by the edge device 20 and provides the response sentence to the edge device 20.

[0063] First, the query acquisition unit 2001 of the edge device 20 acquires a query input by a user operation via the operation unit 207 (step S101). Next, the information transmission / reception unit 2002 of the edge device 20 transmits the acquired query to the edge server 10 (step S102).

[0064] The query receiving unit 1001 of the edge server 10 receives the query transmitted from the edge device 20 (step S103). Next, the prompt generating unit 1002 of the edge server 10 generates a prompt including the received query based on the received query (step S104).

[0065] Next, the machine load calculation unit 1003 of the edge server 10 acquires the utilization rate of the CPU 101 as load information (step S105). Next, the text generation control unit 1004 refers to the load information table 124 and acquires the number of models in use according to the acquired load information (step S106). Next, the text generation control unit 1004 determines whether the acquired number of models in use is "1" (step S107).

[0066] If the number of models used is "1" (step S107; Yes), the text generation control unit 1004 inputs a prompt to the first LLM 122 or the second LLM 123 (step S108). Furthermore, the text generation control unit 1004 acquires the answer sentence output by the first LLM 122 or the second LLM 123 that input the prompt (step S109). Then, the control unit 100 proceeds to step S113.

[0067] On the other hand, if the number of models used is "2" (step S107; No), the text generation control unit 1004 inputs a prompt to the first LLM 122 and the second LLM 123 (step S110). Furthermore, the text generation control unit 1004 acquires the first answer sentence and the second answer sentence output by the first LLM 122 and the second LLM 123, which received the prompt (step S111). Then, the answer providing unit 1005 of the edge server 10 acquires an answer sentence obtained by merging the first answer sentence and the second answer sentence (step S112).

[0068] Next, the answer providing unit 1005 transmits the answer sentence to the edge device 20 (step S113).

[0069] The information transmitting / receiving unit 2002 of the edge device 20 receives the reply message from the edge server 10 (step S114). Then, the display control unit 2003 of the edge device 20 displays the reply message on the display unit 206 (step S115).

[0070] As described above, the edge server 10 (an example of an information processing device) of this embodiment comprises a first acquisition means for acquiring load information representing the load status of the device itself, an input means for inputting a prompt including an inquiry sentence entered by a user to a generation AI that is functionally configured to generate an answer sentence to the prompt, a second acquisition means for acquiring the answer sentence generated by the generation AI, a provision means for providing the answer sentence acquired by the second acquisition means to the user, and a determination means for determining the number of generation AIs to which the prompt will be input or the number of times the prompt will be input to the generation AI depending on the load status, and the input means inputs the prompt based on the number or number of times determined by the determination means.

[0071] The edge server 10 generates a prompt based on a query containing the inquiry content entered by the user, and determines the number of LLMs to input the prompt according to load information indicating the load status of the edge server 10. Then, the edge server 10 obtains the answer sentences output by the determined LLMs.

[0072] As a result, the edge server 10 controls the system so that answer sentences are generated using only one LLM when the edge server 10 is in a high-load state, and so that answer sentences are generated using multiple LLMs when the edge server 10 is in a low-load state. Therefore, by changing the number of models used according to the load information, it is possible to prevent the edge server 10 from being constantly in a high-load state, and to prevent a decrease in the speed and efficiency of answer sentence generation due to a lack of resources. In other words, it is possible to control the system so that answers to user questions are generated efficiently according to the load situation.

[0073] The above-described embodiments can be modified as needed by changing some of the configurations or functions of the above-described devices. Therefore, below, several modifications of the above-described embodiments will be described as other embodiments. Below, differences from the above-described embodiments will be mainly described, and components that are common to the contents already described will be given the same reference numerals, and detailed descriptions will be omitted as appropriate. The modifications described below may be implemented individually or in combination as needed.

[0074] (Variation 1) In the above embodiment, the edge server 10 adjusts the number of LLMs used to generate a response sentence according to the load information. In this modified example, the edge server 10 adjusts the number of times a prompt is input to an LLM, that is, the number of processes related to generating a response sentence, according to the load information.

[0075] In this modified example, a case will be described in which the memory unit 104 of the edge server 10 has one LLM, in other words, the memory unit 104 of the edge server 10 stores the first LLM 122. This modified example is configured to generate answer sentences from multiple perspectives by inputting the same query (prompt) to the LLM multiple times with different prompt conditions.

[0076] First, the load information table 124 according to this modification stores the number of attempts (not shown) instead of the number of models used shown in Fig. 3. Here, the number of attempts is the number of times a prompt is input to the LLM, that is, the number of times a reply sentence is generated.

[0077] In this case, it is preferable to set a smaller number of attempts as the load on the edge server 10 increases. For example, if the CPU utilization rate, which is an example of load information, is 50% or higher, in other words, if the edge server 10 is in a high load state, the number of attempts is set to "1," and if the CPU utilization rate is less than 50%, in other words, if the edge server 10 is in a low load state, the number of attempts is set to "3."

[0078] The text generation control unit 1004 of this modification refers to the load information table 124 and acquires the number of attempts corresponding to the load information acquired by the machine load calculation unit 1003. The text generation control unit 1004 also inputs a prompt to the first LLM 122. The text generation control unit 1004 then updates the number of text inputs stored in the RAM 103. Here, the number of text inputs is the number of times a prompt corresponding to the same query is input to the first LLM 122.

[0079] Furthermore, when the text generation control unit 1004 acquires an answer sentence output by the first LLM 122, it temporarily stores the acquired answer sentence as the xth answer sentence (x: number of text inputs) in RAM 103. Then, if the number of text inputs stored in RAM 103 does not match the number of attempts, the text generation control unit 1004 inputs a prompt to the first LLM 122 again.

[0080] Here, when a prompt is input again to the first LLM 122, the prompt conditions may be changed.

[0081] The answer providing unit 1005 of this modification transmits the answer sentence generated by the text generation control unit 1004 to the edge device 20. Specifically, when the number of text inputs stored in the RAM 103 matches the number of attempts, the text generation control unit 1004 acquires all the answer sentences stored in the RAM 103 and acquires an answer sentence by merging the answer sentences. In addition, the answer providing unit 1005 transmits the answer sentence to the edge device 20.

[0082] 7 is a flowchart showing an example of processing performed by each of the edge server 10 and the edge device 20 according to the first modification of the first embodiment. The flowchart shown in FIG. 7 shows a processing example in which the edge server 10 generates a response sentence to a query acquired by the edge device 20 and provides the response sentence to the edge device 20.

[0083] First, the query acquisition unit 2001 of the edge device 20 acquires a query input by a user operation via the operation unit 207 (step S201). Next, the information transmission / reception unit 2002 of the edge device 20 transmits the acquired query to the edge server 10 (step S202).

[0084] The query receiving unit 1001 of the edge server 10 receives the query transmitted from the edge device 20 (step S203). Next, the prompt generating unit 1002 of the edge server 10 generates a prompt including the received query based on the received query (step S204).

[0085] Next, the machine load calculation unit 1003 of the edge server 10 acquires the utilization rate of the CPU 101 as load information (step S205). Next, the text generation control unit 1004 refers to the load information table 124 and acquires the number of attempts according to the acquired load information (step S206).

[0086] Next, the text generation control unit 1004 of the edge server 10 inputs a prompt to the first LLM 122 (step S207). Then, the text generation control unit 1004 updates the number of times of text input stored in the RAM 103.

[0087] Next, the text generation control unit 1004 acquires the answer sentence output by the first LLM 122 (step S208). Then, the text generation control unit 1004 temporarily stores the acquired answer sentence as the xth answer sentence (x: number of text inputs) in the RAM 103. Next, the text generation control unit 1004 determines whether the number of text inputs stored in the RAM 103 matches the number of attempts (step S209).

[0088] If the number of times the text is input does not match the number of times the attempt is made (step S209; No), the control unit 100 returns the process to step S207.

[0089] If the number of text inputs matches the number of attempts (step S209; Yes), the answer providing unit 1005 of the edge server 10 acquires all answer sentences stored in the RAM 103 and acquires an answer sentence by merging the answer sentences (step S210). Next, the answer providing unit 1005 transmits the answer sentence to the edge device 20 (step S211).

[0090] The information transmitting / receiving unit 2002 of the edge device 20 receives the reply message from the edge server 10 (step S212). Next, the display control unit 2003 of the edge device 20 displays the reply message on the display unit 206 (step S213).

[0091] As described above, in this modification, the edge server 10 generates a prompt based on a query including the inquiry content entered by the user, and determines the number of times to input the prompt to the first LLM 122 according to load information indicating the load status of the edge server 10. Then, the edge server 10 acquires the response sentence output by the first LLM 122.

[0092] As a result, the edge server 10 controls the generation of answer sentences so that the number of attempts is based on the load status of the edge server 10. This prevents the edge server 10 from being constantly under high load, and prevents a decrease in the speed and efficiency of generating answer sentences due to a lack of resources. In other words, the generation of answers to questions from users can be controlled efficiently based on the load status.

[0093] (Second embodiment) Next, a second embodiment will be described. For example, when the edge server 10 is under a high load, the edge server 10 may want to determine the LLM to be used to generate a response sentence.

[0094] In such cases, by determining the LLM to be used to generate the answer sentence based on the user's attributes, it is possible to control the generation of answers to questions from users so that they are generated efficiently.

[0095] In this embodiment, the first LLM 122 provided in the edge server 10 is a model that can generate answers with higher accuracy compared to the second LLM 123, in other words, the first LLM 122 is described as a model with higher performance than the second LLM 123.

[0096] In addition, in this embodiment, a case will be described in which users are broadly classified into either "general members" or "preferential members" as an example of user attributes. Here, a user whose user attribute is "preferential members" can obtain highly accurate answers by using the first LLM 122 (high-performance model). Furthermore, a user whose user attribute is "general members" can use the first LLM 122 (high-performance model) when the edge server 10 is in a low-load state, but will use the second LLM 123 (low-performance model) when the edge server 10 is in a high-load state.

[0097] The following describes a configuration for determining the LLM to be used to generate a response sentence based on the user's attributes and load information.

[0098] In the second embodiment, the edge device 20 acquires a user ID, which is identification information that can uniquely identify a user, and transmits the user ID together with a query to the edge server 10. Then, upon receiving the user ID and the query from the edge device 20, the edge server 10 acquires user attributes corresponding to the user ID and determines an LLM to be used to generate a response sentence based on the load information and the user attributes.

[0099] First, a description will be given of the hardware configuration of the edge server 10. Fig. 8 is a block diagram showing an example of the hardware configuration of the edge server 10 according to the second embodiment.

[0100] As shown in FIG. 8, the memory unit 104 of the edge server 10 includes a user master 125 in addition to the control program 121, the first LLM 122, the second LLM 123, and the load information table 124 described above.

[0101] The user master 125 is a table or database that stores user attributes in association with a user ID, which is identification information that can uniquely identify a user. Fig. 9 is a diagram showing an example of the data configuration of the user master 125 according to the second embodiment. As shown in Fig. 9, the user master 125 stores information such as user attributes in association with a user ID.

[0102] The user attribute is an example of attribute information. The user attribute is information indicating the attributes of a user. For example, as shown in FIG. 9, the user attribute is configured as either a "general member" or a "preferential member."

[0103] Next, a description will be given of the functional configuration of the edge server 10 and the edge device 20. Fig. 10 is a block diagram showing an example of the functional configuration of the edge server 10 and the edge device 20 according to the second embodiment.

[0104] The edge server 10 functions as a query receiving unit 1001, a prompt generating unit 1002, a machine load calculating unit 1003, a text generation control unit 1004, and an answer providing unit 1005, as well as an attribute information acquiring unit 1006, as a result of the CPU 101 operating in accordance with a control program stored in the ROM 102 or the memory unit 104. Note that each of the above functional configurations may be realized by a hardware configuration such as a dedicated circuit mounted on the edge server 10. Note that the functional configuration of the edge device 20 is the same as the configuration described in the first embodiment, and therefore a description thereof will be omitted.

[0105] The attribute information acquisition unit 1006 is an example of a third acquisition unit. The attribute information acquisition unit 1006 acquires attribute information. Specifically, the attribute information acquisition unit 1006 refers to the user master 125 and acquires a user attribute corresponding to a user ID.

[0106] Next, the control processing of the edge server 10 and the edge device 20 will be described with reference to FIG.

[0107] 11 is a flowchart showing an example of processing performed by each of the edge server 10 and the edge device 20 according to the second embodiment. The flowchart shown in FIG. 11 shows an example of processing when the edge server 10 generates a response sentence to a query acquired by the edge device 20 and provides the response sentence to the edge device 20.

[0108] First, the query acquisition unit 2001 of the edge device 20 acquires a user ID input by a user operation via the operation unit 207 (step S301). Next, the query acquisition unit 2001 acquires a query input by a user operation via the operation unit 207 (step S302). Then, the information transmission / reception unit 2002 of the edge device 20 transmits the acquired user ID and query to the edge server 10 (step S303).

[0109] The query receiving unit 1001 of the edge server 10 receives the user ID and the query transmitted from the edge device 20 (step S304). Next, the prompt generating unit 1002 of the edge server 10 generates a prompt including the received query based on the received query (step S305).

[0110] Next, the machine load calculation unit 1003 of the edge server 10 acquires the utilization rate of the CPU 101 as load information (step S306). Next, the text generation control unit 1004 refers to the load information table 124 and acquires the number of models in use according to the acquired load information (step S307). Next, the text generation control unit 1004 of the edge server 10 determines whether the number of models in use acquired by the text generation control unit 1004 is "1" (step S308).

[0111] If the number of models in use is "1" (step S308; Yes), the attribute information acquisition unit 1006 of the edge server 10 refers to the user master 125 and acquires the user attribute corresponding to the user ID (step S309). Then, the text generation control unit 1004 determines whether the user attribute is "preferential member" (step S310).

[0112] If the user attribute is "good member" (step S310; Yes), the text generation control unit 1004 inputs a prompt to the first LLM 122 (step S311). Then, the control unit 100 advances the process to step S313.

[0113] If the user attribute is "general member" (step S310; No), the text generation control unit 1004 inputs a prompt to the second LLM 123 (step S312). Next, the text generation control unit 1004 acquires the answer sentence output by the first LLM 122 or second LLM 123 that input the prompt (step S313). Then, the control unit 100 proceeds to step S317.

[0114] On the other hand, if the number of models used is "2" (step S308; No), the text generation control unit 1004 inputs a prompt to the first LLM 122 and the second LLM 123 (step S314). Furthermore, the text generation control unit 1004 acquires the first answer sentence and the second answer sentence output by the first LLM 122 and the second LLM 123, which received the prompt (step S315). Then, the answer providing unit 1005 of the edge server 10 acquires an answer sentence obtained by merging the first answer sentence and the second answer sentence (step S316).

[0115] Next, the answer providing unit 1005 transmits the answer sentence to the edge device 20 (step S317).

[0116] The information transmitting / receiving unit 2002 of the edge device 20 receives the reply message from the edge server 10 (step S318). Next, the display control unit 2003 of the edge device 20 displays the reply message on the display unit 206 (step S319).

[0117] In step S301, the query acquisition unit 2001 may acquire the user ID from a storage medium such as a membership card read by an imaging unit, a reading unit, or the like (not shown) included in the edge device 20.

[0118] As described above, the edge server 10 of this embodiment further includes an attribute information acquisition unit 1006 that acquires attribute information, and the text generation control unit 1004 determines the LLM to input the prompt based on the load information and the attribute information acquired by the attribute information acquisition unit 1006.

[0119] As a result, the edge server 10 determines the LLM to input the prompt according to the attribute information and controls the generation of the answer sentence. Therefore, by changing the number of models used according to the attribute information, it is possible to prevent the edge server 10 from being constantly in a high load state and to prevent a decrease in the speed and efficiency of generating answer sentences due to a lack of resources. In other words, it is possible to control the generation of answers to questions from users so that they are generated efficiently according to the load situation.

[0120] (Variation 2) In the second embodiment described above, the text generation control unit 1004 is configured to determine the LLM to input the prompt based on the user attributes acquired by the attribute information acquisition unit 1006 from the user master 125, but this is not limited to this, and the text generation control unit 1004 may also be configured to determine the LLM to input the prompt based on the attributes of the prompt extracted by the attribute information acquisition unit 1006.

[0121] For example, if the attribute information acquisition unit 1006 extracts that the language in which the prompt is written is Japanese as an example of an attribute of the prompt, the text generation control unit 1004 may determine that the LLM that inputs the prompt is an LLM with excellent Japanese processing capabilities.

[0122] This is expected to enable quick generation of answer sentences to the prompts, and the edge server 10 can control the generation of answers to questions from users in an efficient manner.

[0123] (Variation 3) In the first and second embodiments described above, the prompt generation unit 1002 of the edge server 10 generates a prompt based on a query received by the query reception unit 1001 of the edge server 10, but this is not limiting. For example, the prompt generation unit 1002 may be provided in the edge device 20. In this case, the edge server 10 receives the prompt generated by the prompt generation unit 1002 of the edge device 20.

[0124] (Variation 4) In the first and second embodiments described above, the query acquisition unit 2001 of the edge device 20 acquires a query input by a user operation via the operation unit 207, but this is not limiting. For example, the query acquisition unit 2001 of the edge device 20 may convert voice input from a voice input device such as a microphone (not shown) provided in the edge device 20 into text data and acquire it as a query.

[0125] (Variation 5) In the second embodiment described above, the edge server 10 is configured to include the first LLM 122 and the second LLM 123, but this is not limiting. Even if the edge server 10 includes only one LLM, it is possible to achieve the same effect as that described in the second embodiment by, for example, varying the number of attempts to input a prompt to the LLM between a user whose user attribute is "general member" and a user whose user attribute is "principal member." In this case, it is preferable to set the number of attempts for a "principal member" to be greater than the number of attempts for a "general member" as the number of attempts stored in the load information table 124.

[0126] The programs executed by the information processing system 1 of the embodiment and the modified example may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the programs executed by the information processing system 1 of the embodiment and the modified example may be provided or distributed via a network such as the Internet.

[0127] The programs executed by each device in the above-described embodiments are provided in a state where they are pre-installed in a ROM, a storage unit, etc. The programs executed by each device in the above-described embodiments may be provided by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD).

[0128] These novel embodiments and their modifications may be embodied in various other forms, and various omissions, substitutions, changes, and combinations may be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the inventions, and are also included in the inventions described in the claims and their equivalents. [Explanation of symbols]

[0129] 1. Information Processing Systems 10 Edge Server 20 Edge Devices 1001 Query receiver 1002 Prompt Generation Unit 1003 Machine Load Calculation Unit 1004 Text generation control unit 1005 Answer provision department 1006 Attribute information acquisition section 2001 Query Acquisition Unit 2002 Information Transmission and Reception Department 2003 Display control unit [Prior art documents] [Patent documents]

[0130] [Patent Document 1] Patent No. 5122486

Claims

1. a first acquisition means for acquiring load information representing a load state of the device itself; an input means for inputting a prompt including a query sentence input by a user to a generation AI that is configured to generate a response sentence to the prompt; A second acquisition means for acquiring the answer sentence generated by the generation AI; providing means for providing the user with the answer sentence acquired by the second acquisition means; a determining means for determining the number of the generation AIs to which the prompts are input or the number of times the prompts are input to the generation AIs according to the load state; Equipped with the input means inputs the prompt based on the number or number of times determined by the determination means. Information processing device.

2. the load information includes at least one of a processor usage rate of the device itself, a number of users currently using the device itself, and a number of the prompts for which the answer sentence is to be generated; The information processing device according to claim 1 .

3. The determining means determines the number of the generation AIs to which the prompts are to be input or the number of times the prompts are to be input based on setting information in which the number of the generation AIs to be used to generate the answer sentence or the number of times the prompts are to be input is set in association with the load condition condition. The information processing device according to claim 1 .

4. The system further includes a third acquisition means for acquiring attribute information indicating an attribute of the user or an attribute of the prompt, determining the number of generated AIs to which the prompt is to be input or the number of times the prompt is to be input based on the attribute information acquired by the third acquisition means; The information processing device according to claim 1 .

5. When there are a plurality of answer sentences acquired by the second acquisition means, the providing means merges the answer sentences and then provides the answer sentences to the user. The information processing device according to claim 1 .

6. The computer of the information processing device, a first acquisition means for acquiring load information representing a load state of the device itself; an input means for inputting a prompt including a query sentence input by a user to a generation AI that is configured to generate a response sentence to the prompt; A second acquisition means for acquiring the answer sentence generated by the generation AI; providing means for providing the user with the answer sentence acquired by the second acquisition means; a determining means for determining the number of the generation AIs to which the prompts are input or the number of times the prompts are input to the generation AIs according to the load state; A program for functioning as the input means inputs the prompt based on the number or number of times determined by the determination means. program.

Images