Question-answering device, method, and program

Abstract

To provide appropriate answers to a questioner without causing inconvenience, even when the accuracy of answers automatically generated using a local large language model (LLM) is low.SOLUTION: A question-answering device includes an answer generation unit, an accuracy determination unit, an answer acquisition unit, and an answer output unit. The answer generation unit generates a first answer which is an answer to a question input from a questioner terminal operated by a questioner, using a local LLM. The accuracy determination unit determines the accuracy of the first answer generated by the answer generation unit. The answer acquisition unit, when the accuracy determined by the accuracy determination unit is equal to or less than a threshold, provides a question to an answer generation device different from the answer generation unit, and acquires a second answer generated by the answer generation device. The answer output unit outputs the first answer and / or the second answer to the questioner terminal.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD An embodiment of the present invention relates to a question answering device, method, and program. [Background technology]

[0002] In recent years, large-scale language models (LLMs), such as chatGPT, have been attracting attention. However, LLMs run on the cloud, but they have the drawback of being unsuitable for services that involve the transfer of personal information or for the high-speed real-time processing that edge devices excel at.

[0003] In response to this, local LLMs that are small in size and can run on devices are also being released. While these local LLMs do not match the functionality of cloud LLM services, they are known to be usable if optimized for specific purposes, and methods such as fine-tuning are provided as a means to achieve this.

[0004] The advantages of using local LLM include the ability to handle data that cannot be uploaded to the cloud, such as personal information, and the ability to ensure immediate responsiveness by running it on the edge.

[0005] One possible use of such a local LLM is a system that automatically generates answers to questions, as disclosed in Patent Document 1, for example.

[0006] In the system disclosed in Patent Document 1, if the accuracy of the automatically generated answer is low, the questioner must directly communicate with the answerer to obtain an appropriate answer. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2022-86817 Summary of the Invention [Problem to be solved by the invention]

[0008] The problem that the embodiments of the present invention aim to solve is to provide a question answering device, method, and program that can provide an appropriate answer to a questioner without bothering the questioner, even if the answer automatically generated using a local LLM has low accuracy. [Means for solving the problem]

[0009] In one embodiment, the question answering apparatus includes an answer generation unit, an accuracy determination unit, an answer acquisition unit, and an answer output unit. The answer generation unit uses a local LLM to generate a first answer that is an answer to a question input from a questioner terminal operated by a questioner. The accuracy determination unit determines the accuracy of the first answer generated by the answer generation unit. If the accuracy determined by the accuracy determination unit is equal to or less than a threshold, the answer acquisition unit supplies the question to an answer generation device different from the answer generation unit and acquires a second answer generated by the answer generation device. The answer output unit outputs the first answer and / or the second answer to the questioner terminal. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a question answering system including a store server to which a question answering device according to the first embodiment is applied. [Figure 2] FIG. 2 is a block diagram illustrating an example of the functional configuration of the store server. [Figure 3] FIG. 3 is a sequence diagram showing an outline of the operation of the question answering system. [Figure 4] FIG. 4 is a sequence diagram showing an outline of the operation of the question answering system. [Figure 5] FIG. 5 is a flowchart showing the main steps of the question answering process executed by the processor of the store server. [Figure 6] FIG. 6 is a sequence diagram showing an outline of the operation of a question answering system including a store server to which the question answering device according to the second embodiment is applied. [Figure 7] FIG. 7 is a flowchart showing the main steps of a question answering process executed by a processor of a store server in the second embodiment. [Figure 8] FIG. 8 is a block diagram showing an example of the functional configuration of a store server to which the question answering device according to the third embodiment is applied. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the settlement device will be described with reference to the drawings.

[0012] [First embodiment] FIG. 1 is a schematic diagram showing a question answering system including a store server SS to which a question answering device according to a first embodiment is applied. In addition to the store server SS, the question answering system also includes a plurality of questioner terminals QT and answerer terminals RT. Although FIG. 1 shows one answerer terminal RT, there may be multiple answerer terminals RT. The store server SS is an edge server connected to the cloud CL, and is installed in a store backyard BY located in a store SH. The questioner terminal QT is installed in the store SH, and the answerer terminal RT is installed in the store backyard BY.

[0013] The store server SS includes a processor 1, a main memory 2, a storage device 3, an external communication device 4, a communication interface 5, a network interface 6, and a bus line 7. The bus line 7 connects the processor 1, the main memory 2, the storage device 3, the external communication device 4, the communication interface 5, and the network interface 6 so that they can communicate with each other. The bus line 7 includes an address bus, a data bus, control signal lines, etc. The bus line 7 connects the processor 1 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The processor 1 and the main memory 2 are connected by the bus line 7 to form a computer that serves as the control unit of the store server SS.

[0014] The processor 1 is a hardware processor that corresponds to the central part of the computer. The processor 1 controls each part to realize various functions of the store server SS in accordance with an operating system and a control program. The processor 1 is, for example, a central processing unit (CPU). The processor 1 may be a microprocessing unit (MPU) instead of a CPU. A multi-core / multi-threaded processor 1 can execute multiple processes in parallel. The processor 1 preferably further includes a graphics processing unit (GPU) for machine learning. The processor 1 may also be realized in various other forms, including integrated circuits such as an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), and a programmable logic device (PLD). Alternatively, the processor 1 may be a combination of these.

[0015] The main memory 2 corresponds to the main storage portion of the computer. The main memory 2 includes a nonvolatile memory area and a volatile memory area. The main memory 2 stores an operating system in the nonvolatile memory area. Furthermore, this nonvolatile memory area can also store various control programs. The main memory 2 also stores data required for the processor 1 to execute processes for controlling each part in a nonvolatile or volatile memory area. The main memory 2 uses the volatile memory area as a work area where data can be rewritten by the processor 1 as appropriate. The nonvolatile memory area is, for example, ROM (Read Only Memory). The volatile memory area is, for example, RAM (Random Access Memory).

[0016] The storage device 3 corresponds to the auxiliary storage portion of the computer. For example, the storage device 3 can be an EEPROM (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disc Drive), or an SSD (Solid State Drive). The storage device 3 stores data used by the processor 1 when performing various processes, data created by the processes in the processor 1, etc. For example, the storage device 3 can store training data for the local LLM. Note that the storage device 3 may also store the above-mentioned control program. For example, the storage device 3 stores the question-answering program according to this embodiment as a control program, the local LLM, training data, etc.

[0017] The external communication device 4 communicates with the cloud CL.

[0018] The communication interface 5 communicates with an interrogator terminal QT located within the store SH via wired or wireless means.

[0019] The network interface 6 communicates with the respondent terminal RT installed in the store's backyard BY via an in-store network such as a LAN (Local Area Network).

[0020] The store server SS may have only one of the communication interface 5 and the network interface 6. That is, the respondent terminal RT may communicate with the communication interface 5. The questioner terminal QT may be connected to the in-store network directly or via a wireless access point or the like.

[0021] The questioner terminal QT is, for example, an information processing terminal such as a personal computer (PC) installed in a location within the store SH or a tablet terminal loaned by the store SH to customers who visit the store SH. The questioner terminal QT may also be a tablet terminal or smartphone owned by the customer. The questioner terminal QT is a terminal capable of executing web browser application software. For example, a customer can access a question acceptance website prepared in advance by the store SH to enter a question and view the answer to that question. Note that if the questioner terminal QT is the customer's own smartphone, the questioner terminal QT can also communicate with the external communication device 4 via the cloud CL instead of the communication interface 5.

[0022] The respondent terminal RT is a PC operated by a respondent such as a store clerk of the store SH.

[0023] Fig. 2 is a block diagram showing an example of the functional configuration of the store server SS. The processor 1 of the store server SS is connected to a main memory 2, and executes a control program stored in the main memory 2, particularly a question-answering program according to this embodiment, thereby constituting a control unit 11 as shown in Fig. 2. The control unit 11 is equipped with an answer generation unit 111, an accuracy determination unit 112, an answer acquisition unit 113, and an answer output unit 114. Each unit implemented in the control unit 11 can also be referred to as a function. Each unit implemented in the control unit 11 can also be referred to as being implemented in the processor 1.

[0024] The answer generation unit 111 constructs a local LLM implementation environment 111M that executes the local LLM stored in the storage device 3. The answer generation unit 111 executes the local LLM in this local LLM implementation environment 111M to generate an answer document to a question input from a questioner terminal QT operated by a customer who is the questioner.

[0025] The accuracy determination unit 112 determines the accuracy of the answer document generated by the answer generation unit 111. Any method may be used to determine the accuracy of the answer document.

[0026] For example, the accuracy determination unit 112 determines the accuracy level based on whether the answer document contains specific words relative to the words included in the question, such as whether the answer document contains ambiguous words such as "I don't know" or "I wonder..." In this case, the accuracy determination unit 112 calculates an accuracy score that decreases as the number of specific words included in the answer document increases, and determines that the accuracy is low if the score value is equal to or less than a predetermined threshold.

[0027] As shown in FIG. 2, the accuracy determination unit 112 may construct local LLM implementation environments 112M1 and 112M2 that execute local LLMs with models different from those of the local LLM implementation environment 111M of the answer generation unit 111, and determine accuracy using each local LLM. In this case, the accuracy determination unit 112 executes different local LLMs in each of the local LLM implementation environments 112M1 and 112M2 to generate answer documents to questions input from the questioner terminal QT. The accuracy determination unit 112 then calculates a similarity score indicating the similarity between each generated answer document and the answer document generated by the answer generation unit 111. If the calculated score is equal to or less than a predetermined threshold, the accuracy determination unit 112 determines that the accuracy is low. Note that, although FIG. 2 illustrates the execution of two different local LLMs, the number of local LLMs may be one or more than two.

[0028] When the accuracy determination unit 112 determines that the accuracy of the answer document is low, the answer acquisition unit 113 acquires the answer document generated by the answerer terminal RT, which is an answer generation device different from the answer generation unit 111. Specifically, the answer acquisition unit 113 transmits the question input from the questioner terminal QT and the answer document generated by the answer generation unit 111 to the answerer terminal RT. The answerer, who is the operator of the answerer terminal RT, views the question and the answer document, and inputs an updated answer document with updated answer content to the answerer terminal RT. The answerer terminal RT returns the updated answer document input by the answerer to the answer acquisition unit 113. The answer acquisition unit 113 acquires this returned updated answer document.

[0029] The answer output unit 114 outputs the answer document generated by the answer generation unit 111 or the answer document acquired by the answer acquisition unit 113 to the questioner terminal QT. That is, if the accuracy determination unit 112 determines that the accuracy of the answer document generated by the answer generation unit 111 is high, the answer output unit 114 outputs the answer document generated by the answer generation unit 111 to the questioner terminal QT. On the other hand, if the accuracy determination unit 112 determines that the accuracy of the answer document generated by the answer generation unit 111 is low, the answer output unit 114 outputs the answer document acquired by the answer acquisition unit 113 to the questioner terminal QT. Furthermore, in this latter case, the answer output unit 114 updates the learning data LD with the question input from the questioner terminal QT and the answer document acquired by the answer acquisition unit 113, thereby updating and training each local LLM.

[0030] The operation of the store server SS to which the question answering device according to this embodiment having such a configuration is applied and the question answering system including the store server SS will be described below.

[0031] 3 and 4 are sequence diagrams showing an outline of the operation of the question answering system. Here, Fig. 3 shows a case where the answer document generated by the answer generation unit 111 is highly accurate, and Fig. 4 shows a case where the answer document generated by the answer generation unit 111 is low in accuracy.

[0032] First, the case where the accuracy of the answer document is high will be described with reference to FIG. The questioner terminal QT accepts input of a question by a customer who will be the questioner (step S1), and then transmits the input question to the store server SS (step S2).

[0033] When the store server SS receives a question transmitted from the questioner terminal QT, it executes the local LLM in the local LLM implementation environment 111M to generate an answer document for the question (step S3). The store server SS then determines the accuracy of the generated answer document (step S4). The store server SS then determines whether the determined accuracy is low (step S5). If the accuracy is not low, that is, if the accuracy is high, the store server SS transmits the answer document generated in step S3 to the questioner terminal QT, which is the sender of the question (step S6).

[0034] The questioner terminal QT displays the answer document sent from the store server SS (step S7).

[0035] Next, a case where the accuracy of the answer document is low will be described with reference to FIG. Steps S1 to S5 are as described above. If the store server SS determines in step S5 that the accuracy of the answer document created in step S3 is low, it creates a notification to be sent to the answerer terminal RT (step S8). This notification includes the question sent from the questioner terminal QT and the created answer document. The store server SS then sends the created notification to the answerer terminal RT (step S9).

[0036] The answerer terminal RT receives the notification sent from the store server SS and displays the notification (step S10). The answerer clerk who will be the answerer checks this notification and operates the answerer terminal RT to input an updated answer document by modifying the answer document included in the notification or creating a different answer document so that it is an appropriate answer document for the question included in the notification (step S11). The answerer terminal RT transmits the updated answer document thus input to the store server SS (step S12).

[0037] The store server SS receives this updated response document and transmits it to the questioner terminal QT that is the sender of the question (step S13).

[0038] The inquirer terminal QT displays and outputs this updated response document transmitted from the store server SS (step S14).

[0039] In addition to transmitting the updated response document in step S13, the store server SS also updates the learning data LD with the updated response document and the question corresponding to it (step S15).

[0040] Next, a specific example of the operation of the store server SS to which the question answering device according to the first embodiment is applied will be described. FIG. 5 is a flowchart showing the main steps of the question answering process executed by the processor 1 of the store server SS. For example, when the processor 1 receives a question from any questioner terminal QT via the communication interface 5, the processor 1 executes this process based on the question answering program stored in the main memory 2. The processor 1 can simultaneously execute these operations in parallel in response to the reception of questions from each questioner terminal QT. Unless otherwise specified, the processing operation of the processor 1 transitions from ACTx (x is a natural number) in FIG. 5 to ACT(x+1). The procedure shown in FIG. 5 is merely an example. The procedure is not particularly limited as long as similar results are obtained.

[0041] The answer generation unit 111 implemented in the processor 1 as ACT11 temporarily stores a question received from the questioner terminal QT via the communication interface 5 in a volatile memory area used as a work area of ​​the main memory 2. At this time, the answer generation unit 111 associates some information for identifying the questioner terminal QT, such as the terminal identification information, IP address, or terminal number of the questioner terminal QT, with the question and temporarily stores the information in the main memory 2, thereby making it possible to identify which questioner terminal QT sent the question.

[0042] As ACT12, the answer generation unit 111 implemented in the processor 1 executes the local LLM in the local LLM implementation environment 111M to create an answer document to the question temporarily stored in the main memory 2. The created answer document is temporarily stored in the main memory 2, just like the question.

[0043] As ACT13, the accuracy determination unit 112 implemented in the processor 1 determines the accuracy of the created response document.

[0044] In ACT14, the accuracy determination unit 112 implemented in the processor 1 determines whether the determined accuracy is equal to or less than a threshold. That is, the accuracy determination unit 112 determines whether the determined accuracy is low. If the accuracy is equal to or less than the threshold, that is, the accuracy is low, the accuracy determination unit 112 determines YES in ACT14. In this case, the processor 1 proceeds to ACT17. On the other hand, if the accuracy is not equal to or less than the threshold, that is, the accuracy is high, the accuracy determination unit 112 determines NO in ACT14. In this case, the processor 1 proceeds to ACT15.

[0045] As ACT15, the answer output unit 114 implemented in the processor 1 transmits the created answer document temporarily stored in the main memory 2 as an answer to the question to the questioner terminal QT, which is the sender of the question, via the communication interface 5.

[0046] As ACT16, the answer output unit 114 implemented in the processor 1 deletes the data temporarily stored in the main memory 2, in this case, the question, the information for identifying the questioner terminal QT that sent the question, and the created answer document. Then, the processor 1 ends this question answering process.

[0047] In ACT17, the answer acquisition unit 113 implemented in the processor 1 creates a notification including the question and answer document temporarily stored in the main memory 2.

[0048] The answer acquisition unit 113 implemented in the processor 1 as ACT 18 transmits the created notification to the answerer terminal RT via the network interface 6.

[0049] In ACT19, the answer acquisition unit 113 implemented in the processor 1 determines whether or not an updated answer document has been received from the answerer terminal RT via the network interface 6. If an updated answer document has not been received, the answer acquisition unit 113 determines NO in ACT19. In this case, the processor 1 executes ACT19 again. Then, if an updated answer document has been received from the answerer terminal RT, the answer acquisition unit 113 determines YES in ACT19. In this case, the processor 1 proceeds to ACT20.

[0050] As ACT20, the response acquisition unit 113 implemented in the processor 1 temporarily stores the received updated response document in the main memory 2.

[0051] As ACT21, the answer output unit 114 implemented in the processor 1 transmits the received updated answer document temporarily stored in the main memory 2 as an answer to the question to the questioner terminal QT, which is the sender of the question, via the communication interface 5.

[0052] As ACT22, the answer output unit 114 implemented in the processor 1 updates the learning data LD stored in the storage device 3 based on the temporarily stored question and the received updated answer document.

[0053] Thereafter, the answer output unit 114 proceeds to ACT16 and deletes the data temporarily stored in the main memory 2. The data deleted in this case includes the question, information for identifying the questioner terminal QT that sent the question, the created answer document, and the updated answer document.

[0054] As described above, the store server SS to which the question answering device according to the first embodiment is applied uses the local LLM in the local LLM implementation environment 111M to generate an answer document as a first answer to a question input from the questioner terminal QT operated by the questioner, by the answer generation unit 111, and the accuracy determination unit 112 determines the accuracy of the first answer generated by the answer generation unit 111. If the accuracy determined by the accuracy determination unit 112 is equal to or less than a threshold, the store server SS supplies the question to an answer generation device different from the answer generation unit 111, by the answer acquisition unit 113, and acquires an updated answer document as a second answer generated by the answer generation device. The store server SS outputs the first answer generated by the answer generation unit 111 or the second answer acquired by the answer acquisition unit 113 to the questioner terminal QT by the answer output unit 114. Thus, according to the store server SS in the first embodiment, if the accuracy of the answer generated by the answer generation unit 111 is equal to or less than the threshold, that is, if the accuracy of the answer document automatically generated using the local LLM is low, the store server SS causes the answer generation device to generate an updated answer document and outputs this to the questioner terminal QT. Therefore, even if the accuracy of the answer automatically generated using the local LLM is low, it is possible to provide an appropriate answer to the questioner without bothering the questioner.

[0055] Here, if the accuracy determined by the accuracy determination unit 112 is not below the threshold, the answer output unit 114 of the store server SS outputs the first answer generated by the answer generation unit 111 to the questioner terminal QT, and if the accuracy determined by the accuracy determination unit 112 is below the threshold, the answer output unit 114 outputs the second answer acquired by the answer acquisition unit 113 to the questioner terminal QT. Therefore, according to the store server SS in the first embodiment, only either the first or second answer is provided to the questioner terminal QT, thereby preventing the questioner from being confused by being presented with two answers.

[0056] In the question answering system according to the first embodiment, the answer generation device that acquires the second answer by the answer acquisition unit 113 can include an answerer terminal RT into which the answerer inputs the answer to the question as the second answer. In this case, the answer acquisition unit 113 of the store server SS includes a network interface 6 that is a communication unit that transmits the question to the answerer terminal RT and receives the second answer from the answerer terminal RT. For example, a question about the ingredients of a certain product, such as "Please tell me the ingredients of the AAA bento," is specific and the answer is also specific, so the local LLM can answer with high accuracy, such as "Ingredients: rice, shrimp, tempura flour, egg, salad oil, tempura sauce (soy sauce, mirin, sake, sugar), pickled radish." On the other hand, for a question like, "Can vegans eat the CCC menu at BBB Restaurant?", the local LLM might generate a response document like, "The ingredients of the menu item in question are rice, shrimp, tempura flour, egg, salad oil, tempura sauce (soy sauce, mirin, sake, sugar), and pickled radish. It contains animal-derived products, so it is not suitable for vegans." However, in this case, the response is likely to vary depending on the conditions. For example, if the menu item in question contains animal-derived products or additives, the default response would be "You cannot eat it." However, if the chef can avoid using animal-derived products, the response might be, "You cannot eat it because it contains animal-derived products. However, if requested, we can replace the animal-derived products with plant-based products." As another example, if the ingredients are free of animal-derived products, but the cooking area for the menu items containing animal-derived products is not clearly separated from the cooking area for the menu items containing animal-derived products, the response might be, "Although we do not use animal-derived products, we do not recommend this restaurant for strict vegans because the cooking area for the menu items containing animal-derived products and the menu items without animal-derived products is not separate." Therefore, when creating an answer to a question whose answer content changes depending on the conditions, the store server SS in the first embodiment can prepare a highly accurate answer by working in cooperation with the person in charge who is the answerer.

[0057] Furthermore, in the question answering system according to the first embodiment, the answer output unit 114 updates the learning data LD by storing the updated answer document, which is the second answer, in the learning data LD together with the question from which the answer was derived. Therefore, it becomes possible to tune the local LLM used in the answer generation unit 111 and the accuracy determination unit 112 using the second answer, and from the next time onwards, it becomes possible to acquire an answer document with higher accuracy.

[0058] [Second embodiment] Next, a second embodiment will be described. Note that the same reference numerals as in the first embodiment are used for the same configurations and operations as in the first embodiment, and the description thereof will be omitted.

[0059] The store server SS to which the question answering device according to the second embodiment is applied and the configuration of the question answering system including the store server SS are the same as those in the first embodiment.

[0060] 6 is a sequence diagram showing an outline of the operation of a question answering system including a store server SS to which a question answering device according to the second embodiment is applied. In this embodiment, after creating an answer document in step S3, the store server SS first transmits the created answer document (step S6). Thereafter, the store server SS determines the accuracy of the answer document in step S4, and determines whether the determined accuracy is low in step S5.

[0061] If it is determined that the accuracy of the created response document is not low, that is, that the accuracy is high, the store server SS ends the process.

[0062] Furthermore, if it is determined that the accuracy of the created answer document is low, the process proceeds to step S8, where an updated answer document is obtained and the answer is sent to the questioner terminal QT, as described in the first embodiment.

[0063] Next, a specific example of the operation of the store server SS to which the question answering device according to the second embodiment is applied will be described. Fig. 7 is a flowchart showing the main steps of the question answering process executed by the processor 1 of the store server SS in the second embodiment.

[0064] In this embodiment, once the answer generator 111 implemented in the processor 1 creates an answer document to the question in ACT12, the processor 1 proceeds to ACT15.

[0065] Then, in ACT15, the answer output unit 114 implemented in the processor 1 transmits the created answer document to the questioner terminal QT as an answer to the question. After that, the processor 1 proceeds to ACT13.

[0066] The accuracy determination unit 112 implemented in the processor 1 determines the accuracy of the created response document in ACT13, and determines whether the determined accuracy is equal to or less than a threshold value in ACT14.

[0067] Here, if the accuracy is not equal to or less than the threshold, that is, the accuracy is high, and the accuracy determination unit 112 determines NO, the processor 1 in this embodiment proceeds to ACT16.

[0068] Also, if the accuracy is equal to or less than the threshold, that is, the accuracy is low, and the accuracy determination unit 112 determines YES, the processor 1 executes the process from ACT18, as in the first embodiment.

[0069] As described above, in the store server SS to which the question answering device according to the second embodiment is applied, the answer output unit 114 outputs the first answer generated by the answer generation unit 111 and the second answer acquired by the answer acquisition unit 113 to the questioner terminal QT. Specifically, the answer output unit 114 of the store server SS outputs the first answer generated by the answer generation unit 111 to the questioner terminal QT regardless of the accuracy determination result by the accuracy determination unit 112, and if the accuracy determined by the accuracy determination unit 112 is equal to or less than a threshold, the answer output unit 114 further outputs the second answer acquired by the answer acquisition unit 113 to the questioner terminal QT. Therefore, according to the store server SS in the second embodiment, once an answer document is generated by the answer generation unit 111, it is first output to the questioner terminal QT, and if the accuracy of the generated answer document is low, an updated answer document, which is a second answer, is obtained and output to the questioner terminal QT, so that the answer can be presented to the questioner with good response and the questioner can be prevented from having to wait. In particular, since it may take time for the answerer to input the updated answer document, which is the second answer, such two-stage answer output is effective.

[0070] In addition, if the accuracy determined by the accuracy determination unit 112 is below a threshold, the answer output unit 114 may, after outputting the first answer, notify the questioner that a more accurate answer is being prepared.

[0071] [Third embodiment] Next, a third embodiment will be described. Note that the same reference numerals as in the first embodiment are used for the same configurations and operations as in the first embodiment, and the description thereof will be omitted.

[0072] 8 is a block diagram showing an example of the functional configuration of a store server SS to which a question answering device according to the third embodiment is applied. In this embodiment, the answer acquisition unit 113 constructs a local LLM implementation environment 113M that executes a local LLM having a model different from that of the local LLM implementation environment 111M of the answer generation unit 111 and the local LLM implementation environments 112M1 and 112M2 of the accuracy determination unit 112, and issues a question to the local LLM to acquire an updated answer document as a second answer.

[0073] In addition, the answer acquisition unit 113 may not simply use the answer obtained using the local LLM in the local LLM implementation environment 113M as the second answer, but may compare the similarity between that answer document and the answer document generated by the answer generation unit 111, and adopt only the part of the answer document generated here that is similar to the answer document of the answer generation unit 111 as the second answer.

[0074] The operation is the same as that of the first or second embodiment, except that the method for acquiring the updated response document is different.

[0075] As described above, according to the store server SS to which the question answering device of the third embodiment is applied, the answer acquisition unit 113 includes a local LLM implementation environment 113M, which is a second answer generation unit that generates an answer to a question using a local LLM whose model is different from the local LLM in the answer generation unit 111, as an answer generation device that generates a second answer. Therefore, the store server SS in the third embodiment makes it possible to provide an updated answer document to the inquirer without an answerer.

[0076] Of course, the response acquisition unit 113 may acquire the updated response document using a high-performance LLM constructed on the cloud CL by the external communication device 4, rather than a local LLM constructed on the store server SS.

[0077] Although the embodiment of the question answering device has been described above, the embodiment is not limited to this. For example, although an example in which the question answering device is applied to a store server SS has been described, it may also be applied to an edge server located separately from the store server SS within the store SH. It is also possible to apply it to other edge devices within the store SH, such as an edge gateway, rather than a server. Furthermore, a question answering device that communicates with the store server SS and executes a local LLM may be configured on the cloud CL side.

[0078] Also, for example, the order of ACT21 and ACT22 in the flowchart of Fig. 5 may be reversed or may be performed in parallel. In this way, the order of processes may be changed or multiple processes may be performed in parallel as long as there is no discrepancy with the preceding or following processes.

[0079] In addition, although the accuracy of an answer is determined based on whether it is below a threshold, it is also possible to determine whether it is above or above the threshold, and determine that the accuracy is low if it is not above or above the threshold or does not exceed the threshold.

[0080] In the above embodiment, the question answering program executed by the processor 1 of the store server SS to which the question answering device is applied may be configured to be provided by being recorded on a computer-readable recording medium such as a CD-ROM. Alternatively, the question answering program may be stored in a program providing server on the cloud CL and downloaded by the external communication device 4 so as to be stored in the storage device 3.

[0081] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope of the invention and the scope of the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0082] 1...processor, 2...main memory, 3...storage device, 4...external communication device, 5...communication interface, 6...network interface, 7...bus line, 11...control unit, 111...answer generation unit, 111M, 112M1, 112M2, 113M...local LLM implementation environment, 112...accuracy determination unit, 113...answer acquisition unit, 114...answer output unit, BY...store back-office, CL...cloud, LD...learning data, QT...questioner terminal, SH...store, SS...store server.

Claims

1. an answer generation unit that generates a first answer, which is an answer to a question input from a questioner terminal operated by a questioner, using a local large-scale language model (LLM); an accuracy determination unit that determines accuracy of the first answer generated by the answer generation unit; an answer acquisition unit that supplies the question to an answer generation device different from the answer generation unit and acquires a second answer generated by the answer generation device when the accuracy determined by the accuracy determination unit is equal to or less than a threshold; a response output unit that outputs the first response and / or the second response to the questioner terminal; A question answering device comprising:

2. The answer output unit When the accuracy determined by the accuracy determination unit is not equal to or less than the threshold, outputting the first answer generated by the answer generation unit to the questioner terminal; When the accuracy determined by the accuracy determination unit is equal to or less than the threshold, the second answer acquired by the answer acquisition unit is output to the questioner terminal. The question answering device according to claim 1 .

3. The answer output unit outputting the first answer generated by the answer generation unit to the questioner terminal, regardless of the result of the accuracy determination by the accuracy determination unit; When the accuracy determined by the accuracy determination unit is equal to or less than the threshold, the second answer acquired by the answer acquisition unit is output to the questioner terminal. The question answering device according to claim 1 .

4. the answer generation device includes a respondent terminal into which a respondent inputs an answer to the question as the second answer, the answer acquisition unit includes a communication unit that transmits the question to the respondent terminal and receives the second answer from the respondent terminal; 4. The question answering device according to claim 1.

5. 4. The question answering apparatus according to claim 1, wherein the answer generation device includes a second answer generation unit that generates the second answer to the question using a local LLM that has a model different from that of the local LLM.

6. A question answering method executed by a processor of a question answering device that provides an answer to a question from a questioner, comprising: generating, by the processor, a first answer to the question input from a questioner terminal operated by the questioner, using a local LLM; determining, by the processor, the accuracy of the first answer; If the accuracy is equal to or less than a threshold, providing the question to an answer generation device by the processor and obtaining a second answer generated by the answer generation device; outputting, by the processor, the first answer and / or the second answer to the questioner terminal; A question-answering method including:

7. A question answering device that provides an answer to a question from a questioner, the question answering device comprising a processor and a memory, when executed by the processor, the processor, storing the question input from a questioner terminal operated by the questioner in the memory; generating a first answer to the question stored in the memory using a local LLM; determining the accuracy of the first answer; If the accuracy is equal to or less than a threshold, supplying the question stored in the memory to an answer generation device and causing the answer generation device to obtain a second answer; outputting the first answer and / or the second answer to the questioner terminal; Question answering program.

Images