Answer support system, answer support method, and answer support program

Abstract

We provide an answer support system, an answer support method, and an answer support program that use generation AI to obtain appropriate answers. [Solution] The assistance device 20 includes a control unit 21 connected to a generative AI system 30. The control unit 21 executes a first process of acquiring a question sentence and text layout data related to the question sentence, a second process of acquiring image data that has been visualized from the text layout data, and a third process of inputting the question sentence, the text layout data, and the image data into the generative AI system 30 to acquire an answer.

Description

[Technical Field]

[0001] The present disclosure relates to an answer support system, an answer support method, and an answer support program for supporting answer acquisition using a generation AI. [Background technology]

[0002] By utilizing generative AI technology that uses large-scale language models, etc., it is possible to obtain answers to questions. Technologies for obtaining appropriate answers are also being considered (see, for example, Patent Documents 1 and 2). The technology described in Patent Document 1 improves search performance for generating appropriate answers to user questions. The technology described in Patent Document 2 uses multimodal information retrieval technology that uses multiple modalities, such as text and images, to generate more appropriate answers to questions. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7564400 [Patent Document 2] US Patent Application Publication No. 2022 / 0230061 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when data with a complex data structure is input to the generation AI, it may be difficult to obtain an appropriate answer. [Means for solving the problem]

[0005] The answer support system for solving the above problem includes a control unit connected to a generation AI, which executes a first process of acquiring a question sentence and text layout data related to the question sentence, a second process of acquiring image data that visualizes the text layout data, and a third process of inputting the question sentence, the text layout data, and the image data into the generation AI to acquire an answer. [Effects of the Invention]

[0006] According to the present disclosure, a more appropriate answer can be obtained using a generation AI. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is an explanatory diagram of a response support system according to a first embodiment. [Figure 2] FIG. 2 is an explanatory diagram of a hardware configuration of the first embodiment. [Figure 3] FIG. 2 is an explanatory diagram of a processing procedure according to the first embodiment. [Figure 4] FIG. 10 is an explanatory diagram of a processing procedure according to a second embodiment. [Figure 5] FIG. 11 is an explanatory diagram of a processing procedure according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] (First embodiment) A first embodiment of an answer support system, an answer support method, and an answer support program will be described with reference to Figures 1 to 4. In this embodiment, it is assumed that an answer to a question is obtained using a generation AI technique.

[0009] As shown in FIG. 1, the response support system of this embodiment uses a user device 10, a support device 20, an information system 25, and a generation AI system 30, all of which are interconnected via a network. (Example of hardware configuration) FIG. 2 shows an example of the hardware configuration of an information processing device H10 that functions as the user device 10, the support device 20, the information system 25, the generation AI system 30, and the like.

[0010] The information processing device H10 includes a communication device H11, an input device H12, a display device H13, a storage device H14, and a processor H15. Note that this hardware configuration is an example, and the information processing device H10 may include other hardware.

[0011] The communication device H11 is an interface that establishes a communication path with another device and executes data transmission and reception, and is, for example, a network interface or a wireless interface.

[0012] The input device H12 is a device that accepts input from a user, etc., and is, for example, a mouse, a keyboard, etc. The display device H13 is a display, a touch panel, etc. that displays various information.

[0013] The storage device H14 is a storage device that stores data and various programs for executing various functions of the user device 10, the assistance device 20, and the generation AI system 30. Examples of the storage device H14 include a ROM, a RAM, and a hard disk.

[0014] The processor H15 uses programs and data stored in the storage device H14 to control each process in the user device 10, the support device 20, the information system 25, and the generation AI system 30. Examples of the processor H15 include a CPU and an MPU. The processor H15 loads programs stored in a ROM or the like into a RAM and executes various processes corresponding to the various processes. For example, when an application program of the user device 10, the support device 20, the information system 25, or the generation AI system 30 is launched, the processor H15 runs a process that executes each process described below.

[0015] The processor H15 is not limited to a processor that performs all of its processing using software. For example, the processor H15 may include a dedicated hardware circuit (e.g., an application-specific integrated circuit (ASIC)) that performs hardware processing for at least part of the processing it performs. That is, the processor H15 may be configured, for example, with the following:

[0016] [1] One or more processors that operate according to a computer program (software). [2] One or more dedicated hardware circuits that perform at least some of the various processes [3] Circuits containing combinations of these The processor includes a CPU and memory, such as RAM and ROM, that stores program code or instructions configured to cause the CPU to perform processes. Memory, or computer-readable media, includes any available media that can be accessed by a general-purpose or special-purpose computer.

[0017] (Functions of each information processing device) The functions of a user device 10, a support device 20, an information system 25, and a generation AI system 30 used in the response support system will be described with reference to FIG.

[0018] The user device 10 is a computer terminal used by a user of this system. The support device 20 is a computer system for utilizing the generation AI. The support device 20 includes a control unit 21.

[0019] The control unit 21 performs the processes described below (processes including an acquisition stage, a generation stage, an operation stage, etc.) By executing a response support program for this purpose, the control unit 21 functions as an acquisition unit 211, a generation unit 212, an operation unit 213, etc.

[0020] The acquisition unit 211 executes a process of acquiring various pieces of information from the user device 10. The generation unit 212 generates a prompt to be input to the generation AI system 30 and executes a process to obtain a response from the generation AI system 30. In this embodiment, the response obtained is an answer script (SQL query) for performing operations such as retrieving, adding, updating, and deleting data from a database.

[0021] The operation unit 213 executes a process of searching the information system 25 using the answer script acquired from the generation AI system 30. The information system 25 is a computer system that stores various types of information. The information system 25 stores tabular data (text layout data) in which text is laid out in a table format for various types of information.

[0022] The generative AI system 30 is a computer system that utilizes generative AI technology (large-scale language model) to generate responses to prompts. The generative AI system 30 automatically generates content such as text, images, and audio based on a given prompt (an input text instruction or request).

[0023] (Answer support processing) The answer support process will be described with reference to FIG. First, the control unit 21 of the assistance device 20 executes a process (first process) for acquiring a question (step S11). Specifically, the acquisition unit 211 of the control unit 21 acquires a question input to the user device 10. In this embodiment, a case will be described in which a question including a condition for acquiring desired information from tabular data specified in the information system 25 is used.

[0024] Next, control unit 21 of support device 20 executes a process for acquiring structured table data (step S12). Specifically, generation unit 212 of control unit 21 acquires the specified tabular data from information system 25 and generates structured table data. This structured table data is data expressed in text generated by assigning a label to each value included in the tabular format according to the arrangement of the value. In this structured table data, the table structure is defined in SQL (Structured Query Language) and data is stored.

[0025] Next, the control unit 21 of the assistance device 20 executes a first prompt generation process including the structured table data (step S13). Specifically, the generation unit 212 of the control unit 21 generates a prompt (first prompt) including the structured table data and conditions for acquiring desired information. In this case, the generation unit 212 includes in the prompt an instruction to create an answer script (here, an SQL query) for acquiring information according to the conditions of the question sentence. Then, the generation unit 212 inputs the first prompt to the generation AI system 30.

[0026] Next, the control unit 21 of the assistance device 20 executes a response acquisition process (step S14). Specifically, the generation unit 212 of the control unit 21 acquires a response corresponding to the first prompt from the generation AI system 30.

[0027] Next, the control unit 21 of the support device 20 executes a determination process as to whether or not the first answer has been acquired (step S15). Specifically, the generation unit 212 of the control unit 21 determines that the first answer has been acquired if the response includes an answer script (here, an operable SQL query) for extracting desired information.

[0028] If it is determined that the first answer has been acquired (YES in step S15), the control unit 21 of the support device 20 executes an answer output process (step S19). Specifically, the acquisition unit 211 of the control unit 21 uses the SQL query included in the response to cause the SQL Agent to acquire desired information from the tabular data and output the information to the user device 10.

[0029] If it is determined that the first answer could not be acquired ("NO" in step S15), the control unit 21 of the support device 20 executes an image data acquisition process (second process) (step S16). Specifically, the generation unit 212 of the control unit 21 generates image data by visualizing the specified tabular data. To generate this image data, for example, pandas.Dataframe of Python (registered trademark) may be used to convert the tabular data into an image, or the image may be output by rendering.

[0030] Next, the control unit 21 of the assistance device 20 executes a second prompt generation process including the image data (step S17). Specifically, the generation unit 212 of the control unit 21 generates a prompt (second prompt) including the conditions for acquiring desired information, the structured table data, and the image data. In this case, the generation unit 212 also includes an instruction to create an SQL query for acquiring information according to the conditions of the question sentence. Then, the generation unit 212 inputs the second prompt to the generation AI system 30.

[0031] Next, the control unit 21 of the assistance device 20 executes a response acquisition process (third process) (step S18). Specifically, the generation unit 212 of the control unit 21 acquires a response corresponding to the second prompt from the generation AI system 30.

[0032] Next, the control unit 21 of the support device 20 executes an answer output process (step S19). Specifically, the acquisition unit 211 of the control unit 21 acquires desired information from the tabular data using the SQL query included in the response, and outputs the information to the user device 10. If the response to the second prompt does not include an SQL query for extracting the desired information, the control unit 21 outputs an error message to the user device 10 indicating that the answer cannot be acquired.

[0033] (Operation of the first embodiment) Structuring tabular data: Table data contains headers and labels. This can lead to complex data structures being input to the generation AI, placing a heavy load on the AI. By handling tabular data together with images, the burden on the generation AI to understand the data structure is reduced.

[0034] (Effects of the first embodiment) (1-1) In this embodiment, the control unit 21 of the assistance device 20 executes a process of acquiring structured table data (step S12) and a process of generating a first prompt including the structured table data (step S13). This allows desired information to be acquired from the tabular data.

[0035] (1-2) In this embodiment, the control unit 21 of the assistance device 20 executes a first prompt generation process including structured table data (step S13). Here, an instruction to create an SQL query is included. This SQL query makes it possible to acquire desired information from the tabular data. Furthermore, since the method of acquiring the information is clear, it is possible to suppress erroneous answers such as hallucinations.

[0036] (1-3) In this embodiment, when it is determined that the first answer has been obtained ("YES" in step S15), the control unit 21 of the assistance device 20 executes an answer output process (step S19). This allows the answer to be quickly obtained using the structured table data.

[0037] (1-4) In this embodiment, the control unit 21 of the assistance device 20 executes an image data acquisition process (step S16) and a second prompt generation process including the image data (step S17). This allows for a more appropriate response to be obtained by referring to the image data rather than complex structured table data. Compared to a case where images are not used, the multimodalization using images has resulted in a more robust response to variations in notation (such as differences in the case of letters between the question and the tabular data).

[0038] (Second embodiment) Next, a second embodiment that embodies an answer support system, an answer support method, and an answer support program will be described with reference to Fig. 4. In the first embodiment, a case where an answer is obtained using a second prompt was described. In the second embodiment, the configuration has been modified to accommodate a case where an answer cannot be obtained even when the second prompt is used, and the same parts as in the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

[0039] First, the control unit 21 of the support device 20 executes a process of acquiring a question (step S11) to a process of acquiring a response (step S18). Next, the control unit 21 of the support device 20 executes a determination process as to whether or not a second answer has been acquired (step S20). Specifically, the generation unit 212 of the control unit 21 determines that a second answer has been acquired if the response includes an SQL query for extracting desired information.

[0040] If it is determined that the second answer has been acquired (YES in step S20), the control unit 21 of the support device 20 executes an answer output process (step S19). On the other hand, if it is determined that a second answer could not be obtained ("NO" in step S20), the control unit 21 of the assistance device 20 executes a question modification process (step S21). Specifically, the generation unit 212 of the control unit 21 modifies the question. For example, it inputs a prompt to the generation AI system 30 instructing it to rewrite the same question content into a different expression. Then, the control unit 21 returns to the question text acquisition process (step S11) and executes the subsequent processes again. In this case, the modified question text is used.

[0041] (Effects of the second embodiment) According to the second embodiment, in addition to the effects (1-1) to (1-4), the following effects can be obtained.

[0042] (2-1) In this embodiment, if it is determined that the second answer could not be obtained (if "NO" in step S20), the control unit 21 of the assistance device 20 executes a question change process (step S21). This allows the question to be rephrased and retried.

[0043] (Third embodiment) Next, a third embodiment that embodies an answer support system, an answer support method, and an answer support program will be described with reference to Fig. 5. In the first and second embodiments, the control unit 21 of the support device 20 executes a process for acquiring a question sentence (step S11) and a process for acquiring structured table data (step S12). In the third embodiment, the configuration is modified so that information related to the question sentence is provided as knowledge, and the same parts as in the first and second embodiments are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0044] First, the control unit 21 of the support device 20 executes a process of acquiring a question sentence (step S11). Next, the control unit 21 of the support device 20 executes a process of acquiring related information (step S30). Specifically, the acquisition unit 211 of the control unit 21 vectorizes the question. Then, the acquisition unit 211 acquires tabular data that has a high similarity to the vector of the question among the vectors of information recorded in the information system 25.

[0045] Then, the control unit 21 of the support device 20 executes the process of acquiring structured table data (step S12) and subsequent processes in the second embodiment. In this case, the generation unit 212 of the control unit 21 acquires tabular data with a high degree of similarity from the information system 25 and generates structured table data.

[0046] (Effects of the third embodiment) According to the third embodiment, in addition to the effects (1-1) to (1-4) and (2-1), the following effects can be obtained.

[0047] (3-1) In this embodiment, the control unit 21 of the assistance device 20 executes a process of acquiring related information (step S30), thereby acquiring information (text layout data) used to generate an answer.

[0048] This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility. In each of the above embodiments, a user device 10, a support device 20, an information system 25, and a generation AI system 30 are used. The hardware configuration is not limited to these. For example, the information system 25 and the generation AI system 30 may be provided within the support device 20.

[0049] In each of the above embodiments, the control unit 21 executes a response support program. A non-transitory storage medium (such as a semiconductor memory, a magnetic disk, or an optical disk) that stores this response support program may be used.

[0050] In each of the above embodiments, the control unit 21 of the support device 20 executes a process of acquiring structured table data (step S12). Here, the tabular data specified in the information system 25 is used, but is not limited to this. For example, the tabular data may be acquired from the user device 10.

[0051] Furthermore, the information used for the search is not limited to a table format; any information arranged in a predetermined format can be used. For example, a knowledge graph or a web page written in HTML may be used. In this knowledge graph, nodes representing entities are connected by edges that represent the relationships between the entities, and labels indicating the type and attributes of the nodes and edges are added. In this case, the relationships between nodes are expressed as a matrix. Here, each element of the matrix is ​​converted into a table format that shows the relationship between two corresponding nodes. Also, general network graphs expressed by edges and nodes, such as tournament graphs, may be used. Furthermore, the method can be applied not only to network graphs, but also to charts such as bar graphs and pie charts.

[0052] In each of the above embodiments, an SQL query is obtained as a response from the generative AI system 30. The response is not limited to an SQL query, and the operation result (answer) itself based on the SQL query may be obtained from the generative AI system 30.

[0053] In each of the above embodiments, the control unit 21 of the support device 20 executes a first prompt generation process including structured table data (step S13). Alternatively, the control unit 21 may execute a second prompt generation process including image data (step S17) from the beginning.

[0054] In each of the above embodiments, the control unit 21 of the assistance device 20 executes a process for acquiring structured table data (step S12). Here, tabular data defined in SQL is used as the structured table data. The structured table data is not limited to SQL. For example, the structured table data may be generated using pandas.DataFrame in Python. In this case, the answer script is acquired in Python.

[0055] In each of the above embodiments, the control unit 21 of the support device 20 executes a process of acquiring structured table data (step S12) and a process of acquiring image data (step S16). In this case, the structured table data and image data may be registered in advance in the information system 25 in association with the text layout data (tabular data).

[0056] In the second embodiment, the control unit 21 of the assistance device 20 executes a question change process (step S21). Here, the wording of the question is changed using the generative AI system 30, but the use of the generative AI system 30 is not limited as long as the wording of the question can be changed.

[0057] In the third embodiment, the control unit 21 of the support device 20 executes a process of acquiring related information (step S30). Specifically, the acquisition unit 211 of the control unit 21 vectorizes the question sentence and acquires the related information. The method of acquiring the related information is not limited to this. For example, keywords included in the question sentence may be extracted, and a keyword search may be performed in the information system 25. [Explanation of symbols]

[0058] 10...user device, 20...assistance device, 21...control unit, 211...acquisition unit, 212...generation unit, 213...operation unit, 25...information system, 30...generation AI system.

Claims

1. An answer support system including a control unit connected to a generation AI, The control unit a first process of acquiring a question sentence and text layout data related to the question sentence; a second process for acquiring image data obtained by imaging the text layout data; and a third process of inputting the question sentence, the text layout data, and the image data into the generating AI to obtain an answer.

2. The answer support system according to claim 1 , wherein the text layout data includes a table.

3. The answer support system according to claim 1 or 2, characterized in that the control unit inputs the question sentence and the text placement data to the generation AI, and if no answer is obtained, executes the third process.

4. 3. The response support system according to claim 1, wherein the control unit converts the text layout data into structured table data.

5. The control unit In the third process, a query for obtaining an answer from the text layout data is obtained from the generating AI; 3. The answer support system according to claim 1, wherein the query is used to acquire an answer from the text layout data.

6. 3. The response support system according to claim 1, wherein the control unit rewrites the question sentence and re-executes the third process if an answer is not obtained in the third process.

7. 2. The answer support system according to claim 1, wherein the control unit retrieves the text layout data related to the question by searching an information system.

8. An answer support method for supporting an answer using an answer support system including a control unit connected to a generation AI, The control unit a first process of acquiring a question sentence and text layout data related to the question sentence; a second process for acquiring image data obtained by imaging the text layout data; and a third process of inputting the question sentence, the text layout data, and the image data into the generating AI to obtain an answer.

9. An answer support program for supporting answers using an answer support system having a control unit connected to a generation AI, The control unit a first process of acquiring a question sentence and text layout data related to the question sentence; a second process for acquiring image data obtained by imaging the text layout data; and a third process of inputting the question, the text layout data, and the image data into the generating AI to obtain an answer.

Images