Response support system, response support method, and response support program

Abstract

This invention provides a response support system, response support method, and response support program that use generation AI to obtain appropriate answers. [Solution] The support device 20 includes a control unit 21 connected to the generation AI system 30. The control unit 21 performs a first process of acquiring a question and text placement data related to the question; a second process of acquiring image data obtained by converting the text placement data into an image; and a third process of inputting the question, text placement data, and image data to the generation AI system 30 to obtain an answer.

Description

Technical Field

[0001] The present disclosure relates to an answer support system, an answer support method, and an answer support program for assisting in obtaining an answer using generative AI.

Background Art

[0002] By using generative AI technologies such as large language models, it is possible to obtain an answer to a question. And technologies for obtaining an appropriate answer have also been studied (see, for example, Patent Documents 1 and 2). In the technology described in Patent Document 1, the search performance for creating an appropriate answer to a user's question is improved. In the technology described in Patent Document 2, a multimodal information retrieval technology that uses a plurality of modalities such as text and images to generate a more appropriate answer to a question is used.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when data having a complex data structure is input to generative AI, it may be difficult to obtain an appropriate answer.

Means for Solving the Problems

[0005] The answer support system that solves the above problems includes a control unit connected to a generation AI. The control unit performs a first process of acquiring a question and text placement data related to the question; a second process of acquiring image data obtained by converting the text placement data into an image; and a third process of inputting the question, the text placement data, and the image data to the generation AI to obtain an answer. [Effects of the Invention]

[0006] According to this disclosure, it is possible to obtain more appropriate answers using generative AI. [Brief explanation of the drawing]

[0007] [Figure 1] This is an explanatory diagram of the answer support system according to the first embodiment. [Figure 2] This is an explanatory diagram of the hardware configuration of the first embodiment. [Figure 3] This is an explanatory diagram of the processing procedure of the first embodiment. [Figure 4] This is an explanatory diagram of the processing procedure according to the second embodiment. [Figure 5] This is an explanatory diagram of the processing procedure of the third embodiment. [Modes for carrying out the invention]

[0008] (First Embodiment) A first embodiment of the answer support system, answer support method, and answer support program will be described in accordance with Figures 1 to 4. In this embodiment, it is assumed that answers to questions are obtained using generation AI technology.

[0009] As shown in Figure 1, the answer support system of this embodiment uses a user device 10, a support device 20, an information system 25, and a generation AI system 30 that are interconnected via a network. (Example hardware configuration) Figure 2 shows an example of the hardware configuration of the information processing device H10, which functions as a user device 10, a support device 20, an information system 25, a generation AI system 30, etc.

[0010] The information processing device H10 includes a communication device H11, an input device H12, a display device H13, a storage device H14, and a processor H15. Note that this hardware configuration is an example, and other hardware may be included.

[0011] Communication device H11 is an interface that establishes a communication path with other devices and performs data transmission and reception, such as a network interface or a wireless interface.

[0012] The input device H12 is a device that receives input from users, such as a mouse or keyboard. The display device H13 is a display or touch panel that displays various types of information.

[0013] The storage device H14 is a storage device that stores data and various programs for executing various functions of the user device 10, the support device 20, and the generation AI system 30. Examples of storage devices H14 include ROM, RAM, and hard disks.

[0014] The processor H15 uses programs and data stored in the memory device H14 to control the various processes in the user device 10, support device 20, information system 25, and generation AI system 30. An example of the processor H15 is a CPU or MPU. This processor H15 loads programs stored in ROM, etc., into RAM and executes various processes corresponding to various operations. For example, when an application program of the user device 10, support device 20, information system 25, or generation AI system 30 is started, the processor H15 operates the processes that execute the operations described later.

[0015] Processor H15 is not limited to performing software processing for all the processes it executes. For example, processor H15 may include a dedicated hardware circuit (e.g., an application-specific integrated circuit: ASIC) that performs hardware processing for at least a part of the processes it executes. That is, processor H15 may be configured, for example, as follows.

[0016] 〔1〕One or more processors that operate according to a computer program (software) 〔2〕One or more dedicated hardware circuits that execute at least a part of various processes 〔3〕A circuit (circuitry) including a combination thereof The processor includes a CPU and memories such as a RAM and a ROM, and the memories store program codes or instructions configured to cause the CPU to execute processes. The memory, that is, the computer-readable medium, includes any available medium that can be accessed by a general-purpose or dedicated computer.

[0017] (Functions of each information processing device) Using FIG. 1, the functions of the user device 10, support device 20, information system 25, and generation AI system 30 used in the answer support system will be described.

[0018] The user device 10 is a computer terminal used by a user who uses this system. The support device 20 is a computer system for using a generation AI. This support device 20 includes a control unit 21.

[0019] The control unit 21 performs processes (processes including an acquisition stage, a generation stage, an operation stage, etc.) described later. By executing an answer support program for this purpose, the control unit 21 functions as an acquisition unit 211, a generation unit 212, an operation unit 213, etc.

[0020] The acquisition unit 211 executes a process of acquiring various information from the user device 10. The generation unit 212 generates prompts to be input to the generation AI system 30 and executes a process to obtain a response from the generation AI system 30. In this embodiment, the response obtained is a response script (SQL query) for performing operations such as retrieving, adding, updating, and deleting data from the database.

[0021] The operation unit 213 executes a process to search the information system 25 using the response script obtained from the generation AI system 30. Information system 25 is a computer system that stores various types of information. This information system 25 stores tabular data (text-based data) in which text is arranged in a table format for each type of information.

[0022] The generative AI system 30 is a computer system that utilizes generative AI technology (large-scale language model) to generate responses to prompts. Based on the given prompts (instructions or requests in the input text), the generative AI system 30 automatically generates content such as text, images, and audio.

[0023] (Answer support processing) The response support process will be explained using Figure 3. First, the control unit 21 of the support device 20 executes the question text acquisition process (first process) (step S11). Specifically, the acquisition unit 211 of the control unit 21 acquires the question text entered in the user device 10. In this embodiment, we will describe the case in which a question text that includes conditions for acquiring desired information from tabular data specified in the information system 25 is used.

[0024] Next, the control unit 21 of the support device 20 executes the process of acquiring structured table data (step S12). Specifically, the generation unit 212 of the control unit 21 acquires the specified tabular data from the information system 25 and generates structured table data. In this structured table data, each value included in the tabular format is represented as text generated by assigning a label according to the arrangement of the values. In this structured table data, the table structure is defined using SQL (Structured Query Language) and the data is stored.

[0025] Next, the control unit 21 of the support device 20 performs a first prompt generation process that includes structured table data (step S13). Specifically, the generation unit 212 of the control unit 21 generates a prompt (first prompt) that includes the conditions for obtaining the desired information and structured table data. In this case, the generation unit 212 includes an instruction in the prompt to create an answer script (in this case, an SQL query) for obtaining information according to the conditions of the question. The generation unit 212 then inputs the first prompt to the generation AI system 30.

[0026] Next, the control unit 21 of the support device 20 executes the response acquisition process (step S14). Specifically, the generation unit 212 of the control unit 21 acquires a response corresponding to the first prompt from the generation AI system 30.

[0027] Next, the control unit 21 of the support device 20 performs a determination process to determine whether the first response has been obtained (step S15). Specifically, the generation unit 212 of the control unit 21 determines that the first response has been obtained if the response contains a response script (in this case, an executable SQL query) for extracting the desired information.

[0028] If it is determined that the first response has been obtained (if the answer is "YES" in step S15), the control unit 21 of the support device 20 executes the response output processing (step S19). Specifically, the acquisition unit 211 of the control unit 21 uses the SQL query included in the response to have the SQL Agent retrieve the desired information from the tabular data and output it to the user device 10.

[0029] If it is determined that the first answer could not be obtained (if the answer is "NO" in step S15), the control unit 21 of the support device 20 executes the image data acquisition process (second process) (step S16). Specifically, the generation unit 212 of the control unit 21 generates image data by converting the specified tabular data into an image. For example, to generate this image data, the tabular data may be converted into an image using pandas.DataFrame in Python®, or the image may be rendered and output.

[0030] Next, the control unit 21 of the support device 20 performs a second prompt generation process that includes image data (step S17). Specifically, the generation unit 212 of the control unit 21 generates a prompt (second prompt) that includes conditions for obtaining desired information, structured table data, and image data. In this case as well, the generation unit 212 includes instructions for creating an SQL query to obtain information according to the conditions of the question. The generation unit 212 then inputs the second prompt to the generation AI system 30.

[0031] Next, the control unit 21 of the support device 20 executes the response acquisition process (third process) (step S18). Specifically, the generation unit 212 of the control unit 21 acquires a response corresponding to the second prompt from the generation AI system 30.

[0032] Next, the control unit 21 of the support device 20 performs the output processing of the response (step S19). Specifically, the acquisition unit 211 of the control unit 21 uses the SQL query included in the response to obtain the desired information from the tabular data and outputs it to the user device 10. If the second prompt does not contain an SQL query to extract the desired information from the response, an error message indicating that the response could not be obtained is output to the user device 10.

[0033] (Operation of the first embodiment) Structured tabular data includes headings and labels. This can make the data structure input to the generating AI complex, potentially increasing its load. By handling tabular data alongside images, the burden on the generating AI in understanding the data structure is reduced.

[0034] (Effects of the first embodiment) (1-1) In this embodiment, the control unit 21 of the support device 20 performs a structured table data acquisition process (step S12) and a first prompt generation process including the structured table data (step S13). This makes it possible to acquire desired information in tabular data.

[0035] (1-2) In this embodiment, the control unit 21 of the support device 20 executes a first prompt generation process that includes structured table data (step S13). This includes an instruction to create an SQL query. This SQL query allows the user to obtain desired information from the tabular data. Furthermore, because the method of obtaining the information is clear, it is possible to suppress erroneous responses such as hallucination.

[0036] (1-3) In this embodiment, if it is determined that the first answer has been obtained (if the answer is "YES" in step S15), the control unit 21 of the support device 20 executes the answer output processing (step S19). This makes it possible to quickly obtain the answer using structured table data.

[0037] (1-4) In this embodiment, the control unit 21 of the support device 20 performs image data acquisition processing (step S16) and second prompt generation processing including image data (step S17). This makes it possible to obtain an appropriate response by referring to image data rather than complex structured tabular data. Compared to not using images, multimodal processing using images resulted in increased resilience to variations in notation (such as discrepancies in capitalization between the question text and tabular data) when using SQL queries.

[0038] (Second Embodiment) Next, a second embodiment of the response support system, response support method, and response support program will be described with reference to Figure 4. In the first embodiment, the case in which a response is obtained using a second prompt was described. In the second embodiment, the configuration has been modified to handle cases in which a response cannot be obtained even when using a second prompt. Parts the same as in the first embodiment are denoted by the same reference numerals, and their detailed description is omitted.

[0039] First, the control unit 21 of the support device 20 executes the process of acquiring the question text (step S11) to the process of acquiring the response (step S18). Next, the control unit 21 of the support device 20 performs a determination process to determine whether a second response has been obtained (step S20). Specifically, the generation unit 212 of the control unit 21 determines that a second response has been obtained if the response contains an SQL query for extracting the desired information.

[0040] If it is determined that a second answer has been obtained (if the answer is "YES" in step S20), the control unit 21 of the support device 20 executes the output processing of the answer (step S19). On the other hand, if it is determined that a second answer could not be obtained (i.e., "NO" in step S20), the control unit 21 of the support device 20 performs a question modification process (step S21). Specifically, the generation unit 212 of the control unit 21 modifies the question. For example, it inputs a prompt to the generation AI system 30 instructing it to rewrite the same question content using different wording. Then, it returns to the question text acquisition process (step S11) and re-executes the subsequent processes. In this case, the modified question text is used.

[0041] (Effects of the second embodiment) According to the second embodiment, in addition to the effects of (1-1) to (1-4), the following effects can be obtained.

[0042] (2-1) In this embodiment, if it is determined that a second answer could not be obtained (if the answer is "NO" in step S20), the control unit 21 of the support device 20 performs a question modification process (step S21). This allows the wording of the question to be changed and the test to be retried.

[0043] (Third embodiment) Next, a third embodiment of the answer support system, answer support method, and answer support program will be described with reference to Figure 5. In the first and second embodiments, the control unit 21 of the support device 20 was described in terms of the case in which it performs the question text acquisition process (step S11) and the structured table data acquisition process (step S12). In the third embodiment, the configuration is modified to provide information related to the question text as knowledge, and the same reference numerals are used for parts that are the same as in the first and second embodiments, and their detailed explanation is omitted.

[0044] First, the control unit 21 of the support device 20 executes the process of acquiring the question text (step S11). Next, the control unit 21 of the support device 20 performs the process of acquiring related information (step S30). Specifically, the acquisition unit 211 of the control unit 21 vectorizes the question text. Then, the acquisition unit 211 acquires tabular data that has a high similarity to the vector of the question text in the vector of information recorded in the information system 25.

[0045] Then, the control unit 21 of the support device 20 executes the processing from the acquisition of structured table data (step S12) onward in the second embodiment. In this case, the generation unit 212 of the control unit 21 acquires tabular data with high similarity from the information system 25 and generates structured table data.

[0046] (Effects of the third embodiment) According to the third embodiment, in addition to the effects of (1-1) to (1-4) and (2-1), the following effects can be obtained.

[0047] (3-1) In this embodiment, the control unit 21 of the support device 20 performs the process of acquiring relevant information (step S30). This makes it possible to acquire information (text layout data) used to generate the answer.

[0048] This embodiment can be implemented with the following modifications. This embodiment and the following modifications can be combined with each other to the extent that they do not contradict each other technically. In each of the above embodiments, a user device 10, a support device 20, an information system 25, and a generation AI system 30 are used. The hardware configuration is not limited to these. For example, the information system 25 and the generation AI system 30 may be installed within the support device 20.

[0049] In each of the above embodiments, the control unit 21 executes the answer support program. A non-temporary storage medium (such as a semiconductor memory, magnetic disk, or optical disk) that stores this answer support program may be used.

[0050] In each of the above embodiments, the control unit 21 of the support device 20 performs the process of acquiring structured tabular data (step S12). Here, tabular data specified in the information system 25 is used, but is not limited to this. For example, tabular data may be acquired from the user device 10.

[0051] Furthermore, the information used for searching is not limited to tabular format; information arranged in a predetermined format can be used. For example, knowledge graphs or HTML web pages can be used. In this knowledge graph, nodes representing entities are connected by edges representing relationships between entities, and labels indicating the type and attributes of nodes and edges are attached. In this case, the relationships between nodes are represented by matrices. Here, each element of the matrix is ​​converted into a tabular format showing the relationship between two corresponding nodes. In addition, any network graph represented by edges and nodes, such as tournament graphs, can be used. Moreover, it can be applied not only to network graphs but also to charts such as bar graphs and pie charts.

[0052] In each of the above embodiments, an SQL query is obtained as a response from the generating AI system 30. The response is not limited to an SQL query; the result of the operation by the SQL query itself (the answer) may also be obtained from the generating AI system 30.

[0053] In each of the above embodiments, the control unit 21 of the support device 20 performs a first prompt generation process that includes structured table data (step S13). Alternatively, a second prompt generation process that includes image data may be performed from the beginning (step S17).

[0054] In each of the above embodiments, the control unit 21 of the support device 20 executes the process of acquiring structured table data (step S12). Here, as structured table data, tabular data defined in SQL is used. The use of SQL is not limited to structured table data. For example, structured table data may be generated using Python's pandas.DataFrame. In this case, the answer script is obtained in Python.

[0055] In each of the above embodiments, the control unit 21 of the support device 20 performs the acquisition of structured table data (step S12) and the acquisition of image data (step S16). In this case, the structured table data and image data may be registered in advance in the information system 25 in association with the text layout data (tabular data).

[0056] In the second embodiment described above, the control unit 21 of the support device 20 performs a question modification process (step S21). Here, the wording of the question is modified using the generation AI system 30, but the process is not limited to using the generation AI system 30 as long as the wording of the question can be modified.

[0057] In the third embodiment described above, the control unit 21 of the support device 20 performs the process of acquiring related information (step S30). Specifically, the acquisition unit 211 of the control unit 21 vectorizes the question text and acquires related information. The method of acquiring related information is not limited to this. For example, keywords contained in the question text may be extracted and a keyword search may be performed in the information system 25. [Explanation of Symbols]

[0058] 10...User device, 20...Support device, 21...Control unit, 211...Acquisition unit, 212...Generation unit, 213...Operation unit, 25...Information system, 30...Generation AI system.

Claims

1. A response support system comprising a control unit connected to a generation AI, The control unit, A first process for obtaining a question and text placement data related to the question, A second process to obtain image data by converting the aforementioned text arrangement data into an image, An answer support system characterized by performing a third process of inputting the aforementioned question text, the aforementioned text arrangement data, and the aforementioned image data into the generating AI to obtain an answer.

2. The response support system according to claim 1, characterized in that the aforementioned text arrangement data includes a table.

3. The answer support system according to claim 1 or 2, characterized in that the control unit inputs the question and the text arrangement data to the generating AI, and if an answer is not obtained, it executes the third process.

4. The response support system according to claim 1 or 2, characterized in that the control unit converts the text arrangement data into structured table data.

5. The control unit, In the third process, the generating AI obtains a query to obtain an answer from the text placement data, The response support system according to claim 1 or 2, characterized in that it obtains a response from the text placement data using the aforementioned query.

6. The answer support system according to claim 1 or 2, characterized in that if the control unit does not obtain an answer in the third process, it rewrites the question and re-executes the third process.

7. The answer support system according to claim 1, characterized in that the control unit retrieves the text arrangement data related to the question statement using an information system.

8. A response support method that uses a response support system equipped with a control unit connected to a generation AI to support responses, The control unit, A first process for obtaining a question and text placement data related to the question, A second process to obtain image data by converting the aforementioned text arrangement data into an image, A method for providing answers, characterized by performing a third process of inputting the aforementioned question, the aforementioned text arrangement data, and the aforementioned image data into the generating AI to obtain an answer.

9. A response support program for assisting responses using a response support system equipped with a control unit connected to a generation AI, The control unit, A first process for obtaining a question and text placement data related to the question, A second process to obtain image data by converting the aforementioned text arrangement data into an image, An answer support program that functions as a means for performing a third process of inputting the aforementioned question text, the aforementioned text arrangement data, and the aforementioned image data into the generating AI to obtain an answer.

Images