Information processing device, information processing method, and program

Abstract

To obtain and output appropriate information using a generative AI based on non-text information.SOLUTION: An information processing device 4 obtains and outputs appropriate information using a generative AI based on non-text information. The information processing device 4 includes: a definition management unit 411 that stores prompt definition information for creating a prompt to be provided to the generative AI; an input reception unit 421 that receives a user input which includes input information including non-text information; a text conversion unit 431 that obtains text based on the non-text information; a prompt creation unit 432 that creates a prompt to be provided to the generative AI by using both one or more pieces of the input information using the text obtained by the text conversion unit 431 and the prompt definition information; an estimation information acquisition unit 132 that provides the prompt to the generative AI and obtains one or more pieces of estimation information obtained by the generative AI; and an information output unit 141 that outputs output information based on the one or more pieces of estimation information.SELECTED DRAWING: Figure 19

Description

[Technical field]

[0001] The present invention relates to an information processing device that outputs information based on information acquired using a generation AI. [Background technology]

[0002] Recently, generative AI such as ChatGPT has been attracting attention (see Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] OpenAI, ChatGPT [Retrieved September 2, 2023], Internet [URL: https: / / openai.com / chatgpt, https: / / chat.openai.com / ] Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the prior art, there was no means to appropriately use the inference results of the generative AI. In particular, there was no means to give the generative AI appropriate prompts and to appropriately process and output the information obtained from the generative AI.

[0005] Furthermore, conventional technology was not able to obtain and output appropriate information using generative AI based on non-text information. [Means for solving the problem]

[0006] The information processing device of the first invention is an information processing device that includes a definition management unit in which prompt definition information for creating a prompt to be given to a generation AI is stored, an input receiving unit that accepts user input including input information including non-text information, a text conversion unit that acquires text based on the non-text information, a prompt creation unit that creates a prompt to be given to the generation AI using one or more pieces of input information using the text acquired by the text conversion unit and the prompt definition information, an estimated information acquisition unit that provides the prompt to the generation AI and acquires one or more pieces of estimated information acquired by the generation AI, and an information output unit that outputs output information based on the one or more pieces of estimated information.

[0007] With this configuration, appropriate information can be obtained and output using a generation AI based on non-text information.

[0008] In addition, the information processing device of the second invention is an information processing device in which, compared to the first invention, the non-text information is image data, and the text conversion unit is equipped with character recognition means for performing character recognition processing on the image data to obtain text.

[0009] With this configuration, appropriate information can be obtained and output using the generation AI based on the image data.

[0010] In addition, the information processing device of the third invention is an information processing device in which, compared to the first invention, the text conversion unit is equipped with an explanatory text acquisition means that provides image data to the generation AI and acquires text explaining the image data from the generation AI.

[0011] With this configuration, appropriate information can be obtained and output using the generation AI based on the image data.

[0012] In addition, in the information processing device of the fourth invention, compared to the third invention, the text conversion unit further includes a character recognition means which performs character recognition processing on image data and acquires text, and a text construction means which constructs text using both the text acquired by the character recognition means and the text acquired by the explanatory text acquisition means, and the prompt creation unit is an information processing device which creates a prompt to be given to the generation AI using one or more pieces of input information using the text constructed by the text construction means and the prompt definition information.

[0013] With this configuration, appropriate information can be obtained and output using the generation AI based on the image data.

[0014] In addition, the information processing device of the fifth invention is an information processing device in which, compared to the second invention, the prompt definition information has location identification information that identifies a location at which to place partial information based on a non-text image portion which is a table, graph, or image, and includes a non-text image portion which is a table, graph, or image in the image data, and further includes a non-text image detection unit that detects the non-text image portion in the image data, and the prompt creation unit obtains the partial information based on the non-text image portion detected by the non-text image detection unit and creates a prompt using the partial information.

[0015] With this configuration, appropriate information can be obtained and output by appropriately using tables, graphs, or images in the image data.

[0016] In addition, the information processing device of the sixth invention is an information processing device in which, compared to the second invention, the input information is a text file including non-text image portions which are tables, graphs or images, and further includes a non-text image detection unit which detects the non-text image portions in the text file, and the prompt creation unit obtains partial information based on the non-text image portions detected by the non-text image detection unit and creates a prompt using the partial information as well.

[0017] With such an arrangement, tables or graphs or images in the text file can be used as appropriate to obtain and output the appropriate information.

[0018] In addition, the information processing device of the seventh invention is an information processing device in which, compared to the first invention, the non-text information is a video, and the text conversion unit is equipped with a video text acquisition means for acquiring text which is an explanatory text related to the video.

[0019] With this configuration, appropriate information can be obtained and output using a generating AI based on the video.

[0020] In addition, compared to the seventh invention, the information processing device of the eighth invention is an information processing device in which the video text acquisition means acquires two or more representative still images from the video, acquires explanatory text for each of the two or more representative still images, and acquires text based on the two or more explanatory texts.

[0021] With this configuration, appropriate information can be obtained and output using a generating AI based on the video.

[0022] Furthermore, an information processing device according to a ninth aspect of the present invention is an information processing device according to the first aspect of the present invention, in which the non-text information is voice, and the text conversion unit includes voice recognition means for recognizing voice and acquiring text.

[0023] With this configuration, appropriate information can be obtained and output using a generation AI based on the voice.

[0024] In addition, the information processing device of the tenth invention is an information processing device in which, compared to the first invention, the text conversion unit is equipped with an attribute value acquisition means for acquiring a data attribute value of the non-text information, and acquires text by performing different processing depending on the data attribute value.

[0025] With this configuration, appropriate information based on the non-text information can be obtained and output, based on the data attribute values ​​of the non-text information.

[0026] In addition, compared to the first invention, the information processing device of the eleventh invention is an information processing device in which the definition management unit stores parameter definition information, which is information for obtaining parameters to be used when obtaining a prompt, using input information, and the prompt creation unit includes a parameter acquisition means for obtaining one or more parameters using the input information and the parameter definition information, and a prompt creation means for creating a prompt using the prompt definition information and one or more parameters.

[0027] With this configuration, appropriate information can be output using generation AI.

[0028] Furthermore, the information processing device of the twelfth invention is an information processing device in which, compared to the eleventh invention, the parameter definition information has an information identifier that identifies input information or parameters and a processing identifier that identifies processing for the input information or parameters, and the parameter acquisition means acquires input information or parameters identified by the information identifier held by the parameter definition information, performs processing identified by the processing identifier using the input information or parameters, and acquires one or more parameters.

[0029] With this configuration, appropriate prompts can be given to the generation AI to output appropriate information.

[0030] In addition, the information processing device of the thirteenth invention is an information processing device in which, compared to the first invention, the prompt definition information has partial prompt definition information for constructing a prompt in correspondence with each of one or more selection conditions using input information or parameters, and the prompt creation unit determines the selection condition satisfied by the input information or parameters, obtains the partial prompt definition information that pairs with the selection condition, and creates a prompt using the partial prompt definition information.

[0031] With this configuration, appropriate prompts can be given to the generation AI to output appropriate information.

[0032] Furthermore, the information processing device of the fourteenth invention is an information processing device according to the first invention, in which the definition management unit stores prompt definition information corresponding to two or more types of category information that specify a type of input information, the user input includes two or more input information corresponding to the type information, the prompt creation unit creates a prompt for each of the two or more types of category information included in the user input using the input information corresponding to the type information and the prompt definition information corresponding to the type information, and the estimated information acquisition unit provides a prompt to the generation AI for each of the two or more types of category information, and acquires one or more estimated information.

[0033] With this configuration, appropriate prompts can be given to the generation AI to output appropriate information.

[0034] In addition, the information processing device of the fifteenth invention is an information processing device in which, compared to the first invention, the estimated information acquisition unit acquires two or more pieces of estimated information, an output management unit in which output information corresponding to each of two or more output conditions using the two or more pieces of estimated information is stored, and an output information acquisition unit acquires from the output management unit output information corresponding to the output condition satisfied by the two or more pieces of estimated information, and the information output unit outputs the output information acquired by the output information acquisition unit.

[0035] With this configuration, more appropriate information can be output using generation AI.

[0036] In addition, the information processing device of the sixteenth invention is an information processing device that, compared to the first invention, further includes a validity judgment unit that judges whether the estimated information acquired by the estimated information acquisition unit satisfies a validity condition, and if the validity condition is not satisfied, the information output unit outputs error information to the effect that the validity condition is not satisfied.

[0037] With this configuration, more appropriate information can be output using generation AI. Effect of the Invention

[0038] The information processing device according to the present invention can provide a means for appropriately utilizing the inference results of the generation AI. [Brief description of the drawings]

[0039] [Figure 1] Conceptual diagram of information system A in embodiment 1. [Diagram 2] Block diagram of Information System A [Diagram 3] Block diagram of the information processing device 1 [Figure 4] A flowchart illustrating an example of the operation of the information processing device 1. [Diagram 5] A flowchart illustrating an example of the prompt creation process. [Figure 6] A flowchart for explaining an example of a process for acquiring the parameters. [Figure 7] A flowchart illustrating an example of the partial prompt acquisition process. [Figure 8] A flowchart illustrating an example of the estimated information acquisition process. [Figure 9] A flowchart for explaining an example of the validity determination process [Figure 10] 11 is a flowchart illustrating an example of the output information acquisition process. [Figure 11] A diagram showing the prompt definition management table [Figure 12] The parameter definition management table is shown in the figure. [Figure 13] The processing management table [Figure 14] The processing management table [Figure 15] The output definition management table [Figure 16] A diagram showing an example of the input screen [Figure 17] The intermediate value management table [Figure 18] A diagram showing an example of the output [Figure 19] Block diagram of information system B in embodiment 2 [Figure 20] Block diagram of the information processing device 4 [Figure 21] A flowchart for explaining an example of the operation of the information processing device 4. [Figure 22] A flowchart illustrating an example of the text conversion process. [Figure 23] A flowchart illustrating an example of the image-to-text process. [Figure 24] A flowchart illustrating an example of the non-text image part processing. [Diagram 25] A flowchart illustrating an example of the video text conversion process. [Figure 26] A diagram showing an example of the prompt definition information [Figure 27] FIG. 13 shows an example of the parameter management table. [Figure 28] An example of the output definition management table [Figure 29] FIG. 13 shows an example of the input screen. [Diagram 30] FIG. 13 shows an example of the receipt image. [Diagram 31] FIG. 13 shows an example of text obtained from the image data. [Diagram 32] A diagram showing an example of a response from the AI [Diagram 33] A diagram showing an example of the output [Diagram 34] Block diagram of a computer system according to the above embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0040] Hereinafter, an embodiment of an information processing device and the like will be described with reference to the drawings. Note that, in the embodiments, components with the same reference numerals perform similar operations, and therefore repeated description may be omitted.

[0041] (Embodiment 1) In this embodiment, an information processing device is described that accepts user input, uses the user input and prompt definition information to obtain a prompt to be given to a generation AI, provides the prompt to the generation AI, obtains estimated information from the generation AI, and outputs output information based on the estimated information.

[0042] In addition, in this embodiment, we will describe an information processing device that acquires one or more parameters using parameter definition information that defines a method of acquiring parameters to be used when creating a prompt, and creates a prompt using the one or more parameters and the prompt definition information.

[0043] In this embodiment, an information processing device will be described that performs processing using a user input, acquires parameters, and creates a prompt using the parameters and prompt definition information.

[0044] In addition, in this embodiment, an information processing device that creates different prompts depending on input information or parameters included in a user input will be described.

[0045] In this embodiment, an information processing device will be described that selects one or more prompt definition information corresponding to input information from two or more prompt definition information, and creates a prompt using each of the one or more prompt definition information.

[0046] In this embodiment, an information processing device that uses two or more pieces of estimated information acquired from a generation AI to construct and output one piece of information will be described.

[0047] Furthermore, in this embodiment, an information processing device that determines the validity of estimated information acquired from a generation AI based on rules will be described.

[0048] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0049] Furthermore, in this specification, selecting or determining information Z means obtaining information Z, obtaining a pointer to information Z, obtaining the ID of information Z, setting a flag for information Z, etc., and is sufficient as long as information Z can be accessed.

[0050] Fig. 1 is a conceptual diagram of an information system A in this embodiment. The information system A includes an information processing device 1, one or more terminal devices 2, and one or more generation AI devices 3. Note that in Fig. 1, there may be two or more generation AI devices 3.

[0051] The information processing device 1 is a device that accepts input from a user and outputs output information using the input and a generating AI. The information processing device 1 is, for example, a cloud server or an ASP server, but the type does not matter. If the information processing device 1 is a terminal, it may be considered that the terminal device 2 is not necessary in the information system A, or that the information processing device 1 also serves as the terminal device 2. If the information processing device 1 is a terminal, the information processing device 1 is, for example, a so-called personal computer, smartphone, or tablet terminal, but the type does not matter.

[0052] The terminal device 2 is a terminal used by a user. A user is a person who inputs input information. The terminal device 2 is, for example, a so-called personal computer, a smartphone, or a tablet terminal, but the type is not important.

[0053] The generation AI device 3 is a device having the function of a generation AI. Here, the generation AI device 3 usually has the function of a text generation AI. The generation AI is, for example, ChatGPT or Google Bard, but it does not matter. Google is a registered trademark. The generation AI device 3 is, for example, a cloud server or an ASP server, but it does not matter the type.

[0054] FIG. 2 is a block diagram of an information system A according to the present embodiment. FIG. 3 is a block diagram of an information processing device 1.

[0055] The information processing device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and an output unit 14. The storage unit 11 includes a definition management unit 111, an output management unit 112, and a condition management unit 113. The reception unit 12 includes an input reception unit 121. The processing unit 13 includes a prompt creation unit 131, an estimated information acquisition unit 132, a validity determination unit 133, and an output information acquisition unit 134. The prompt creation unit 131 includes a parameter acquisition means 1311 and a prompt creation means 1312. The output unit 14 includes an information output unit 141.

[0056] The terminal device 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 25, and a terminal output unit .

[0057] Various types of information are stored in the storage unit 11 constituting the information processing device 1. The various types of information are, for example, prompt definition information (to be described later), output information (to be described later), and various conditions. It goes without saying that the various conditions may be embedded in a program.

[0058] The definition management unit 111 stores one or more pieces of prompt definition information. It is preferable that type information is associated with each of the one or more pieces of prompt definition information. The type information is information that identifies the type of the prompt. Typically, the type information is associated with the prompt to be acquired.

[0059] One or more pieces of parameter definition information are stored in the definition management unit 111. It is preferable that type information is associated with each of the one or more pieces of parameter definition information.

[0060] Prompt definition information is information for creating a prompt to be given to a generation AI. Prompt definition information is, for example, template information for creating a prompt. A prompt is completed using the prompt definition information and input information. The prompt definition information includes, for example, one or more variables. Input information or parameters corresponding to the variable are given to each of the one or more variables in the prompt definition information, and the prompt is completed.

[0061] It is preferable that the prompt definition information has two or more partial prompt definition information. It is preferable that each of the two or more partial prompt definition information corresponds to a selection condition. The selection condition is a condition for selecting the partial prompt definition information. The selection condition is a condition using one or more pieces of information among the input information or parameters.

[0062] A parameter is information based on input information, which will be described later. A parameter may be information based on other parameters. A parameter is usually information obtained by performing processing using one or more pieces of input information or one or more parameters, but may be input information. A parameter may also be referred to as information, a value, etc.

[0063] The type information is information that identifies the type of input information.

[0064] The parameter definition information is information for acquiring a parameter. The parameter definition information has, for example, one or more information identifiers. The information identifier is an input information identifier or a parameter identifier. The input information identifier is information for identifying input information, for example, an input information name. The parameter identifier is information for identifying a parameter, for example, a parameter name. Note that a parameter is information acquired based on input information.

[0065] The output management unit 112 stores, for example, one or more pieces of output information. It is preferable that an output condition is associated with each of the one or more pieces of output information. The output condition is a condition for selecting the output information. The output condition is a condition related to one or more pieces of information among the input information or parameters. The output condition is, for example, a condition using two or more pieces of estimated information.

[0066] For example, one or more pieces of output definition information are stored in the output management unit 112. It is preferable that the output definition information has two or more pieces of partial output definition information.

[0067] The output definition information is information for acquiring output information. The output definition information is, for example, a character string or a character string including one or more variables. For example, input information or a parameter is substituted for the variable.

[0068] Partial output definition information is information for acquiring partial output information. Partial output definition information is, for example, a character string or a character string including one or more variables. Partial output information is a part of information that constitutes output information. Partial output definition information is, for example, associated with an output condition. The output condition here is a condition for selecting partial output definition information. The output condition is a condition related to one or more pieces of information among the input information or parameters.

[0069] The reception unit 12 receives various instructions and information. The various instructions and information are, for example, user input, which will be described later. The user input includes one or more pieces of input information.

[0070] The input reception unit 121 receives a user input. The input reception unit 121 usually receives a user input from the terminal device 2. That is, here, reception usually means reception of information transmitted via a wired or wireless communication line, but may also be a concept including reception of information input from an input device such as a keyboard, a mouse, or a touch panel, reception of information read from a recording medium such as an optical disk, a magnetic disk, or a semiconductor memory, etc.

[0071] It is preferable that the user input includes two or more pieces of input information. It is preferable that each of the two or more pieces of input information corresponds to one or two or more types of information.

[0072] The processing unit 13 performs various types of processing. The various types of processing are, for example, processing performed by a prompt creating unit 131, an estimated information acquiring unit 132, a validity determining unit 133, and an output information acquiring unit .

[0073] The prompt creation unit 131 creates a prompt to be given to the generation AI using one or more pieces of input information accepted by the input acceptance unit 121 and prompt definition information. The prompt to be given to the generation AI is a prompt to be passed to the generation AI device 3. The prompt creation unit 131 passes the prompt to the generation AI, for example, using the API of the generation AI. The prompt may be called an inquiry, a query, or the like. The prompt is usually a character string.

[0074] It is preferable that the prompt creation unit 131 determines the selection conditions that the input information or parameters satisfy, obtains partial prompt definition information that pairs with the selection conditions from the definition management unit 111, obtains one or more pieces of partial prompt information using the partial prompt definition information, and creates a prompt that includes the one or more pieces of partial prompt information.

[0075] It is preferable that the prompt creation unit 131 creates a prompt to be given to the generation AI for each of two or more types of category information included in the user input, using input information corresponding to the category information and prompt definition information corresponding to the category information.

[0076] The parameter acquiring means 1311 acquires one or more parameters using the input information accepted by the input accepting unit 121 and the parameter definition information of the definition managing unit 111. More specifically, the parameter acquiring means 1311 acquires, for example, input information identified by an input identifier included in the parameter definition information, and performs a process identified by a process identifier using the input information to acquire one or more parameters.

[0077] The parameter acquiring means 1311 acquires, for example, for each of one or more types of classification information, one or more parameters using one or more pieces of input information corresponding to the type information and parameter definition information corresponding to the type information.

[0078] The prompt creation means 1312 creates a prompt using the prompt definition information and one or more parameters. The prompt creation means 1312 creates a prompt by replacing one or more variables in the prompt definition information with input information or parameters corresponding to each variable. Such parameters are information acquired by the parameter acquisition means 1311.

[0079] The estimated information acquisition unit 132 provides the generation AI with the prompt acquired by the prompt creation unit 131, and acquires one or more pieces of information acquired by the generation AI. Each of the one or more pieces of information is estimated information acquired by the generation AI.

[0080] It is preferable that the estimated information acquisition unit 132 gives a prompt to the generation AI for each of two or more types of information, and acquires one or more pieces of estimated information.

[0081] The validity determination unit 133 determines whether the estimated information acquired by the estimated information acquisition unit 132 satisfies a validity condition. The validity condition is, for example, a condition for identifying appropriate estimated information, a condition for identifying inappropriate estimated information, a condition for identifying a range of appropriate estimated information, and a condition for identifying a range of inappropriate estimated information. The validity condition is, for example, "the estimated information does not include specific information" and "the estimated information is any one of one or more pieces of information determined in advance." The specific information is, for example, "unknown."

[0082] The output information acquisition unit 134 determines an output condition satisfied by one or more pieces of estimated information, and acquires output information corresponding to the output condition from the output management unit 112, for example.

[0083] The output information acquisition unit 134 determines output conditions that are satisfied by, for example, one or more of the input information and parameters and one or more pieces of estimated information acquired from the generation AI, and acquires output information corresponding to the output conditions from the output management unit 112.

[0084] The output unit 14 outputs various information. The various information is, for example, output information. Here, output usually means transmission to the terminal device 2, but may also be a concept including display on a display, projection using a projector, printing on a printer, sound output, storage on a recording medium, delivery of processing results to other processing devices, other programs, etc.

[0085] The information output unit 141 outputs the output information acquired by the output information acquisition unit 134. The information output unit 141 outputs output information based on, for example, one or more pieces of estimated information. The output information may be one or more pieces of estimated information. When the validity condition is not satisfied, the information output unit 141 preferably outputs error information indicating that the validity condition is not satisfied.

[0086] Various types of information are stored in the terminal storage unit 21 constituting the terminal device 2. The various types of information are, for example, a user identifier. Note that the user identifier in the terminal storage unit 21 may be information input by a user and temporarily stored.

[0087] The terminal reception unit 22 receives various information, instructions, etc. The various information, instructions, etc. are, for example, user input, one or more pieces of input information,

[0088] The means for inputting various information and instructions may be anything, such as a touch panel, a keyboard, a mouse, a menu screen, a camera, a microphone, etc.

[0089] The terminal processing unit 23 performs various types of processing. For example, the various types of processing are processing for converting received information, instructions, etc. into information or instructions with a structure to be transmitted. For example, the various types of processing are processing for converting received information into information with a structure to be output.

[0090] The terminal transmission unit 24 transmits various information, instructions, etc. to the information processing device 1. The various information, instructions, etc. are, for example, a user input, one or more pieces of input information,

[0091] The terminal receiving unit 25 receives various types of information, such as screen information and output information.

[0092] The terminal output unit 26 outputs various types of information, such as a screen and output information.

[0093] The storage unit 11, definition management unit 111, output management unit 112, condition management unit 113, and terminal storage unit 21 are preferably non-volatile recording media, but may also be realized using volatile recording media.

[0094] There is no restriction on the process by which information is stored in the storage unit 11, etc. For example, information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information inputted via an input device may be stored in the storage unit 11, etc.

[0095] It is preferable that the reception unit 12 and the input reception unit 121 are realized by wireless or wired communication means, but they may also be realized by a means for receiving broadcasts, a device driver for an input means such as a touch panel or keyboard, or control software for a menu screen.

[0096] The processing unit 13, the prompt creation unit 131, the estimated information acquisition unit 132, the validity determination unit 133, the output information acquisition unit 134, the parameter acquisition means 1311, the prompt creation means 1312, and the terminal processing unit 23 can usually be realized by a processor, a memory, etc. The processing procedure of the processing unit 13, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0097] The output unit 14 and the information output unit 141 are usually realized by wireless or wired communication means, but may be realized by broadcasting means. Also, the output unit 14 etc. may be realized by driver software for an output device such as a display or speaker, or by driver software for an output device and an output device etc.

[0098] The terminal reception unit 22 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0099] The terminal transmitting unit 24 is usually realized by a wireless or wired communication means, but may also be realized by a broadcasting means.

[0100] The terminal receiving unit 25 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast.

[0101] The terminal output unit 26 may be considered to include, or may not include, an output device such as a display, a speaker, etc. The terminal output unit 26 may be realized by driver software for an output device, or a combination of driver software for an output device and an output device, etc.

[0102] Next, an example of the operation of the information processing device 1 will be described with reference to the flowchart of FIG.

[0103] (Step S401) The input receiving unit 121 judges whether or not a user input has been received. If a user input has been received, the process proceeds to step S402, and if not, the process returns to step S401. Here, the input receiving unit 121 judges, for example, whether or not input information has been received from the terminal device 2. Here, the user input is associated with one or more types of information.

[0104] (Step S402) Prompt generator 131 generates a prompt using one or more pieces of input information contained in the user input accepted in step S401. An example of such a prompt generation process will be described with reference to the flowchart of FIG.

[0105] (Step S403) The estimated information acquisition unit 132 acquires one or more pieces of estimated information using each of the one or more prompts acquired in step S402. An example of such estimated information acquisition processing will be described with reference to the flowchart of FIG.

[0106] (Step S404) The validity determination unit 133 determines whether or not the one or more pieces of estimated information acquired in step S403 are valid. An example of such validity determination processing will be described with reference to the flowchart of FIG.

[0107] (Step S405) If the result of the determination in step S404 is "valid", the process proceeds to step S406, and if not, the process proceeds to step S408.

[0108] (Step S406) The output information acquisition unit 134 acquires the output information. An example of such output information acquisition processing will be described with reference to the flowchart of FIG.

[0109] (Step S407) The information output unit 141 outputs the output information acquired in step S406. Return to step S401. Note that the information output unit 141 transmits the output information to the terminal device 2, for example.

[0110] (Step S408) The output information acquiring unit 134 acquires error information indicating that the estimation information is invalid.

[0111] (Step S409) The information output unit 141 outputs the error information acquired in step S408. Return to step S401. The information output unit 141 transmits the error information to the terminal device 2, for example.

[0112] In the flowchart of FIG. 4, the process ends when the power is turned off or an interrupt is issued to end the process.

[0113] Next, an example of the prompt creation process in step S402 will be described with reference to the flowchart in FIG.

[0114] (Step S501) The parameter acquiring means 1311 acquires one or more types of information corresponding to a received user input.

[0115] (Step S502) The parameter acquiring means 1311 assigns 1 to a counter i.

[0116] (Step S503) The parameter acquisition means 1311 judges whether or not the i-th type information exists among the one or more types of information acquired in step S501. If the i-th type information exists, the process proceeds to step S504, and if not, the process returns to the upper process.

[0117] (Step S504) The parameter acquiring unit 1311 acquires parameter definition information corresponding to the i-th type information from the definition managing unit 111.

[0118] (Step S505) The parameter acquiring means 1311 assigns 1 to a counter j.

[0119] (Step S506) The parameter acquiring means 1311 judges whether or not the j-th variable exists in the parameter definition information acquired in step S504. If the j-th variable exists, the process proceeds to step S507, and if not, the process proceeds to step S509.

[0120] (Step S507) The parameter acquiring means 1311 acquires a parameter corresponding to the j-th variable. An example of such a parameter acquiring process will be described with reference to the flowchart in Fig. 6. The acquired parameter is temporarily stored in association with the i-th type information.

[0121] (Step S508) The parameter acquiring means 1311 increments the counter j by 1. The process returns to step S506.

[0122] (Step S509) The prompt creating unit 1312 acquires prompt definition information corresponding to the i-th type information from the definition managing unit 111.

[0123] (Step S510) The prompt generating means 1312 assigns 1 to a counter k.

[0124] (Step S511) The prompt creation unit 1312 determines whether or not the kth partial prompt definition information exists in the prompt definition information acquired in step S509. If the kth partial prompt definition information exists, the process proceeds to step S512; if not, the process proceeds to step S514.

[0125] (Step S512) The prompt generating unit 1312 acquires the k-th partial prompt using the k-th partial prompt definition information. An example of such partial prompt acquisition processing will be described with reference to the flowchart of FIG.

[0126] (Step S513) The prompt generating means 1312 increments the counter k by 1. The process returns to step S511.

[0127] (Step S514) The prompt generating unit 1312 composes an ith prompt having one or more acquired partial prompts. The ith prompt corresponds to the ith type information.

[0128] (Step S515) The parameter acquiring means 1311 increments the counter i by 1. The process returns to step S503.

[0129] Next, an example of the parameter acquisition process in step S507 will be described with reference to the flowchart in FIG.

[0130] (Step S601) The parameter acquiring means 1311 acquires the variable name of the j-th variable in step S506.

[0131] (Step S602) The parameter acquiring means 1311 judges whether the parameter corresponding to the variable name acquired in step S601 is input information or not. If it is input information, the process proceeds to step S603, and if it is not input information, the process proceeds to step S604.

[0132] (Step S603) The parameter acquiring means 1311 acquires input information corresponding to the variable name acquired in step S601 from the input information contained in the user input, and returns to the upper level process.

[0133] (Step S604) The parameter acquiring means 1311 acquires a processing identifier corresponding to the variable name acquired in step S601.

[0134] (Step S605) The parameter acquisition means 1311 acquires one or more pieces of information among one or more pieces of input information or one or more parameters used in the process identified by the process identifier. Note that each of the one or more parameters here is a parameter that has already been acquired.

[0135] (Step S606) The parameter acquiring means 1311 executes a process using the one or more pieces of information acquired in step S605, acquires a parameter, associates the parameter with a process identifier, and temporarily stores the parameter, and then returns to the upper level process.

[0136] Next, an example of the partial prompt acquisition process in step S512 will be described with reference to the flowchart in FIG.

[0137] (Step S701) The prompt generating unit 1312 judges whether the k-th partial prompt definition information corresponds to the selection conditions. If it corresponds to the selection conditions, the process proceeds to step S702. If it does not correspond to the selection conditions, the process proceeds to step S704.

[0138] (Step S702) The prompt generating unit 1312 acquires one or more pieces of information used to determine a selection condition corresponding to the k-th partial prompt definition information. Each of the one or more pieces of information is input information or a parameter.

[0139] (Step S703) The prompt creation means 1312 uses one or more pieces of information acquired in step S702 to determine whether or not the selection condition is satisfied. If the selection condition is satisfied, the process proceeds to step S704; if not, the process returns to the upper level process. Note that if the selection condition is not satisfied, partial prompt information corresponding to the partial prompt definition information paired with the selection condition is not acquired.

[0140] (Step S704) The prompt generating unit 1312 obtains the k-th partial prompt definition information of step S511.

[0141] (Step S705) The prompt creation unit 1312 acquires variables from the partial prompt definition information acquired in step S704. Note that here, there may be cases where the partial prompt definition information does not have variables and the variables cannot be acquired.

[0142] (Step S706) The prompt generating means 1312 assigns 1 to a counter i.

[0143] (Step S707) The prompt generating means 1312 determines whether or not the i-th variable exists among the variables acquired in step S705. If the i-th variable exists, the process proceeds to step S708; if not, the process returns to the upper level process.

[0144] (Step S708) The prompt generating means 1312 acquires a parameter corresponding to the i-th variable. An example of such parameter acquisition processing has been described with reference to the flowchart of FIG.

[0145] (Step S709) The prompt generating means 1312 assigns the parameter acquired in step S708 to the i-th variable.

[0146] (Step S710) The prompt generating means 1312 increments the counter i by 1. The process returns to step S707.

[0147] Next, an example of the estimated information acquisition process in step S403 will be described with reference to the flowchart in FIG.

[0148] (Step S801) The estimated information acquiring unit 132 assigns 1 to a counter i.

[0149] (Step S802) Estimated information acquisition unit 132 determines whether or not the i-th prompt exists among the generated prompts. If the i-th prompt exists, the process proceeds to step S803, and if not, the process returns to the upper process.

[0150] (Step S803) The estimated information acquisition unit 132 acquires the i-th prompt and passes the i-th prompt to the generation AI. Note that passing a prompt to the generation AI usually means sending the prompt to the generation AI device 3. The estimated information acquisition unit 132 also passes the i-th prompt to the generation AI, for example, by using an API provided by the generation AI. The estimated information acquisition unit 132 may also pass the i-th prompt to one of the two or more generation AIs. Such one generation AI is, for example, a generation AI that corresponds to type information.

[0151] (Step S804) The estimated information acquisition unit 132 determines whether or not an estimated result has been received from the generation AI device 3. If an estimated result has been received, the process proceeds to step S805; if an estimated result has not been received, the process returns to step S804. The estimated result is the response of the generation AI to the prompt passed to the generation AI. The response of the generation AI is information returned from the generation AI.

[0152] (Step S805) The estimated information acquisition unit 132 acquires one or more pieces of estimated information from the estimation result acquired in step S804. The estimated information may be the estimation result itself or may be a part of the estimation result. The estimated information is, for example, information that is paired with a specific label (tag) in the estimation result.

[0153] (Step S806) The estimated information acquiring unit 132 increments the counter i by 1. The process returns to step S802.

[0154] Next, an example of the validity determination process in step S404 will be described with reference to the flowchart in FIG.

[0155] (Step S901) The validity determination unit 133 assigns 1 to a counter i.

[0156] (Step S902) The validity determination unit 133 determines whether or not the i-th validity condition exists in the condition management unit 113. If the i-th validity condition exists, the process proceeds to step S903, and if not, the process proceeds to step S907.

[0157] (Step S903) The validity determination unit 133 acquires one or more pieces of information to be used for determining the i-th validity condition. Note that the one or more pieces of information are, for example, one or more pieces of estimated information, one or more pieces of input information, and one or more parameters.

[0158] (Step S904) The validity determination unit 133 determines whether the i-th validity condition is satisfied by using the one or more pieces of information acquired in step S903. If the validity condition is satisfied, the process proceeds to step S905, and if the validity condition is not satisfied, the process proceeds to step S906.

[0159] (Step S905) The validity determining unit 133 increments the counter i by 1. The process returns to step S902.

[0160] (Step S906) The validity determination unit 133 assigns the "information indicating that the result is invalid" to the variable "result." Then, the process returns to the upper level process.

[0161] (Step S907) The validity determination unit 133 assigns the "information indicating validity" to the variable "result." Then, the process returns to the upper level process.

[0162] Next, an example of the output information acquisition process in step S406 will be described with reference to the flowchart in FIG.

[0163] (Step S1001) The output information acquiring unit 134 assigns 1 to a counter i.

[0164] (Step S1002) The output information acquisition unit 134 judges whether or not the i-th partial output definition information exists in the definition management unit 111. If the i-th partial output definition information exists, the process proceeds to step S1003, and if not, the process returns to the upper level process.

[0165] (Step S1003) The output information acquisition unit 134 judges whether the i-th partial output definition information corresponds to the output condition. If it corresponds to the output condition, the process proceeds to step S1004, and if it does not correspond to the output condition, the process proceeds to step S1006.

[0166] (Step S1004) The output information acquisition unit 134 acquires one or more pieces of information used to determine the output condition corresponding to the i-th partial output definition information. The one or more pieces of information here are, for example, one or more pieces of estimated information, one or more pieces of input information, and one or more pieces of parameters.

[0167] (Step S1005) The output information acquisition unit 134 uses the one or more pieces of information acquired in step S1004 to determine whether or not the output condition corresponding to the i-th partial output definition information is satisfied. If the output condition is satisfied, the process proceeds to step S1006, and if not, the process proceeds to step S1013.

[0168] (Step S1006 ) The output information acquiring unit 134 acquires the i-th partial output definition information from the definition managing unit 111 .

[0169] (Step S1007) The output information acquisition unit 134 acquires a variable in the i-th partial output definition information. Note that here, there may be a case where the variable does not exist in the i-th partial output definition information, in which case the variable is not acquired.

[0170] (Step S1008) The output information acquiring unit 134 assigns 1 to a counter j.

[0171] (Step S1009) The output information acquiring unit 134 judges whether or not the j-th variable exists. If the j-th variable exists, the process proceeds to step S1010, and if not, the process proceeds to step S1013.

[0172] (Step S1010) The output information acquiring unit 134 acquires a parameter corresponding to the j-th variable. Note that an example of the parameter acquiring process has been described with reference to the flowchart of FIG.

[0173] (Step S1011) The output information acquiring unit 134 assigns the parameter acquired in step S1010 to the j-th variable in the i-th partial output definition information.

[0174] (Step S1012) The output information acquiring unit 134 increments the counter j by 1. The process returns to step S1009.

[0175] (Step S1013) The output information acquiring unit 134 increments the counter i by 1. The process returns to step S1002.

[0176] Next, an example of the operation of the terminal device 2 will be described. First, the terminal reception unit 22 receives a user input having one or more pieces of input information. The device processing unit 23 converts the received user input into information structured for transmission. The terminal transmission unit 24 transmits the user input to the information processing device 1. In response to the transmission of the user input, the terminal reception unit 25 receives output information from the information processing device 1. The device processing unit 23 converts the received output information into information structured for output. The terminal output unit 26 outputs the output information.

[0177] A specific example of the operation of the information system A in this embodiment will be described below.

[0178] The definition management unit 111 of the information processing device 1 stores a prompt definition management table shown in FIG. 11. The prompt definition management table is a table for managing prompt definition information. The prompt definition management table manages one or more records having "ID", "type information", "selection condition", and "partial prompt definition information". The prompt definition management table manages partial prompt definition information for each prompt definition information. It indicates that no selection condition is associated with partial prompt definition information corresponding to the selection condition "NULL". The selection condition for "ID=2" indicates that "disease evaluation" is not "high blood pressure". In other words, the operator "!=" indicates a mismatch. In addition, in the table, {@···} indicates a variable. {@Dietary status} indicates a variable to which the input information "Dietary status" or the parameter "Dietary status" is assigned. {@Medication status} indicates a variable to which the input information "Medication status" or the parameter "Medication status" is assigned.

[0179] The definition management unit 111 stores a parameter definition management table shown in FIG. 12. The parameter definition management table is a table that manages parameter definition information. The parameter definition management table manages one or more records having "ID," "type information," and "parameter definition information." The parameter definition information of "ID=1" indicates that input information or parameters to be substituted for the variables {@dietary status} and {@disease evaluation} are to be obtained. The parameter definition information of "ID=2" indicates that input information or parameters to be substituted for the variable {@medication status} are to be obtained.

[0180] Definition management unit 111 stores processing management tables shown in Fig. 13 and Fig. 14. The processing management table is a table for managing parameters when processing identified by a processing identifier is performed. The processing management table is information used when acquiring parameters. The processing management table is a table for managing one or more parameters associated with two or more information conditions related to information that is input information or a parameter.

[0181] The process management table in Figure 13 corresponds to the process identifier "disease assessment." The process management table in Figure 13 is information used when acquiring parameters to be substituted for the variable {@disease assessment}. The process management table in Figure 13 manages one or more records having "ID," "information condition," and "parameter." The "information condition" has "systolic blood pressure" and "diastolic blood pressure." The "systolic blood pressure" and "diastolic blood pressure" are input information here. The "parameter" is "disease assessment" here.

[0182] The module identified by the process identifier "Disease Evaluation" is stored in the storage unit 11. The module "Disease Evaluation" refers to the process management table of FIG. 13 and acquires the parameter "Disease Evaluation". Based on the record of "ID=1" in FIG. 13, if the input information "systolic blood pressure" is 140 or more, the module "Disease Evaluation" acquires the parameter "high blood pressure" of "Disease Evaluation", based on the record of "ID=2" if the input information "diastolic blood pressure" is 90 or more, the module "Disease Evaluation" acquires the parameter "high blood pressure" of "Disease Evaluation", based on the record of "ID=3" if the input information "systolic blood pressure" is 130 or more and 139 or less, based on the record of "ID=4" if the input information "diastolic blood pressure" is 85 or more and 89 or less, the module "Disease Evaluation" acquires the parameter "high normal blood pressure". Based on the record "ID=5", if the input information "systolic blood pressure" is less than 130, the parameter "normal blood pressure" of "Disease evaluation" is obtained; based on the record "ID=6", if the input information "diastolic blood pressure" is less than 85, the parameter "normal blood pressure" of "Disease evaluation" is obtained; based on the record "ID=7", if the input information "systolic blood pressure" is less than 120, the parameter "optimal blood pressure" of "Disease evaluation" is obtained; and based on the record "ID=8", if the input information "diastolic blood pressure" is less than 80, the parameter "optimal blood pressure" of "Disease evaluation" is obtained.

[0183] The process management table in FIG. 14 corresponds to the process identifier "Need for lifestyle guidance." The process management table in FIG. 14 is the parameters substituted for the variable {@Need for lifestyle guidance} and the information used when acquiring the variable {@Unknown}. The process management table in FIG. 14 manages one or more records having "ID," "information condition," and "parameter." The "information condition" includes "disease evaluation," "hospital visit: medication status," and "salt restriction." The "parameter" includes "Need for lifestyle guidance" and "Unknown."

[0184] The module identified by the process identifier "Need for lifestyle guidance" is stored in the storage unit 11. The module "Need for lifestyle guidance" refers to the process management table in FIG. 14 and acquires the parameter "Need for lifestyle guidance" and the parameter "Unknown". For example, based on the record "ID=1" in FIG. 14, when the parameter "Disease assessment" is "High blood pressure", the module "Need for lifestyle guidance" acquires the parameter "Need for lifestyle guidance" and the parameter "Unknown" "x", and based on the record "ID=2", when the parameter "Disease assessment" is "High normal blood pressure" and "Hospital visit="Yes" OR Medication status="Yes" and "Salt restriction="Yes", the module "Need for lifestyle guidance" acquires the parameter "Not required" and the parameter "Unknown" "x".

[0185] The definition management unit 111 stores an output definition management table shown in Fig. 15. The output definition management table is a table for managing output definition information. The output definition management table manages one or more records having "ID", "output condition", and "partial output definition information". The "output condition" is a condition based on one or more pieces of information out of one or more pieces of input information, one or more parameters, and one or more pieces of estimated information, and here includes "unknown", "disease evaluation", "need for lifestyle guidance", and "salt restriction".

[0186] In such a situation, it is assumed that the user's terminal device 2 receives screen information for inputting input information related to the user's own blood pressure from the information processing device 1, and outputs the screen of Fig. 16. In the screen of Fig. 16, 1601 is a field for inputting the input information "systolic blood pressure", 1602 is a field for inputting the input information "diastolic blood pressure", 1603 is a field for inputting the input information "hospital visit", 1604 is a field for inputting the input information "dietary situation", and 1605 is a field for inputting the input information "medication situation". The screen of Fig. 16 is an input screen for providing the user with advice related to blood pressure.

[0187] Then, it is assumed that the user U inputs input information into each field on the screen of FIG.

[0188] Then, the terminal device 2 accepts the user input "<Systolic blood pressure> 140 <Diastolic blood pressure> 120 <Visiting the hospital> Yes <Dietary status> I am not particularly careful about my diet. <Medication status> I am taking Cetabil." and transmits the user input to the information processing device 1. It is assumed that the user input is associated with the type information "Extraction of salt restriction status" and the type information "Extraction of medication status." Such type information may be associated with each input information in each field.

[0189] Next, the input receiving unit 121 of the information processing device 1 receives the user input from the terminal device 2.

[0190] Next, the prompt generating unit 131 acquires the type information "extraction of salt restriction status" and "extraction of medication status" associated with the user input.

[0191] Next, the parameter acquisition means 1311 acquires the parameter definition information "Dietary status: @Dietary status & Disease evaluation: @Disease evaluation" paired with the type information "Extraction of salt restriction status" from the parameter definition management table (FIG. 12). Next, the parameter acquisition means 1311 acquires the input information "I don't pay particular attention to my diet" corresponding to the parameter identifier "@Dietary status" and assigns the input information to the variable "Dietary status". In addition, the parameter acquisition means 1311 detects that there is no input information corresponding to the parameter identifier "@Disease evaluation", and acquires the module "Disease evaluation" identified by the process identifier "Disease evaluation" corresponding to the parameter identifier "@Disease evaluation". Then, the parameter acquisition means 1311 executes the module "Disease evaluation". That is, the parameter acquisition means 1311 refers to the process management table (FIG. 13) associated with the process identifier "Disease evaluation", and acquires the parameter "Disease evaluation=high blood pressure" corresponding to the input information "<Systolic blood pressure> 140 <Diastolic blood pressure> 120". Such "disease evaluation=hypertension" is the information of "ID=2" in the intermediate value management table of Fig. 17. The intermediate value management table is a buffer that stores intermediate values, which are information acquired in the process of acquiring output information.

[0192] Next, the prompt creation means 1312 acquires prompt definition information (partial prompt definition information for "ID=1" to "ID=6") that is paired with the type information "extract salt restriction status" from the prompt definition table (FIG. 11). The prompt creation means 1312 then determines that the selection condition "disease evaluation="high blood pressure"" is met, but the selection condition "disease evaluation!="high blood pressure"" is not met. The prompt creation means 1312 then acquires the partial prompt definition information for "ID=1,3,4,5,6" in FIG. 11. The prompt creation means 1312 also assigns the received input information "I am not particularly careful about my diet" to the variable {@dietary situation} of the partial prompt definition information for "ID=6". The prompt creation means 1312 then acquires the salt restriction status extraction prompt for "ID=3" in FIG. 17, "If it is clearly determined from the following text that salt restriction is being implemented, then only 'Yes'...Currently taking medication."

[0193] Furthermore, the parameter acquisition means 1311 acquires parameter definition information "medication status: @medication status" that pairs with the type information "medication status extraction" from the parameter definition management table (FIG. 12). Next, the parameter acquisition means 1311 acquires input information "Taking cetabil" that corresponds to the parameter identifier "@medication status" and assigns the input information to the variable "medication status".

[0194] The prompt creation means 1312 also obtains prompt definition information (partial prompt definition information from "ID=7" to "ID=11") that pairs with the type information "medication status extraction" from the prompt definition table (FIG. 11). The prompt creation means 1312 then assigns the information of the variable "medication status" "Currently taking cetabiril" to the variable {@medication status} of the partial prompt definition information for "ID=11". The prompt creation means 1312 then obtains the medication status extraction prompt for "ID=1" in FIG. 17, "If the following text includes hypertension treatment, only answer 'Yes'...Currently taking cetabiril."

[0195] Next, the estimated information acquisition unit 132 acquires one or more pieces of estimated information by the process described using the flowchart in Fig. 8. That is, the estimated information acquisition unit 132 gives the salt restriction status extraction prompt of "ID=3" in Fig. 17 to ChatGPT, which is the generation AI, and acquires the estimated information "No" corresponding to the salt restriction status from ChatGPT. Such information is the intermediate value of "ID=5" in Fig. 17.

[0196] In addition, the estimated information acquisition unit 132 provides the medication status extraction prompt of "ID=1" in Fig. 17 to the generation AI ChatGPT, and acquires the estimated information "Yes" corresponding to the medication status from ChatGPT. This information is the intermediate value of "ID=4" in Fig. 17.

[0197] Next, it is assumed that the validity determining unit 133 determines that the inferred information "No" for "ID=5" in FIG. 17 and the inferred information "Yes" for "ID=4" in FIG. 17 satisfy all the validity conditions.

[0198] 14, and determines that the information conditions satisfied by the acquired disease evaluation "high blood pressure", input information "hospital visits="Yes", estimated information "medication status="Yes", and estimated information "salt restriction="No" are the information conditions for "ID=1", and acquires the parameters "need for lifestyle guidance="need for guidance"" and "unknown="x". The parameter "need for lifestyle guidance="need for guidance"" is the intermediate value of "ID=6" in FIG. 17, and the parameter "unknown="x"" is the intermediate value of "ID=7" in FIG. 17.

[0199] Next, the output information acquisition unit 134 acquires output information by the process described with reference to the flowchart of Fig. 10. That is, the output information acquisition unit 134 refers to the output definition management table of Fig. 15 and the intermediate values ​​"unknown="x"", "disease evaluation="high blood pressure", "need for lifestyle guidance="need guidance", and "salt restriction="no"" of Fig. 15, and determines that the output condition matching "unknown="x", "disease evaluation="high blood pressure", "need for lifestyle guidance="need guidance", and "salt restriction="no"" is "ID=1,2,4,5,6,7,8,9,11", and acquires partial output definition information paired with each output condition from Fig. 15. Then, the output information acquisition unit 134 assigns input information or parameters "140", "120", and "high blood pressure" to each variable of the variables {@systolic blood pressure} and {@diastolic blood pressure} of "ID=6" including variables, and {@disease evaluation} of "ID=7", completing the partial output information. Furthermore, the output information acquisition unit 134 configures output information by sequentially concatenating the nine pieces of partial output information. Next, the information output unit 141 transmits the output information acquired by the output information acquisition unit 134 to the terminal device 2.

[0200] Then, the terminal device 2 receives and outputs the output information. An example of such output is shown in FIG.

[0201] As described above, according to this embodiment, appropriate information can be output using the generation AI.

[0202] Furthermore, according to this embodiment, by giving an appropriate prompt to the generation AI, it is possible to output appropriate information.

[0203] The processing in this embodiment may be realized by software. This software may be distributed by software download or the like. This software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software for realizing the information processing device 1 in this embodiment is a program as follows. That is, this program is a program for making a computer that can access a definition management unit in which prompt definition information for creating a prompt to be given to the generation AI is stored function as an input receiving unit that receives user input including one or more pieces of input information, a prompt creating unit that uses the one or more pieces of input information and the prompt definition information to create a prompt to be given to the generation AI, an estimated information acquiring unit that provides the prompt to the generation AI and acquires one or more pieces of estimated information acquired by the generation AI, and an information output unit that outputs output information based on the one or more pieces of estimated information.

[0204] (Embodiment 2) In this embodiment, an information processing device is described that accepts user input including non-text information, converts the non-text information into text, acquires a prompt to be given to a generation AI using the acquired text and prompt definition information, provides the prompt to the generation AI, acquires estimated information from the generation AI, and outputs output information based on the estimated information. This embodiment differs from embodiment 1 in that it is capable of accepting user input including non-text information.

[0205] Non-text information is information that is not text data. Examples of non-text information include image data, video data (hereinafter referred to as "video" where appropriate), and audio data (hereinafter referred to as "audio" where appropriate). Image data is usually a still image. Image data is, for example, a gif, jpeg, or png data type, but may also be PDF or the like, and the data type does not matter. Video is, for example, an mp4, but the data type does not matter. It is preferable for video to include audio data. Audio may be, for example, WAV or AIFF, and the data type does not matter. Non-text information may include tables, graphs, and images in a text file. Images are usually images (still images), but may also be videos. Tables, graphs, and images in a text file are, for example, tables, graphs, and images in a WORD file.

[0206] The conceptual diagram of an information system B in this embodiment is the same as FIG. 1 except for the reference numerals.

[0207] Fig. 19 is a block diagram of an information system B in this embodiment. Fig. 20 is a block diagram of an information processing device 4.

[0208] The information system B includes an information processing device 4, one or more terminal devices 2, and one or more generation AI devices 3.

[0209] The information processing device 4 is a device that accepts input from a user and outputs output information using the input and a generating AI. The information processing device 4 is, for example, a cloud server or an ASP server, but the type does not matter. If the information processing device 4 is a terminal, it may be considered that the terminal device 2 is not necessary in the information system B, or that the information processing device 4 also functions as the terminal device 2. If the information processing device 4 is a terminal, the information processing device 4 is, for example, a so-called personal computer, smartphone, or tablet terminal, but the type does not matter.

[0210] The information processing device 4 includes a storage unit 41, a reception unit 42, a processing unit 43, and an output unit 14. The storage unit 41 includes a definition management unit 411, an output management unit 112, and a condition management unit 113. The reception unit 42 includes an input reception unit 421. The processing unit 43 includes a text conversion unit 431, a prompt creation unit 432, an estimated information acquisition unit 132, a validity determination unit 133, and an output information acquisition unit 134. The text conversion unit 431 includes an attribute value acquisition means 4310, a non-text image detection means 4311, a character recognition means 4312, an explanatory text acquisition means 4313, a video text acquisition means 4314, a voice recognition means 4315, and a text composition means 4316. The prompt creation unit 432 includes a parameter acquisition means 1311, and a prompt creation means 4322. The output unit 14 includes an information output unit 141.

[0211] Prompt definition information for creating a prompt to be given to the generating AI is stored in the definition management unit 411 of the information processing device 4. The definition management unit 411 may be the same as the definition management unit 111.

[0212] The definition management section 411 stores parameter definition information, which is information for acquiring parameters used when acquiring a prompt using input information.

[0213] The definition management unit 411 stores prompt definition information associated with two or more types of information that specify the type of input information.

[0214] The prompt definition information here may include location-specific information that specifies a location for locating partial information based on a non-text image portion. A non-text image portion is a portion that is not text. A non-text image portion is, for example, a table, a graph, or an image. An image is, for example, a photograph or an illustration. An image is not information that is an image of a sentence.

[0215] The prompt definition information here may be the same as the prompt definition information described in the first embodiment, or may have the characteristics and functions of the prompt definition information described in the first embodiment.

[0216] The reception unit 42 receives various instructions and information. The various instructions and information are, for example, user input. The user input includes one or more pieces of input information.

[0217] The input receiving unit 421 receives a user input including input information. The user input preferably includes two or more pieces of input information. Each of the two or more pieces of input information preferably corresponds to one or more types of information. The input information here may also include one or more pieces of non-text information.

[0218] The input reception unit 421 normally receives a user input from the terminal device 2. That is, here, reception normally means reception of information transmitted via a wired or wireless communication line, but may also be a concept including reception of information input from an input device such as a keyboard, mouse, or touch panel, reception of information read from a recording medium such as an optical disk, a magnetic disk, or a semiconductor memory, etc.

[0219] The processing unit 43 performs various types of processing, such as processing performed by a text conversion unit 431, a prompt creation unit 432, the estimated information acquisition unit 132, the validity determination unit 133, and the output information acquisition unit .

[0220] The text conversion unit 431 acquires text based on the non-text information. The text conversion unit 431 usually acquires text by performing different processing depending on the data type of the non-text information. The text conversion unit 431 may acquire text by performing different processing depending on one or more data attribute values ​​other than the data type of the non-text information. The data attribute value is, for example, the size of the non-text information.

[0221] The attribute value acquiring means 4310 acquires one or more data attribute values ​​of the non-text information. The data attribute values ​​are, for example, the data type, the size, the number or the ratio of characters that can be acquired by character recognition processing of the image data, and the presence or absence of audio in the video.

[0222] The non-text image detection means 4311 detects non-text image parts in the image data received by the input receiving unit 421. A non-text image part is a part that is not composed of text only. A non-text image part is, for example, a table, a graph, or an image. Note that text may exist in a table.

[0223] The non-text image detection means 4311 detects, for example, a non-text image portion in a text file accepted by the input acceptance unit 421. The text file is, for example, a WORD file. The character recognition means 4312 performs character recognition processing on the image data accepted by the input acceptance unit 421 to acquire text.

[0224] The explanation acquisition means 4313 provides the image data received by the input reception unit 421 to the generation AI, and acquires text explaining the image data from the generation AI. The prompt provided to the generation AI is, for example, "Please explain the following image data. <image data>". Note that <image data> is a variable in which the image data is placed.

[0225] The explanation acquisition means 4313 acquires partial information, which is, for example, text explaining the non-text image part detected by the non-text image detection means 4311, from the generation AI. The explanation acquisition means 4313 passes, for example, a prompt "Please explain the following non-text image part. <non-text image part>" to the generation AI, and acquires partial information, which is, text, from the generation AI. Note that <non-text image part> is a variable in which the non-text image part (for example, a table, graph, image) is placed.

[0226] The partial information may be text obtained from the generation AI using the non-text image portion, or the non-text image portion. Partial information based on the non-text image portion may be, for example, text analyzing or summarizing a table, text resulting from analyzing a graph, or text describing an image.

[0227] The explanatory text acquisition means 4313 constructs a prompt by, for example, substituting the table detected and acquired by the non-text image detection means 4311 for the variable of the prompt definition information "Tell us the results of analyzing or summarizing the following table! ". Next, the explanatory text acquisition means 4313 passes the prompt to a generation AI, for example, and acquires text that analyzes or summarizes the table from the generation AI.

[0228] The explanatory text acquisition means 4313 constructs a prompt by, for example, substituting the graph detected and acquired by the non-text image detection means 4311 for the variable <graph> of the prompt definition information "Tell us the results of analyzing or summarizing the following graph! <graph>". Next, the explanatory text acquisition means 4313 passes the prompt to the generation AI, for example, and acquires from the generation AI the text that analyzed or summarized the graph.

[0229] The explanatory text acquisition means 4313, for example, assigns the image detected and acquired by the non-text image detection means 4311 to the variable <image> of the prompt definition information "Tell us the result of analyzing or summarizing the image below! <image>" to construct a prompt. Next, the explanatory text acquisition means 4313, for example, passes the prompt to the generation AI and acquires text that analyzes or summarizes the graph from the generation AI. The video text acquisition means 4314 acquires text, which is explanatory text regarding the video accepted by the input acceptance unit 421. The video text acquisition means 4314, for example, passes the video accepted by the input acceptance unit 421 to the generation AI and acquires text explaining the video from the generation AI. The prompt to be given to the generation AI is, for example, "Please explain the video below. <video>". Note that <video> is a variable in which image data is placed.

[0230] The video text acquisition means 4314, for example, acquires two or more representative still images from the video accepted by the input acceptance unit 421, acquires explanatory text for each of the two or more representative still images, and acquires text based on the two or more explanatory texts. For example, the video text acquisition means 4314 provides the representative still image to the generation AI for each representative still image, and acquires text explaining the representative still image from the generation AI. The prompt provided to the generation AI is, for example, "Please explain the image data below. <representative still image>". Note that <representative still image> is a variable in which the representative still image is placed. For example, the video text acquisition means 4314 performs character recognition processing on the representative still image for each representative still image, and acquires text.

[0231] The representative still image is, for example, a representative still image of each of two or more scenes in a video. The technique of dividing a video into two or more scenes is a well-known technique. The representative still image is, for example, any image in the video.

[0232] The voice recognition means 4315 recognizes the voice received by the input receiving unit 421 and acquires text. The voice recognition means 4315 recognizes, for example, the voice contained in the video received by the input receiving unit 421 and acquires text.

[0233] The text composing means 4316 acquires text based on the input information accepted by the input accepting unit 421. The text composing means 4316 composes one text using, for example, two or more pieces of text acquired from one or more pieces of non-text information accepted by the input accepting unit 421. It is preferable that the text composing means 4316 composes one text using two or more pieces of text acquired from different sources. It is preferable that the text composing means 4316 composes one text using two or more pieces of text acquired from one or more sources by different algorithms. The source is the input information. The source is, for example, text, image data, one or more still images in a video, audio in a video, and audio.

[0234] The text construction means 4316 constructs text using both a first text acquired by the character recognition means 4312 from the image data accepted by the input accepting unit 421 and a second text acquired by the explanatory text acquisition means 4313 from the image data accepted by the input accepting unit 421.

[0235] The prompt creation unit 432 creates a prompt to be given to the generation AI, for example, by using the text acquired by the text conversion unit 431 and the prompt definition information. Information including the text acquired by the text conversion unit 431, and information for constructing the prompt, may also be called input information. The prompt creation unit 432 performs the same process as the prompt creation unit 131, for example.

[0236] The prompt generator 432 determines whether or not non-text information is included in one or more pieces of input information among the user inputs accepted by the input accepting unit 121. If non-text information is not included, the prompt generator 432 performs the same process as the prompt generator 131.

[0237] The prompt creation unit 432 creates a prompt to be given to the generation AI by using, for example, one or more pieces of input information using the text created by the text creation unit 4316 and the prompt definition information. Such input information may directly include information given by the user.

[0238] Prompt generator 432 obtains partial information based on the non-text image portion detected by non-text image detector 4311, and generates a prompt using the partial information as well. The partial information is usually text.

[0239] Prompt creator 432 obtains partial information based on the non-text image portion detected by non-text image detection means 4311, and creates a prompt using the partial information. Note that the processing performed by prompt creator 432 after obtaining the text may be the same as that performed by prompt creator 131.

[0240] The prompt generator 432, for example, determines a selection condition that is satisfied by the input information or a parameter, obtains partial prompt definition information that pairs with the selection condition, and generates a prompt using the partial prompt definition information.

[0241] The prompt generator 432 generates a prompt for each of two or more types of information included in the user input, for example, by using input information corresponding to the type information and prompt definition information corresponding to the type information.

[0242] The prompt generator 4322 generates a prompt using the prompt definition information and one or more parameters.

[0243] The storage unit 41 and the definition management unit 411 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0244] There is no restriction on the process by which information is stored in the storage unit 41, etc. For example, information may be stored in the storage unit 41, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 41, etc., or information inputted via an input device may be stored in the storage unit 41, etc.

[0245] The reception unit 42 and the input reception unit 421 are typically realized by wireless or wired communication means, but may also be realized by a means for receiving broadcasts, a device driver for an input means such as a touch panel or keyboard, or control software for a menu screen.

[0246] The processing unit 43, the text conversion unit 431, the prompt creation unit 432, the attribute value acquisition means 4310, the non-text image detection means 4311, the character recognition means 4312, the explanatory text acquisition means 4313, the animated text acquisition means 4314, the voice recognition means 4315, the text composition means 4316, and the prompt creation means 4322 can usually be realized by a processor, a memory, etc. The processing procedure of the processing unit 43, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may also be realized by hardware (dedicated circuit). The processor is a CPU, an MPU, a GPU, etc., and the type does not matter.

[0247] Next, an example of the operation of the information processing device 4 will be described with reference to the flowchart of Fig. 21. In the flowchart of Fig. 21, the description of the same steps as those in the flowchart of Fig. 4 will be omitted.

[0248] (Step S2101) Prompt generator 432 determines whether or not one or more pieces of input information in the user input accepted by input accepting unit 121 include non-text information. If non-text information is included, the process proceeds to step S2102. If non-text information is not included, the process proceeds to step S402.

[0249] (Step S2102) Text conversion unit 431 performs processing for converting non-text information in the input information of the user input into text. An example of such text conversion processing will be described with reference to the flowchart of FIG.

[0250] In the flowchart of FIG. 21, the process ends when the power is turned off or an interrupt occurs to end the process.

[0251] Next, an example of the text conversion process in step S2102 will be described with reference to the flowchart in FIG.

[0252] (Step S2201) The attribute value acquiring means 4310 acquires one or more data attribute values ​​of non-text information in the input information. The one or more data attribute values ​​include a data type.

[0253] (Step S2202) Text conversion section 431 judges whether or not the data type is image data. If it is image data, the process proceeds to step S2203, and if it is not image data, the process proceeds to step S2204.

[0254] (Step S2203) Text conversion section 431 performs processing to convert image data into text. An example of such image-to-text conversion processing will be described with reference to the flowchart of FIG.

[0255] (Step S2204) Text conversion unit 431 judges whether the data type is text data (e.g., a text file). If it is text data, the process proceeds to step S2205, and if it is not text data, the process proceeds to step S2208. The text data is, for example, a WORD file. The text data here includes a non-text image portion.

[0256] (Step S2205) Non-text image detecting unit 4311 obtains one or more non-text image parts from the text data.

[0257] (Step S2206) Text generation section 431 obtains text from the non-text image portion. An example of such non-text image portion processing will be described with reference to the flowchart of FIG.

[0258] (Step S2207) Text generation unit 431 generates text for forming a prompt, using the text contained in the text data and the text acquired in step S2206, and then returns to the upper-level process.

[0259] (Step S2208) Text conversion unit 431 determines whether the data type is a moving image. If it is a moving image, the process proceeds to step S2209, and if it is not a moving image, the process proceeds to step S2210.

[0260] (Step S2209) Text conversion unit 431 acquires text based on the video. The process returns to the upper level process. An example of such video text conversion process will be described with reference to the flowchart in FIG.

[0261] (Step S2210) Text conversion unit 431 judges whether or not the data type is voice. If it is voice, the process proceeds to step S2211, and if it is not voice, the process returns to the upper level process.

[0262] (Step S2211) The voice recognition means 4315 performs voice recognition processing on the voice and obtains text.

[0263] (Step S2212) Text composing means 4316 judges whether or not the text acquired in step S2211 satisfies the summarization condition. If the summarization condition is satisfied, the process proceeds to step S2213, and if the summarization condition is not satisfied, the process returns to the upper level process. Note that the summarization condition may be, for example, that the number of characters in the text acquired in step S2211 is equal to or greater than a threshold value, or that the text acquired in step S2211 does not contain any specific terminology.

[0264] (Step S2213) The text composing means 4316 obtains text that is a summary of the text obtained in step S2211. Note that the method for summarizing the text is not important. For example, the text composing means 4316 gives a prompt "Please summarize the following text in X characters or less (X is, for example, "400"). <text>" to the generation AI, and obtains the summarized text from the generation AI. The text composing means 4316 may obtain the summarized text using, for example, a known summarization processing technique for summarizing text.

[0265] In the flowchart of FIG. 22, if the user input includes two or more pieces of non-text information, the above process is carried out for each piece of non-text information.

[0266] Next, an example of the image-to-text process in step S2203 will be described with reference to the flowchart in FIG.

[0267] (Step S2301) The character recognition means 4312 performs character recognition processing on the image data to obtain text.

[0268] (Step S2302) The caption obtaining unit 4313 judges whether or not the image data satisfies the image caption obtaining condition. If the image caption obtaining condition is satisfied, the process proceeds to step S2303, and if the image caption obtaining condition is not satisfied, the process proceeds to step S2304.

[0269] The image description acquisition condition is a condition for acquiring a sentence that explains image data. For example, the image description acquisition condition is that the number of characters in the text acquired by performing character recognition processing on the image data is less than or equal to a threshold value. For example, the image description acquisition condition is that the ratio of the area other than the characters in the image data is greater than or equal to a threshold value.

[0270] (Step S2303) The explanation acquisition means 4313 acquires an explanation of the image data from the generation AI. For example, the explanation acquisition means 4313 places image data in the variable <image data> of the prompt definition information “Explain the following image! <image data>”, configures a prompt, passes the prompt to the generation AI, and acquires the image explanation from the generation AI.

[0271] (Step S2304) Non-text image detection unit 4311 obtains a non-text image portion in the image data.

[0272] (Step S2305) Text conversion unit 431 determines whether or not a non-text image portion was acquired in step S2304. If a non-text image portion was acquired, the process proceeds to step S2306; if not, the process proceeds to step S2307. Note that the non-text image portion here is, for example, a table or graph. Also, the inclusion of a non-text image portion (for example, a table, graph, or image) in image data may be considered a condition for acquiring an image description.

[0273] (Step S2306) Text generation section 431 performs non-text image partial processing. An example of the non-text image partial processing will be described with reference to the flowchart of FIG.

[0274] (Step S2307) Text composition means 4316 uses the acquired text to compose text for constructing a prompt. The process returns to the upper level process. The acquired text is, for example, one or more of the text acquired in step S2301, the text acquired in step S2303, and the text acquired in step S2306. Text composition means 4316 combines, for example, the two or more acquired texts to compose text for constructing a prompt.

[0275] Next, an example of the non-text image partial processing in steps S2206 and S2306 will be described with reference to the flowchart in FIG.

[0276] (Step S2401) Text conversion unit 431 judges whether or not a table is present in the acquired non-text image portion. If a table is present, the process proceeds to step S2402, and if no table is present, the process proceeds to step S2403.

[0277] (Step S2402) The explanation obtaining means 4313 passes the table to the generation AI and obtains the explanation of the table from the generation AI.

[0278] (Step S2403) Text conversion unit 431 judges whether or not a graph is present in the acquired non-text image portion. If a graph is present, the process proceeds to step S2404, and if a graph is not present, the process proceeds to step S2405.

[0279] (Step S2404) The explanation obtaining means 4313 passes the graph to the generation AI and obtains an explanation of the graph from the generation AI.

[0280] (Step S2405) Text conversion unit 431 judges whether or not an image is present in the acquired non-text image portion. If an image is present, the process proceeds to step S2406, and if an image is not present, the process returns to the upper process.

[0281] (Step S2406) The explanation acquisition means 4313 passes the image to the generation AI, and acquires the explanation of the image from the generation AI. The process returns to the upper level process.

[0282] Next, an example of the video-to-text conversion process in step S2209 will be described with reference to the flowchart in FIG.

[0283] (Step S2501) Text generation unit 431 judges whether or not audio is included in the video. If audio is included, the process proceeds to step S2502, and if audio is not included, the process proceeds to step S2505.

[0284] (Step S2502) The voice recognition means 4315 performs voice recognition processing on the voice included in the video and acquires text.

[0285] (Step S2503) The video text acquisition means 4314 determines whether the text acquired in step S2502 satisfies the summarization condition. If the summarization condition is satisfied, the process proceeds to step S2504, and if the summarization condition is not satisfied, the process proceeds to step S2505. Note that the summarization condition may be, for example, that the number of characters in the text acquired in step S2211 is equal to or greater than a threshold value, or that the text acquired in step S2211 contains or does not contain a specific term.

[0286] (Step S2504) The video text acquisition means 4314 acquires text that summarizes the text acquired in step S2211. Note that there is no restriction on the method for summarizing the text.

[0287] (Step S2505) The video description acquisition means 4314 judges whether or not the video description acquisition condition is satisfied. If the video description acquisition condition is satisfied, the process proceeds to step S2506, and if the video description acquisition condition is not satisfied, the process proceeds to step S2507.

[0288] The video description acquisition condition is a condition for acquiring a video description. For example, the video description acquisition condition is that the video does not include sound, and the amount or length of text in the still images that make up the video is equal to or less than a threshold.

[0289] (Step S2506) The video text acquisition means 4314 provides the video to the generation AI and acquires an explanatory text of the video from the generation AI.

[0290] (Step S2507) The moving image text acquisition unit 4314 determines whether or not the moving image text conversion conditions are satisfied. If the moving image text conversion conditions are satisfied, the process proceeds to step S2508, and if the moving image text conversion conditions are not satisfied, the process proceeds to step S2509.

[0291] The video text conversion condition is a condition for obtaining text from still images constituting a video, such as that the amount or length of text in the still images constituting the video is equal to or greater than a threshold, or that the video does not contain audio.

[0292] (Step S2508) The moving image text acquiring means 4314 acquires one or more representative still images from the moving image.

[0293] (Step S2509) The video text obtaining means 4314 assigns 1 to a counter i.

[0294] (Step S2510) The moving image text acquisition means 4314 judges whether or not the i-th representative still image exists. If the i-th representative still image exists, the process proceeds to step S2511, and if not, the process proceeds to step S2513.

[0295] (Step S2511) The description obtaining means 4313 converts the i-th representative still image into text. An example of such image-to-text processing has been described with reference to the flowchart in FIG.

[0296] (Step S2512) The moving image text obtaining means 4314 increments the counter i by 1. The process returns to step S2510.

[0297] (Step S2513) The text composing means 4316 composes a text to be passed to the generation AI based on one or more acquired texts, and returns to the upper process.

[0298] A specific example of the operation of the information system B in this embodiment will be described below.

[0299] 26 is stored in the definition management unit 111 of the information processing device 4. This prompt definition information is associated with type information "receipt item extraction."

[0300] Furthermore, the definition management unit 111 stores a parameter management table shown in FIG. 27. The parameter management table is a table that manages records having "parameter", "type", and "fx: calculation content". "Parameter" is the name of the parameter. "Type" indicates whether it is a parameter to be calculated or a parameter for a final conclusion. "fx: calculation content" indicates the method for acquiring the parameter.

[0301] Furthermore, the definition management unit 111 stores an output definition management table shown in FIG. 28. The output definition management table is a table for managing output definition information. According to FIG. 28, the information that is output is the "journalization result" and "journalization tax deduction" corresponding to the type "conclusion" in FIG. 27. Furthermore, the attribute value of the attribute "journalization result" in FIG. 27 is a value that can become the "journalization result". The attribute value of the attribute "journalization tax deduction" in FIG. 27 is a value that can become the "journalization tax deduction". The four parameters 2801 in FIG. 28 are conditions for obtaining the conclusions "journalization result" and "journalization tax deduction".

[0302] In the above situation, it is assumed that the user inputs the number of people "3" (2901 in FIG. 29) and the URL (2902) of the receipt image into the input screen of FIG. 29 displayed on the terminal device 2, and selects the execute button 2903. The receipt image corresponding to the URL (2902) of the receipt image is shown in FIG. 30.

[0303] Then, the terminal device 2 accepts a user input including the number of people “3” and the receipt image, and transmits the user input to the information processing device 4.

[0304] Next, the input receiving unit 421 of the information processing device 4 receives a user input having the number of people "3" and the receipt image from the terminal device 2.

[0305] Next, the prompt generator 432 determines that one or more pieces of input information among the user inputs received by the input receiving unit 121 include non-text information (here, the image data in FIG. 30).

[0306] Next, the text conversion unit 431 converts the image data (FIG. 30) in the input information of the user input into text, and obtains the text in FIG. 31. Note that this text is obtained by performing character recognition processing on the receipt image in FIG. 30.

[0307] Next, the prompt creation unit 432 obtains the prompt definition information of Fig. 26 from the definition management unit 111. The prompt creation unit 432 then constructs a prompt by substituting the text of Fig. 31 for the variable {@document text} of 2601 in Fig. 26. Next, the prompt creation unit 432 provides the prompt to the generation AI, and obtains from the generation AI the character string that is the answer shown in Fig. 32.

[0308] Next, the validity determination unit 133 determines whether or not the information for each item included in the reply shown in Fig. 32 is valid. Here, it is assumed that all the information is determined to be valid.

[0309] Next, the output information acquisition unit 134 acquires output information as follows. That is, the output information acquisition unit 134 acquires the number of people "3" included in the user input. Also, the output information acquisition unit 134 acquires the value "24838" of the item "amount" in FIG. 32 according to "fx: calculation content" in 2701 in FIG. 27. Next, the output information acquisition unit 134 acquires the amount per person "24838 / 3=8279.3333" according to "@amount / @number of people" in 2702 in FIG. 27. Also, the output information acquisition unit 134 refers to 2703 in FIG. 27 and acquires the value "normal" of "registration number_inspection result" from the inspection result "normal..." of the item "registration number" in FIG. 32.

[0310] 28, the output information acquisition unit 134 determines that it matches "Registration number_Test result "Normal", "fx: Amount per person" =

[5000] or more" (record 2802 in FIG. 28). Then, the output information acquisition unit 134 acquires the journal entry result "Entertainment expenses" and journal entry tax deductible "Possible" from the record 2802 in FIG. 28, and transmits the journal entry result "Entertainment expenses" and journal entry tax deductible "Possible" to the terminal device 2.

[0311] The terminal device 2 receives and outputs the journalization result "entertainment expenses" and the journalization tax deduction "possible." An example of such output is shown in FIG.

[0312] As described above, according to the present embodiment, appropriate information can be acquired and output using a generating AI based on non-text information. Note that non-text information is, for example, image data (still images), video, and audio.

[0313] Furthermore, according to this embodiment, appropriate information can be acquired and output by appropriately using tables, graphs, or images in the image data.

[0314] Furthermore, according to the present invention, tables or graphs or images in the text file can be appropriately used to obtain and output the appropriate information.

[0315] The software for realizing the information processing device 4 in this embodiment is the following program. That is, this program causes a computer that can access a definition management unit in which prompt definition information for creating a prompt to be given to a generation AI to function as an input receiving unit that receives user input including input information including non-text information, a text conversion unit that acquires text based on the non-text information, a prompt creation unit that creates a prompt to be given to the generation AI using one or more pieces of input information using the text acquired by the text conversion unit and the prompt definition information, an estimated information acquisition unit that provides the prompt to the generation AI and acquires one or more pieces of estimated information acquired by the generation AI, and an information output unit that outputs output information based on the one or more pieces of estimated information.

[0316] FIG. 34 is a block diagram of a computer system 300 that executes the programs described in this specification to realize the information processing device 1, the information processing device 4, and the like according to the various embodiments described above.

[0317] In FIG. 34, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0318] 34, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0319] A program for causing computer system 300 to execute the functions of information processing device 1 and the like of the above-mentioned embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and further transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 at the time of execution. The program may be loaded directly from CD-ROM 3101 or the network.

[0320] The program does not necessarily include an operating system (OS) or a third party program that causes the computer 301 to execute the functions of the information processing device 1 of the above-mentioned embodiment. The program only needs to include an instruction portion that calls appropriate functions (modules) in a controlled manner to obtain a desired result. How the computer system 300 operates is well known, and a detailed description will be omitted.

[0321] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmitting step (processing that is performed only by hardware).

[0322] The program may be executed by a single computer or a plurality of computers. That is, the program may be executed by a centralized processing or a distributed processing.

[0323] Furthermore, in each of the above embodiments, it goes without saying that two or more communication means present in one device may be physically realized by one medium.

[0324] In each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0325] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0326] As described above, the information processing device 4 according to the present invention has the effect of being able to obtain and output appropriate information using a generation AI based on non-text information, and is useful as a server or the like that appropriately handles the generation AI. [Explanation of symbols]

[0327] A, B Information Systems 1, 4 Information processing device 2 Terminal Equipment 3 Generation AI device 11, 41 Storage area 12, 42 Reception 13, 43 Processing section 14 Output section 21 Terminal storage section 22 Terminal Reception 23 Terminal Processing Section 24 Terminal transmitter 25 Terminal receiving section 26 Terminal Output Section 111, 411 Definition management department 112 Output management section 113 Condition Management Department 121, 421 Input reception section 131, 432 Prompt Creation Department 132 Estimated information acquisition unit 133 Validity judgment unit 134 Output information acquisition unit 141 Information output section 431 Text Conversion Department 1311 Parameter Acquisition Method 1312, 4322 Prompt Creation Method 4310 Attribute value acquisition means 4311 Non-text image detection means 4312 Character recognition means 4313 Means of obtaining explanatory text 4314 Video text acquisition means 4315 Voice recognition means 4316 Text Composition Methods

Claims

1. a definition management section in which prompt definition information for creating a prompt to be given to the generation AI is stored; an input receiving unit that receives user input including input information including non-text information; a text generation unit for obtaining text based on the non-text information; a prompt creation unit that creates a prompt to be provided to a generation AI using one or more pieces of input information using the text acquired by the text conversion unit and the prompt definition information; an estimated information acquisition unit that provides the prompt to the generation AI and acquires one or more pieces of estimated information acquired by the generation AI; and an information output unit that outputs output information based on the one or more pieces of estimated information.

2. the non-text information is image data; The text conversion unit includes:

2. The information processing apparatus according to claim 1, further comprising character recognition means for performing character recognition processing on said image data to obtain said text.

3. The text conversion unit includes:

2. An information processing apparatus according to claim 1, further comprising an explanatory text acquisition means for providing the image data to a generating AI and acquiring the text explaining the image data from the generating AI.

4. The text conversion unit includes: character recognition means for performing character recognition processing on the image data to obtain the text; a text construction means for constructing a text using both the text acquired by the character recognition means and the text acquired by the explanatory sentence acquisition means; The prompt creation unit is 4. The information processing apparatus according to claim 3, further comprising: a prompt generating unit configured to generate a prompt for the AI ​​by using one or more pieces of input information using the text configured by the text configuration unit and the prompt definition information.

5. The prompt definition information is having location-specific information that specifies a location to place the portion information based on the non-text image portion, which is a table or a graph or an image; The image data includes a non-text image portion that is a table, a graph, or an image, a non-text image detection means for detecting the non-text image portions in the image data; The prompt creation unit is 3. An information processing apparatus according to claim 2, further comprising: acquiring partial information based on the non-text image portion detected by said non-text image detection means; and generating said prompt using said partial information as well.

6. the input information is a text file containing a non-text image portion which is a table or a graph or an image; a non-text image detection means for detecting the non-text image portions in the text file; The prompt creation unit is 3. An information processing apparatus according to claim 2, further comprising: acquiring partial information based on the non-text image portion detected by said non-text image detection means; and generating said prompt using said partial information as well.

7. the non-text information is a video; The text conversion unit includes:

2. The information processing apparatus according to claim 1, further comprising a video text acquisition means for acquiring the text, which is an explanatory text relating to the video.

8. The video text acquisition means includes: The information processing apparatus according to claim 7 , wherein two or more representative still images are acquired from the moving image, a description for each of the two or more representative still images is acquired, and the text is acquired based on the two or more descriptions.

9. the non-text information is audio; The text conversion unit includes:

2. An information processing apparatus according to claim 1, further comprising a voice recognition unit for recognizing the voice and acquiring the text.

10. The text conversion unit includes: The method further comprises: acquiring data attribute values ​​of non-text information; 2. The information processing apparatus according to claim 1, wherein different processes are performed to acquire the text depending on the data attribute value.

11. The definition management unit includes: Parameter definition information is stored, which is information for acquiring parameters to be used when acquiring a prompt using the input information; The prompt creation unit is a parameter acquisition means for acquiring one or more parameters using the input information and the parameter definition information; 2. The information processing apparatus according to claim 1, further comprising: a prompt creating means for creating a prompt using the prompt definition information and the one or more parameters.

12. the parameter definition information has an information identifier for identifying input information or a parameter and a processing identifier for identifying a processing for the input information or the parameter, The parameter acquisition means The information processing apparatus according to claim 11 , wherein input information or a parameter identified by the information identifier contained in the parameter definition information is acquired, and the input information or the parameter is used to perform a process identified by the process identifier, thereby acquiring the one or more parameters.

13. The prompt definition information is partial prompt definition information for configuring a prompt in association with one or more selection conditions using the input information or the parameters; The prompt creation unit is 2. The information processing apparatus according to claim 1, further comprising: determining a selection condition satisfied by the input information or the parameter; acquiring partial prompt definition information paired with the selection condition; and creating the prompt using the partial prompt definition information.

14. The definition management unit includes: Prompt definition information corresponding to two or more types of information that specify the type of input information is stored, the user input includes two or more pieces of input information corresponding to type information; The prompt creation unit is creating the prompt for each of two or more types of category information included in the user input, using input information corresponding to the category information and prompt definition information corresponding to the category information; The estimated information acquisition unit 2 . The information processing apparatus according to claim 1 , wherein the prompt is given to the generating AI for each of the two or more types of information, and the one or more pieces of estimated information are obtained.

15. The estimated information acquisition unit Obtain two or more pieces of presumptive information; an output management unit in which output information corresponding to each of two or more output conditions using two or more pieces of estimated information is stored; an output information acquisition unit that acquires output information corresponding to an output condition satisfied by the two or more pieces of estimated information from the output management unit; The information output unit is The information processing apparatus according to claim 1 , wherein the output information acquisition unit outputs the acquired output information.

16. a validity determination unit that determines whether the estimated information acquired by the estimated information acquisition unit satisfies a validity condition; The information output unit is 2. The information processing apparatus according to claim 1, further comprising: if the validity condition is not satisfied, outputting error information indicating that the validity condition is not satisfied.

17. An information processing method implemented by a definition management unit in which prompt definition information for creating a prompt to be given to a generation AI is stored, an input reception unit, a text conversion unit, a prompt creation unit, and an information output unit, an input receiving step in which the input receiving unit receives user input including input information including non-text information; a text generation step in which the text generation unit obtains text based on the non-text information; a prompt creation step in which the prompt creation unit creates a prompt to be provided to the generation AI using one or more pieces of input information using the text acquired by the text conversion unit and the prompt definition information; an estimated information acquisition step in which the estimated information acquisition unit provides the prompt to the generation AI and acquires one or more pieces of estimated information acquired by the generation AI; and an information output step in which the information output unit outputs output information based on the one or more pieces of estimated information.

18. A computer that can access a definition management unit that stores prompt definition information for creating a prompt to be given to a generation AI, an input receiving unit that receives user input including input information including non-text information; a text generation unit for obtaining text based on the non-text information; a prompt creation unit that creates a prompt to be provided to a generation AI using one or more pieces of input information using the text acquired by the text conversion unit and the prompt definition information; an estimated information acquisition unit that provides the prompt to the generation AI and acquires one or more pieces of estimated information acquired by the generation AI; A program for causing the information output unit to function as an information output unit that outputs output information based on the one or more pieces of estimated information.

Images