Information processing system, information processing method, and program

Abstract

An information processing system and the like that can efficiently update system prompts in a system that supports work performed by a user is provided. [Solution] In an information processing system 10, a business information acquisition unit 110 acquires business information from a business execution system. A system prompt generation unit 120 generates a system prompt based on the business information to cause a generative artificial intelligence to respond to a business inquiry from a user. A log information acquisition unit 130 acquires log information including the answer output by the generative artificial intelligence and the user's operation status in response to the answer. A determination unit 140 determines whether to update the content of the system prompt based on the update status of the business information or the log information. An update unit 150 updates the system prompt when the determination result indicates that the content of the system prompt should be updated.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] In recent years, companies have been introducing business support systems that utilize generative artificial intelligence (also known as generative AI (Artificial Intelligence)) as part of efforts to improve business efficiency. Prompt engineering, a technology for designing or optimizing the instructions (prompts) input into generative AI, has been attracting attention in the use of generative AI.

[0003] For example, according to Patent Document 1, a prompt engineering service receives system configuration data, generates a prompt that queries a large-scale language model for a category, and generates and displays a user interface message for configuration support based on the response. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent Application No. 2025 / 0005224 Summary of the Invention [Problem to be solved by the invention]

[0005] Meanwhile, in business support systems that use generative AI, a mechanism is required to efficiently update system prompts when information about procedures in business systems such as online business manuals and ERP (Enterprise Resource Planning) is updated.

[0006] In view of the above-mentioned problems, the present disclosure aims to provide an information processing system and the like that can efficiently update system prompts in a system that supports tasks performed by users. [Means for solving the problem]

[0007] The information processing system according to the present disclosure includes a business information acquisition unit, a system prompt generation unit, a log information acquisition unit, a determination unit, and an update unit. The business information acquisition unit acquires business information from a business execution system. The system prompt generation unit generates a system prompt based on the business information to cause the generative artificial intelligence to respond to business inquiries from users. The log information acquisition unit acquires log information including answers output by the generative artificial intelligence and the user's operation status in response to the answers. The determination unit determines whether to update the content of the system prompt based on the update status of the business information or the log information. The update unit updates the system prompt when the determination result indicates that the content of the system prompt should be updated.

[0008] The information processing method according to the present disclosure is executed by a computer as follows: The computer acquires business information from a business execution system. The computer generates a system prompt based on the business information to cause the generative artificial intelligence to respond to a business inquiry from a user. The computer acquires log information including the answer output by the generative artificial intelligence and the user's operation status in response to the answer. The computer determines whether to update the content of the system prompt based on the update status of the business information or the log information. The computer updates the system prompt if the determination result indicates that the content of the system prompt should be updated.

[0009] A program according to the present disclosure causes a computer to execute the following information processing method. The computer acquires business information from a business execution system. The computer generates a system prompt based on the business information to cause a generative artificial intelligence to respond to a business inquiry from a user. The computer acquires log information including an answer output by the generative artificial intelligence and the user's operation status in response to the answer. The computer determines whether to update the content of the system prompt based on the update status of the business information or the log information. The computer updates the system prompt if the determination result indicates that the content of the system prompt should be updated. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to provide an information processing system, an information processing method, and a program that can efficiently update system prompts in a system that supports tasks performed by users. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram of a business support system according to the present disclosure. [Figure 2] 10 is a flowchart of a process executed by the answering system. [Figure 3] FIG. 2 is a sequence diagram showing a process executed by the business support system. [Figure 4] 1 is a flowchart of an information processing method according to a first embodiment. [Figure 5] FIG. 10 illustrates a process executed by a system prompt generating unit according to the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating a process executed by a system prompt update unit according to the first embodiment. [Figure 7] 10 is a flowchart of an information processing method according to a second embodiment. [Figure 8] 10 is a flowchart of an information processing method according to a third embodiment. [Figure 9]10 is a flowchart of an information processing method according to a fourth embodiment. [Figure 10] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present invention will be described below through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential means for solving the problems. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In each drawing, the same elements are assigned the same reference numerals, and duplicate explanations are omitted as necessary.

[0013] <First Embodiment> An information processing system 10 will be described with reference to FIG. 1. FIG. 1 is a block diagram of a business support system according to the present disclosure. The business support system 1 receives inquiries about business from users who perform predetermined business using a business execution system, and supports the business by answering the received inquiries. The business support system 1 mainly comprises an information processing system 10, a generation AI 20, a response system 30, a business execution system 40, and a user terminal 50. Each of the above components is communicably connected to a network N1.

[0014] (Information Processing System 10) The information processing system 10 is a computer or server having a communication function, and mainly includes a business information acquisition unit 110, a system prompt generation unit 120, a log information acquisition unit 130, a determination unit 140, and an update unit 150.

[0015] The business information acquisition unit 110 acquires business information 421 from a business database 420 included in the business execution system 40. The business information 421 includes, for example, information about the procedure of processing executed by the business execution system 40. Alternatively, the business information 421 includes a manual for a user to use the business execution system 40.

[0016] The system prompt generation unit 120 generates a system prompt 341 for causing the generation AI 20 to respond to a business inquiry from a user based on the business information 421. The system prompt generation unit 120 causes a predetermined language model to generate the system prompt 341. The system prompt generation unit 120 may include a language model for generating the system prompt 341 from the business information 421. The system prompt generation unit 120 may also cause the generation AI 20 to generate the system prompt 341. The system prompt generation unit 120 has an instruction sentence (a prompt for generating a system prompt) for causing the language model to generate the system prompt 341.

[0017] The system prompt generator 120 does not necessarily generate one system prompt 341. The system prompt generator 120 may generate a plurality of different system prompts 341.

[0018] The log information acquisition unit 130 acquires log information 342 from the response system 30, which includes the answer output by the generation AI 20 and the user's operational status regarding the answer. The user's operational status includes, for example, information on whether the user made an additional inquiry after receiving the answer from the generation AI 20, whether the user entered a positive or negative evaluation of the answer content, or whether the user performed a business operation in accordance with the presented instructions.

[0019] The determination unit 140 determines whether to update the contents of the system prompt 341 based on the update status of the business information 421 or the log information 342. For example, the determination unit 140 determines that the system prompt 341 needs to be updated when the business information 421 registered in the business database 420 is updated. The determination unit 140 also determines whether the system prompt 341 needs to be updated based on the interaction between the generation AI 20 and the user in the log information 342. The determination unit 140 may include, in the determination result, information on the item of the business information 421 that has been determined to need to be updated. The determination unit 140 may also include, in the determination result, a part of the log information 342 as the reason for determining that the update is necessary. The determination unit 140 supplies the determination result to the update unit 150. The determination result may also be referred to as determination result information.

[0020] Specifically, the determining unit 140 determines whether to update the system prompt 341 based on at least one of the user's evaluation of the answer included in the log information 342, the occurrence of additional inquiries, and the accuracy score of the answer.

[0021] More specifically, for example, the determination unit 140 may determine to update the system prompt 341 when the user's evaluation of an answer included in the log information 342 is negative a predetermined number of times or more. The determination unit 140 may also determine to update the system prompt 341 when the occurrence rate of additional inquiries exceeds a predetermined threshold. The determination unit 140 may also determine to update the system prompt 341 when the accuracy score of the answer is less than a predetermined value.

[0022] When an update of the task information 421 is detected, the determination unit 140 may determine whether to update the system prompt 341 based on the difference. In this case, the determination unit 140 quantitatively calculates the degree of change in the content of the task information 421 before and after the update, for example. If the amount of change exceeds a predetermined threshold, the determination unit 140 determines that an update of the system prompt 341 is necessary. The determination unit 140 may also compare the task information 421 before and after the update with the current system prompt 341 to identify changes that are not reflected in the prompt. The determination unit 140 may cause the generation AI 20 to perform such a comparison process. The determination unit 140 may individually set a threshold for the amount of change depending on the type of task execution system 40, which will be described later.

[0023] If the determination result output by the determination unit 140 indicates that the content of the system prompt 341 should be updated, the update unit 150 updates the system prompt 341. For example, if an update to the business information 421 is detected and the determination unit 140 determines that the system prompt 341 should be updated based on the difference, the update unit 150 updates the system prompt 341 corresponding to the difference. Also, for example, if the determination unit 140 determines that the system prompt 341 should be updated based on the log information 342, the update unit 150 performs an update process for the system prompt 341 based on the log information 342.

[0024] The update unit 150 may include a language model for performing the above-described processing. Alternatively, the update unit 150 may cause the generation AI 20 to update the system prompt 341. The update unit 150 has a command statement (a prompt for updating the system prompt) for causing the language model to update the system prompt 341.

[0025] (Generation AI20) The generation AI 20 responds to prompts provided to the generation AI 20 by the response system 30. The generation AI 20 is, for example, a large-scale language model. The generation AI 20 may be a multimodal model that accepts inputs such as image information and audio signals in addition to accepting text input. The generation AI 20 can also be called generative artificial intelligence. The generation AI 20 may have a Retrieval-Augmented Generation (RAG) function that searches for information from a specified database and uses it in the response.

[0026] As described above, the generation AI 20 may also receive a predetermined prompt from the information processing system 10 and provide the information processing system 10 with a response to the received prompt.

[0027] The generation AI 20 may also have a plug-in function for linking with external applications or services. In this case, the generation AI 20 may link with various functional modules of the business execution system 40 or an external business database via a plug-in, for example, to obtain necessary information and generate a response. The generation AI 20 may also have an agent function that autonomously executes multiple processes according to user instructions and business conditions.

[0028] (Answer System 30) The answering system 30 is, for example, a computer or a server, and includes a user prompt receiving unit 310, an answer generating unit 320, an answer presenting unit 330, and a storage unit 340 as its main components.

[0029] The user prompt receiving unit 310 receives a query, that is, a user prompt, input by the user from the user terminal 50. The user prompt receiving unit 310 supplies the received user prompt to the answer generating unit 320.

[0030] The answer generation unit 320 receives a user prompt and inputs the received user prompt to the generation AI 20. At this time, the answer generation unit 320 may also supply a system prompt 341 and task information 421 to the generation AI 20. Whether the answer generation unit 320 inputs the system prompt 341 together with the user prompt may be set via an operation of the user terminal 50. That is, the answer generation unit 320 may accept a setting for the system prompt 341 from the user terminal 50. Alternatively, the answer generation unit 320 may receive a user prompt and determine whether the received user prompt is related to the system prompt 341.

[0031] When inputting the system prompt 341 together with the user prompt to the generation AI 20, the answer generation unit 320 may read the business information 421 in accordance with the contents of the system prompt 341 and input the read business information 421 to the generation AI 20. This allows the answer system 30 to generate an answer suitable for the generation AI 20.

[0032] The answer generation unit 320 also stores the history of a series of interactions with the user terminal 50 that input the system prompt 341 to the generation AI 20 as log information 342 in the storage unit 340. The log information 342 is configured with each historical unit being a group of conversations between the user terminal 50 and the generation AI 20. The log information 342 may include histories related to multiple different conversations.

[0033] The answer generation unit 320 receives an answer to a query from the generation AI 20 by inputting at least a user prompt to the generation AI 20. Alternatively, the answer generation unit 320 receives an answer to a query from the generation AI 20 by inputting a user prompt, a system prompt 341, and business information 421 provided from the user prompt reception unit 310 to the generation AI 20. The answer generation unit 320 provides the answer received from the generation AI 20 to the answer presentation unit 330.

[0034] The answer presenting unit 330 transmits the answer received from the answer generating unit 320 to the user terminal 50 via the network N1 and presents it.

[0035] The storage unit 340 is a storage device including a nonvolatile memory, and stores at least a system prompt 341 and log information 342.

[0036] The system prompt 341 is an instruction statement set for the purpose of making the generation AI 20 respond to a user inquiry in accordance with a business procedure. The system prompt 341 is generated by the system prompt generation unit 120 of the information processing system 10 and updated by the update unit 150.

[0037] The log information 342 includes at least the content of the user's inquiry and the content of the answer provided by the generation AI 20. In addition to the above, the log information 342 may also include the presentation history of the answer presentation unit 330 and the user's operations and evaluation results. The log information 342 is provided to the log information acquisition unit 130 of the information processing system 10.

[0038] (Business Execution System 40) The business execution system 40 is a business system for executing and managing daily business in a company, such as an ERP system, a sales management system, a production management system, or a customer management system. The business execution system 40 is a computer or server managed by the company that executes the business. The business execution system 40 mainly comprises a business server 410 and a business database 420.

[0039] The business server 410 is a device that executes various business processes in a company or organization, and executes various business applications based on business procedures. The business processes that the business applications process include, for example, transactions, accounting, inventory management, personnel management, and production management. The business server 410 presents operation screens and processing results to users who are responsible for the business via the user terminal 50.

[0040] The business database 420 is a storage device including a nonvolatile memory. The business database 420 stores at least business information 421.

[0041] The business information 421 is information relating to the execution of business procedures, such as procedures for various business processes in the business execution system 40, operation methods, input items, screen configurations, authority settings, and dependencies between data items.

[0042] The business information 421 is updated when system settings or manuals are changed due to, for example, the addition of new functions, revision of business flows, changes to operation screens, or responses to changes in laws and regulations or company rules. The business information 421 is updated by an administrator. The business execution system 40 may automatically update the business information 421 by having the business server 410 analyze log data and operation records.

[0043] The above describes each component of the business support system 1. A user who performs a business makes an inquiry about the business procedure to the answering system 30 via the user terminal 50. The answering system 30 cooperates with the business execution system 40 and the generation AI 20 to generate an answer to the user's inquiry and outputs the generated answer to the user terminal 50. At this time, the answering system 30 causes the generation AI 20 to generate an answer using a system prompt generated or updated by the information processing system 10. The information processing system 10 generates and updates a system prompt 341 used by the answering system 30. With the above-described configuration, the business support system 1 efficiently operates the business execution system 40. Furthermore, the information processing system 10 favorably supports the answering system 30 so that the answer generation executed by the answering system 30 can be performed efficiently.

[0044] Next, the processing executed by the reply system 30 will be described with reference to Fig. 2. Fig. 2 is a flowchart of the processing executed by the reply system. The processing shown in Fig. 2 starts when an inquiry is received from the user terminal 50, for example.

[0045] First, the user prompt receiving unit 310 of the answer system 30 receives a user prompt, which is an inquiry from a user (step S101). The user prompt receiving unit 310 receives the user prompt from the user terminal 50. The user prompt receiving unit 310 supplies the received user prompt to the answer generating unit 320.

[0046] Next, the answer generation unit 320 analyzes the received user prompt and determines whether or not to use a system prompt (step S102). If it is determined not to use a system prompt (step S102: NO), the answer system 30 proceeds to step S106. If it is determined to use a system prompt (step S102: YES), the answer system 30 proceeds to step S103.

[0047] In step S103, the answer generating unit 320 starts generating the log information 342 (step S103). The log information 342 generated by the answer generating unit 320 is stored in the storage unit 340.

[0048] In step S104, the answer generator 320 inputs the user prompt, the system prompt 341, and the business information 421 to the generation AI 20 (step S104). The answer generator 320 receives an answer from the generation AI 20 by inputting the above information, and provides the received answer to the answer presenter 330.

[0049] In step S105, the answer presenting unit 330 transmits the answer of the generated AI 20 to the user terminal 50 (step S105). After transmitting the answer to the user terminal 50, the answering system 30 ends the series of processes.

[0050] In step S106, the answer generation unit 320 inputs the user prompt to the generation AI 20 (step S106). The answer generation unit 320 receives the answer from the generation AI 20 and provides the received answer to the answer presentation unit 330. After that, the answer system 30 proceeds to step S105.

[0051] The above describes the processing executed by the answering system 30. Through the above processing, the answering system 30 can select a system prompt according to the content of the user's inquiry and automatically generate an appropriate answer that corresponds to the business procedure. The answering system 30 also accumulates log information 342 related to the inquiry and its answer. This allows the answering system 30 to improve the system prompt.

[0052] Next, the processing of the task support system 1 when the reply system 30 uses the system prompt 341 will be described with reference to Fig. 3. Fig. 3 is a sequence diagram showing the processing executed by the task support system.

[0053] First, the user terminal 50 transmits an inquiry (ie, a user prompt) to the answering system 30 (step S201).

[0054] Next, upon receiving the user prompt, the answering system 30 starts generating log information 342 (step S202). At this time, if the user prompt is due to a new conversation, the answering system 30 starts generating new log information 342. On the other hand, if the user prompt is sent as a continuation of a past conversation, the answering system 30 adds and records the conversation to the previously generated log information 342.

[0055] Next, the answering system 30 transmits the user prompt, the system prompt 341, and the business information 421 to the generating AI 20 (step S203).

[0056] Next, the generation AI 20 generates a first response to the received user prompt (step S204). At this time, the generation AI 20 generates the first response in a format that conforms to the system prompt 341. The generation AI 20 also generates the first response by referring to the provided business information 421.

[0057] Next, the generation AI 20 transmits the first answer to the response system 30 (step S205). Subsequently, the response system 30 transmits the first answer received from the generation AI 20 to the user terminal 50 (step S206).

[0058] 3, a case will be described in which the user makes an additional inquiry after receiving the first answer. After receiving the answer in step S206, the user sends an additional inquiry to the answering system 30 (step S207).

[0059] Next, the reply system 30 inputs the user prompt, which is an additional inquiry, the system prompt 341, and the business information 421 again to the generation AI 20 (step S208). The generation AI 20 generates a second reply to the received information (step S209).

[0060] Next, the generation AI 20 transmits the generated second answer to the answering system 30 (step S210). Subsequently, the answering system 30 transmits the second answer to the user terminal 50 (step S211).

[0061] The above has described the processing of the business support system 1 when the answering system 30 uses the system prompt 341. The answering system 30 records the above-mentioned interactions between the user terminal 50 and the answering system 30 as log information 342. That is, the log information 342 includes the user prompt, the first answer, an additional user prompt, and the second answer. If the user makes an additional inquiry, or if the user inputs an impression or evaluation of the first answer or the second answer, the answering system 30 adds this information to the log information 342.

[0062] Next, a process executed by the information processing system 10 will be described with reference to Fig. 4. Fig. 4 is a flowchart of the information processing method according to the first embodiment.

[0063] First, the business information acquisition unit 110 acquires business information 421 from the business execution system 40 (step S10). The business information acquisition unit 110 supplies the acquired business information 421 to the system prompt generation unit 120.

[0064] Next, the system prompt generation unit 120 generates a system prompt 341 based on the business information to cause the generative artificial intelligence to respond to a business-related inquiry from the user (step S11). The system prompt generation unit 120 supplies the generated system prompt 341 to the answering system 30. The answering system 30 uses the system prompt 341 received from the information processing system 10 to generate an answer to the inquiry from the user terminal 50 as described above.

[0065] Next, the log information acquisition unit 130 acquires, from the answering system 30, the log information 342 including the answer output by the generating AI 20 and the user's operation status for the answer (step S12).

[0066] Next, the determination unit 140 determines whether or not to update the contents of the system prompt 341 based on the update status of the business information 421 or the log information 342 (step S13). If it is determined not to update the contents of the system prompt 341 (step S13: NO), the information processing system 10 ends the series of processes. On the other hand, if it is determined to update the contents of the system prompt 341 (step S13: YES), the information processing system 10 proceeds to step S14. At this time, the determination unit 140 supplies information on the result of the determination (determination result information) to the update unit 150.

[0067] In step S14, if the determination result indicates that the content of the system prompt should be updated, update unit 150 updates system prompt 341 (step S14). Once update unit 150 has updated system prompt 341, information processing system 10 ends the series of processes.

[0068] The above describes the processing executed by the information processing system 10. Through the above processing, the information processing system 10 automatically and appropriately updates the system prompt 341 in accordance with changes in business procedures and user usage conditions. This eliminates the need for users of the business execution system 40 to manually update the prompt. Therefore, the information processing system 10 can prevent a decrease in the operational efficiency and response accuracy of the business execution system 40 and the response system 30.

[0069] Next, the process executed by the system prompt generating unit 120 will be described with reference to Fig. 5. Fig. 5 is a diagram illustrating the process executed by the system prompt generating unit according to the first embodiment.

[0070] The system prompt generation unit 120 has a prompt for generating a system prompt 121. The prompt for generating a system prompt 121 includes an instruction such as "Please anticipate user questions from the contents of the acquired business procedures and generate a system prompt for each item."

[0071] The system prompt generation unit 120 acquires business information 421 and inputs the acquired business information 421 and the prompt for generating a system prompt 121 to the generation AI 20. Furthermore, the system prompt generation unit 120 receives the system prompt 341 generated by the generation AI 20 and outputs the received system prompt 341.

[0072] As described above, the system prompt generation unit 120 has the prompt 121 for generating a system prompt, and is therefore able to generate a system prompt 341 from business information 421. The system prompt generation unit 120 may generate multiple system prompts 341. The system prompt generation unit 120 may generate a different system prompt 341 for each business type, such as sales management, inventory management, or personnel management.

[0073] The system prompt generation unit 120 can also support multiple business systems. In this case, the system prompt generation unit 120 may have a system prompt generation prompt 121 customized for each business system. The system prompt generation unit 120 may generate the system prompt 341 using a language model different from that of the generation AI 20.

[0074] Next, the processing executed by the update unit 150 will be described with reference to Fig. 6. Fig. 6 is a diagram showing the processing executed by the update unit 150 according to the first embodiment. The update unit 150 shown in Fig. 6 executes processing to convert the pre-update system prompt 341A into the post-update system prompt 341B.

[0075] The update unit 150 has a system prompt update prompt 151. The system prompt update prompt 151 includes an instruction such as "Please update the existing system prompt from the contents of the determination result information."

[0076] The update unit 150 receives the judgment result information, the business information 421, and the system prompt 341A before the update, and inputs the received information and the system prompt update prompt 151 to the generation AI 20. Furthermore, the update unit 150 receives the system prompt 341B updated by the generation AI 20, and outputs the updated system prompt 341B.

[0077] The above has described the processing executed by the update unit 150. Through the above processing, the update unit 150 performs processing to update the system prompt 341A that the determination unit 140 determined to be updated, in accordance with the determination result information. In the example shown in FIG. 6, the update unit 150 updated the pre-update system prompt 341A to the updated system prompt 341B. However, the processing executed by the update unit 150 is not limited to this. For example, when new information is added to the business information 421, the update unit 150 generates a new system prompt 341.

[0078] Although the first embodiment has been described above, the business support system 1 according to the first embodiment is not limited to the above-described configuration. For example, the business support system 1 may have a plurality of different generation AIs. In this case, the user may be allowed to select a generation AI to respond to the user's inquiry. Alternatively, the answering system 30 may have a function to select a generation AI to respond depending on the content of the user's inquiry. The generation AI 20 may include a plurality of selectable types or versions.

[0079] The information processing system 10 may include a generation AI 20. The information processing system 10 may also include at least a part of a response system 30. For example, the information processing system 10 may have a storage unit and store at least one of a system prompt 341 and log information 342. Similarly, the information processing system 10 may also include at least a part of a business execution system 40.

[0080] The information processing system 10 may include a processor and a storage device as components not shown. In this case, the storage device may include a nonvolatile memory such as a flash memory or a solid-state drive (SSD). In this case, the storage device may store a computer program (hereinafter simply referred to as a program) for executing the above-described method. The processor loads the computer program from the storage device into a buffer memory such as a dynamic random access memory (DRAM) and executes the program.

[0081] Each component of the information processing system 10 may be realized by dedicated hardware. Furthermore, some or all of the components may be realized by general-purpose or dedicated circuits, processors, etc., or a combination thereof. These may be configured by a single chip, or by multiple chips connected via a bus. Some or all of the components may be realized by a combination of the above-mentioned circuits, etc., and a program. Furthermore, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), etc. may be used as the processor. Furthermore, at least some of the functions of this embodiment may be provided in the form of infrastructure as a service (IaaS), platform as a service (PaaS), software as a service (SaaS), etc.

[0082] As described above, according to the present embodiment, it is possible to provide an information processing system, an information processing method, and a program that can efficiently update system prompts in a system that supports tasks performed by users.

[0083] <Embodiment 2> Next, a description will be given of a second embodiment. In the information processing system 10 according to the second embodiment, when the determination unit 140 determines that the log information 342 includes a possibility of updating the business information 421, the information processing system 10 obtains the latest business information 421 and determines whether the system prompt 341 should be updated.

[0084] Fig. 7 is a flowchart of the information processing method according to the embodiment 2. The flowchart shown in Fig. 7 differs from the flowchart shown in Fig. 4 in the processing between step S12 and step S13.

[0085] In step S12, the log information acquisition unit 130 acquires, from the answering system 30, the log information 342 including the answer output by the generating AI 20 and the user's operation status for the answer (step S12).

[0086] Next, the determination unit 140 determines whether the log information 342 includes a possibility of updating the business information 421 (step S21). If it is determined that the log information 342 does not include a possibility of updating the business information 421 (step S21: NO), the information processing system 10 proceeds to step S13. If it is determined that the log information 342 includes a possibility of updating the business information 421 (step S21: YES), the information processing system 10 proceeds to step S22.

[0087] In step S22, the determination unit 140 acquires the latest business information 421. After acquiring the latest business information 421, the determination unit 140 proceeds to step S13 and determines whether the system prompt 341 has been updated.

[0088] The processing according to the second embodiment has been described above. The information processing system 10 according to the second embodiment detects the possibility that the business information 421 has been updated by analyzing the log information 342, which includes the user's operation status and the response content of the generation AI 20. For example, if the user's comments or ratings included in the log information 342 include information containing a predetermined number or more of negative comments or ratings, it is inferred that the business information 421 has been updated. In such a situation, the determination unit 140 acquires the latest business information 421 and then determines whether the business information 421 has been updated. With this configuration, the determination unit 140 quickly detects an update to the business information 421 and updates the system prompt 341.

[0089] As described above, according to this embodiment, the system prompt 341 is updated before receiving notification of an explicit change to the task information 421. Therefore, according to this embodiment, it is possible to provide an information processing system, an information processing method, and a program that can quickly and efficiently update the system prompt in a system that supports tasks performed by users.

[0090] <Third Embodiment> Next, a third embodiment will be described. In the information processing system 10 according to the third embodiment, the determination unit 140 determines whether to update the system prompt 341 based on a reward signal derived from a user's evaluation included in the log information 342. In the information processing system 10 according to the third embodiment, the update unit 150 updates the system prompt 341 based on reinforcement learning using the reward signal.

[0091] The reward signal is an index that indicates how well the answer of the generation AI 20 matches the user's expectations. The determination unit 140 calculates this index from the user's evaluation comments. The determination unit 140 in this embodiment determines whether to update the system prompt 341 based on the reward signal. This allows the determination unit 140 to determine whether to update the system prompt 341 based on a quantitative index.

[0092] Fig. 8 is a flowchart of an information processing method according to the third embodiment. The flowchart shown in Fig. 8 differs from the flowchart shown in Fig. 7 in that steps S31 and S32 are provided between steps S22 and S13.

[0093] In step S21, if it is not determined that the log information 342 includes a possibility that the business information 421 will be updated (step S21: NO), the information processing system 10 proceeds to step S31. If it is determined that the log information 342 includes a possibility that the business information 421 will be updated (step S21: YES), the information processing system 10 proceeds to step S22 and then to step S31.

[0094] In step S31, the determination unit 140 determines whether the log information 342 has been updated (step S31). If it is determined that the log information 342 has not been updated (step S31: NO), the information processing system 10 proceeds to step S13. If it is determined that the log information 342 has been updated (step S31: YES), the information processing system 10 proceeds to step S32.

[0095] In step S32, the determination unit 140 generates a reward signal (step S32). That is, the determination unit 140 derives the reward signal from the user evaluation or usage status included in the log information 342. When the determination unit 140 generates the reward signal, the information processing system 10 proceeds to step S13.

[0096] In step S13, the determination unit 140 determines whether or not to update the system prompt 341 based on reinforcement learning using the reward signal. That is, the determination unit 140 uses the reward signal as a signal for learning.

[0097] The update unit 150 updates the system prompt 341 in a direction that improves the quality of the answer output by the generation AI 20. Specifically, for example, when the reward signal indicates a positive value, the update unit 150 maintains the content of the system prompt used to generate the answer, and when the reward signal indicates a negative value, the update unit 150 updates the content of the system prompt 341 to correct the expression or instruction that caused the quality of the answer to be reduced.

[0098] The third embodiment has been described above. The information processing system 10 according to this embodiment can optimize the content of the system prompt 341 by feeding back the user's evaluation as a reward signal. This allows the information processing system 10 according to this embodiment to improve the quality of answers through the user's evaluation. Therefore, this embodiment can provide an information processing system, an information processing method, and a program that can efficiently update system prompts in a system that supports a user's work through self-learning.

[0099] <Fourth Embodiment> Next, a description will be given of a fourth embodiment. In the information processing system 10 according to the fourth embodiment, the update unit 150 has an approval request function for receiving approval from an administrator.

[0100] The update unit 150 according to this embodiment has an approval request function for receiving approval from an administrator for the system prompt 341 to be updated. The update unit 150 applies the update content only when the administrator's approval is received. This prevents the update unit 150 from reflecting erroneous updates or inappropriate prompt content.

[0101] Fig. 9 is a flowchart of the information processing method according to the fourth embodiment. The flowchart shown in Fig. 9 differs from the flowchart shown in Fig. 8 in that steps S41 and S42 are provided between steps S13 and S14.

[0102] In step S13, if the determination unit 140 determines that the contents of the system prompt 341 should be updated (step S13: YES), the information processing system 10 proceeds to step S41.

[0103] In step S41, the update unit 150 requests approval from the administrator (step S41). Subsequently, the update unit 150 determines whether or not approval for the update has been given (step S42). If it is not determined that approval for the update has been given (step S42: NO), the information processing system 10 does not perform the process of updating the system prompt 341 and ends the series of processes. On the other hand, if it is determined that approval for the update has been given (step S42: YES), the information processing system 10 proceeds to step S14.

[0104] In step S14, if the determination result indicates that the content of the system prompt should be updated and approval has been given by the administrator, update unit 150 updates system prompt 341 (step S14). Once update unit 150 has updated system prompt 341, information processing system 10 ends the series of processes.

[0105] The above describes the fourth embodiment. The approval function according to this embodiment can be applied to all of the above-mentioned embodiments. By having the update unit 150 receive approval from the administrator, the information processing system 10 can prevent the generation AI 20 from generating an incorrect system prompt and reflecting the generated system prompt in the answering system 30.

[0106] As described above, according to the present embodiment, it is possible to provide an information processing system, an information processing method, and a program that can efficiently and safely update system prompts in a system that supports tasks performed by users.

[0107] <Example of hardware configuration> Below, an example will be described in which each functional configuration of the information processing system according to the present disclosure is realized by a combination of hardware and software.

[0108] 10 is a block diagram illustrating an example of the hardware configuration of a computer. The information processing system 10 according to the present disclosure can achieve the above-described functions using a computer 500 including the hardware configuration shown in the figure.

[0109] The computer 500 may be a portable computer such as a smartphone or tablet terminal, or may be a stationary computer such as a PC. The computer 500 may be a dedicated computer designed to realize each device, or may be a general-purpose computer. The computer 500 can realize desired functions by installing a predetermined application.

[0110] The computer 500 has a bus 501, a processor 510, a memory 511, a storage device 512, an input / output interface 513, and a network interface 514. The bus 501 is a data transmission path through which the processor 510, the memory 511, the storage device 512, the input / output interface 513, and the network interface 514 transmit and receive data to and from each other. However, the method of connecting the processor 510 and the like to each other is not limited to bus connection.

[0111] The processor 510 is one of various processors such as a CPU, a GPU, an FPGA, etc. The memory 511 is a main storage device realized using a RAM (Random Access Memory) or the like.

[0112] The storage device 512 is an auxiliary storage device realized using a hard disk, an SSD, a memory card, a ROM (Read Only Memory), etc. The storage device 512 stores programs for realizing desired functions. The processor 510 reads the programs into the memory 511 and executes them to realize the respective functional components of each device.

[0113] The input / output interface 513 is an interface for connecting the computer 500 to an input / output device. For example, an input device such as a keyboard and an output device such as a display device are connected to the input / output interface 513. The network interface 514 is an interface for connecting the computer 500 to a network.

[0114] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0115] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate. [Explanation of symbols]

[0116] 1. Business support system 10 Information Processing Systems 20 Generation AI 30 Answer System 40 Business Execution System 50 User Terminals 110 Business Information Acquisition Department 120 System prompt generation unit 121 System prompt generation prompt 130 Log information acquisition unit 140 Judgment section 150 Update section 151 System prompt update prompt 310 User prompt reception unit 320 Answer generation part 330 Answer presentation section 340 Storage section 341 System Prompt 342 Log Information 410 Business Server 420 Business Database 421 Business Information 500 computers 501 Bus 510 processor 511 memory 512 storage devices 513 Input / Output Interface 514 network interface N1 Network

Claims

1. a business information acquisition unit that acquires business information from a business execution system; a system prompt generation unit that generates a system prompt based on the business information to cause the generative artificial intelligence to respond to a business inquiry from a user; a log information acquisition unit that acquires log information including an answer output by the generative artificial intelligence and an operation status of the user with respect to the answer; a determination unit that determines whether to update the content of the system prompt based on an update status of the business information or the log information; an update unit that updates the system prompt when the determination result indicates that the content of the system prompt should be updated; the determination unit, when determining that the log information includes a possibility of updating the business information, determines whether to update the system prompt by acquiring the latest business information. Information processing system.

2. the determination unit determines whether to update the system prompt based on at least one of an evaluation of the answer by the user, an occurrence status of an additional inquiry, or an accuracy score of the answer, which are included in the log information. The information processing system according to claim 1 .

3. the determination unit determines to update the system prompt when the user's evaluation of the answer included in the log information is negative a predetermined number of times or more, when the occurrence rate of additional inquiries exceeds a predetermined threshold, or when the accuracy score of the answer is less than a predetermined value. The information processing system according to claim 2 .

4. the determination unit, when an update of the business information is detected, determines whether the system prompt is updated based on the difference; the updating unit updates the system prompt corresponding to the difference. The information processing system according to claim 1 .

5. the determination unit determines whether to update the system prompt based on a reward signal derived from the user's evaluation included in the log information; the update unit updates the system prompt based on reinforcement learning using the reward signal. The information processing system according to claim 1 .

6. the updating unit further has an approval request function for receiving approval from an administrator for the system prompt to be updated, and applies the updated content when approval from the administrator is received. The information processing system according to any one of claims 1 to 5.

7. The computer Acquire business information from the business execution system, generating system prompts based on the business information to cause the generative artificial intelligence to respond to business inquiries from users; Acquire log information including the answer output by the generative artificial intelligence and the user's operation status regarding the answer; determining whether to update the content of the system prompt based on the update status of the business information or the log information; updating the system prompt if the determination result indicates that the content of the system prompt should be updated; When it is determined that the log information includes a possibility of updating the business information, the latest business information is acquired to determine whether the system prompt should be updated. Information processing methods.

8. a business information acquisition step of acquiring business information from a business execution system; a system prompt generating step of generating a system prompt based on the business information to cause the generative artificial intelligence to respond to a business inquiry from a user; a log information acquisition step of acquiring log information including an answer output by the generative artificial intelligence and an operation status of the user with respect to the answer; a determination step of determining whether to update the content of the system prompt based on the update status of the business information or the log information; an updating step of updating the system prompt when the determination result indicates that the content of the system prompt should be updated; the determining step, when it is determined that the log information includes a possibility of updating the business information, determines whether the system prompt should be updated by acquiring the latest business information; and program.

Images