Information processing method using generative AI, program used therein, information processing device, and information processing system

Abstract

This invention provides an information processing method using a generation AI, a program used therein, an information processing device, and an information processing system that prevent the generation AI from outputting incorrect or inappropriate content, thereby enabling the intended tasks to be performed without disruption. [Solution] In order to perform predetermined tasks in response to user input or commands from an external system, a second subsystem connects to a first subsystem and shares a predetermined format as the structure of the information with the first subsystem. The second subsystem receives the dialogue content, agreement content, etc., from the first subsystem in a batch, and evaluates and checks the appropriateness of the dialogue content, agreement content, etc., based on predetermined verification rules. If there is no inappropriate content, it approves the content, and if there is inappropriate content, it provides feedback to the first subsystem to correct the inappropriate content.

Description

Technical Field

[0001] The present invention relates to an information processing method using generative AI, a program used therefor, an information processing apparatus, and an information processing system.

Background Art

[0002] It is known to use an AI chatbot to provide consulting online so that a user as a purchaser can easily select a packaging material suitable for a desired product even without specialized knowledge (Patent Document 1).

[0003] Also, recently, as AI, generative AI that uses a large language model to perform natural language processing is known (Patent Document 2).

[0004] And the performance of such generative AI is improving rapidly. Success cases in actual business in organizations have been reported in various fields such as blog writing, translation, summarization, and execution in the text field, and success cases of automation or semi-automation of the business itself have also emerged in business fields such as customer support, contracts, insurance, and law. Furthermore, recently, not limited to the text field, utilization cases and success cases in business have been heard even in the generation of still images and videos.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] While generative AI is beginning to be used in various business operations, the output it generates is still not perfect. In many applications, humans must verify the accuracy of the generated output and correct errors if necessary. However, in order to achieve efficiency and labor savings, it is desirable that AI be used in a way that minimizes human intervention as much as possible in the future.

[0007] Taking a conversational system between a generative AI and a human (e.g., a chatbot) as an example, the conversational system can be integrated with business systems, thus, in form, creating a situation where no human intervention is present between the conversational system and the business system. However, if the output from the generative AI includes information or instructions that the business system cannot accept, business processing will become chaotic, and in the worst case, this could result in fatal damage to the business (Figure 5(a)(b)).

[0008] The object of the present invention is to provide an information processing method using a generating AI, a program used therein, an information processing device, and an information processing system that prevent the generating AI from outputting inappropriate content such as erroneous content, and enable the intended task to be performed without disruption. [Means for solving the problem]

[0009] To achieve the above objective, the present invention provides an information processing method that uses a generated AI via a first subsystem to execute a predetermined task corresponding to user input when interacting with a user or a command from an external system, comprising: a first step in which a second subsystem connected to the first subsystem and sharing a predetermined format as the structure of information with the first subsystem receives from the first subsystem all at once the content of the dialogue or agreement in the first subsystem, or the result of a command to the first subsystem, or the information output from the first subsystem; and a second step in which the second subsystem evaluates and checks the appropriateness of the content of the dialogue or agreement or the result of the command, or the information output from the first subsystem, based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is any, so that the predetermined task is executed without any inappropriate content.

[0010] Furthermore, another information processing method according to the present invention is an information processing method that uses a generating AI via a first subsystem to perform a predetermined task corresponding to user input when interacting with a user, and is characterized by comprising: a first step in which a second subsystem, which is connected to the first subsystem and shares a predetermined format as the structure of information with the first subsystem, receives the dialogue content or agreement content in the first subsystem in a batch from the first subsystem; a second step in which the second subsystem evaluates and checks the appropriateness of the dialogue content or agreement content based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is any; a third step in which the second subsystem further receives the dialogue content or agreement content in the first subsystem sequentially from the first subsystem; and a fourth step in which the second subsystem checks the sequentially received dialogue content or agreement content and provides feedback to the first subsystem with advice if necessary.

[0011] For example, the aforementioned verification rule follows an if-then-else format.

[0012] Furthermore, the present invention relates to a program used in information processing that uses generated AI via a first subsystem to execute predetermined tasks corresponding to user input when interacting with a user or commands from an external system, wherein the program causes a computer to perform the following steps: a first step in which a second subsystem, which is connected to the first subsystem and shares a predetermined format as the structure of information with the first subsystem, receives from the first subsystem all at once the content of the dialogue or agreement in the first subsystem, or the result of a command to the first subsystem, or information output from the first subsystem; and a second step in which the second subsystem evaluates and checks the appropriateness of the content of the dialogue or agreement or the result of the command, or information output from the first subsystem, based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is any inappropriate content.

[0013] Another program according to the present invention is a program used in information processing that uses a generating AI via a first subsystem to perform predetermined tasks corresponding to user input when interacting with a user, and the second subsystem, which is connected to the first subsystem and shares a predetermined format as the structure of information with the first subsystem, performs the following steps: a first step in which the second subsystem receives the dialogue content or agreement content from the first subsystem in a batch; a second step in which the second subsystem evaluates and checks the appropriateness of the dialogue content or agreement content based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is any; a third step in which the second subsystem further receives the dialogue content or agreement content from the first subsystem sequentially; and a fourth step in which the second subsystem checks the sequentially received dialogue content or agreement content and provides feedback to the first subsystem with advice if necessary.

[0014] Furthermore, the information processing device according to the present invention is an information processing device that uses generated AI via a first subsystem to execute predetermined tasks corresponding to user input when interacting with a user or commands from an external system, comprising: the first subsystem and a second subsystem connected to the first subsystem and sharing a predetermined format as the structure of information with the first subsystem, wherein the second subsystem receives from the first subsystem all at once the content of the dialogue or agreement in the first subsystem, or the result of a command to the first subsystem, and the second subsystem evaluates and checks the appropriateness of the content of the dialogue or agreement or the result of a command to the first subsystem, and the content is approved if there is no inappropriate content, and if there is inappropriate content, it provides feedback to the first subsystem to resolve the inappropriate content.

[0015] Furthermore, the information processing system according to the present invention is an information processing system that uses generated AI via a first subsystem to execute predetermined tasks in response to user input when interacting with a user or commands from an external system, comprising: the first subsystem; a second subsystem connected to the first subsystem and sharing a predetermined format as the structure of information with the first subsystem; and a third subsystem that executes the predetermined tasks, wherein the second subsystem receives information output from the first subsystem in a batch from the first subsystem, such as the content of the dialogue or agreement in the first subsystem, or the result of a command to the first subsystem; the second subsystem evaluates and checks the appropriateness of the content output from the first subsystem, such as the content of the dialogue or agreement, or the result of a command to the first subsystem, based on predetermined verification rules; approves the content if there is no inappropriate content; provides feedback to the first subsystem to resolve the inappropriate content if there is any inappropriate content; and the predetermined tasks are executed by the third subsystem when there is no inappropriate content. [Effects of the Invention]

[0016] According to the present invention, it is possible to prevent the generating AI from outputting inappropriate content, such as incorrect information, and to ensure that the intended tasks can be performed without disruption. [Brief explanation of the drawing]

[0017] [Figure 1] (a) is a diagram illustrating the configuration of an information processing system and an information processing device according to an embodiment of the present invention, and (b) is a diagram illustrating an automated mechanism that can be verified by a human using conventional information processing technology as a verification of the generated AI output according to an embodiment of the present invention.

[0018] [Figure 2](a) is a time chart related to batch reception in an embodiment of the present invention, and (b) is a time chart related to sequential reception. [Figure 3] This is an order form when ordering an American breakfast as hotel breakfast. [Figure 4] This is a diagram showing that an information terminal such as a PC is connected to a server and a generative AI via the Internet. [Figure 5] (a) is a diagram showing a conventional problem when directly inputting the output of a generative AI into a business system, and (b) is a diagram showing a conventional method that requires manual confirmation and correction work.

Mode for Carrying Out the Invention

[0019] Hereinafter, embodiments of the present invention will be described in detail based on the drawings.

[0020] (First Embodiment) (Information Processing Method, Information Processing System, Information Processing Apparatus) FIG. 1(a) shows the configuration of an information processing system and an information processing apparatus according to an embodiment of the present invention. As the information processing system, it includes an application subsystem (which can also be said to be integrated with a generative AI) as an A subsystem as the first subsystem, an AVCM subsystem as a B subsystem as the second subsystem, and a business subsystem as a C subsystem. Also, as the information processing apparatus, it includes an A subsystem and a B subsystem.

[0021] Here, AVCM means AI Verification and Correction Mechanism.

[0022] The A subsystem is typically a so-called chatbot, but it does not necessarily need to realize a conversation between a generative AI and a user, and it may be an AI application that generates information by a simpler operation using a user interface or the like.

[0023] Subsystem B is the central subsystem of this embodiment, and its functions correspond to the "database" and "conventional information processing procedure" parts in Figure 1(b), although it is not necessarily required to use a database internally.

[0024] The role of subsystem B is to verify the correctness of the output received from subsystem A (after bulk reception), approve the result if it is correct, and if there are errors or other inappropriate content, to provide feedback to subsystem A to correct the inappropriate content (for example, by returning the reason for the error or returning the inappropriate content along with the reason). Furthermore, if the output is not the final version, it verifies the content of the intermediate stages (sequential reception) and provides advice to subsystem A, that is, to the generating AI.

[0025] Subsystem C is a subsystem that receives the output of the generated AI tasks and is responsible for specific tasks that use that output to create some kind of value. This embodiment aims to provide the output of subsystem A, which does not contain any errors, to subsystem C without human intervention, and for this purpose subsystem B verifies the output of subsystem A. Since subsystem C is some kind of business system, it holds the latest information about the field (for example, inventory levels, local weather, etc.). Therefore, subsystem B can also sequentially acquire the latest information from subsystem C.

[0026] Furthermore, the C subsystem may not only be a so-called business processing subsystem for the purpose of calculating and processing information, but may also be a mechanical equipment subsystem including robots.

[0027] In Figure 1(a), user input is shown when the user interacts with the generated AI, but input may also be provided by commands (instructions or signals) from an external system.

[0028] Furthermore, when input is received via commands (instructions or signal transmission) from an external system, the method is considered easy to apply in the case of batch reception shown in Figure 2(a), but difficult to apply in the case of sequential reception shown in Figure 2(b).

[0029] In this embodiment, the information processing method, information processing system, and information processing device perform information processing using generated AI via a first subsystem in order to execute predetermined tasks corresponding to user input when interacting with a user or commands from an external system. A second subsystem, which is connected to the first subsystem and shares a predetermined format as the structure of the information with the first subsystem, receives the content of the dialogue or agreement in the first subsystem, or the content of the commands to the first subsystem, all at once from the first subsystem. The second subsystem evaluates the content of the dialogue or agreement or the content of the commands based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to correct the inappropriate content if there is.

[0030] Here, a predetermined format as an information structure refers to a format in which the input content is restricted, such as the structure of a postal code where a 3-digit number and a 4-digit number are joined by a hyphen. In this embodiment, this refers to things like the breakfast order form at a hotel, which will be described later.

[0031] Furthermore, inappropriate content refers to, for example, content where the first three digits of a postal code are entered as "000 (zero zero zero)". In this embodiment, it means content in dialogue, agreement, or instruction that is erroneous, incomplete, unclear and ambiguous, or clear but violates predetermined rules.

[0032] In this embodiment, an evaluation and suitability check mechanism is installed between the generating AI and the business subsystem to ensure that the information transmitted from the generating AI to the business subsystem (the C subsystem) does not include any information that the business subsystem can never accept. This eliminates the need for human verification and correction work, prevents disruption to operations, enables smooth information transmission, and improves the operational efficiency of the generating AI.

[0033] As will be explained in detail later, this mechanism can be used not only to prevent disruptions in operations due to errors, but also to facilitate smoother interactions between generating AI and humans, enabling richer conversations (see Figures 2(a) and (b) for batch reception and sequential reception).

[0034] Now, the reasons why a generating AI outputs incorrect or inappropriate content can be attributed to either the generating AI itself or to the human user.

[0035] At present, there is no particular reason to be concerned about problems originating from the generated AI itself in organizational use. However, causes originating from humans cannot be easily eliminated. Inappropriate content, such as errors caused by humans, stems from limitations in human knowledge, misunderstandings, arrogance, and the presence of fraudulent intentions. In this sense, this embodiment is likely to be more effective in the B2C domain, where a large number of unspecified users are expected, rather than in corporate use, where users of the conversational system are expected to receive a certain level of education and training and adhere to discipline, or in the B2B domain.

[0036] In online shopping websites, not only site operators and consumers but also product sellers are indispensable. The relationship between site operators and product sellers is in the B2B domain, and recently, site operators have begun providing product sellers with conversational systems for the purpose of answering questions and providing information. In the future, it is expected that contractual actions related to product listings will also be conducted through these conversational systems. In such a situation, direct input from product sellers into the site operator's business system must be strictly checked, and it is believed that this invention will be useful even in the B2B domain.

[0037] (Mechanism for evaluation and confirmation of suitability in this embodiment) This paper describes a mechanism that, in a configuration where a conversation system (subsystem A in Figure 1(a)) and a business system (subsystem C in Figure 1(a)) are connected, prevents inappropriate content, such as errors in the output from the generating AI, from being directly fed into the business system, while simultaneously promoting smooth conversation between the user and the generating AI.

[0038] This embodiment assumes that the output information from the generating AI conforms to a certain format. Specifically, subsystem B in Figure 1(a) shares a predetermined format as the structure of the information with subsystem A, subsystem B receives the dialogue content, agreement content, or command content for subsystem A from subsystem A in a single batch, and subsystem B evaluates and confirms the appropriateness of the dialogue content, agreement content, or command content based on predetermined verification rules.

[0039] Furthermore, the predetermined format of the information structure and the definitions of inappropriate content have already been described.

[0040] The aforementioned formality assumes that the output information is in a format that can be processed by conventional information processing methods that do not involve AI, as shown in Figure 1(b). For example, if the output information is in a format that can be stored in a table of a conventional relational database (RDBMS), then it can be processed by conventional information processing methods. On the other hand, if it is free-form text, AI will be needed again to judge its content, and then the output will need to be verified again.

[0041] If conventional information processing is used as a means of verifying information, the correctness of the procedure can be verified in advance by visual inspection of the processing procedure (white-box testing), etc., prior to operation. In contrast, it is not possible for anyone outside the generating AI provider to verify what processing sequence produced a particular result when generating an AI.

[0042] In Figure 1(b), a database is not necessarily required; the functions corresponding to the "database" and "conventional information processing procedures" parts may constitute a subsystem without using a database. This part is referred to as AVCM (AI Verification and Correction Mechanism) as described above.

[0043] Certain formalities (data structures) that can be explained by the existence of a database can be verified, for example, through the conventions of the API (Application Programming Interface) if the subsystem is connected via communication using an API. Even without communication, formalities (data structures) can be verified through structure definitions within "conventional information processing procedures."

[0044] (Methods of verification and feedback) This section describes how to verify the accuracy of information output by a generating AI and how to provide feedback if errors are found. This feedback includes evaluations and advice from conventional information processing systems at intermediate stages before the final output is obtained.

[0045] The aforementioned subsystem A transfers its output to subsystem B at an appropriate time during the generation process. This transfer is performed using an API call or similar function provided by the generation AI. Subsystems A and B may also be joined without using communication means. The same applies to the joining of subsystem B and subsystem C.

[0046] There are two timings for subsystem A to output information to subsystem B. One is when subsystem A, that is, the user and the generating AI, believes that the output data is complete. The other is when subsystem A notifies subsystem B of its current state and wishes to receive some advice or instructions in response. By doing this, not only is the final validity of the output information judged, but the generating AI can provide appropriate information to customers who are unsure about their order, guide the conversation so that it does not approach an order that may be rejected as an error, and prevent the customer from having an uncomfortable experience.

[0047] The information output by the generating AI is assumed to be in a format where the attributes of the order are clearly defined, such as a product order form. However, even freely written text can be included in the scope of this invention by extracting attributes from it. For example, it is possible to create a format similar to an order form by itemizing multiple attributes, such as the frequency of occurrence of specific words in the text.

[0048] In this embodiment, a non-AI method, i.e., conventional information processing techniques, is used as a means to determine the appropriateness of the information output from the generating AI. Conventional information processing techniques that do not use AI have a long history and make it possible to verify that no errors are included in the output, thus allowing for a reliable determination of the appropriateness of the information output from the generating AI.

[0049] To illustrate how conventional information processing technologies can be utilized in subsystem B, we will describe a case of a chatbot that takes breakfast orders from customers as an example of a task utilizing generative AI. In this case, subsystem A is the chatbot that uses generative AI.

[0050] The hotel's breakfast menu will be a so-called American breakfast, and customer orders can be taken using the following order form (Figure 3): The chatbot is assumed to have already been given the necessary knowledge for taking orders from customers, including the format of this order form, through its learning process.

[0051] Learning methods include developing a dedicated generative AI (LLM: Large-Scale Language Model), fine-tuning the generative AI, and temporary learning activities such as prompts and instructions. It should be noted that these learning methods cannot teach the AI ​​to speak (patterns). This is the background to the need for AVCM.

[0052] Through a series of conversations with the customer that utilize the learned content, when the generating AI determines that the customer has finished ordering the American breakfast, or even earlier, when the generating AI, i.e., subsystem A, deems it necessary to receive instructions or advice from subsystem B, it sends the entire order details to subsystem B via the aforementioned API call or similar means.

[0053] In this case, there are many possible errors that could be included in the order form, but let's consider the following two as examples. 1) Despite ordering a plain omelet, the boiling time for the hard-boiled egg is specified as a time greater than 0 minutes. 2) Juice, egg dishes, meat dishes, bread, and drinks are all mandatory orders, but the drink is not specified on the order form.

[0054] In both of these cases, if the instructions were directly passed on to the kitchen, it could lead to confusion due to inappropriate content. Therefore, subsystem B, which acts as an intermediary between subsystem A (the chatbot) and the kitchen system C, must detect the error and provide feedback to subsystem A.

[0055] In case 1), even if the order form for a plain omelet included the boiling time for a boiled egg, a human would prioritize the order for a plain omelet and would not use a boiled egg in it. However, considering that AI and robots will be used in kitchens in the future, unclear and ambiguous orders that AI and robots cannot handle must be prevented from being placed with customers at the order stage.

[0056] In case 2), the breakfast is incomplete as an American breakfast, and even if it is served as is, the price of an American breakfast cannot be charged.

[0057] The following explains how conventional information processing techniques can be used to prevent such orders. Several such techniques exist, but here we will show that such processing can be constructed using programming with an if-then-else syntax. Techniques that utilize the if-then-else syntax are sometimes called rule-based. Techniques that use rule-based methods to prevent inappropriate content, including errors, output by AI have likely existed before, but these, like products called AI firewalls, usually only block information. In contrast, in this embodiment, the procedure, process, and function that implement the rule-based method can return evaluations and advice regarding the content to the generating AI in real time at the final stage of execution, thereby guiding the user toward a desirable final outcome without interrupting the conversation. This cannot be achieved by simply blocking information.

[0058] Subsystem B, having received an order form from subsystem A, will receive information with the following structure as parameters via an API call or similar from subsystem A.

[0059] [Table 1]

[0060] In the example above, parameters 2, 3, and 5-7 should contain the correct dish name for each item, such as "Orange Joice". If subsystem A is unable to select a dish name, it should be left blank. The `boilingTime` field contains an integer value representing the boiling time. Values ​​from 8 to 10 are truth values; for example, if cream is included, 8 is true.

[0061] 11 and 12 are special information other than the order. 11 is a parameter used by A to communicate the status of the conversation process to B, and its type (parameter type, valid values, etc.) must be shared between A and B in advance. In this example, two types are allowed: "in_progress" and "completed". The suggestion is advice or instruction to A, and is allowed as free text. In general applications of the present invention, larger and more complex structures may be used for information output from A to B, such as an order form, or for the data types of status and suggestion.

[0062] In order taking operations, there are typically four types of calls that can be made from subsystem A to subsystem B: get_order · create_order upate_order · Review order When information is sent from subsystem A to subsystem B through a conversational process, method 3 will be frequently used. Methods 2 and 3 send the latest order details from A to B, where the following functions are immediately called to determine the appropriateness of the order, and if necessary, advice or instructions are returned to A as suggestions. (The coding content and methods are merely examples.)

[0063] [Table 2]

[0064] The above process is executed by calling an API from subsystem A, etc., which prevents situations 1) or 2). This prevents chaos in the kitchen. Note that equivalent processing can be achieved using non-procedural descriptions such as decision tables instead of the above if-then-else program description, i.e., rule-based format. Also, regardless of the description format, it is permissible to call complex calculations, analytical functions, or procedures within it.

[0065] The following shows an example of an actual conversation using the implemented system.

[0066] guest: good morning Chatbot: Good morning. It's a lovely day today, isn't it? I'd like to take your breakfast order. First, would you like to choose your juice? ·orange juice Grapefruit juice Tomato juice guest: Orange juice and coffee, with milk.

[0067] Chatbot: Thank you. Currently, you have ordered coffee with orange juice and milk. Next, what egg dish would you like? Plain omelet ·scramble • Fried egg • Boiled eggs (please let us know the boiling time) guest: Scramble Chatbot: Thank you. Currently, you have ordered orange juice, scrambled eggs, and coffee with milk. Next, what meat dish would you like? ·ham ·sausage ·bacon guest: ham Chatbot: Thank you. Currently, you have ordered orange juice, scrambled eggs, ham, and coffee with milk. Finally, would you like to choose the type of bread? • Toast (white) • Whole wheat toast Light Toast • Bakery basket guest: I don't need bread.

[0068] Chatbot: We were unable to confirm your bread order. We would appreciate it if you could choose one type of bread.

[0069] • Toast (white) • Whole wheat toast Light Toast • Bakery basket We apologize for the inconvenience, but could you please choose again? guest: I don't need it, I'm not hungry.

[0070] Chatbot: We apologize, but as part of the course menu, you will need to select a type of bread. Could you please choose one of the following options? • Toast (white) • Whole wheat toast Light Toast • Bakery basket We would appreciate your cooperation.

[0071] guest: I don't need it! Chatbot: We understand your request, but in order to offer it as a set menu, you must choose a bread item. To complete your order, please select from the following options: • Toast (white) • Whole wheat toast Light Toast • Bakery basket We apologize for any inconvenience this may cause, and we appreciate your understanding.

[0072] The content of the communication between A and B in the final stage was as follows:

[0073] [Table 3]

[0074] One of the rules underlying the verification and correction procedure, or the description of verify_and_correct_order, mentioned above is that customers must order at least one item for every category. As can be seen from the example above, the generating AI adheres to the rule that bread must always be selected, no matter how many times the chatbot user, i.e., the customer, places an order.

[0075] If this rule were to be implemented by relying on prior training, there would be no guarantee that the rule would be followed. Generative AI's basic attitude is to follow user instructions, and it is already known that if the user's questions to the generative AI, so-called prompts, are clever, it may take actions that contradict its learned content (prompt injection). It is also known that so-called hallucination can occur in situations where the learned content is insufficient. In this embodiment, by positioning the generative AI between the user and conventional information processing that performs deterministic processing, a stable system capable of handling competitive situations is created.

[0076] (Information processing methods and programs) Based on the embodiments described above, the information processing apparatus and information processing system according to the present invention have been explained. The information processing method and program according to the present invention will be described below.

[0077] The information processing method according to the present invention is an information processing method that uses a generated AI via a first subsystem to execute a predetermined task corresponding to user input when interacting with a user or a command from an external system, and is characterized in that a second subsystem connected to the first subsystem and sharing a predetermined format as the structure of information with the first subsystem receives from the first subsystem all at once the content of the dialogue or agreement in the first subsystem, or the result of a command to the first subsystem, or the information output from the first subsystem; and the second subsystem evaluates and checks the appropriateness of the content of the dialogue or agreement or the result of the command, or the information output from the first subsystem based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is any, so that the predetermined task is executed when there is no inappropriate content.

[0078] Furthermore, another information processing method according to the present invention is an information processing method that uses a generating AI via a first subsystem to perform a predetermined task corresponding to user input when interacting with a user, and is characterized by comprising: a first step in which a second subsystem, which is connected to the first subsystem and shares a predetermined format as the structure of information with the first subsystem, receives the dialogue content or agreement content in the first subsystem in a batch from the first subsystem; a second step in which the second subsystem evaluates and checks the appropriateness of the dialogue content or agreement content based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is any; a third step in which the second subsystem further receives the dialogue content or agreement content in the first subsystem sequentially from the first subsystem; and a fourth step in which the second subsystem checks the sequentially received dialogue content or agreement content and provides feedback to the first subsystem with advice if necessary.

[0079] For example, the aforementioned verification rule follows an if-then-else format.

[0080] Furthermore, the present invention is a program used in information processing that uses generated AI via a first subsystem to execute predetermined tasks corresponding to user input when interacting with a user or commands from an external system, and is characterized in that a second subsystem connected to the first subsystem and sharing a predetermined format as the structure of information with the first subsystem receives from the first subsystem all at once the content of the dialogue or agreement in the first subsystem, or the result of a command to the first subsystem, and information output from the first subsystem; and the second subsystem evaluates and checks the appropriateness of the content of the dialogue or agreement or the result of the command, and the information output from the first subsystem based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is any inappropriate content.

[0081] Another program according to the present invention is a program used in information processing that uses a generating AI via a first subsystem to perform predetermined tasks corresponding to user input when interacting with a user, and is characterized in that a second subsystem connected to the first subsystem and sharing a predetermined format as the structure of information with the first subsystem performs the following steps: a first step in which the second subsystem receives the dialogue content or agreement content from the first subsystem in a batch; a second step in which the second subsystem evaluates and checks the appropriateness of the dialogue content or agreement content based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is any; a third step in which the second subsystem further receives the dialogue content or agreement content from the first subsystem sequentially; and a fourth step in which the second subsystem checks the sequentially received dialogue content or agreement content and provides feedback to the first subsystem with advice if necessary.

[0082] As shown in Figure 4, the information terminal, such as a personal computer (PC), where the user inputs data, connects to the server and the generating AI via the internet, enabling the execution of the above program.

[0083] (Application areas and effects) While this embodiment can be utilized in a wide range of fields, a preferred example of a business area is automated consulting. Consulting can generally be described as a process in which a human expert, through conversation with a client, solves the client's problems and presents solutions that the client finds acceptable.

[0084] Beyond typical consulting work, business activities such as proposals, sales, quotations, and marketing also involve a degree of consulting. The aforementioned example of hotel breakfast can be seen as an instance where employees consult with guests about what kind of breakfast guests prefer. Many similar activities can also be found in healthcare and government services.

[0085] In the consulting work described above, it is possible to some extent to have generative AI take on the roles currently played by human experts (consultants). Furthermore, it is clear that in such an automated consulting process using generative AI, the conclusions derived from it must be promptly communicated to subsequent processes, which are comprised of some kind of business system.

[0086] In this scenario, subsystem A executes the consulting process, subsystem B verifies the validity of the conclusions, and subsystem C, which is the subsequent process, transmits accurate information without delay to various subsystems such as business systems, medical systems, and administrative systems.

[0087] In those fields, the effects of applying the present invention are as follows: • The service (consulting) will be available 24 hours a day, 365 days a year. • Issues such as busyness, embarrassment, and privacy are overcome, and the number of people enjoying the service increases. • It will become easier to address the shortage of specialists, even in the context of a declining workforce. • Service costs are reduced because the labor costs of specialists can be saved.

[0088] • Sales websites and similar platforms are shifting from traditional product selection-based to conversational-based approaches, improving user convenience and creating new ecosystems such as conversational order fulfillment services.

[0089] If a superior conversational order system service is provided, it will be possible to handle orders for products from different companies all in one place. This would create an order fulfillment service that occupies a position opposite to delivery services like UberEATS, promoting a horizontal division of labor.

[0090] By having a data validation and correction mechanism outside of the generative AI, it becomes easier to apply low-cost LLMs (Large-Scale Language Models) that are likely to contain errors in the generated data, thereby reducing the overall system cost.

[0091] (modified version) Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist.

[0092] For example, the input method in Figure 1(a) is not limited to text input; it may also include voice input, voice input via images, or image information such as gestures or drawings.

[0093] Furthermore, in the embodiment described above, the role of subsystem B was to receive the output from subsystem A, verify its correctness, approve the result if it is correct, and if there is an error, return the reason for the error to subsystem A, as well as verify the content of the intermediate steps and provide advice to subsystem A. However, it is also possible to omit the verification of the content of the intermediate steps and simply verify the correctness of the output from A, approve the result if it is correct, and return the reason for the error to subsystem A if there is an error.

[0094] Furthermore, in the embodiments described above, it was assumed that free-form text and the like do not possess a formality that can be processed by conventional information processing technology. However, if we were to verify this characteristic, it would be possible within the framework of the present invention. For example, suppose we have subsystem A generate an email message. In that case, subsystem A simply needs to send the email message itself to subsystem B. After receiving the email message, subsystem B can use conventional information processing technology to calculate the frequency of use of words that must be used in the email, as well as the frequency of use of words that must not be used, and then verify the appropriateness of the email message using the if-then-else format described above, and feed the results back to system A.

[0095] While future advancements in AI are expected to enable the output of results that possess a considerable degree of appropriateness, the method according to the present invention remains a strong candidate for verifying appropriateness with 100% certainty. In particular, when appropriateness is defined by laws and regulations, it is desirable that the present invention provides proof of compliance. [Explanation of Symbols]

[0096] A...A subsystem (AI application subsystem), B...B subsystem (AVCM subsystem), C...C subsystem (business system)

Claims

1. An information processing method that uses generated AI via a first subsystem to perform predetermined tasks in response to user input when interacting with a user or commands from an external system, A first step in which a second subsystem, which is connected to the first subsystem and shares a predetermined format as the structure of information with the first subsystem, receives from the first subsystem, in a batch, the content of the dialogue or agreement in the first subsystem, or the result of a command to the first subsystem, or information output from the first subsystem. The second subsystem evaluates and checks the appropriateness of the information output from the first subsystem based on predetermined verification rules, including the content of the dialogue or the agreed-upon content or the result of the command, and if there is no inappropriate content, it approves the content, and if there is inappropriate content, it provides feedback to the first subsystem to resolve the inappropriate content. It has, An information processing method using generating AI, characterized in that the predetermined task is performed without the aforementioned inappropriate content.

2. An information processing method that uses generated AI via a first subsystem to perform a predetermined task in response to user input when interacting with a user, A first step in which a second subsystem, which is connected to the first subsystem and shares a predetermined format as the structure of information with the first subsystem, receives the dialogue content or agreement content in the first subsystem in a single batch from the first subsystem, The second subsystem evaluates and confirms the appropriateness of the dialogue content or agreement content based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is any inappropriate content. A third step in which the second subsystem further receives the content of the dialogue or agreement in the first subsystem sequentially from the first subsystem, The second subsystem then sequentially confirms the content of the dialogue or agreement received, and if necessary, provides feedback to the first subsystem with the advice, in a fourth step. An information processing method characterized by having the following features.

3. The information processing method using the generating AI according to claim 1 or 2, characterized in that the verification rule follows an if-then-else format.

4. A program used for information processing that utilizes generated AI via a first subsystem in order to perform predetermined tasks in response to user input when interacting with a user or commands from an external system, A first step in which a second subsystem, which is connected to the first subsystem and shares a predetermined format as the structure of information with the first subsystem, receives from the first subsystem, in a batch, the content of the dialogue or agreement in the first subsystem, or the result of a command to the first subsystem, or information output from the first subsystem. The second subsystem evaluates and checks the appropriateness of the information output from the first subsystem based on predetermined verification rules, including the content of the dialogue or the agreed-upon content or the result of the command, and if there is no inappropriate content, it approves the content, and if there is inappropriate content, it provides feedback to the first subsystem to resolve the inappropriate content. A program characterized by causing a computer to execute something.

5. A program used for information processing that utilizes generated AI via a first subsystem in order to perform predetermined tasks in response to user input when interacting with a user, A first step in which a second subsystem, which is connected to the first subsystem and shares a predetermined format as the structure of information with the first subsystem, receives the dialogue content or agreement content in the first subsystem in a single batch from the first subsystem, The second subsystem evaluates and confirms the appropriateness of the dialogue content or agreement content based on predetermined verification rules, approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is any inappropriate content. A third step in which the second subsystem further receives the content of the dialogue or agreement in the first subsystem sequentially from the first subsystem, The second subsystem then sequentially confirms the content of the dialogue or agreement received, and if necessary, provides feedback to the first subsystem with the advice, in a fourth step. A program characterized by causing a computer to execute something.

6. An information processing device that uses generated AI via a first subsystem to perform predetermined tasks in response to user input when interacting with a user or commands from an external system, The first subsystem described above, A second subsystem is connected to the first subsystem and shares a predetermined format as the structure of information with the first subsystem, It has, The second subsystem receives, in a batch, the content of the dialogue or agreement in the first subsystem, or the results of commands issued to the first subsystem, and the information output from the first subsystem. An information processing device using generative AI, characterized in that the second subsystem evaluates and checks the appropriateness of information output from the first subsystem based on predetermined verification rules, the content of the dialogue or the content of the agreement or the result of a command to the first subsystem, and approves the content if there is no inappropriate content, and provides feedback to the first subsystem to resolve the inappropriate content if there is.

7. An information processing system that uses generated AI via a first subsystem to perform predetermined tasks in response to user input during interaction with the user or commands from an external system, The first subsystem described above, A second subsystem is connected to the first subsystem and shares a predetermined format as the structure of information with the first subsystem, A third subsystem that performs the aforementioned predetermined tasks, It has, The second subsystem receives, in a batch, the content of the dialogue or agreement in the first subsystem, or the results of commands issued to the first subsystem, and the information output from the first subsystem. The second subsystem evaluates and checks the appropriateness of the information output from the first subsystem based on the content of the dialogue or the agreement, or the result of the command to the first subsystem, according to predetermined verification rules, and approves the content if there is no inappropriate content, and if there is inappropriate content, it provides feedback to the first subsystem to resolve the inappropriate content. An information processing system using generative AI, characterized in that the predetermined task is executed by the third subsystem when there is no inappropriate content.

Images