system

Abstract

Provide a system. [Solution] An information storage means for collecting expertise information and usage information regarding multiple types of automatic response elements using multiple generative learning models and storing the expertise information and usage information for each of the multiple types of automatic response elements; an evaluation means for inputting an instruction statement for providing performance information and function information to the generation learning model for each of the plurality of types of automatic response elements, and evaluating the performance information and function information based on the response; an analysis means for analyzing user evaluation information regarding the plurality of types of automatic response elements and calculating a satisfaction index; an analysis means for integrating the expertise information, use information, performance information, function information, and satisfaction index for each of the plurality of types of automatic response elements and performing a comparative analysis of the plurality of types of automatic response elements; a selection means for acquiring business process information and purpose of use information of a user, and selecting an optimum automatic response element based on the business process information and purpose of use information from the comparative analysis results of the analysis means; a providing means for automatically proposing a method of using and setting conditions for the optimum automatic response element selected by the selecting means, and providing consulting to a user using the optimum automatic response element; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, a variety of automated response elements (hereafter referred to as "automated response elements") have been put into practical use, but each has different specializations, uses, performance, and functions, making it difficult for users to select the appropriate automated response element based on their own business process and purpose of use, and to set up the optimal usage and configuration conditions.In addition, while there is a demand for objective evaluation based on user reviews and satisfaction indicators, this information is dispersed and changes dynamically, making it difficult to perform appropriate comparative analysis or automated consulting. [Means for solving the problem]

[0005] In this invention, the server includes means for accumulating information about automatic response elements using multiple generative learning models, means for acquiring and evaluating performance information, function information, and satisfaction indexes for multiple types of automatic response elements, and means for selecting the optimal automatic response element based on the user's business process information and purpose of use information and for providing automatic consulting. This enables users to accurately and efficiently select and set automatic response elements and utilize them in an optimal manner for their own business process.

[0006] An "automatic response element" is an element that has the function of automatically generating a response in response to a request from a user. A "generative learning model" is a trained data processing structure that learns tasks such as language processing and information extraction and generates responses based on instructional statements. "Expertise information" is information that indicates in what field or area the autoresponder element is specialized or useful. "Usage information" is information that indicates the purpose or usage for which the automatic response element is primarily suited. "Performance information" is information that includes quantitative or qualitative evaluation indicators such as the processing speed, response accuracy, and scalability of an automatic response element. "Feature information" is information about specific functions or operational characteristics that an autoresponder element may provide. The "satisfaction index" is an evaluation value that indicates the user's satisfaction with the automatic response element, obtained by analyzing user reviews and feedback. "Business process information" is information that indicates the workflow and processing procedures when a user uses the automatic response element. "Intent of use information" is information about the purpose or goal that a user is trying to achieve by using an automatic response element. An "instruction statement" is a statement that is input to the generative learning model to extract, evaluate, analyze, suggest, or perform other specific information. The "evaluation means" is a means having a processing function for comparing and evaluating the performance information and function information of the automatic response element based on the response obtained from the generative learning model. The "analysis means" is a means having a processing function of analyzing text information such as user reviews and calculating a satisfaction index. The "analysis means" is a means having a processing function for comprehensively comparing and analyzing the expertise information, use information, performance information, function information and satisfaction index. The "selection means" is a means having a processing function of selecting the optimum automatic response element from the comparative analysis results obtained by the analysis means, based on the business process information and the purpose of use information. The "provision means" is a means having a processing function of automatically presenting the usage method and setting conditions of the optimum automatic response element selected by the selection means and providing consulting to the user. The "information storage means" is a means having a processing function for recording and storing the expertise information, application information, performance information, function information, and satisfaction index related to a plurality of types of automatic response elements. The "control means" is a means having a processing function of dynamically changing the instruction sentences input to the generative learning model in response to updates of information by the evaluation means and analysis means, and periodically reevaluating information regarding automatic response elements. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 2 is a sequence diagram showing the flow of processing of the system in this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0009] <System configuration>

[0010] An example of an embodiment of the present invention will be described below. The present invention relates to a system for evaluating, analyzing, selecting, and proposing multiple automatic response elements, and this system is mainly composed of an information processing environment including a server and a terminal. The system of this embodiment includes a server and a terminal. The functional configuration and processing flow of the server and terminal will be described below.

[0011] (1) Overall system configuration The system is configured to accumulate information on multiple types of auto-response elements, select the optimal auto-response element that matches the user's business process and purpose, and automatically provide consulting using that auto-response element. Here, an auto-response element is an element that can respond to requests from users using a specific generative learning model, and may have different specialties and uses such as text processing, image processing, and voice processing.

[0012] (2) Server configuration The server has an information storage unit, an evaluation unit, an analysis unit, a selection unit, a provision unit, and, if necessary, a control unit. These units are realized by functional modules that are program-controlled within the server.

[0013] Specifically, the server has an internal database and an information storage means for storing expertise information, usage information, performance information, function information, and satisfaction indexes for multiple types of automatic response elements. The server communicates with the generative learning model via an interface that can be connected to an external device, and acquires and updates the information for each automatic response element by inputting instruction statements into the generative learning model. The instruction statements are generated internally on the server, and are constructed as text that the generative learning model can respond to.

[0014] The server also uses an evaluation means to obtain performance information and function information for each automatic response element. The evaluation means inputs instructions for performance evaluation into the generative learning model, analyzes the response, and quantifies evaluation items such as processing speed, response accuracy, and scalability. The analysis means inputs instructions for analysis into the generative learning model using user reviews and feedback information as input, and extracts a satisfaction index from the response. This allows qualitative user reviews to be accumulated as a quantitative satisfaction index.

[0015] The server uses the analysis means to perform an integrated comparative analysis of the expertise information, application information, performance information, function information, and satisfaction index. The analysis means generates a relative evaluation report using data on each automatic response element stored in the information storage means. The analysis results are used by the selection means, described below, to select the optimal automatic response element. Furthermore, if a control means is provided, instructions can be dynamically regenerated in response to information updates by the evaluation means and analysis means, and periodic re-evaluations can be performed. This configuration enables the server to keep information on automatic response elements always up to date.

[0016] The selection means refers to the business process information and purpose of use information entered by the user from the terminal, and automatically selects the most suitable auto-response element from the results of the comparative analysis by the analysis means. In this process, the most suitable auto-response element is determined according to the user's requirements, such as "business that requires rapid proofreading of text" or "design process whose main purpose is image generation."

[0017] The provision means automatically inputs the usage method and optimal setting conditions for the optimal auto response element identified by the selection means into the generative learning model as instructions, and presents consulting content to the user using the response results. This consulting automatically provides advice such as what parameters the auto response element should be used with and in what situations it will be most effective.

[0018] (3) Terminal configuration The terminal is a device operated by the user for inputting and outputting information, and includes, for example, a display, keyboard, mouse, and touch panel. The terminal is connected to the server via a network, and the user uses the terminal to send business process information and purpose of use information to the server. The terminal also receives analysis reports, comparison results, satisfaction indicators, recommended auto-response elements and their setting conditions, etc. sent from the server, and displays them on the screen for the user.

[0019] Users can easily select the auto-response element that best suits their workflow based on the optimal agent information received via their terminal. Furthermore, by referring to the consulting content provided by the means of provision, optimal settings and usage can be immediately reflected.

[0020] (4) An example of processing flow First, the user uses a terminal to input their own business process information and purpose of use information. This information is sent to the server. The server references information on multiple types of auto-response elements stored in the information storage means, and updates performance information, function information, and satisfaction indicators by inputting instructions into the generative learning model as needed. The updated information is compared and analyzed by the analysis means, and based on the results, the selection means identifies the optimal auto-response element. The selection results are presented to the user via the provision means, and the user can refer to the proposed method of using the auto-response element and the optimal setting conditions. This allows the user to immediately improve business efficiency.

[0021] (5) Variations In this embodiment, the server and terminal may be configured as an integrated unit, or may be distributed using a cloud environment. Furthermore, the instructions sent to the generative learning model can be dynamically optimized using natural language processing technology, and by adding or changing different evaluation criteria and analysis methods, the system can be easily applied to other types of automatic response elements. Furthermore, the types of performance information and functional information to be evaluated, the method for calculating the satisfaction index, the content and granularity of the consulting, and so on can be appropriately changed according to practical requirements.

[0022] According to this embodiment configured as described above, users can automatically and efficiently select appropriate elements from a vast variety of auto-response elements and easily acquire their usage methods and optimal settings, resulting in a significant improvement in productivity and convenience in the user's business process.

[0023] <Example of system configuration>

[0024] Each component will be described in detail below as an example of an embodiment of the present invention. The system of this embodiment is an information processing environment primarily composed of a server and a terminal. The server accumulates and analyzes information about multiple automatic response elements, automatically selects the most appropriate automatic response element based on the business process information and purpose of use information specified by the user, and presents its usage method and setting conditions. On the other hand, the terminal functions as an input / output device operated by the user, sending and receiving information to and from the server.

[0025] (1) Server The server has information storage means, evaluation means, analysis means, selection means, provision means, and, if necessary, control means. Each of these means can be realized as a program (software module) that runs inside the server, and the server can run on any information processing platform, such as a cloud environment or within a local network.

[0026] (a) Information storage means The information storage means is constructed using a database and records expertise information, application information, performance information, function information, and satisfaction index for multiple types of automatic response elements. For example, the information storage means can use a relational database, in which agent IDs, expertise (e.g., text processing, image processing, voice processing, etc.), main applications (e.g., text proofreading, image generation, voice recognition, etc.), performance parameters (e.g., processing speed, response accuracy, scalability index), and user satisfaction scores (e.g., on a scale of 1 to 5) are stored in a table structure.

[0027] By accumulating this information, the server can hold quantitative and qualitative information about the characteristics of each automatic response element. The information storage means continuously updates the data in response to updates in the results of the evaluation means and analysis means (described later), and always holds the latest and most consistent information.

[0028] (b) Evaluation tools The evaluation means has the function of acquiring and updating performance information and function information of each automatic response element using the generative learning model. Specifically, the server inputs instructions to the generative learning model and acquires information such as the processing speed, response accuracy, and scalability of each automatic response element. At this time, the instructions describe the evaluation criteria and evaluation method, and the generative learning model outputs a response corresponding to this. The server analyzes the output and records the evaluation results in the information storage means.

[0029] This evaluation means allows the server to periodically grasp the performance status of the automatic response element, and makes it possible to compare the merits and demerits of a plurality of automatic response elements.

[0030] (c) Analysis means The analysis means has a function of analyzing text-based evaluation information such as user reviews and feedback, and calculating it as a quantitative satisfaction index. The analysis means sends instructions to input review sentences and feedback sentences to the generative learning model, and analyzes the satisfaction ratings and summary information obtained in response.

[0031] This process allows reviews that were previously merely subjective impressions to be accumulated as quantitative scores (e.g., a rating of 1 to 5) and combined with other evaluation indicators to perform comprehensive analysis.

[0032] (d) Analytical tools The analysis means has the function of comprehensively comparing and analyzing the expertise information, application information, performance information, function information, and satisfaction index stored in the information storage means. At this time, the analysis means compares multiple automatic response elements with each other and generates a report that ranks them based on specific evaluation criteria (e.g., emphasis on processing speed, emphasis on accuracy, emphasis on user satisfaction, etc.) and extracts the characteristic advantages and disadvantages of each agent. These analysis results are used by the selection means (described later) to select the optimal automatic response element and by the provision means to generate consulting content.

[0033] (e) Selection method The selection means has a function of selecting the optimum auto-response element based on the comparative analysis results obtained from the analysis means, by referring to the business process information and the purpose of use information input by the user from the terminal. For example, if the user presents a request such as "I want to perform text proofreading processing quickly" or "I want to prioritize the quality of image generation," the selection means compares the request with the results of the analysis means and identifies the auto-response element that best suits the request.

[0034] The selection method involves inputting instructions into a generative learning model to execute a procedure to extract the optimal agent that meets the conditions, allowing users to easily find an agent that suits their business and purpose from the vast number of automatic response elements available.

[0035] (f) Means of provision The provision means has a function to input the usage methods and setting conditions for the selected optimal auto-response elements as instructions to the generative learning model and extract them from the response results. This allows users not only to know the optimal elements but also to automatically obtain specific instructions and recommended settings for making the most of those elements.

[0036] For example, the providing means inputs instructions such as "Show the optimal parameters for proofreading text using this automatic response element" into the generative learning model, and based on the output results, provides the user with suggestions such as "If the text length is less than X characters, using parameter Y will improve accuracy."

[0037] (g) Control measures (optional) The control means has the function of dynamically changing the update frequency of information by the evaluation means and analysis means and the rules for generating instruction sentences, and controls the entire system so that the latest and most appropriate evaluations are always performed. For example, when new reviews are accumulated, it is possible to automatically start the analysis means, regenerate instruction sentences, and calculate the latest satisfaction index.

[0038] (2) Terminal The terminal is an information processing device equipped with a display device and an input device that can communicate with the server, and the user transmits business process information and purpose of use information to the server via the terminal. The terminal also receives analysis results, selection results, consulting information on optimal usage methods and setting conditions, etc. transmitted from the server, and presents them to the user by displaying them on the screen.

[0039] A web browser, mobile application, or dedicated user interface application runs on the terminal, allowing users to freely operate the system. Information is input from the terminal to the server using the desired interface, such as text input, pull-down menu selection, or file upload. The terminal also displays acquired analysis reports, quantified satisfaction indicators, and comparison results in intuitive graphs and charts, providing an environment in which users can easily understand the optimal auto-response elements.

[0040] As explained above, this embodiment exchanges information between the server and the terminal, and inputs instructions into the generative learning model to analyze and evaluate auto-response elements, making it possible to provide the optimal auto-response element that meets the user's needs. This allows users to easily select an auto-response element that matches their own business process and instantly learn how to use it and its setting conditions, thereby contributing to improved business efficiency and productivity.

[0041] <System Operation>

[0042] The server operates in a cloud environment (e.g., a virtual machine running on a general-purpose information processing device) and realizes programs by executing a web application framework (e.g., Flask (registered trademark) or Django (registered trademark)) built using Python (registered trademark) on a Linux (registered trademark) operating system. The server accumulates information on multiple types of automatic response elements using a relational database such as MySQL (registered trademark) or PostgreSQL (registered trademark), and provides this data to an external generative learning model via instructions to obtain performance information, function information, satisfaction indicators, etc.

[0043] The server sends instructions to the generative learning model (for example, an external large-scale language processing platform API) and analyzes the response to evaluate performance and analyze reviews. The server issues an HTTP request within a Python program and receives a response from the model in JSON or text format. The server then analyzes the response using Python, stores the information in a database, and uses it to select the optimal automated response element based on the user's business process information and purpose of use information. The server then converts the selection results and analysis report into a format such as HTML or JSON and sends it to the terminal.

[0044] The terminal is a general information processing device equipped with a display means such as a web browser and an input means. The terminal receives the analysis results and the optimal auto-response element suggestions sent from the server and presents them to the user via a graphical user interface. The display layout is controlled on the terminal using a scripting language such as JavaScript (registered trademark), and the user can input their purpose and request and send it back to the server.

[0045] Users can input their own business process information and purpose of use information using a terminal, and refer to the results analyzed and selected by the server based on this information. Users can check the presented agent selection results and consulting information and reflect them in their work, enabling them to make effective use of automatic response elements.

[0046] As a specific example, the server can send the following directive to the generative learning model: "For Agent_A, Agent_B, and Agent_C, summarize their respective specialties and main uses in JSON format." "Calculate the processing speed, response accuracy, and scalability of Agent_A, Agent_B, and Agent_C on a 5-point scale and return the results in JSON format." "Calculate your satisfaction with the agent (score 1-5) based on the user review below, and briefly explain the reasons. Review: 'Agent_A's grammar is highly accurate and very helpful, but sometimes the process is slow.'" "Output an analysis report for Agent_A, Agent_B, and Agent_C that combines expertise, usage, performance evaluation, and user satisfaction indicators. In conclusion, indicate the recommended situations for each agent." "Choose one agent that is best suited to the following workflow: 'Mainly proofreading, but needs to be corrected immediately for short sentences'. Explain your reasons in JSON format. The agents in question are Agent_A, Agent_B, and Agent_C."

[0047] Using these directives, the server analyzes responses from the generative learning model and updates information by executing insert statements into multiple database tables, etc. This allows the server to dynamically generate processing results suited to user requests received from the terminal, and enables users to receive suggestions for optimal auto-response elements and their usage settings via their terminal.

[0048] <Example of system operation (Figure 1)>

[0049] Step 1: The user enters their own business process information and purpose of use information into an input form on the terminal and issues a request to send to the server. Here, the user's input (e.g., "I would like to have my text proofread quickly") is captured by the terminal, which then sends it to the server as an HTTP request (e.g., JSON format). The input data is the user's request data, and by passing it to the server, the server can obtain basic information for selecting the optimal auto-response element that matches the request in the subsequent processing step.

[0050] Step 2: The terminal sends the business process information and purpose of use information entered by the user to the server. The input is data including the business content and purpose (e.g., "proofreading" and "instant correction of short sentences"), which the terminal transfers to the server using the HTTP protocol. The server receives this request, processes the data (e.g., JSON parsing) to extract the business process information and purpose of use information, and prepares it for comparison with the automatic response element information stored internally. The user request data is stored internally as output.

[0051] Step 3: The server references multiple types of automatic response element information stored in an information storage means and dynamically generates prompts for the generative AI model to evaluate its performance and identify its intended use. The input here is a list of agent candidates (e.g., Agent_A, Agent_B, Agent_C) stored internally on the server, and based on this, the server sends prompts such as "Summarize your expertise and intended use in JSON format" to the generative AI model. The output is a response text obtained from the generative AI model, and the server processes this response using Python string processing, JSON parsing, and other data processing before storing it in a database.

[0052] Step 4: The server analyzes the responses received from the generative AI model and extracts performance information (e.g., processing speed, response accuracy) and satisfaction indicators. The input is the response text from the generative AI model, and the server performs natural language analysis and regular expression processing using Python to perform numerical evaluation and tagging. As a result, performance information and satisfaction indicators are extracted and recorded as output in the server's internal database using insert statements. This enables the server to accumulate quantitative evaluation data for multiple automated response elements.

[0053] Step 5: The server performs an integrated comparative analysis of the expertise information, usage information, performance information, function information, and satisfaction indexes stored in the database. The input is evaluation data for multiple agents in the database, and the server executes SQL queries in a Python program to obtain this information and performs statistical calculations and ranking processes. As a result (output), the server can identify the agent candidates that are best suited to the purpose and clarify the characteristics of those agents (e.g., the fastest grammar proofreading ability).

[0054] Step 6: After identifying the optimal agent, the server generates a prompt to obtain usage instructions and setting conditions for that agent, and sends it back to the generative AI model. The input is the selection result (the name of the optimal agent), and the server generates a prompt such as "Indicate the optimal parameters when using Agent_A for proofreading purposes." The response (output) from the generative AI model is text describing the setting conditions and specific usage procedures, which the server analyzes and formats into a format (JSON or HTML) that can be sent back to the terminal.

[0055] Step 7: The server sends the formatted analysis report, the optimal agent selection results, and their usage and configuration conditions to the terminal. The input is the text information obtained and analyzed from the generative AI model and the database information within the server, which the server returns to the terminal in JSON or HTML format as an HTTP response. The output can be formatted and displayed on the terminal using HTML templates or JavaScript for visualization, allowing the user to check the optimal agent and configuration procedures on the screen.

[0056] Step 8: The terminal displays the optimal agent candidates and their settings received from the server on a GUI. Inputs include HTML, JSON, and analysis results sent from the server, and the terminal uses JavaScript to display tables and graphs, creating an interface that allows users to easily understand the results. The output is a visual presentation to the user, who can refer to this information to make agent selection decisions and apply them to business processes.

Claims

[Claim 1] an analysis means for analyzing user evaluation information input about a plurality of types of agents that automatically generate responses in response to requests from users using a specific generation learning model, and calculating a satisfaction index using a generation AI model for each of the agents; an information storage means for storing, for each of the plurality of types of agents, expertise information, use information, performance information, and the satisfaction index related to the agent; a selection means for acquiring the business process information and the purpose of use information of the user, and selecting an agent that is optimal for the business process information and the purpose of use information using the generative AI model based on the expertise information, the application information, the performance information, and the satisfaction index for each of the plurality of types of agents stored in the information storage means; generating a prompt sentence for obtaining a usage method and setting conditions for the agent selected by the selection means, and inputting the prompt sentence into the generation AI model; providing means for acquiring a text describing the method of use or the setting conditions, and providing the agent selection result and the text describing the method of use or the setting conditions to the user; A system including:

Images