system

Abstract

We provide the system. [Solution] An information storage means that collects expertise information and application information related to multiple types of automated response elements using multiple generation learning models, and stores the expertise information and application information for each of the multiple types of automated response elements, An evaluation means inputs instructions to provide performance information and functional information to the generation learning model for each of the multiple types of automatic response elements, and evaluates the performance information and functional information based on the response. An analysis means for analyzing user evaluation information related to the aforementioned multiple types of automated response elements and calculating a satisfaction index, An analysis means for integrating the expertise information, application information, performance information, function information, and satisfaction index for each of the multiple types of automated response elements, and for comparing and analyzing the multiple types of automated response elements, A selection means that acquires information on the user's business process and purpose of use, and selects the optimal automated response element from the comparative analysis results of the analysis means based on the business process and purpose of use information, A providing means that automatically proposes the optimal method of use and setting conditions for the automated response element selected by the selection means, and provides the user with consulting using the optimal automated response element. A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, various automatic response elements (hereinafter referred to as automatic response elements) have been put into practical use, but they each have different specializations, uses, performances and functions. It is difficult for users to select appropriate automatic response elements according to their own business processing processes and usage purposes and to arrange their optimal usage conditions and setting conditions. In addition, while objective evaluations based on user reviews and satisfaction indicators are required, there is a problem that these information are scattered and dynamically changing, making it difficult to perform appropriate comparative analysis and automatic consulting.

Means for Solving the Problems

[0005] In this invention, the server includes means for accumulating information on automated response elements using multiple generation learning models, means for acquiring and evaluating performance information, functional information, and satisfaction indicators for multiple types of automated response elements, and means for selecting the optimal automated response element and providing automated consulting based on the user's business process information and usage purpose information. This enables users to accurately and efficiently select and configure automated response elements and utilize them in a way that is optimal for their own business process.

[0006] An "automatic response element" is an element that has the function of automatically generating a response in response to a request from a user. A "generative learning model" is a pre-trained data processing structure that learns tasks such as language processing and information extraction, and generates responses based on instructions. "Specialized information" refers to information indicating which fields or areas an automated response element is specialized in or useful in. "Usage information" refers to information indicating the primary purpose or usage scenario for which an automated response element is best suited. "Performance information" refers to information that includes quantitative or qualitative evaluation indicators such as processing speed, response accuracy, and scalability of the automated response element. "Functional information" refers to information about specific functions or operational characteristics that an automated response element may provide. A "satisfaction index" is an evaluation value that indicates user satisfaction with automated response elements, obtained by analyzing user reviews and feedback. "Business process information" refers to information that shows the workflow and processing procedures when a user uses the automated response element. "Information on purpose of use" refers to information about the purpose or goal that the user intends to achieve using the automated response elements. An "instruction statement" is a text input to a generative learning model to perform specific tasks such as extracting, evaluating, analyzing, or suggesting information. "Evaluation means" refers to means that have a processing function to compare and evaluate performance information and functional information of automated response elements based on the responses obtained from the generation learning model. "Analysis means" refers to means that have a processing function to analyze text information such as user reviews and calculate a satisfaction index. "Analysis means" refers to means having a processing function that integrates and compares the aforementioned expertise information, application information, performance information, functional information, and satisfaction indicators. The "selection means" is a means that has a processing function to select the optimal automated response element from the comparative analysis results obtained by the analysis means, based on business process information and usage purpose information. "Providing means" refers to means that have a processing function that automatically presents the optimal usage method and setting conditions for the automated response elements selected by the selection means, and provides consulting to the user. "Information storage means" refers to means having a processing function that records and stores expertise information, application information, performance information, functional information, and satisfaction indicators related to multiple types of automated response elements. "Control means" refers to means having a processing function that dynamically changes the instruction sentences input to the generation learning model in response to information updates by the evaluation means and analysis means, and periodically re-evaluates information related to the automatic response elements. [Brief explanation of the drawing]

[0007] [Figure 1] This is a sequence diagram showing the processing flow of the system in this embodiment. [Modes for carrying out the invention]

[0008] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0009] <System Configuration>

[0010] An example of an embodiment for carrying out the present invention is described below. The present invention relates to a system for evaluating, analyzing, selecting, and proposing multiple automated response elements, and this system mainly consists of an information processing environment including a server and terminals. The system of this embodiment includes a server and terminals. The functional configuration and processing flow of the server and terminals are described below.

[0011] (1) Overall system configuration The system is configured to store information on multiple types of automated response elements, select the optimal automated response element that matches the user's business process and purpose, and automatically provide consulting using that element. Here, an automated response element is an element that can respond to user requests using a specific generation learning model, and may have different specialties and applications such as text processing, image processing, and speech processing.

[0012] (2) Server configuration The server includes means for storing information, means for evaluation, means for analysis, means for selection, means for providing information, and, if necessary, means for control. These means are implemented by functional modules that are programmatically controlled within the server.

[0013] Specifically, the server has an internal database and information storage means for accumulating expertise information, application information, performance information, functional information, and satisfaction indicators related to multiple types of automated response elements. The server communicates with the generation learning model through an externally connectable interface and acquires and updates information for each automated response element by inputting instruction sentences to the generation learning model. The instruction sentences are generated internally by the server and constructed as text that the generation learning model can respond to.

[0014] Furthermore, the server uses an evaluation means to acquire performance and functional information for each automated response element. The evaluation means inputs instruction sentences for performance evaluation into the generation learning model, analyzes the response, and quantifies evaluation items such as processing speed, response accuracy, and scalability. The analysis means inputs instruction sentences to the generation learning model, taking user reviews and feedback information as input, and extracts satisfaction indicators from the response. In this way, qualitative user reviews are accumulated as quantitative satisfaction indicators.

[0015] The server integratively compares and analyzes expertise information, usage information, performance information, function information, and satisfaction indicators by means of analysis. The analysis means generates a relative evaluation report using the data regarding each self-response element held by the information storage means. This analysis result is used for the selection of the optimal self-response element by the selection means described later. Also, when a control means is provided, in response to information updates by the evaluation means or the analysis means, the instruction text can be dynamically regenerated and a periodic re-evaluation can be performed. With such a configuration, the server can always keep the information regarding the self-response element in the latest state.

[0016] The selection means refers to the business process information and usage purpose information input by the user from the terminal, and automatically selects the optimal self-response element from the comparative analysis results by the analysis means. In this process, for example, the most suitable self-response element is determined according to user requirements such as "business that requires rapid text correction" or "design process mainly aimed at image generation".

[0017] The providing means inputs, as an instruction text, the usage method and optimal setting conditions of the optimal self-response element specified by the selection means into the automatic generation learning model, and presents the consulting content to the user using the response result. This consultation automatically gives advice on what parameters should be used for the self-response element and in what scenarios it will exert the maximum effect.

[0018] (3) Configuration of the terminal The terminal is a device for information output and input operated by the user, and includes, for example, a display, a keyboard, a mouse, a touch panel, etc. The terminal is connected to the server via a network, and the user transmits business process information and usage purpose information to the server using the terminal. The terminal also receives the analysis report, comparison result, satisfaction indicator, recommended self-response element, and its setting conditions, etc. transmitted from the server, and displays them on the screen for the user.

[0019] Users can easily select automated response elements that correspond to their workflow based on information about the optimal agent received via their device. Furthermore, by referring to the consulting content provided, they can immediately implement the optimal settings and usage.

[0020] (4) Example of a processing flow First, the user inputs information about their work process and purpose of use using a terminal. This information is sent to the server. The server refers to information on multiple types of automated response elements stored in the information storage means and updates performance information, functional information, and satisfaction indicators by inputting instructions into the generation learning model as needed. The updated information is compared and analyzed by the analysis means, and based on the results, the selection means identifies the optimal automated response element. The selection results are presented to the user via the provision means, and the user refers to the proposed use methods and optimal setting conditions for the automated response elements. This allows the user to immediately improve their work efficiency.

[0021] (5) Variant In this embodiment, the server and terminal may be configured integrally or distributed using a cloud environment. Furthermore, the instruction sentences sent to the generation learning model can be dynamically optimized using natural language processing technology, and can be easily applied to other types of automated response elements by adding or changing different evaluation criteria and analysis methods. In addition, the types of performance information and functional information to be evaluated, the method for calculating satisfaction indicators, and the content and granularity of consulting can be appropriately changed according to practical requirements.

[0022] According to this embodiment, configured as described above, users can automatically and efficiently select appropriate elements from a vast number of automated response elements, and easily obtain information on how to use them and their optimal settings. As a result, the productivity and convenience of the user's work processes are significantly improved.

[0023] <Specific examples of system configuration>

[0024] The following describes in detail each component as an example of an embodiment of the present invention. The system of this embodiment is an information processing environment mainly composed of a server and a terminal. The server stores and analyzes information on multiple automated response elements, and based on the business process information and usage purpose information specified by the user, it automatically selects the optimal automated response element and presents its usage method and setting conditions. On the other hand, the terminal functions as an input / output device operated by the user and sends and receives information to and from the server.

[0025] (1) Server The server includes means for storing information, means for evaluation, means for analysis, means for selection, means for providing information, and, if necessary, means for control. Each of these means can be implemented as a program (software module) that operates within the server, and the server can operate on any information processing infrastructure, such as a cloud environment or within a local network.

[0026] (a) Information storage means The information storage system is constructed using a database and records expertise information, application information, performance information, function information, and satisfaction indicators for multiple types of automated response elements. For example, the information storage system can use a relational database, and the agent ID, expertise (e.g., text processing, image processing, speech processing, etc.), main application (e.g., proofreading, image generation, speech recognition, etc.), performance parameters (e.g., processing speed, response accuracy, scalability indicator), and user satisfaction score (e.g., 1-5 scale) are stored in a table structure.

[0027] By accumulating this kind of information, the server can quantitatively and qualitatively maintain the characteristics of each automated response element. The information storage means continuously updates the data in accordance with the updates of the results from the evaluation and analysis means described later, and always maintains the latest and most consistent information.

[0028] (b) Evaluation means The evaluation means has the function of acquiring and updating performance information and functional information of each automated response element using a generation learning model. Specifically, the server inputs instructions to the generation learning model and acquires information such as the processing speed, response accuracy, and scalability of each automated response element. At this time, the instructions describe evaluation criteria and evaluation methods, and the generation learning model outputs a response corresponding to these. The server analyzes this output and records the evaluation results in the information storage means.

[0029] This evaluation method allows the server to periodically monitor the performance status of automated response elements and compare their relative strengths and weaknesses.

[0030] (c) Analysis means The analysis method has the function of analyzing text-based evaluation information, such as user reviews and feedback, and calculating it as a quantitative satisfaction index. The analysis method sends instruction sentences to a generation learning model to input review sentences and feedback sentences, and analyzes the satisfaction evaluation and summary information obtained as a response.

[0031] This process allows reviews, which were previously merely subjective opinions, to be accumulated as quantitative scores (e.g., a 1-5 scale), enabling comprehensive analysis by combining them with other evaluation metrics.

[0032] (d) Analytical tools The analysis means has the function of comprehensively comparing and analyzing expertise information, application information, performance information, function information, and satisfaction indicators stored in the information storage means. In this process, the analysis means compares multiple automated response elements with each other and generates rankings based on specific evaluation axes (e.g., emphasis on processing speed, emphasis on accuracy, emphasis on user satisfaction, etc.) and reports that extract characteristic strengths and weaknesses for each agent. These analysis results are used to select the optimal automated response elements by the selection means described later and to generate consulting content by the provision means.

[0033] (e) Selection method The selection means has the function of selecting the optimal automated response element based on comparative analysis results obtained from the analysis means, by referring to business process information and usage purpose information entered by the user from the terminal. For example, if the user presents a request such as "I want to perform text proofreading processing quickly" or "I want to prioritize the quality of image generation," the selection means will compare these requests with the results of the analysis means and identify the automated response element that best suits the request.

[0034] The selection method involves inputting instruction sentences into a learning model for generation, which then extracts the most suitable agent based on the given conditions. This allows users to easily find an automated response that matches their specific work content and objectives from a vast array of options.

[0035] (f) Means of provision The service provider has the functionality to input usage methods and setting conditions for selected optimal automated response elements as instructions into a generation learning model, and then extract them from the response results. This allows users not only to know the optimal elements, but also to automatically obtain specific instructions and recommended settings for making the most of those elements.

[0036] For example, the service provider inputs an instruction such as "Show the optimal parameters for proofreading text using this automated response element" into a learning model for generating instructions, and based on the output, provides the user with suggestions such as "If the text length is less than X characters, using parameter Y will improve accuracy."

[0037] (g) Control means (optional) The control means has the function of dynamically changing the information update frequency and instruction generation rules by the evaluation means and analysis means, and controls the entire system to ensure that the latest and most appropriate evaluations are always performed. For example, when new reviews are accumulated, the analysis means can be automatically activated, the instruction statements can be regenerated, and the latest satisfaction index can be calculated.

[0038] (2) terminal The terminal is an information processing device equipped with a display device and input device capable of communicating with the server. Users transmit business process information and usage purpose information to the server via the terminal. The terminal also receives analysis results, selection results, and consulting information regarding optimal usage methods and settings transmitted from the server, and presents them to the user by displaying them on the screen.

[0039] On the terminal, a web browser, mobile application, or dedicated user interface application runs, allowing users to operate freely through these. Information entered from the terminal to the server is done using the desired interface, such as text input, pull-down menu selection, or file upload. The terminal also displays acquired analysis reports, quantified satisfaction indicators, and comparison results in intuitive graphs and charts, providing users with an environment where they can easily understand the optimal automated response elements.

[0040] As described above, this embodiment can exchange information via a server and a terminal, analyze and evaluate automated response elements by inputting instruction sentences into a generation learning model, and provide the optimal automated response elements that meet user needs. As a result, users can easily select automated response elements that match their own work processes and immediately learn how to use them and their setting conditions, thereby contributing to improved work efficiency and productivity.

[0041] <System Operation>

[0042] The server operates in a cloud environment (for example, a virtual machine running on a general-purpose information processing device) and implements programs by executing a web application framework (e.g., Flask or Django) built using Python on a Linux® operating system. The server stores information on multiple types of automated response elements using a relational database such as MySQL® or PostgreSQL®, and obtains performance information, functional information, satisfaction indicators, etc., by providing this data to an external generation learning model via instructions.

[0043] The server sends instructions to a generative learning model (e.g., an external large-scale language processing platform API), analyzes the response, and performs performance evaluation and review analysis. During this process, the server issues HTTP requests within a Python program and retrieves the model's response in JSON or text format. The server then analyzes the obtained response using Python, stores the information in a database, and uses it to select the optimal automated response elements based on the user's business process information and usage purpose information. The server converts the selection results and analysis reports into formats such as HTML or JSON and sends them to the terminal.

[0044] The terminal is a general information processing device equipped with display and input means such as a web browser. The terminal receives analysis results and suggestions for optimal automated response elements transmitted from the server and presents them to the user via a graphical user interface. On the terminal, the display layout can be controlled using a scripting language such as JavaScript (registered trademark), and the user can input their purpose and requests and send them back to the server.

[0045] Users can input their own work process information and usage purpose information using a terminal, and then refer to the results of the server's analysis and selection based on this information. By reviewing the presented agent selection results and consulting information and reflecting them in their work, users can effectively utilize automated response elements.

[0046] As a concrete example, the server can send the following instruction to the generation learning model. "For Agent_A, Agent_B, and Agent_C, summarize their respective areas of expertise and main uses in JSON format." "For Agent_A, Agent_B, and Agent_C, calculate a 5-point rating for processing speed, response accuracy, and scalability, and return the results in JSON format." "Based on the user review below, calculate the satisfaction level (score 1-5) for the agent and briefly summarize the reason. Review: 'Agent_A has high accuracy in proofreading and is very helpful, but sometimes it is slow.'" "Generate an analysis report for Agent_A, Agent_B, and Agent_C, comprehensively evaluating their expertise, applications, performance, and user satisfaction metrics. In conclusion, indicate the recommended scenarios for each agent." "Select the one agent best suited to the following workflow, 'primarily proofreading documents, requiring immediate correction of short sentences,' and explain your reasoning in JSON format. The agents involved are Agent_A, Agent_B, and Agent_C."

[0047] Using these instructions, the server analyzes the response from the generation learning model and updates the information by executing INSERT statements into multiple database tables. This allows the server to dynamically generate processing results suitable for the user's request received from the terminal, and the user can receive suggestions regarding the optimal automated response elements and their usage settings via the terminal.

[0048] <Specific example of system operation (Figure 1)>

[0049] Step 1: The user enters information about their work process and purpose of use into an input form on the terminal and issues a request to send it to the server. The user's input (e.g., "I want to quickly proofread my documents") is then taken into the terminal and sent to the server as an HTTP request (in JSON format, etc.). The input data is user request data, and by passing this to the server, the server can obtain basic information to select the most suitable automated response element for the request in subsequent processing steps.

[0050] Step 2: The terminal sends user-entered business process information and usage purpose information to the server. Input includes data containing business details and purpose (e.g., "proofreading" and "immediate correction of short sentences"), which the terminal transfers to the server using the HTTP protocol. The server receives this request, performs data processing to extract the business process information and usage purpose information (e.g., JSON parsing), and prepares to compare it with internally stored automated response element information. The user request data is then stored internally as output.

[0051] Step 3: The server references multiple types of automated response element information stored by the information storage system and dynamically generates prompt messages for performance evaluation and application identification for the generating AI model. The input is a list of candidate agents (e.g., Agent_A, Agent_B, Agent_C) stored internally by the server, and based on this, the server sends prompt messages such as "Summarize expertise and application in JSON format" to the generating AI model. The output is the response text obtained from the generating AI model, and the server processes this response using Python string processing, JSON parsing, and other data processing before storing it in the database.

[0052] Step 4: The server analyzes the responses received from the generative AI model and extracts performance information (e.g., processing speed, response accuracy) and satisfaction metrics. The input is the response text from the generative AI model, and the server performs numerical evaluation and tagging using Python's natural language processing and regular expression processing. As a result, performance information and satisfaction metrics are extracted and recorded as output in the server's internal database using INSERT statements. This allows the server to accumulate quantitative evaluation data for multiple automated response elements.

[0053] Step 5: The server comprehensively compares and analyzes expertise, application, performance, functionality, and satisfaction indicators stored in the database. The input consists of evaluation data for multiple agents within the database. The server retrieves this information by executing SQL queries within a Python program, performing statistical calculations and ranking. As a result (output), the server identifies the most suitable agent candidate for the purpose and clarifies the characteristics of that agent (e.g., the fastest text proofreading capability).

[0054] Step 6: The server identifies the optimal agent, generates prompts to obtain usage instructions and configuration conditions for that agent, and sends them back to the generating AI model. The input is the selection result (optimal agent name), and the server generates prompts such as "Show the optimal parameters for using Agent_A for text proofreading purposes." The response (output) from the generating AI model is text describing the configuration conditions and specific usage procedures, which the server parses and formats into a format (JSON or HTML) that can be sent back to the terminal.

[0055] Step 7: The server sends formatted analysis reports, optimal agent selection results, and their usage and configuration conditions to the terminal. Input consists of text information obtained and analyzed from the generated AI model and database information within the server. The server returns these to the terminal as an HTTP response in JSON or HTML format. On the terminal side, the output can be formatted and displayed using HTML templates or JavaScript for visualization, allowing the user to check the optimal agent and configuration procedure on the screen.

[0056] Step 8: The terminal displays the optimal agent candidates and their configuration conditions received from the server on a GUI. Inputs include HTML, JSON, and analysis results sent from the server. The terminal uses JavaScript to display tables and graphs, creating an interface that allows users to easily understand the results. The output is a visual presentation to the user, who can then refer to this information to make decisions regarding agent selection and application to business processes.

Claims

1. Information storage means that collects expertise information and application information related to multiple types of automated response elements using multiple generation learning models, and stores the expertise information and application information for each of the multiple types of automated response elements, An evaluation means inputs instructions to provide performance information and functional information to the generation learning model for each of the multiple types of automatic response elements, and evaluates the performance information and functional information based on the response. An analysis means for analyzing user evaluation information related to the aforementioned multiple types of automated response elements and calculating a satisfaction index, An analysis means for integrating the expertise information, application information, performance information, function information, and satisfaction index for each of the multiple types of automated response elements, and for comparing and analyzing the multiple types of automated response elements, A selection means that acquires information on the user's business process and purpose of use, and selects the optimal automated response element from the comparative analysis results of the analysis means based on the business process and purpose of use information, A providing means that automatically proposes the optimal method of use and setting conditions for the automated response element selected by the selection means, and provides the user with consulting using the optimal automated response element. A system that includes this.

2. The system according to claim 1, further comprising control means for dynamically generating instruction sentences to be input to the generation learning model in response to information updates by the evaluation means and analysis means, and for periodically re-evaluating the performance information or satisfaction index of each automatic response element by executing the instruction sentences.

3. The system according to claim 1, wherein the providing means includes a process of inputting an instruction sentence to the generation learning model based on the selection result of the optimal automatic response element by the selection means, and automatically extracting usage conditions or setting conditions suitable for the automatic response element from the execution result of the instruction sentence.

Images