System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a request from a user; means for transmitting the request to a plurality of natural language generation systems; means for receiving outputs from the plurality of natural language generation systems; means for comparing and evaluating the received outputs; means for removing outliers based on the evaluation results; means for selecting an optimal output; and means for providing the optimal output to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, multiple natural language generation systems are offered by various companies and countries, but because each system uses a different training dataset, the output often differs. As a result, the output obtained by the user is not necessarily accurate and may contain misinformation or biased information. Another issue is the lack of a mechanism for centrally evaluating the output of different systems and selecting the most accurate output. [Means for solving the problem]

[0005] The present invention provides a system that sends requests received from a user to multiple natural language generation systems and aggregates and evaluates the outputs returned from each system. Specifically, the system sends requests received from a user to multiple natural language generation systems in parallel and receives the outputs returned from these systems. The received outputs are then compared and evaluated, and outliers are removed using statistical techniques or machine learning models. Finally, the optimal output is selected and provided to the user. This minimizes the risk of misinformation or biased information and provides more accurate output to the user.

[0006] "User" means any person or entity that submits a request to the System.

[0007] A "request" refers to a specific question or request that a user makes to the system.

[0008] "Natural language generation system" refers to algorithms and software that generate responses or text in response to input natural language data.

[0009] "Output" refers to the response or generated text returned by a natural language generation system.

[0010] "Comparison" refers to the process of juxtaposing multiple outputs and evaluating them against each other.

[0011] "Evaluation" refers to the means and process of measuring the reliability and accuracy of outputs.

[0012] An "outlier" is a result that is significantly different from other outputs and is considered to be less accurate.

[0013] "Statistical methods" refer to methods of analyzing data that use statistical theory and computational methods to find outliers and patterns.

[0014] A "machine learning model" refers to an algorithm that learns from past data and makes predictions and classifications for new data.

[0015] "Optimal output" refers to the output that is determined to be the most reliable and accurate after comparison and evaluation.

[0016] The term "system" refers to the entire mechanism that combines the multiple means described as the summary of the present invention. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention is a system for unifying the outputs from multiple natural language generation systems and providing the most appropriate response to the user. A specific form of this system is shown below.

[0039] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems. The user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[0040] User Action

[0041] Users access the system from their own devices using a dedicated application or a web browser. After accessing the system, the user enters a request, such as a question or a request to generate a sentence, and presses the send button. This operation sends the request to the server.

[0042] Server Processing

[0043] The server analyzes requests received from users and sends the request contents to multiple natural language generation systems in parallel. For example, requests are sent simultaneously to natural language generation systems A, B, and C. Each generation system generates an output based on the request and sends it back to the server.

[0044] The server collects the outputs returned by each generation system, compares and evaluates each output, and uses statistical methods and machine learning models as criteria for evaluation to eliminate outliers and inaccurate outputs. For example, it processes the output that receives the fewest votes overall. Through this process, the output deemed to be the most reliable is selected.

[0045] Specific examples

[0046] Consider a case where a user sends a request such as "What year did Napoleon Bonaparte die?" The server sends this request to natural language generation systems A, B, and C. Each generation system returns an answer. For example, if generation systems A and B return "1821" and generation system C returns "1820," the server evaluates these answers using statistical methods and machine learning models and selects the majority answer, "1821," as the optimal output. The server then returns this optimal output to the user.

[0047] In this way, the present invention is a system that can eliminate erroneous or biased information and provide users with accurate and reliable output.

[0048] The processing flow will be explained below.

[0049] Step 1:

[0050] The user uses a terminal to launch a dedicated application or a web browser, enters a question or a request for sentence generation in a request form, and clicks a send button.

[0051] Step 2:

[0052] The server receives a request from the user, which is sent to the server as an HTTP request and parsed.

[0053] Step 3:

[0054] The server analyzes the received request and sends it to multiple natural language generation systems in parallel, sending messages in the appropriate format to the API endpoints of each generation system.

[0055] Step 4:

[0056] Multiple natural language generation systems receive requests from the server, generate output through their own internal processing, and send it back to the server.

[0057] Step 5:

[0058] The server receives responses from each generation system. The received responses are obtained in a data format such as JSON.

[0059] Step 6:

[0060] The server aggregates the received multiple outputs, compares and evaluates the content of each output, and performs preprocessing of the data using statistical methods and machine learning models to evaluate its reliability.

[0061] Step 7:

[0062] Eliminate inaccurate outputs that are outliers based on the evaluation results, for example, eliminating the output that received the fewest votes.

[0063] Step 8:

[0064] The server selects the optimal output that is evaluated as the most reliable, i.e., the output with the highest score from the majority opinion or evaluation model.

[0065] Step 9:

[0066] The server returns the optimal output it has selected to the user, returning it to the user's device as an HTTP response and displaying it in the browser or application.

[0067] Step 10:

[0068] The user checks the optimal output returned on the device and receives the answer.

[0069] This system makes it possible to efficiently compare and evaluate the results of multiple generative AI services and provide users with highly reliable output.

[0070] Example 1

[0071] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0072] Current natural language generation systems rely on a single system, resulting in problems with the accuracy and consistency of responses. Furthermore, even when responses are obtained from multiple natural language generation systems, there is a lack of effective means for integrating these responses and selecting the optimal response. This makes it difficult to provide users with reliable information. Therefore, there is a need for a system that can compare and evaluate responses from multiple natural language generation systems, remove outliers, and provide the optimal response to the user.

[0073] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0074] In this invention, the server includes means for receiving a request from a user, means for transmitting a request to a plurality of natural language generation systems, means for receiving responses from the plurality of natural language generation systems, means for comparing and evaluating the received responses, means for removing outliers based on the evaluation result, means for selecting an optimal response from the set as a result of the evaluation, and means for providing the optimal response to the user. This makes it possible to effectively integrate responses obtained from the plurality of natural language generation systems and provide the user with reliable information.

[0075] A "user" is an entity that sends requests to the system and receives responses.

[0076] A "request" is information such as a question or a request for sentence generation that a user sends to the system.

[0077] "Multiple natural language generation systems" refers to a collection of multiple systems that generate natural language based on different algorithms or models.

[0078] A "response" is information that multiple natural language generation systems generate based on a request and return to the server.

[0079] "Comparing and evaluating means" refers to methods and techniques for comparing received responses with each other and evaluating their accuracy and reliability.

[0080] "Means for removing outliers" are methods or techniques that, based on the evaluation results, eliminate inaccurate responses that differ significantly from other responses.

[0081] "Means for selecting optimal responses" are methods or techniques for selecting the most appropriate and reliable responses based on the evaluation.

[0082] The "means for providing" refers to the method or technology for transmitting and displaying the selected optimal response to the user.

[0083] The present invention is a system that unifies responses from multiple natural language generation systems and provides the most appropriate response to the user. A specific form of this system is shown below.

[0084] composition

[0085] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems, and the user terminal provides an interface for users to input questions or requests and communicate with the server to obtain responses.

[0086] Hardware and software used

[0087] Hardware: Servers with high-performance processors and large amounts of memory, and devices for users to access the system (PCs, smartphones, tablets, etc.)

[0088] Software: Multiple natural language generation systems (generative AI models A, B, and C), statistical analysis software, and machine learning model evaluation software

[0089] Processing flow

[0090] User operations

[0091] Users access the system from their own devices using a dedicated application or web browser. After accessing the system, the user enters a question or request and presses the send button. This action sends the request to the server.

[0092] Specific examples

[0093] A user enters "What year did Napoleon Bonaparte die?" into the text input field of a web browser and clicks the submit button.

[0094] Server Processing

[0095] The server analyzes the request data received from the user and converts it into an appropriate format. It then sends the analyzed request to multiple natural language generation systems (generative AI models A, B, and C) in parallel and receives responses from each generation system.

[0096] The server collects the received responses, compares them with each other, and evaluates them. Statistical methods and machine learning models are used to filter out inaccurate responses and outliers. The server then selects the most reliable response and sends it back to the user.

[0097] Specific examples

[0098] The server sends the prompt "What year did Napoleon Bonaparte die?" to generative AI models A, B, and C. If generative AI models A and B respond with "1821" and generative AI model C responds with "1820," the server evaluates these responses using statistical methods and machine learning models, selects the majority response, "1821," as the optimal response, and returns it to the user device.

[0099] effect

[0100] This allows the system to effectively integrate information obtained from multiple natural language generation systems and provide users with highly reliable information, making the overall operation more efficient and significantly improving the user experience.

[0101] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0102] Step 1: User submits request

[0103] Specific operation: A user accesses the system using a dedicated application or a web browser from their own device (PC, smartphone, tablet, etc.), enters the request content (e.g., a question or a request to generate a sentence) in the text input field, and presses the send button.

[0104] Input: A user question or request for text generation (e.g., "What year did Napoleon Bonaparte die?")

[0105] Output: Request data sent to the server

[0106] Step 2: The server parses the request

[0107] Specific operation: The server analyzes the request data received from the user, understands the type and content of the request, and converts the request into an appropriate format based on this.

[0108] Input: Request data received from the user

[0109] Output: Parsed request data

[0110] Step 3: The server sends the request to the natural language generation system

[0111] Specific operation: The server sends parsed requests in parallel to multiple natural language generation systems (generative AI models A, B, and C).

[0112] Input: Parsed request data

[0113] Output: A prompt sent to each generative AI model (e.g., "What year did Napoleon Bonaparte die?")

[0114] Step 4: The natural language generation system generates a response

[0115] Specific operation: Each generative AI model generates a response based on the sent prompt sentence and sends it back to the server.

[0116] Input: Prompt sent to each generative AI model

[0117] Output: Response data returned by each generative AI model (e.g., Generative AI Models A and B respond "1821," while Generative AI Model C responds "1820")

[0118] Step 5: The server collects and compares the responses

[0119] What it does: The server collects the responses from each generation system, compares and evaluates them against each other, and uses statistical techniques and machine learning models to filter out inaccurate responses and outliers.

[0120] Input: Response data from each generative AI model

[0121] Output: Evaluated response data with incorrect responses and outliers removed

[0122] Step 6: The server picks the best response

[0123] Specific behavior: The server selects the single most authoritative response based on the evaluation results, e.g., the majority response.

[0124] Input: Evaluated response data

[0125] Output: Best response (e.g. "1821")

[0126] Step 7: The server provides the best response to the user

[0127] Specific operation: The server sends the selected best response to the user's terminal, where the user can receive and confirm it.

[0128] Input: Best response

[0129] Output: Response data sent to the user device

[0130] (Application example 1)

[0131] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0132] On conventional online shopping sites, it has been difficult to obtain a quick and accurate answer when users have questions about products. This problem can lead to a decrease in user satisfaction and a decrease in purchasing intent. Another issue is that there has been no effective method for integrating answers from multiple natural language generation systems and selecting the most appropriate answer.

[0133] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0134] In this invention, the server includes means for receiving requests from a user, means for transmitting requests to a plurality of natural language generation systems, means for receiving outputs from the plurality of natural language generation systems, means for comparing and evaluating the received outputs, means for removing outliers based on the evaluation results, means for selecting an optimal output, means for providing the optimal output to a user, means for communicating with an external content providing system, and means for responding with the optimal output in response to a product-related question from a user on an online shopping site. This makes it possible to provide a quick and accurate answer to a product-related question from a user, thereby increasing user satisfaction.

[0135] The "means for receiving requests from a user" is a function for sending questions or requests input by a user through an application to a server.

[0136] The "means for transmitting requests to a plurality of natural language generation systems" is a function that allows the server to simultaneously transmit requests received from users to a plurality of natural language generation systems.

[0137] The "means for receiving outputs from a plurality of natural language generation systems" refers to a function by which the server collects responses generated from each natural language generation system.

[0138] "Means for comparing and evaluating received outputs" refers to a function in which the server compares and evaluates multiple outputs using statistical methods or machine learning models.

[0139] The "means for removing outliers based on the evaluation results" is a function for removing unreliable outputs based on the evaluation results.

[0140] The "means for selecting the optimal output" is a function that selects the most reliable output through evaluation and removal of outliers.

[0141] The "means for providing the optimal output to the user" is a function that returns the selected optimal output to the user.

[0142] The "means for communicating with an external content providing system" is a communication function that enables the server to exchange information with an external service.

[0143] "Means for providing the optimal output in response to a question about a product from a user on an online shopping site" is a function that provides the optimal answer when a user asks a question about a product on an online shopping site.

[0144] The present invention provides a system for providing quick and accurate answers to user questions and requests. This system is particularly effective for online shopping sites and aims to improve the user experience. Specific embodiments of the system are described below.

[0145] The system consists of a server, a user terminal, and multiple natural language generation systems. The entire system operates with the following components:

[0146] Hardware and Software Configuration

[0147] Server: The server receives requests from users and has an interface for communicating with multiple natural language generation systems. Specifically, a cloud server such as AWS (registered trademark) EC2 is used.

[0148] User terminal: The user terminal provides an interface for inputting user questions and requests through a smartphone application.

[0149] Natural language generation systems: External systems equipped with generative AI models (e.g., BERT, GPT-3 (registered trademark)) are used.

[0150] Data processing and calculation

[0151] 1. User input processing: The user enters a question or request through the application. This input data is sent from the user's device to the server.

[0152] 2. Request distribution: The server sends the received user request to multiple natural language generation systems in parallel, forming an appropriate prompt for each system.

[0153] For example: Question: "Is this T-shirt machine washable?"

[0154] 3. Receiving and evaluating the output: The server receives the answers from each generation system and evaluates and compares these answers using statistical methods and machine learning models.

[0155] 4. Outlier Removal: This process removes unreliable outputs based on the evaluation results. Specifically, it removes the output with the fewest votes.

[0156] 5. Selecting and providing the optimal output: Finally, the output with the highest rating is selected and returned to the user.

[0157] Specific examples

[0158] A user types a question into an online shopping site: "Is this T-shirt machine washable?" The system works as follows:

[0159] The server receives user requests and sends the requests to multiple natural language generation systems.

[0160] Each generating system returns the answer "Yes, this T-shirt is machine washable" to the server.

[0161] The server compares and evaluates these answers and provides the best answer to the user.

[0162] This system allows users to quickly obtain accurate and reliable answers, which is expected to improve the online shopping experience and increase user satisfaction.

[0163] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0164] Step 1:

[0165] A user accesses an online shopping site and inputs a question or request about a product through the application. The input information is sent from the user's terminal to the server.

[0166] Input: "Is this t-shirt machine washable?"

[0167] Output: The question is sent to the server.

[0168] Step 2:

[0169] The server receives the user's request, parses and understands it, and formats it in a format suitable for multiple natural language generation systems.

[0170] Input: User question

[0171] Output: Formatted prompt (e.g., "Question: Is this T-shirt machine washable?")

[0172] Step 3:

[0173] The server sends formatted prompt sentences to multiple natural language generation systems in parallel, sending requests to each generation system independently.

[0174] Input: Formatted prompt text

[0175] Output: The requests sent to each NLG system

[0176] Step 4:

[0177] Each natural language generation system generates a response based on the request and sends it back to the server.

[0178] Input: The request sent to each natural language generation system

[0179] Output: The answer from each generating system (e.g., "Yes, this T-shirt is machine washable.")

[0180] Step 5:

[0181] The server receives and collects responses from multiple natural language generation systems, storing all responses in a temporary database.

[0182] Input: Answers from each generating system

[0183] Output: Save all responses to a temporary database

[0184] Step 6:

[0185] The responses collected by the server are evaluated using statistical methods and machine learning models, and their reliability and relevance are assessed, with outlier responses being removed based on statistical methods.

[0186] Input: Response stored in temporary database

[0187] Output: Evaluation score and outlier removal results

[0188] Step 7:

[0189] Select the most reliable response based on the evaluation results. Sort by highest evaluation score to select the most appropriate response.

[0190] Input: Evaluation scores and outlier removal results

[0191] Output: Best response

[0192] Step 8:

[0193] The server formats the best response for return to the user and sends it to the user's terminal, which receives it and displays it on its screen.

[0194] Input: Best response

[0195] Output: The optimal response that will be displayed to the user (e.g., "Yes, this T-shirt is machine washable.")

[0196] The above are the detailed processing steps from when the user inputs a question about a product to when the most suitable response is obtained.

[0197] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0198] This invention is a system that unifies the outputs from multiple natural language generation systems and provides the most appropriate response to the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide an output that is adapted to the user's emotions. Specific forms of this system are shown below.

[0199] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems and an emotion engine. The user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[0200] User Action

[0201] Users access the system from their own devices using a dedicated application or web browser. After accessing the system, they input requests such as questions or requests for sentence generation. At this time, the emotion engine recognizes the user's emotional state through voice or text input. The request and emotional state are then sent to the server.

[0202] Server Processing

[0203] The server analyzes the request content and emotional state received from the user. The request content is sent to multiple natural language generation systems in parallel, and the server receives the response output from each generation system. The server then takes into account the emotional state recognized by the emotion engine in its evaluation and applies emotion adjustment logic to ultimately select the optimal output.

[0204] Specifically, if a user requests "What year did Napoleon Bonaparte die?" and the emotion engine recognizes the emotional state as "interesting," the server sends this information in parallel to natural language generation systems A, B, and C. Each generation system returns answers such as "1821," "1821," and "1820," respectively. At this time, the emotional information recognized by the emotion engine is taken into account by the emotion adjustment logic, and an output that provides a detailed explanation of "1821" in a way that matches the emotion of "interesting" is selected.

[0205] Emotion Engine

[0206] The emotion engine uses speech and text analysis techniques to recognize the user's emotions. For voice input, it analyzes the tone, pitch, and speed of the voice to determine whether the user is excited or calm. For text input, it analyzes keywords and context in the content to identify emotions. The recognized emotion information is sent to the server along with the request and is used to evaluate and optimize the output.

[0207] Specific examples

[0208] If the emotion engine recognizes that the user is in an "uneasy" state when making a request, the system will select a generation system that is more likely to generate sentences that "provide a sense of security" to the user. Also, if the user makes a request in a "happy" state, the system will tend to select "fun and attractive" output.

[0209] This system can efficiently compare and evaluate the results of multiple generative AI services and provide output that is adapted to the user's emotional state, improving the user experience and providing information that better meets the user's needs.

[0210] The processing flow will be explained below.

[0211] Step 1:

[0212] The user uses a device to launch a dedicated application or a web browser, enters a question or a request for sentence generation into the request form, and clicks the send button. At this time, the device sends the user's emotion data to the emotion engine via voice or text input.

[0213] Step 2:

[0214] The server receives a request from the user, along with the request content and the user's emotion data analyzed by the emotion engine from the device.

[0215] Step 3:

[0216] The server analyzes the received request content and sends the request content and emotion data to multiple natural language generation systems in parallel. It sends messages in an appropriate format to the API endpoint of each generation system.

[0217] Step 4:

[0218] Multiple natural language generation systems receive requests from the server, generate output through their own internal processing, and send it back to the server. The response from each system is sent back to the server as generated text data.

[0219] Step 5:

[0220] The server receives responses from each generation system. The received responses are obtained in a data format such as JSON. For example, answers from natural language generation systems A, B, and C are obtained.

[0221] Step 6:

[0222] The server aggregates the received multiple outputs, compares and evaluates the content of each output, and performs preprocessing of the data using statistical methods and machine learning models to evaluate its reliability.

[0223] Step 7:

[0224] The server uses the emotion data from the emotion engine to filter out inaccurate outputs that are outliers based on the evaluation results, and then applies emotion adjustment logic to select outputs that reflect the emotion data.

[0225] Step 8:

[0226] The server selects the optimal output that is evaluated as the most reliable, specifically, the output with the highest score on majority opinion and emotion regulation logic.

[0227] Step 9:

[0228] The server returns the optimal output it has selected to the user, returning it to the user's device as an HTTP response and displaying it in the browser or application.

[0229] Step 10:

[0230] The user checks the best output on their device and receives the answer. For example, the response "Napoleon Bonaparte died in 1821" is displayed along with an additional explanation based on the emotion recognized by the emotion engine (such as "He was a great leader of his time").

[0231] This system can efficiently compare and evaluate the results of multiple generative AI services and provide output that is adapted to the user's emotional state, improving the user experience and providing information that better meets the user's needs.

[0232] Example 2

[0233] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0234] Current natural language generation systems have a mechanism for evaluating the outputs of multiple systems to obtain the optimal response to a user request, but because they do not take into account the user's emotional state, they are unable to provide the optimal response that meets the user's needs.In addition, there is no emotion-based evaluation in output selection, which results in a suboptimal user experience.

[0235] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0236] In this invention, the server includes means for receiving a user request, means for recognizing the user's emotion, means for transmitting the user's request content and emotional state, means for transmitting the request to a plurality of natural language generation systems, means for receiving outputs from the plurality of natural language generation systems, means for comparing and evaluating the received outputs based on the emotional state, means for selecting an optimal output based on the evaluation result, and means for providing the optimal output to the user. This makes it possible to select an optimal response from the outputs of the plurality of generation systems while taking the user's emotional state into consideration, thereby improving the user experience.

[0237] The "means for receiving requests" is a function that provides an interface for receiving questions or information requests from users and inputting them into the system.

[0238] "Means for recognizing emotions" is a function that analyzes the user's voice and text input to identify their emotional state at that time.

[0239] The "means for transmitting the request content and emotional state" is a function for transferring the request content received from the user and the recognized emotional state to the server.

[0240] The "means for sending a request to a natural language generation system" is a function that enables the server to send the request contents to multiple natural language generation engines in parallel.

[0241] The "means for receiving output" is a function for receiving response results generated from multiple natural language generation engines.

[0242] The "means for comparing and evaluating based on emotional state" is a function for analyzing each received output based on the user's emotional state and selecting the most suitable one.

[0243] The "means for selecting the optimal output" is a function for comparing and examining response results from multiple natural language generation systems and selecting the optimal response based on evaluation criteria (e.g., emotional state).

[0244] The "means for providing output to the user" is a function for sending the selected optimal response to the user's terminal and displaying the result so that the user can confirm it.

[0245] The present invention is a system that unifies the outputs of multiple natural language generation systems in response to a user request and provides the most appropriate response. This system is composed of a server and a user terminal, and operates as follows.

[0246] System Configuration

[0247] In this system, the server has an interface for communicating with multiple natural language generation systems and emotion engines, and the user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[0248] Hardware and software used

[0249] The server requires a high-performance processor and a large amount of memory. In addition, a "generative AI model" (Model A, Model B, Model C) is used as the natural language generation system. The emotion engine requires an engine with voice analysis and text analysis technology. The user terminal is a smartphone or PC with an internet connection.

[0250] Process Overview

[0251] 1. Accepting user input

[0252] Users access the system from their own devices using a dedicated application or a web browser, and input requests such as questions and sentence generation requests by text or voice.

[0253] 2. User Emotion Recognition

[0254] The emotion engine analyzes the user's input (text or voice) to recognize the user's emotional state. For voice input, it analyzes the tone, pitch, and speed of the voice, and for text input, it analyzes keywords and context of the content to identify emotions.

[0255] 3. Sending request data

[0256] The device sends the user's input and emotional state to the server, which receives and stores them as log data.

[0257] 4. Processing of Natural Language Generation Systems

[0258] The server sends the request content to multiple natural language generation systems (generative AI models A, B, and C) in parallel. Each generation system independently generates an answer to this request.

[0259] 5. Receiving the output

[0260] The server receives the output returned from each natural language generation system. For example, it may return outputs such as "1821," "1821," and "1820."

[0261] 6. Applying emotion regulation logic

[0262] Based on the recognition results of the emotion engine, the server applies emotion adjustment logic to the received output to select the best response, for example, "1821" with a detailed description that matches the emotion "interesting."

[0263] 7. Providing a response to the user

[0264] The final selected output is sent to the user's terminal and displayed to the user.

[0265] Specific examples

[0266] For example, if the emotion engine recognizes that the user is in an "unsettled" state when making a request, the system will select a generation system that is likely to generate sentences that "provide a sense of security" to the user. Also, if the user makes a request while in a "happy" state, the system will tend to select "fun and attractive" output.

[0267] Prompt Sentence Examples

[0268] Below are some examples of prompt sentences to input to the generative AI model.

[0269] Please provide more details about the life of a historical figure that you find interesting.

[0270] "Users are worried, so please provide some comforting words."

[0271] This system allows users to quickly obtain the most appropriate information adapted to their emotional state, improving the user experience and increasing the accuracy and relevance of the information provided.

[0272] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0273] Step 1:

[0274] When a user opens a dedicated application or web browser on their device, an input form appears, where the user can enter their question or request via text or voice. Once the input is complete, the device sends the data to the next processing step.

[0275] Input: A user question or text generation request (e.g., "What year did Napoleon Bonaparte die?")

[0276] Output: Text or audio data is sent to the device.

[0277] Specific behavior:

[0278] The user can type "What year did Napoleon Bonaparte die?" into the text field or say "What year did Napoleon Bonaparte die?"

[0279] The terminal sends the entered data to a dedicated application.

[0280] Step 2:

[0281] The emotion engine analyzes text or voice data sent from the device to recognize the user's emotional state. For voice input, it analyzes the tone, pitch, and speed of the voice, and for text input, it analyzes keywords and context of the content to identify emotions.

[0282] Input: Text or audio data

[0283] Output: Analyzed user sentiment information (e.g., "interesting")

[0284] Specific behavior:

[0285] When voice input is used, the emotion engine analyzes the tone, pitch, and speed of the voice in real time to recognize emotions such as "interesting."

[0286] When text is entered, the emotion engine analyzes emotions from keywords and context in the entered text.

[0287] Step 3:

[0288] The device sends the user's input and emotional state to the server, which receives this data and stores it as log data.

[0289] Input: Input content (text and voice) and emotional information

[0290] Output: Input content and emotion information received by the server

[0291] Specific behavior:

[0292] The terminal sends the user's input data ("What year did Napoleon Bonaparte die?") and the emotion engine's analysis results ("Interesting") to the server.

[0293] The server receives this and stores it as log data.

[0294] Step 4:

[0295] The server sends the request content to multiple natural language generation systems (generative AI models A, B, and C) in parallel. Each generation system independently generates an answer to this request.

[0296] Input: User request

[0297] Output: Requests sent to multiple natural language generation systems

[0298] Specific behavior:

[0299] The server sends the request "What year did Napoleon Bonaparte die?" to generation AI models A, B, and C in parallel.

[0300] Each generative AI model generates its own answer to this request.

[0301] Step 5:

[0302] The server receives the output returned from each natural language generation system. For example, it may return outputs such as "1821," "1821," and "1820."

[0303] Input: Output from each generating system

[0304] Output: The output received by the server from each generating system.

[0305] Specific behavior:

[0306] The server receives outputs such as "1821" from Generative AI Model A, "1821" from Generative AI Model B, and "1820" from Generative AI Model C.

[0307] The server temporarily stores these outputs.

[0308] Step 6:

[0309] Based on the recognition results of the emotion engine, the server applies emotion adjustment logic to the received output to select the best response, for example, "1821" with a detailed description that matches the emotion "interesting."

[0310] Input: Emotional information and output from each generative system

[0311] Output: Optimal output (e.g., a detailed description of "1821")

[0312] Specific behavior:

[0313] The server takes into account the sentiment of "interesting" and selects "1821" with a detailed description as the best response.

[0314] Emotion regulation logic selects outputs that are adapted to the user's emotional state.

[0315] Step 7:

[0316] The final selected output is sent to the user's terminal and displayed to the user.

[0317] Input: Optimal Output

[0318] Output: The output sent to the user's terminal.

[0319] Specific behavior:

[0320] The server sends the optimal output (a detailed description of "1821") to the user's terminal.

[0321] The terminal displays this to the user so that the user can check the results.

[0322] (Application example 2)

[0323] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0324] Conventional natural language generation systems have limitations in providing appropriate output in response to user requests. Furthermore, they lack a mechanism for selecting output that corresponds to the user's emotional state, preventing improvements in user experience. In particular, in the advertising field, there is a demand for providing advertisements that are customized according to the user's emotions, but there has been a problem in that no appropriate system exists to achieve this.

[0325] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a request from a user, means for sending a request to a plurality of natural language generation systems, means for receiving output from the plurality of natural language generation systems, means for comparing and evaluating the received output, emotion recognition means for recognizing the user's emotion, advertisement selection means for selecting an optimal advertisement based on the output of the emotion recognition means, and means for providing the optimal advertisement to the user terminal. This makes it possible to provide optimal output and advertisements adapted to the user's emotional state.

[0326] The "means for receiving requests from users" is a function for sending questions or requests entered by users to the system.

[0327] The "means for sending requests to a plurality of natural language generation systems" is a function for sending requests received from a user to a plurality of natural language generation models in parallel.

[0328] The "means for receiving output from a plurality of natural language generation systems" is a function for receiving responses and generated text from each natural language generation system.

[0329] "Means for comparing and evaluating received outputs" refers to a function for comparing outputs obtained from multiple generative AI models to evaluate their quality and suitability.

[0330] "Means for removing outliers based on evaluation results" refers to techniques for removing outputs that lack validity as a result of evaluation.

[0331] "Means for selecting the optimal output" is a function for selecting the most appropriate output from the results of comparison and evaluation.

[0332] The "means for providing the optimal output to the user" is a function for returning the selected optimal output to the user.

[0333] The "emotion recognition means for recognizing the user's emotions" is a function for analyzing and recognizing the user's emotional state from their statements and text.

[0334] The "advertisement selection means for selecting the most suitable advertisement based on the output of the emotion recognition means" is a function for selecting an advertisement that is suited to the emotional state of the user determined by the emotion recognition means.

[0335] The "means for providing optimal advertisements to user terminals" is a function for displaying selected advertisements on the user's device.

[0336] The present invention provides a system for providing optimal output and advertisements adapted to the emotional state of a user. Specific embodiments of the system are described in detail below.

[0337] This system mainly includes a server and a user terminal, and is composed of the following hardware and software.

[0338] Hardware

[0339] server

[0340] Computational resources that operate under multiple generative AI models

[0341] Computing resources to run the emotion recognition engine

[0342] User terminal

[0343] Smartphone (iOS or ANDROID (registered trademark))

[0344] Input devices such as microphones and cameras

[0345] software

[0346] Python

[0347] Emotion recognition engine (e.g., a model built with Keras)

[0348] API communication (e.g., requests library)

[0349] Generative AI Models

[0350] Natural language generation engines (e.g., GPT-3)

[0351] Data processing and calculation

[0352] Server Processing

[0353] 1. Receiving a request from a user

[0354] The server receives requests from the user terminal in the form of voice or text, which are requests for information or advertisements that the user wishes to view.

[0355] 2. Emotion judgment

[0356] The received voice or text data is analyzed using an emotion recognition engine to recognize the user's emotional state. For voice data, the tone, pitch, and speed of the voice are analyzed, and for text data, keywords and context are analyzed.

[0357] 3. Sending a request

[0358] The server sends the user's request to multiple natural language generation systems (generative AI models) in parallel. Specifically, it sends requests to the API endpoints of the generative AI models.

[0359] 4. Receiving Output

[0360] The server receives the output returned from each generative AI system and temporarily stores it in storage.

[0361] 5. Output evaluation and outlier removal

[0362] Each received output is compared and evaluated using an evaluation model, and outliers are removed using statistical methods based on the evaluation results.

[0363] 6. Selecting the optimal output

[0364] From the evaluated outputs, the one that best suits the user's emotional state is selected, based on emotion regulation logic.

[0365] 7. Ad Selection

[0366] Based on the output of the emotion recognition engine, an algorithm is run to select the most suitable advertisement.

[0367] User terminal processing

[0368] 1. Submit a request

[0369] Users enter questions or requests through a dedicated application or web browser and send them to the server in voice or text format.

[0370] 2. Optimal output and display of ads

[0371] The optimal output and advertisement returned from the server are displayed on the user terminal.

[0372] Specific examples

[0373] Prompt Sentence Examples

[0374] The user speaks the following prompt into their smartphone:

[0375] "I'm in a mood to chill out. I want to see ads that help me relax."

[0376] Based on these prompts, the generative AI model generates and serves relaxing ads to users, such as ads for "relaxing scented candles" or "meditation apps."

[0377] As described above, the system of the present invention aims to improve the user experience by providing optimal information and advertisements to users based on the output and emotional state from multiple generative AI models.

[0378] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0379] Step 1:

[0380] The user inputs a request in voice or text format through a dedicated application or web browser using a device. The input request includes a specific question or request from the user. For example, a prompt such as "I feel like I want to relax. Please show me a relaxing advertisement" may be input.

[0381] Step 2:

[0382] The terminal receives a request from the user and sends the data to the server. Here, the request data is sent to the server via the network. The input is the user's request content, and the output is communication data to the server.

[0383] Step 3:

[0384] The server receives requests sent by users and passes them to the emotion recognition engine. The emotion recognition engine analyzes the input voice or text data to identify the user's emotional state. Specifically, it analyzes the tone, pitch, and speed of the voice, as well as keywords. The input is the user's request data, and the output is the user's emotional state.

[0385] Step 4:

[0386] Based on the emotional state output by the emotion recognition engine, the server sends requests to multiple generative AI models. Here, a request including a prompt sentence is sent to the API endpoint of the generative AI model. The input is the emotional state and the prompt sentence, and the output is the request data for the generative AI model.

[0387] Step 5:

[0388] The server receives output from multiple generative AI models. Each generative AI model returns the text it generates in response to the prompt. The input is the response data from the generative AI model, and the output is the generated output.

[0389] Step 6:

[0390] The server passes the received outputs to the evaluation model for comparison and evaluation. The evaluation model uses a machine learning algorithm to evaluate the quality and relevance of the outputs. The input is the generated multiple outputs, and the output is an evaluation score or ranking.

[0391] Step 7:

[0392] Based on the evaluation results, the server uses statistical methods to remove outliers, where significantly mismatched outputs are removed. The input is the output with the evaluation score, and the output is the list of outputs with the outliers removed.

[0393] Step 8:

[0394] The server selects the optimal output and advertisement based on the output of the emotion recognition engine. In this process, the output and advertisement that best fit the user's emotional state are selected based on the emotion adjustment logic. The input is the output list after outlier removal and the emotional state, and the output is the optimal output and advertisement.

[0395] Step 9:

[0396] The server sends the selected optimal output and advertisement to the user terminal. The input is the optimal output and advertisement, and the output is the communication data to the user terminal.

[0397] Step 10:

[0398] The terminal displays the optimal output and advertisement received from the server to the user. Here, the optimal response to the request entered by the user and the advertisement are displayed. The input is the communication data from the server, and the output is the display content of the user terminal.

[0399] Through the above steps, a system that adapts to the emotional state of the user and provides optimal output and advertisements is specifically realized.

[0400] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0401] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0402] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0403] [Second embodiment]

[0404] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0405] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0406] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0407] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0408] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0409] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0410] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0411] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0412] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0413] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0414] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0415] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0416] The present invention is a system for unifying the outputs from multiple natural language generation systems and providing the most appropriate response to the user. A specific form of this system is shown below.

[0417] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems. The user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[0418] User Action

[0419] Users access the system from their own devices using a dedicated application or a web browser. After accessing the system, the user enters a request, such as a question or a request to generate a sentence, and presses the send button. This operation sends the request to the server.

[0420] Server Processing

[0421] The server analyzes requests received from users and sends the request contents to multiple natural language generation systems in parallel. For example, requests are sent simultaneously to natural language generation systems A, B, and C. Each generation system generates an output based on the request and sends it back to the server.

[0422] The server collects the outputs returned by each generation system, compares and evaluates each output, and uses statistical methods and machine learning models as criteria for evaluation to eliminate outliers and inaccurate outputs. For example, it processes the output that receives the fewest votes overall. Through this process, the output deemed to be the most reliable is selected.

[0423] Specific examples

[0424] Consider a case where a user sends a request such as "What year did Napoleon Bonaparte die?" The server sends this request to natural language generation systems A, B, and C. Each generation system returns an answer. For example, if generation systems A and B return "1821" and generation system C returns "1820," the server evaluates these answers using statistical methods and machine learning models and selects the majority answer, "1821," as the optimal output. The server then returns this optimal output to the user.

[0425] In this way, the present invention is a system that can eliminate erroneous or biased information and provide users with accurate and reliable output.

[0426] The processing flow will be explained below.

[0427] Step 1:

[0428] The user uses a terminal to launch a dedicated application or a web browser, enters a question or a request for sentence generation in a request form, and clicks a send button.

[0429] Step 2:

[0430] The server receives a request from the user, which is sent to the server as an HTTP request and parsed.

[0431] Step 3:

[0432] The server analyzes the received request and sends it to multiple natural language generation systems in parallel, sending messages in the appropriate format to the API endpoints of each generation system.

[0433] Step 4:

[0434] Multiple natural language generation systems receive requests from the server, generate output through their own internal processing, and send it back to the server.

[0435] Step 5:

[0436] The server receives responses from each generation system. The received responses are obtained in a data format such as JSON.

[0437] Step 6:

[0438] The server aggregates the received multiple outputs, compares and evaluates the content of each output, and performs preprocessing of the data using statistical methods and machine learning models to evaluate its reliability.

[0439] Step 7:

[0440] Eliminate inaccurate outputs that are outliers based on the evaluation results, for example, eliminating the output that received the fewest votes.

[0441] Step 8:

[0442] The server selects the optimal output that is evaluated as the most reliable, i.e., the output with the highest score from the majority opinion or evaluation model.

[0443] Step 9:

[0444] The server returns the optimal output it has selected to the user, returning it to the user's device as an HTTP response and displaying it in the browser or application.

[0445] Step 10:

[0446] The user checks the optimal output returned on the device and receives the answer.

[0447] This system makes it possible to efficiently compare and evaluate the results of multiple generative AI services and provide users with highly reliable output.

[0448] Example 1

[0449] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0450] Current natural language generation systems rely on a single system, resulting in problems with the accuracy and consistency of responses. Furthermore, even when responses are obtained from multiple natural language generation systems, there is a lack of effective means for integrating these responses and selecting the optimal response. This makes it difficult to provide users with reliable information. Therefore, there is a need for a system that can compare and evaluate responses from multiple natural language generation systems, remove outliers, and provide the optimal response to the user.

[0451] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0452] In this invention, the server includes means for receiving a request from a user, means for transmitting a request to a plurality of natural language generation systems, means for receiving responses from the plurality of natural language generation systems, means for comparing and evaluating the received responses, means for removing outliers based on the evaluation result, means for selecting an optimal response from the set as a result of the evaluation, and means for providing the optimal response to the user. This makes it possible to effectively integrate responses obtained from the plurality of natural language generation systems and provide the user with reliable information.

[0453] A "user" is an entity that sends requests to the system and receives responses.

[0454] A "request" is information such as a question or a request for sentence generation that a user sends to the system.

[0455] "Multiple natural language generation systems" refers to a collection of multiple systems that generate natural language based on different algorithms or models.

[0456] A "response" is information that multiple natural language generation systems generate based on a request and return to the server.

[0457] "Comparing and evaluating means" refers to methods and techniques for comparing received responses with each other and evaluating their accuracy and reliability.

[0458] "Means for removing outliers" are methods or techniques that, based on the evaluation results, eliminate inaccurate responses that differ significantly from other responses.

[0459] "Means for selecting optimal responses" are methods or techniques for selecting the most appropriate and reliable responses based on the evaluation.

[0460] The "means for providing" refers to the method or technology for transmitting and displaying the selected optimal response to the user.

[0461] The present invention is a system that unifies responses from multiple natural language generation systems and provides the most appropriate response to the user. A specific form of this system is shown below.

[0462] composition

[0463] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems, and the user terminal provides an interface for users to input questions or requests and communicate with the server to obtain responses.

[0464] Hardware and software used

[0465] Hardware: Servers with high-performance processors and large amounts of memory, and devices for users to access the system (PCs, smartphones, tablets, etc.)

[0466] Software: Multiple natural language generation systems (generative AI models A, B, and C), statistical analysis software, and machine learning model evaluation software

[0467] Processing flow

[0468] User operations

[0469] Users access the system from their own devices using a dedicated application or web browser. After accessing the system, the user enters a question or request and presses the send button. This action sends the request to the server.

[0470] Specific examples

[0471] A user enters "What year did Napoleon Bonaparte die?" into the text input field of a web browser and clicks the submit button.

[0472] Server Processing

[0473] The server analyzes the request data received from the user and converts it into an appropriate format. It then sends the analyzed request to multiple natural language generation systems (generative AI models A, B, and C) in parallel and receives responses from each generation system.

[0474] The server collects the received responses, compares them with each other, and evaluates them. Statistical methods and machine learning models are used to filter out inaccurate responses and outliers. The server then selects the most reliable response and sends it back to the user.

[0475] Specific examples

[0476] The server sends the prompt "What year did Napoleon Bonaparte die?" to generative AI models A, B, and C. If generative AI models A and B respond with "1821" and generative AI model C responds with "1820," the server evaluates these responses using statistical methods and machine learning models, selects the majority response, "1821," as the optimal response, and returns it to the user device.

[0477] effect

[0478] This allows the system to effectively integrate information obtained from multiple natural language generation systems and provide users with highly reliable information, making the overall operation more efficient and significantly improving the user experience.

[0479] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0480] Step 1: User submits request

[0481] Specific operation: A user accesses the system using a dedicated application or a web browser from their own device (PC, smartphone, tablet, etc.), enters the request content (e.g., a question or a request to generate a sentence) in the text input field, and presses the send button.

[0482] Input: A user question or request for text generation (e.g., "What year did Napoleon Bonaparte die?")

[0483] Output: Request data sent to the server

[0484] Step 2: The server parses the request

[0485] Specific operation: The server analyzes the request data received from the user, understands the type and content of the request, and converts the request into an appropriate format based on this.

[0486] Input: Request data received from the user

[0487] Output: Parsed request data

[0488] Step 3: The server sends the request to the natural language generation system

[0489] Specific operation: The server sends parsed requests in parallel to multiple natural language generation systems (generative AI models A, B, and C).

[0490] Input: Parsed request data

[0491] Output: A prompt sent to each generative AI model (e.g., "What year did Napoleon Bonaparte die?")

[0492] Step 4: The natural language generation system generates a response

[0493] Specific operation: Each generative AI model generates a response based on the sent prompt sentence and sends it back to the server.

[0494] Input: Prompt sent to each generative AI model

[0495] Output: Response data returned by each generative AI model (e.g., Generative AI Models A and B respond "1821," while Generative AI Model C responds "1820")

[0496] Step 5: The server collects and compares the responses

[0497] What it does: The server collects the responses from each generation system, compares and evaluates them against each other, and uses statistical techniques and machine learning models to filter out inaccurate responses and outliers.

[0498] Input: Response data from each generative AI model

[0499] Output: Evaluated response data with incorrect responses and outliers removed

[0500] Step 6: The server picks the best response

[0501] Specific behavior: The server selects the single most authoritative response based on the evaluation results, e.g., the majority response.

[0502] Input: Evaluated response data

[0503] Output: Best response (e.g. "1821")

[0504] Step 7: The server provides the best response to the user

[0505] Specific operation: The server sends the selected best response to the user's terminal, where the user can receive and confirm it.

[0506] Input: Best response

[0507] Output: Response data sent to the user device

[0508] (Application example 1)

[0509] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0510] On conventional online shopping sites, it has been difficult to obtain a quick and accurate answer when users have questions about products. This problem can lead to a decrease in user satisfaction and a decrease in purchasing intent. Another issue is that there has been no effective method for integrating answers from multiple natural language generation systems and selecting the most appropriate answer.

[0511] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0512] In this invention, the server includes means for receiving requests from a user, means for transmitting requests to a plurality of natural language generation systems, means for receiving outputs from the plurality of natural language generation systems, means for comparing and evaluating the received outputs, means for removing outliers based on the evaluation results, means for selecting an optimal output, means for providing the optimal output to a user, means for communicating with an external content providing system, and means for responding with the optimal output in response to a product-related question from a user on an online shopping site. This makes it possible to provide a quick and accurate answer to a product-related question from a user, thereby increasing user satisfaction.

[0513] The "means for receiving requests from a user" is a function for sending questions or requests input by a user through an application to a server.

[0514] The "means for transmitting requests to a plurality of natural language generation systems" is a function that allows the server to simultaneously transmit requests received from users to a plurality of natural language generation systems.

[0515] The "means for receiving outputs from a plurality of natural language generation systems" refers to a function by which the server collects responses generated from each natural language generation system.

[0516] "Means for comparing and evaluating received outputs" refers to a function in which the server compares and evaluates multiple outputs using statistical methods or machine learning models.

[0517] The "means for removing outliers based on the evaluation results" is a function for removing unreliable outputs based on the evaluation results.

[0518] The "means for selecting the optimal output" is a function that selects the most reliable output through evaluation and removal of outliers.

[0519] The "means for providing the optimal output to the user" is a function that returns the selected optimal output to the user.

[0520] The "means for communicating with an external content providing system" is a communication function that enables the server to exchange information with an external service.

[0521] "Means for providing the optimal output in response to a question about a product from a user on an online shopping site" is a function that provides the optimal answer when a user asks a question about a product on an online shopping site.

[0522] The present invention provides a system for providing quick and accurate answers to user questions and requests. This system is particularly effective for online shopping sites and aims to improve the user experience. Specific embodiments of the system are described below.

[0523] The system consists of a server, a user terminal, and multiple natural language generation systems. The entire system operates with the following components:

[0524] Hardware and Software Configuration

[0525] Server: The server receives requests from users and has an interface for communicating with multiple natural language generation systems. Specifically, a cloud server such as AWS EC2 is used.

[0526] User terminal: The user terminal provides an interface for inputting user questions and requests through a smartphone application.

[0527] Natural language generation systems: External systems equipped with generative AI models (e.g., BERT, GPT-3) are used.

[0528] Data processing and calculation

[0529] 1. User input processing: The user enters a question or request through the application. This input data is sent from the user's device to the server.

[0530] 2. Request distribution: The server sends the received user request to multiple natural language generation systems in parallel, forming an appropriate prompt for each system.

[0531] For example: Question: "Is this T-shirt machine washable?"

[0532] 3. Receiving and evaluating the output: The server receives the answers from each generation system and evaluates and compares these answers using statistical methods and machine learning models.

[0533] 4. Outlier Removal: This process removes unreliable outputs based on the evaluation results. Specifically, it removes the output with the fewest votes.

[0534] 5. Selecting and providing the optimal output: Finally, the output with the highest rating is selected and returned to the user.

[0535] Specific examples

[0536] A user types a question into an online shopping site: "Is this T-shirt machine washable?" The system works as follows:

[0537] The server receives user requests and sends the requests to multiple natural language generation systems.

[0538] Each generating system returns the answer "Yes, this T-shirt is machine washable" to the server.

[0539] The server compares and evaluates these answers and provides the best answer to the user.

[0540] This system allows users to quickly obtain accurate and reliable answers, which is expected to improve the online shopping experience and increase user satisfaction.

[0541] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0542] Step 1:

[0543] A user accesses an online shopping site and inputs a question or request about a product through the application. The input information is sent from the user's terminal to the server.

[0544] Input: "Is this t-shirt machine washable?"

[0545] Output: The question is sent to the server.

[0546] Step 2:

[0547] The server receives the user's request, parses and understands it, and formats it in a format suitable for multiple natural language generation systems.

[0548] Input: User question

[0549] Output: Formatted prompt (e.g., "Question: Is this T-shirt machine washable?")

[0550] Step 3:

[0551] The server sends formatted prompt sentences to multiple natural language generation systems in parallel, sending requests to each generation system independently.

[0552] Input: Formatted prompt text

[0553] Output: The requests sent to each NLG system

[0554] Step 4:

[0555] Each natural language generation system generates a response based on the request and sends it back to the server.

[0556] Input: The request sent to each natural language generation system

[0557] Output: The answer from each generating system (e.g., "Yes, this T-shirt is machine washable.")

[0558] Step 5:

[0559] The server receives and collects responses from multiple natural language generation systems, storing all responses in a temporary database.

[0560] Input: Answers from each generating system

[0561] Output: Save all responses to a temporary database

[0562] Step 6:

[0563] The responses collected by the server are evaluated using statistical methods and machine learning models, and their reliability and relevance are assessed, with outlier responses being removed based on statistical methods.

[0564] Input: Response stored in temporary database

[0565] Output: Evaluation score and outlier removal results

[0566] Step 7:

[0567] Select the most reliable response based on the evaluation results. Sort by highest evaluation score to select the most appropriate response.

[0568] Input: Evaluation scores and outlier removal results

[0569] Output: Best response

[0570] Step 8:

[0571] The server formats the best response for return to the user and sends it to the user's terminal, which receives it and displays it on its screen.

[0572] Input: Best response

[0573] Output: The optimal response that will be displayed to the user (e.g., "Yes, this T-shirt is machine washable.")

[0574] The above are the detailed processing steps from when the user inputs a question about a product to when the most suitable response is obtained.

[0575] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0576] This invention is a system that unifies the outputs from multiple natural language generation systems and provides the most appropriate response to the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide an output that is adapted to the user's emotions. Specific forms of this system are shown below.

[0577] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems and an emotion engine. The user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[0578] User Action

[0579] Users access the system from their own devices using a dedicated application or web browser. After accessing the system, they input requests such as questions or requests for sentence generation. At this time, the emotion engine recognizes the user's emotional state through voice or text input. The request and emotional state are then sent to the server.

[0580] Server Processing

[0581] The server analyzes the request content and emotional state received from the user. The request content is sent to multiple natural language generation systems in parallel, and the server receives the response output from each generation system. The server then takes into account the emotional state recognized by the emotion engine in its evaluation and applies emotion adjustment logic to ultimately select the optimal output.

[0582] Specifically, if a user requests "What year did Napoleon Bonaparte die?" and the emotion engine recognizes the emotional state as "interesting," the server sends this information in parallel to natural language generation systems A, B, and C. Each generation system returns answers such as "1821," "1821," and "1820," respectively. At this time, the emotional information recognized by the emotion engine is taken into account by the emotion adjustment logic, and an output that provides a detailed explanation of "1821" in a way that matches the emotion of "interesting" is selected.

[0583] Emotion Engine

[0584] The emotion engine uses speech and text analysis techniques to recognize the user's emotions. For voice input, it analyzes the tone, pitch, and speed of the voice to determine whether the user is excited or calm. For text input, it analyzes keywords and context in the content to identify emotions. The recognized emotion information is sent to the server along with the request and is used to evaluate and optimize the output.

[0585] Specific examples

[0586] If the emotion engine recognizes that the user is in an "uneasy" state when making a request, the system will select a generation system that is more likely to generate sentences that "provide a sense of security" to the user. Also, if the user makes a request in a "happy" state, the system will tend to select "fun and attractive" output.

[0587] This system can efficiently compare and evaluate the results of multiple generative AI services and provide output that is adapted to the user's emotional state, improving the user experience and providing information that better meets the user's needs.

[0588] The processing flow will be explained below.

[0589] Step 1:

[0590] The user uses a device to launch a dedicated application or a web browser, enters a question or a request for sentence generation into the request form, and clicks the send button. At this time, the device sends the user's emotion data to the emotion engine via voice or text input.

[0591] Step 2:

[0592] The server receives a request from the user, along with the request content and the user's emotion data analyzed by the emotion engine from the device.

[0593] Step 3:

[0594] The server analyzes the received request content and sends the request content and emotion data to multiple natural language generation systems in parallel. It sends messages in an appropriate format to the API endpoint of each generation system.

[0595] Step 4:

[0596] Multiple natural language generation systems receive requests from the server, generate output through their own internal processing, and send it back to the server. The response from each system is sent back to the server as generated text data.

[0597] Step 5:

[0598] The server receives responses from each generation system. The received responses are obtained in a data format such as JSON. For example, answers from natural language generation systems A, B, and C are obtained.

[0599] Step 6:

[0600] The server aggregates the received multiple outputs, compares and evaluates the content of each output, and performs preprocessing of the data using statistical methods and machine learning models to evaluate its reliability.

[0601] Step 7:

[0602] The server uses the emotion data from the emotion engine to filter out inaccurate outputs that are outliers based on the evaluation results, and then applies emotion adjustment logic to select outputs that reflect the emotion data.

[0603] Step 8:

[0604] The server selects the optimal output that is evaluated as the most reliable, specifically, the output with the highest score on majority opinion and emotion regulation logic.

[0605] Step 9:

[0606] The server returns the optimal output it has selected to the user, returning it to the user's device as an HTTP response and displaying it in the browser or application.

[0607] Step 10:

[0608] The user checks the best output on their device and receives the answer. For example, the response "Napoleon Bonaparte died in 1821" is displayed along with an additional explanation based on the emotion recognized by the emotion engine (such as "He was a great leader of his time").

[0609] This system can efficiently compare and evaluate the results of multiple generative AI services and provide output that is adapted to the user's emotional state, improving the user experience and providing information that better meets the user's needs.

[0610] Example 2

[0611] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0612] Current natural language generation systems have a mechanism for evaluating the outputs of multiple systems to obtain the optimal response to a user request, but because they do not take into account the user's emotional state, they are unable to provide the optimal response that meets the user's needs.In addition, there is no emotion-based evaluation in output selection, which results in a suboptimal user experience.

[0613] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0614] In this invention, the server includes means for receiving a user request, means for recognizing the user's emotion, means for transmitting the user's request content and emotional state, means for transmitting the request to a plurality of natural language generation systems, means for receiving outputs from the plurality of natural language generation systems, means for comparing and evaluating the received outputs based on the emotional state, means for selecting an optimal output based on the evaluation result, and means for providing the optimal output to the user. This makes it possible to select an optimal response from the outputs of the plurality of generation systems while taking the user's emotional state into consideration, thereby improving the user experience.

[0615] The "means for receiving requests" is a function that provides an interface for receiving questions or information requests from users and inputting them into the system.

[0616] "Means for recognizing emotions" is a function that analyzes the user's voice and text input to identify their emotional state at that time.

[0617] The "means for transmitting the request content and emotional state" is a function for transferring the request content received from the user and the recognized emotional state to the server.

[0618] The "means for sending a request to a natural language generation system" is a function that enables the server to send the request contents to multiple natural language generation engines in parallel.

[0619] The "means for receiving output" is a function for receiving response results generated from multiple natural language generation engines.

[0620] The "means for comparing and evaluating based on emotional state" is a function for analyzing each received output based on the user's emotional state and selecting the most suitable one.

[0621] The "means for selecting the optimal output" is a function for comparing and examining response results from multiple natural language generation systems and selecting the optimal response based on evaluation criteria (e.g., emotional state).

[0622] The "means for providing output to the user" is a function for sending the selected optimal response to the user's terminal and displaying the result so that the user can confirm it.

[0623] The present invention is a system that unifies the outputs of multiple natural language generation systems in response to a user request and provides the most appropriate response. This system is composed of a server and a user terminal, and operates as follows.

[0624] System Configuration

[0625] In this system, the server has an interface for communicating with multiple natural language generation systems and emotion engines, and the user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[0626] Hardware and software used

[0627] The server requires a high-performance processor and a large amount of memory. In addition, a "generative AI model" (Model A, Model B, Model C) is used as the natural language generation system. The emotion engine requires an engine with voice analysis and text analysis technology. The user terminal is a smartphone or PC with an internet connection.

[0628] Process Overview

[0629] 1. Accepting user input

[0630] Users access the system from their own devices using a dedicated application or a web browser, and input requests such as questions and sentence generation requests by text or voice.

[0631] 2. User Emotion Recognition

[0632] The emotion engine analyzes the user's input (text or voice) to recognize the user's emotional state. For voice input, it analyzes the tone, pitch, and speed of the voice, and for text input, it analyzes keywords and context of the content to identify emotions.

[0633] 3. Sending request data

[0634] The device sends the user's input and emotional state to the server, which receives and stores them as log data.

[0635] 4. Processing of Natural Language Generation Systems

[0636] The server sends the request content to multiple natural language generation systems (generative AI models A, B, and C) in parallel. Each generation system independently generates an answer to this request.

[0637] 5. Receiving the output

[0638] The server receives the output returned from each natural language generation system. For example, it may return outputs such as "1821," "1821," and "1820."

[0639] 6. Applying emotion regulation logic

[0640] Based on the recognition results of the emotion engine, the server applies emotion adjustment logic to the received output to select the best response, for example, "1821" with a detailed description that matches the emotion "interesting."

[0641] 7. Providing a response to the user

[0642] The final selected output is sent to the user's terminal and displayed to the user.

[0643] Specific examples

[0644] For example, if the emotion engine recognizes that the user is in an "unsettled" state when making a request, the system will select a generation system that is likely to generate sentences that "provide a sense of security" to the user. Also, if the user makes a request while in a "happy" state, the system will tend to select "fun and attractive" output.

[0645] Prompt Sentence Examples

[0646] Below are some examples of prompt sentences to input to the generative AI model.

[0647] Please provide more details about the life of a historical figure that you find interesting.

[0648] "Users are worried, so please provide some comforting words."

[0649] This system allows users to quickly obtain the most appropriate information adapted to their emotional state, improving the user experience and increasing the accuracy and relevance of the information provided.

[0650] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0651] Step 1:

[0652] When a user opens a dedicated application or web browser on their device, an input form appears, where the user can enter their question or request via text or voice. Once the input is complete, the device sends the data to the next processing step.

[0653] Input: A user question or text generation request (e.g., "What year did Napoleon Bonaparte die?")

[0654] Output: Text or audio data is sent to the device.

[0655] Specific behavior:

[0656] The user can type "What year did Napoleon Bonaparte die?" into the text field or say "What year did Napoleon Bonaparte die?"

[0657] The terminal sends the entered data to a dedicated application.

[0658] Step 2:

[0659] The emotion engine analyzes text or voice data sent from the device to recognize the user's emotional state. For voice input, it analyzes the tone, pitch, and speed of the voice, and for text input, it analyzes keywords and context of the content to identify emotions.

[0660] Input: Text or audio data

[0661] Output: Analyzed user sentiment information (e.g., "interesting")

[0662] Specific behavior:

[0663] When voice input is used, the emotion engine analyzes the tone, pitch, and speed of the voice in real time to recognize emotions such as "interesting."

[0664] When text is entered, the emotion engine analyzes emotions from keywords and context in the entered text.

[0665] Step 3:

[0666] The device sends the user's input and emotional state to the server, which receives this data and stores it as log data.

[0667] Input: Input content (text and voice) and emotional information

[0668] Output: Input content and emotion information received by the server

[0669] Specific behavior:

[0670] The terminal sends the user's input data ("What year did Napoleon Bonaparte die?") and the emotion engine's analysis results ("Interesting") to the server.

[0671] The server receives this and stores it as log data.

[0672] Step 4:

[0673] The server sends the request content to multiple natural language generation systems (generative AI models A, B, and C) in parallel. Each generation system independently generates an answer to this request.

[0674] Input: User request

[0675] Output: Requests sent to multiple natural language generation systems

[0676] Specific behavior:

[0677] The server sends the request "What year did Napoleon Bonaparte die?" to generation AI models A, B, and C in parallel.

[0678] Each generative AI model generates its own answer to this request.

[0679] Step 5:

[0680] The server receives the output returned from each natural language generation system. For example, it may return outputs such as "1821," "1821," and "1820."

[0681] Input: Output from each generating system

[0682] Output: The output received by the server from each generating system.

[0683] Specific behavior:

[0684] The server receives outputs such as "1821" from Generative AI Model A, "1821" from Generative AI Model B, and "1820" from Generative AI Model C.

[0685] The server temporarily stores these outputs.

[0686] Step 6:

[0687] Based on the recognition results of the emotion engine, the server applies emotion adjustment logic to the received output to select the best response, for example, "1821" with a detailed description that matches the emotion "interesting."

[0688] Input: Emotional information and output from each generative system

[0689] Output: Optimal output (e.g., a detailed description of "1821")

[0690] Specific behavior:

[0691] The server takes into account the sentiment of "interesting" and selects "1821" with a detailed description as the best response.

[0692] Emotion regulation logic selects outputs that are adapted to the user's emotional state.

[0693] Step 7:

[0694] The final selected output is sent to the user's terminal and displayed to the user.

[0695] Input: Optimal Output

[0696] Output: The output sent to the user's terminal.

[0697] Specific behavior:

[0698] The server sends the optimal output (a detailed description of "1821") to the user's terminal.

[0699] The terminal displays this to the user so that the user can check the results.

[0700] (Application example 2)

[0701] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0702] Conventional natural language generation systems have limitations in providing appropriate output in response to user requests. Furthermore, they lack a mechanism for selecting output that corresponds to the user's emotional state, preventing improvements in user experience. In particular, in the advertising field, there is a demand for providing advertisements that are customized according to the user's emotions, but there has been a problem in that no appropriate system exists to achieve this.

[0703] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a request from a user, means for sending a request to a plurality of natural language generation systems, means for receiving output from the plurality of natural language generation systems, means for comparing and evaluating the received output, emotion recognition means for recognizing the user's emotion, advertisement selection means for selecting an optimal advertisement based on the output of the emotion recognition means, and means for providing the optimal advertisement to the user terminal. This makes it possible to provide optimal output and advertisements adapted to the user's emotional state.

[0704] The "means for receiving requests from users" is a function for sending questions or requests entered by users to the system.

[0705] The "means for sending requests to a plurality of natural language generation systems" is a function for sending requests received from a user to a plurality of natural language generation models in parallel.

[0706] The "means for receiving output from a plurality of natural language generation systems" is a function for receiving responses and generated text from each natural language generation system.

[0707] "Means for comparing and evaluating received outputs" refers to a function for comparing outputs obtained from multiple generative AI models to evaluate their quality and suitability.

[0708] "Means for removing outliers based on evaluation results" refers to techniques for removing outputs that lack validity as a result of evaluation.

[0709] "Means for selecting the optimal output" is a function for selecting the most appropriate output from the results of comparison and evaluation.

[0710] The "means for providing the optimal output to the user" is a function for returning the selected optimal output to the user.

[0711] The "emotion recognition means for recognizing the user's emotions" is a function for analyzing and recognizing the user's emotional state from their statements and text.

[0712] The "advertisement selection means for selecting the most suitable advertisement based on the output of the emotion recognition means" is a function for selecting an advertisement that is suited to the emotional state of the user determined by the emotion recognition means.

[0713] The "means for providing optimal advertisements to user terminals" is a function for displaying selected advertisements on the user's device.

[0714] The present invention provides a system for providing optimal output and advertisements adapted to the emotional state of a user. Specific embodiments of the system are described in detail below.

[0715] This system mainly includes a server and a user terminal, and is composed of the following hardware and software.

[0716] Hardware

[0717] server

[0718] Computational resources that operate under multiple generative AI models

[0719] Computing resources to run the emotion recognition engine

[0720] User terminal

[0721] Smartphone (iOS or Android)

[0722] Input devices such as microphones and cameras

[0723] software

[0724] Python

[0725] Emotion recognition engine (e.g., a model built with Keras)

[0726] API communication (e.g., requests library)

[0727] Generative AI Models

[0728] Natural language generation engines (e.g., GPT-3)

[0729] Data processing and calculation

[0730] Server Processing

[0731] 1. Receiving a request from a user

[0732] The server receives requests from the user terminal in the form of voice or text, which are requests for information or advertisements that the user wishes to view.

[0733] 2. Emotion judgment

[0734] The received voice or text data is analyzed using an emotion recognition engine to recognize the user's emotional state. For voice data, the tone, pitch, and speed of the voice are analyzed, and for text data, keywords and context are analyzed.

[0735] 3. Sending a request

[0736] The server sends the user's request to multiple natural language generation systems (generative AI models) in parallel. Specifically, it sends requests to the API endpoints of the generative AI models.

[0737] 4. Receiving Output

[0738] The server receives the output returned from each generative AI system and temporarily stores it in storage.

[0739] 5. Output evaluation and outlier removal

[0740] Each received output is compared and evaluated using an evaluation model, and outliers are removed using statistical methods based on the evaluation results.

[0741] 6. Selecting the optimal output

[0742] From the evaluated outputs, the one that best suits the user's emotional state is selected, based on emotion regulation logic.

[0743] 7. Ad Selection

[0744] Based on the output of the emotion recognition engine, an algorithm is run to select the most suitable advertisement.

[0745] User terminal processing

[0746] 1. Submit a request

[0747] Users enter questions or requests through a dedicated application or web browser and send them to the server in voice or text format.

[0748] 2. Optimal output and display of ads

[0749] The optimal output and advertisement returned from the server are displayed on the user terminal.

[0750] Specific examples

[0751] Prompt Sentence Examples

[0752] The user speaks the following prompt into their smartphone:

[0753] "I'm in a mood to chill out. I want to see ads that help me relax."

[0754] Based on these prompts, the generative AI model generates and serves relaxing ads to users, such as ads for "relaxing scented candles" or "meditation apps."

[0755] As described above, the system of the present invention aims to improve the user experience by providing optimal information and advertisements to users based on the output and emotional state from multiple generative AI models.

[0756] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0757] Step 1:

[0758] The user inputs a request in voice or text format through a dedicated application or web browser using a device. The input request includes a specific question or request from the user. For example, a prompt such as "I feel like I want to relax. Please show me a relaxing advertisement" may be input.

[0759] Step 2:

[0760] The terminal receives a request from the user and sends the data to the server. Here, the request data is sent to the server via the network. The input is the user's request content, and the output is communication data to the server.

[0761] Step 3:

[0762] The server receives requests sent by users and passes them to the emotion recognition engine. The emotion recognition engine analyzes the input voice or text data to identify the user's emotional state. Specifically, it analyzes the tone, pitch, and speed of the voice, as well as keywords. The input is the user's request data, and the output is the user's emotional state.

[0763] Step 4:

[0764] Based on the emotional state output by the emotion recognition engine, the server sends requests to multiple generative AI models. Here, a request including a prompt sentence is sent to the API endpoint of the generative AI model. The input is the emotional state and the prompt sentence, and the output is the request data for the generative AI model.

[0765] Step 5:

[0766] The server receives output from multiple generative AI models. Each generative AI model returns the text it generates in response to the prompt. The input is the response data from the generative AI model, and the output is the generated output.

[0767] Step 6:

[0768] The server passes the received outputs to the evaluation model for comparison and evaluation. The evaluation model uses a machine learning algorithm to evaluate the quality and relevance of the outputs. The input is the generated multiple outputs, and the output is an evaluation score or ranking.

[0769] Step 7:

[0770] Based on the evaluation results, the server uses statistical methods to remove outliers, where significantly mismatched outputs are removed. The input is the output with the evaluation score, and the output is the list of outputs with the outliers removed.

[0771] Step 8:

[0772] The server selects the optimal output and advertisement based on the output of the emotion recognition engine. In this process, the output and advertisement that best fit the user's emotional state are selected based on the emotion adjustment logic. The input is the output list after outlier removal and the emotional state, and the output is the optimal output and advertisement.

[0773] Step 9:

[0774] The server sends the selected optimal output and advertisement to the user terminal. The input is the optimal output and advertisement, and the output is the communication data to the user terminal.

[0775] Step 10:

[0776] The terminal displays the optimal output and advertisement received from the server to the user. Here, the optimal response to the request entered by the user and the advertisement are displayed. The input is the communication data from the server, and the output is the display content of the user terminal.

[0777] Through the above steps, a system that adapts to the emotional state of the user and provides optimal output and advertisements is specifically realized.

[0778] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0779] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0780] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0781] [Third embodiment]

[0782] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0783] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0784] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0785] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0786] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0787] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0788] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0789] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0790] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0791] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0792] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0793] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0794] The present invention is a system for unifying the outputs from multiple natural language generation systems and providing the most appropriate response to the user. A specific form of this system is shown below.

[0795] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems. The user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[0796] User Action

[0797] Users access the system from their own devices using a dedicated application or a web browser. After accessing the system, the user enters a request, such as a question or a request to generate a sentence, and presses the send button. This operation sends the request to the server.

[0798] Server Processing

[0799] The server analyzes requests received from users and sends the request contents to multiple natural language generation systems in parallel. For example, requests are sent simultaneously to natural language generation systems A, B, and C. Each generation system generates an output based on the request and sends it back to the server.

[0800] The server collects the outputs returned by each generation system, compares and evaluates each output, and uses statistical methods and machine learning models as criteria for evaluation to eliminate outliers and inaccurate outputs. For example, it processes the output that receives the fewest votes overall. Through this process, the output deemed to be the most reliable is selected.

[0801] Specific examples

[0802] Consider a case where a user sends a request such as "What year did Napoleon Bonaparte die?" The server sends this request to natural language generation systems A, B, and C. Each generation system returns an answer. For example, if generation systems A and B return "1821" and generation system C returns "1820," the server evaluates these answers using statistical methods and machine learning models and selects the majority answer, "1821," as the optimal output. The server then returns this optimal output to the user.

[0803] In this way, the present invention is a system that can eliminate erroneous or biased information and provide users with accurate and reliable output.

[0804] The processing flow will be explained below.

[0805] Step 1:

[0806] The user uses a terminal to launch a dedicated application or a web browser, enters a question or a request for sentence generation in a request form, and clicks a send button.

[0807] Step 2:

[0808] The server receives a request from the user, which is sent to the server as an HTTP request and parsed.

[0809] Step 3:

[0810] The server analyzes the received request and sends it to multiple natural language generation systems in parallel, sending messages in the appropriate format to the API endpoints of each generation system.

[0811] Step 4:

[0812] Multiple natural language generation systems receive requests from the server, generate output through their own internal processing, and send it back to the server.

[0813] Step 5:

[0814] The server receives responses from each generation system. The received responses are obtained in a data format such as JSON.

[0815] Step 6:

[0816] The server aggregates the received multiple outputs, compares and evaluates the content of each output, and performs preprocessing of the data using statistical methods and machine learning models to evaluate its reliability.

[0817] Step 7:

[0818] Eliminate inaccurate outputs that are outliers based on the evaluation results, for example, eliminating the output that received the fewest votes.

[0819] Step 8:

[0820] The server selects the optimal output that is evaluated as the most reliable, i.e., the output with the highest score from the majority opinion or evaluation model.

[0821] Step 9:

[0822] The server returns the optimal output it has selected to the user, returning it to the user's device as an HTTP response and displaying it in the browser or application.

[0823] Step 10:

[0824] The user checks the optimal output returned on the device and receives the answer.

[0825] This system makes it possible to efficiently compare and evaluate the results of multiple generative AI services and provide users with highly reliable output.

[0826] Example 1

[0827] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0828] Current natural language generation systems rely on a single system, resulting in problems with the accuracy and consistency of responses. Furthermore, even when responses are obtained from multiple natural language generation systems, there is a lack of effective means for integrating these responses and selecting the optimal response. This makes it difficult to provide users with reliable information. Therefore, there is a need for a system that can compare and evaluate responses from multiple natural language generation systems, remove outliers, and provide the optimal response to the user.

[0829] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0830] In this invention, the server includes means for receiving a request from a user, means for transmitting a request to a plurality of natural language generation systems, means for receiving responses from the plurality of natural language generation systems, means for comparing and evaluating the received responses, means for removing outliers based on the evaluation result, means for selecting an optimal response from the set as a result of the evaluation, and means for providing the optimal response to the user. This makes it possible to effectively integrate responses obtained from the plurality of natural language generation systems and provide the user with reliable information.

[0831] A "user" is an entity that sends requests to the system and receives responses.

[0832] A "request" is information such as a question or a request for sentence generation that a user sends to the system.

[0833] "Multiple natural language generation systems" refers to a collection of multiple systems that generate natural language based on different algorithms or models.

[0834] A "response" is information that multiple natural language generation systems generate based on a request and return to the server.

[0835] "Comparing and evaluating means" refers to methods and techniques for comparing received responses with each other and evaluating their accuracy and reliability.

[0836] "Means for removing outliers" are methods or techniques that, based on the evaluation results, eliminate inaccurate responses that differ significantly from other responses.

[0837] "Means for selecting optimal responses" are methods or techniques for selecting the most appropriate and reliable responses based on the evaluation.

[0838] The "means for providing" refers to the method or technology for transmitting and displaying the selected optimal response to the user.

[0839] The present invention is a system that unifies responses from multiple natural language generation systems and provides the most appropriate response to the user. A specific form of this system is shown below.

[0840] composition

[0841] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems, and the user terminal provides an interface for users to input questions or requests and communicate with the server to obtain responses.

[0842] Hardware and software used

[0843] Hardware: Servers with high-performance processors and large amounts of memory, and devices for users to access the system (PCs, smartphones, tablets, etc.)

[0844] Software: Multiple natural language generation systems (generative AI models A, B, and C), statistical analysis software, and machine learning model evaluation software

[0845] Processing flow

[0846] User operations

[0847] Users access the system from their own devices using a dedicated application or web browser. After accessing the system, the user enters a question or request and presses the send button. This action sends the request to the server.

[0848] Specific examples

[0849] A user enters "What year did Napoleon Bonaparte die?" into the text input field of a web browser and clicks the submit button.

[0850] Server Processing

[0851] The server analyzes the request data received from the user and converts it into an appropriate format. It then sends the analyzed request to multiple natural language generation systems (generative AI models A, B, and C) in parallel and receives responses from each generation system.

[0852] The server collects the received responses, compares them with each other, and evaluates them. Statistical methods and machine learning models are used to filter out inaccurate responses and outliers. The server then selects the most reliable response and sends it back to the user.

[0853] Specific examples

[0854] The server sends the prompt "What year did Napoleon Bonaparte die?" to generative AI models A, B, and C. If generative AI models A and B respond with "1821" and generative AI model C responds with "1820," the server evaluates these responses using statistical methods and machine learning models, selects the majority response, "1821," as the optimal response, and returns it to the user device.

[0855] effect

[0856] This allows the system to effectively integrate information obtained from multiple natural language generation systems and provide users with highly reliable information, making the overall operation more efficient and significantly improving the user experience.

[0857] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0858] Step 1: User submits request

[0859] Specific operation: A user accesses the system using a dedicated application or a web browser from their own device (PC, smartphone, tablet, etc.), enters the request content (e.g., a question or a request to generate a sentence) in the text input field, and presses the send button.

[0860] Input: A user question or request for text generation (e.g., "What year did Napoleon Bonaparte die?")

[0861] Output: Request data sent to the server

[0862] Step 2: The server parses the request

[0863] Specific operation: The server analyzes the request data received from the user, understands the type and content of the request, and converts the request into an appropriate format based on this.

[0864] Input: Request data received from the user

[0865] Output: Parsed request data

[0866] Step 3: The server sends the request to the natural language generation system

[0867] Specific operation: The server sends parsed requests in parallel to multiple natural language generation systems (generative AI models A, B, and C).

[0868] Input: Parsed request data

[0869] Output: A prompt sent to each generative AI model (e.g., "What year did Napoleon Bonaparte die?")

[0870] Step 4: The natural language generation system generates a response

[0871] Specific operation: Each generative AI model generates a response based on the sent prompt sentence and sends it back to the server.

[0872] Input: Prompt sent to each generative AI model

[0873] Output: Response data returned by each generative AI model (e.g., Generative AI Models A and B respond "1821," while Generative AI Model C responds "1820")

[0874] Step 5: The server collects and compares the responses

[0875] What it does: The server collects the responses from each generation system, compares and evaluates them against each other, and uses statistical techniques and machine learning models to filter out inaccurate responses and outliers.

[0876] Input: Response data from each generative AI model

[0877] Output: Evaluated response data with incorrect responses and outliers removed

[0878] Step 6: The server picks the best response

[0879] Specific behavior: The server selects the single most authoritative response based on the evaluation results, e.g., the majority response.

[0880] Input: Evaluated response data

[0881] Output: Best response (e.g. "1821")

[0882] Step 7: The server provides the best response to the user

[0883] Specific operation: The server sends the selected best response to the user's terminal, where the user can receive and confirm it.

[0884] Input: Best response

[0885] Output: Response data sent to the user device

[0886] (Application example 1)

[0887] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0888] On conventional online shopping sites, it has been difficult to obtain a quick and accurate answer when users have questions about products. This problem can lead to a decrease in user satisfaction and a decrease in purchasing intent. Another issue is that there has been no effective method for integrating answers from multiple natural language generation systems and selecting the most appropriate answer.

[0889] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0890] In this invention, the server includes means for receiving requests from a user, means for transmitting requests to a plurality of natural language generation systems, means for receiving outputs from the plurality of natural language generation systems, means for comparing and evaluating the received outputs, means for removing outliers based on the evaluation results, means for selecting an optimal output, means for providing the optimal output to a user, means for communicating with an external content providing system, and means for responding with the optimal output in response to a product-related question from a user on an online shopping site. This makes it possible to provide a quick and accurate answer to a product-related question from a user, thereby increasing user satisfaction.

[0891] The "means for receiving requests from a user" is a function for sending questions or requests input by a user through an application to a server.

[0892] The "means for transmitting requests to a plurality of natural language generation systems" is a function that allows the server to simultaneously transmit requests received from users to a plurality of natural language generation systems.

[0893] The "means for receiving outputs from a plurality of natural language generation systems" refers to a function by which the server collects responses generated from each natural language generation system.

[0894] "Means for comparing and evaluating received outputs" refers to a function in which the server compares and evaluates multiple outputs using statistical methods or machine learning models.

[0895] The "means for removing outliers based on the evaluation results" is a function for removing unreliable outputs based on the evaluation results.

[0896] The "means for selecting the optimal output" is a function that selects the most reliable output through evaluation and removal of outliers.

[0897] The "means for providing the optimal output to the user" is a function that returns the selected optimal output to the user.

[0898] The "means for communicating with an external content providing system" is a communication function that enables the server to exchange information with an external service.

[0899] "Means for providing the optimal output in response to a question about a product from a user on an online shopping site" is a function that provides the optimal answer when a user asks a question about a product on an online shopping site.

[0900] The present invention provides a system for providing quick and accurate answers to user questions and requests. This system is particularly effective for online shopping sites and aims to improve the user experience. Specific embodiments of the system are described below.

[0901] The system consists of a server, a user terminal, and multiple natural language generation systems. The entire system operates with the following components:

[0902] Hardware and Software Configuration

[0903] Server: The server receives requests from users and has an interface for communicating with multiple natural language generation systems. Specifically, a cloud server such as AWS EC2 is used.

[0904] User terminal: The user terminal provides an interface for inputting user questions and requests through a smartphone application.

[0905] Natural language generation systems: External systems equipped with generative AI models (e.g., BERT, GPT-3) are used.

[0906] Data processing and calculation

[0907] 1. User input processing: The user enters a question or request through the application. This input data is sent from the user's device to the server.

[0908] 2. Request distribution: The server sends the received user request to multiple natural language generation systems in parallel, forming an appropriate prompt for each system.

[0909] For example: Question: "Is this T-shirt machine washable?"

[0910] 3. Receiving and evaluating the output: The server receives the answers from each generation system and evaluates and compares these answers using statistical methods and machine learning models.

[0911] 4. Outlier Removal: This process removes unreliable outputs based on the evaluation results. Specifically, it removes the output with the fewest votes.

[0912] 5. Selecting and providing the optimal output: Finally, the output with the highest rating is selected and returned to the user.

[0913] Specific examples

[0914] A user types a question into an online shopping site: "Is this T-shirt machine washable?" The system works as follows:

[0915] The server receives user requests and sends the requests to multiple natural language generation systems.

[0916] Each generating system returns the answer "Yes, this T-shirt is machine washable" to the server.

[0917] The server compares and evaluates these answers and provides the best answer to the user.

[0918] This system allows users to quickly obtain accurate and reliable answers, which is expected to improve the online shopping experience and increase user satisfaction.

[0919] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0920] Step 1:

[0921] A user accesses an online shopping site and inputs a question or request about a product through the application. The input information is sent from the user's terminal to the server.

[0922] Input: "Is this t-shirt machine washable?"

[0923] Output: The question is sent to the server.

[0924] Step 2:

[0925] The server receives the user's request, parses and understands it, and formats it in a format suitable for multiple natural language generation systems.

[0926] Input: User question

[0927] Output: Formatted prompt (e.g., "Question: Is this T-shirt machine washable?")

[0928] Step 3:

[0929] The server sends formatted prompt sentences to multiple natural language generation systems in parallel, sending requests to each generation system independently.

[0930] Input: Formatted prompt text

[0931] Output: The requests sent to each NLG system

[0932] Step 4:

[0933] Each natural language generation system generates a response based on the request and sends it back to the server.

[0934] Input: The request sent to each natural language generation system

[0935] Output: The answer from each generating system (e.g., "Yes, this T-shirt is machine washable.")

[0936] Step 5:

[0937] The server receives and collects responses from multiple natural language generation systems, storing all responses in a temporary database.

[0938] Input: Answers from each generating system

[0939] Output: Save all responses to a temporary database

[0940] Step 6:

[0941] The responses collected by the server are evaluated using statistical methods and machine learning models, and their reliability and relevance are assessed, with outlier responses being removed based on statistical methods.

[0942] Input: Response stored in temporary database

[0943] Output: Evaluation score and outlier removal results

[0944] Step 7:

[0945] Select the most reliable response based on the evaluation results. Sort by highest evaluation score to select the most appropriate response.

[0946] Input: Evaluation scores and outlier removal results

[0947] Output: Best response

[0948] Step 8:

[0949] The server formats the best response for return to the user and sends it to the user's terminal, which receives it and displays it on its screen.

[0950] Input: Best response

[0951] Output: The optimal response that will be displayed to the user (e.g., "Yes, this T-shirt is machine washable.")

[0952] The above are the detailed processing steps from when the user inputs a question about a product to when the most suitable response is obtained.

[0953] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0954] This invention is a system that unifies the outputs from multiple natural language generation systems and provides the most appropriate response to the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide an output that is adapted to the user's emotions. Specific forms of this system are shown below.

[0955] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems and an emotion engine. The user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[0956] User Action

[0957] Users access the system from their own devices using a dedicated application or web browser. After accessing the system, they input requests such as questions or requests for sentence generation. At this time, the emotion engine recognizes the user's emotional state through voice or text input. The request and emotional state are then sent to the server.

[0958] Server Processing

[0959] The server analyzes the request content and emotional state received from the user. The request content is sent to multiple natural language generation systems in parallel, and the server receives the response output from each generation system. The server then takes into account the emotional state recognized by the emotion engine in its evaluation and applies emotion adjustment logic to ultimately select the optimal output.

[0960] Specifically, if a user requests "What year did Napoleon Bonaparte die?" and the emotion engine recognizes the emotional state as "interesting," the server sends this information in parallel to natural language generation systems A, B, and C. Each generation system returns answers such as "1821," "1821," and "1820," respectively. At this time, the emotional information recognized by the emotion engine is taken into account by the emotion adjustment logic, and an output that provides a detailed explanation of "1821" in a way that matches the emotion of "interesting" is selected.

[0961] Emotion Engine

[0962] The emotion engine uses speech and text analysis techniques to recognize the user's emotions. For voice input, it analyzes the tone, pitch, and speed of the voice to determine whether the user is excited or calm. For text input, it analyzes keywords and context in the content to identify emotions. The recognized emotion information is sent to the server along with the request and is used to evaluate and optimize the output.

[0963] Specific examples

[0964] If the emotion engine recognizes that the user is in an "uneasy" state when making a request, the system will select a generation system that is more likely to generate sentences that "provide a sense of security" to the user. Also, if the user makes a request in a "happy" state, the system will tend to select "fun and attractive" output.

[0965] This system can efficiently compare and evaluate the results of multiple generative AI services and provide output that is adapted to the user's emotional state, improving the user experience and providing information that better meets the user's needs.

[0966] The processing flow will be explained below.

[0967] Step 1:

[0968] The user uses a device to launch a dedicated application or a web browser, enters a question or a request for sentence generation into the request form, and clicks the send button. At this time, the device sends the user's emotion data to the emotion engine via voice or text input.

[0969] Step 2:

[0970] The server receives a request from the user, along with the request content and the user's emotion data analyzed by the emotion engine from the device.

[0971] Step 3:

[0972] The server analyzes the received request content and sends the request content and emotion data to multiple natural language generation systems in parallel. It sends messages in an appropriate format to the API endpoint of each generation system.

[0973] Step 4:

[0974] Multiple natural language generation systems receive requests from the server, generate output through their own internal processing, and send it back to the server. The response from each system is sent back to the server as generated text data.

[0975] Step 5:

[0976] The server receives responses from each generation system. The received responses are obtained in a data format such as JSON. For example, answers from natural language generation systems A, B, and C are obtained.

[0977] Step 6:

[0978] The server aggregates the received multiple outputs, compares and evaluates the content of each output, and performs preprocessing of the data using statistical methods and machine learning models to evaluate its reliability.

[0979] Step 7:

[0980] The server uses the emotion data from the emotion engine to filter out inaccurate outputs that are outliers based on the evaluation results, and then applies emotion adjustment logic to select outputs that reflect the emotion data.

[0981] Step 8:

[0982] The server selects the optimal output that is evaluated as the most reliable, specifically, the output with the highest score on majority opinion and emotion regulation logic.

[0983] Step 9:

[0984] The server returns the optimal output it has selected to the user, returning it to the user's device as an HTTP response and displaying it in the browser or application.

[0985] Step 10:

[0986] The user checks the best output on their device and receives the answer. For example, the response "Napoleon Bonaparte died in 1821" is displayed along with an additional explanation based on the emotion recognized by the emotion engine (such as "He was a great leader of his time").

[0987] This system can efficiently compare and evaluate the results of multiple generative AI services and provide output that is adapted to the user's emotional state, improving the user experience and providing information that better meets the user's needs.

[0988] Example 2

[0989] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0990] Current natural language generation systems have a mechanism for evaluating the outputs of multiple systems to obtain the optimal response to a user request, but because they do not take into account the user's emotional state, they are unable to provide the optimal response that meets the user's needs.In addition, there is no emotion-based evaluation in output selection, which results in a suboptimal user experience.

[0991] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0992] In this invention, the server includes means for receiving a user request, means for recognizing the user's emotion, means for transmitting the user's request content and emotional state, means for transmitting the request to a plurality of natural language generation systems, means for receiving outputs from the plurality of natural language generation systems, means for comparing and evaluating the received outputs based on the emotional state, means for selecting an optimal output based on the evaluation result, and means for providing the optimal output to the user. This makes it possible to select an optimal response from the outputs of the plurality of generation systems while taking the user's emotional state into consideration, thereby improving the user experience.

[0993] The "means for receiving requests" is a function that provides an interface for receiving questions or information requests from users and inputting them into the system.

[0994] "Means for recognizing emotions" is a function that analyzes the user's voice and text input to identify their emotional state at that time.

[0995] The "means for transmitting the request content and emotional state" is a function for transferring the request content received from the user and the recognized emotional state to the server.

[0996] The "means for sending a request to a natural language generation system" is a function that enables the server to send the request contents to multiple natural language generation engines in parallel.

[0997] The "means for receiving output" is a function for receiving response results generated from multiple natural language generation engines.

[0998] The "means for comparing and evaluating based on emotional state" is a function for analyzing each received output based on the user's emotional state and selecting the most suitable one.

[0999] The "means for selecting the optimal output" is a function for comparing and examining response results from multiple natural language generation systems and selecting the optimal response based on evaluation criteria (e.g., emotional state).

[1000] The "means for providing output to the user" is a function for sending the selected optimal response to the user's terminal and displaying the result so that the user can confirm it.

[1001] The present invention is a system that unifies the outputs of multiple natural language generation systems in response to a user request and provides the most appropriate response. This system is composed of a server and a user terminal, and operates as follows.

[1002] System Configuration

[1003] In this system, the server has an interface for communicating with multiple natural language generation systems and emotion engines, and the user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[1004] Hardware and software used

[1005] The server requires a high-performance processor and a large amount of memory. In addition, a "generative AI model" (Model A, Model B, Model C) is used as the natural language generation system. The emotion engine requires an engine with voice analysis and text analysis technology. The user terminal is a smartphone or PC with an internet connection.

[1006] Process Overview

[1007] 1. Accepting user input

[1008] Users access the system from their own devices using a dedicated application or a web browser, and input requests such as questions and sentence generation requests by text or voice.

[1009] 2. User Emotion Recognition

[1010] The emotion engine analyzes the user's input (text or voice) to recognize the user's emotional state. For voice input, it analyzes the tone, pitch, and speed of the voice, and for text input, it analyzes keywords and context of the content to identify emotions.

[1011] 3. Sending request data

[1012] The device sends the user's input and emotional state to the server, which receives and stores them as log data.

[1013] 4. Processing of Natural Language Generation Systems

[1014] The server sends the request content to multiple natural language generation systems (generative AI models A, B, and C) in parallel. Each generation system independently generates an answer to this request.

[1015] 5. Receiving the output

[1016] The server receives the output returned from each natural language generation system. For example, it may return outputs such as "1821," "1821," and "1820."

[1017] 6. Applying emotion regulation logic

[1018] Based on the recognition results of the emotion engine, the server applies emotion adjustment logic to the received output to select the best response, for example, "1821" with a detailed description that matches the emotion "interesting."

[1019] 7. Providing a response to the user

[1020] The final selected output is sent to the user's terminal and displayed to the user.

[1021] Specific examples

[1022] For example, if the emotion engine recognizes that the user is in an "unsettled" state when making a request, the system will select a generation system that is likely to generate sentences that "provide a sense of security" to the user. Also, if the user makes a request while in a "happy" state, the system will tend to select "fun and attractive" output.

[1023] Prompt Sentence Examples

[1024] Below are some examples of prompt sentences to input to the generative AI model.

[1025] Please provide more details about the life of a historical figure that you find interesting.

[1026] "Users are worried, so please provide some comforting words."

[1027] This system allows users to quickly obtain the most appropriate information adapted to their emotional state, improving the user experience and increasing the accuracy and relevance of the information provided.

[1028] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1029] Step 1:

[1030] When a user opens a dedicated application or web browser on their device, an input form appears, where the user can enter their question or request via text or voice. Once the input is complete, the device sends the data to the next processing step.

[1031] Input: A user question or text generation request (e.g., "What year did Napoleon Bonaparte die?")

[1032] Output: Text or audio data is sent to the device.

[1033] Specific behavior:

[1034] The user can type "What year did Napoleon Bonaparte die?" into the text field or say "What year did Napoleon Bonaparte die?"

[1035] The terminal sends the entered data to a dedicated application.

[1036] Step 2:

[1037] The emotion engine analyzes text or voice data sent from the device to recognize the user's emotional state. For voice input, it analyzes the tone, pitch, and speed of the voice, and for text input, it analyzes keywords and context of the content to identify emotions.

[1038] Input: Text or audio data

[1039] Output: Analyzed user sentiment information (e.g., "interesting")

[1040] Specific behavior:

[1041] When voice input is used, the emotion engine analyzes the tone, pitch, and speed of the voice in real time to recognize emotions such as "interesting."

[1042] When text is entered, the emotion engine analyzes emotions from keywords and context in the entered text.

[1043] Step 3:

[1044] The device sends the user's input and emotional state to the server, which receives this data and stores it as log data.

[1045] Input: Input content (text and voice) and emotional information

[1046] Output: Input content and emotion information received by the server

[1047] Specific behavior:

[1048] The terminal sends the user's input data ("What year did Napoleon Bonaparte die?") and the emotion engine's analysis results ("Interesting") to the server.

[1049] The server receives this and stores it as log data.

[1050] Step 4:

[1051] The server sends the request content to multiple natural language generation systems (generative AI models A, B, and C) in parallel. Each generation system independently generates an answer to this request.

[1052] Input: User request

[1053] Output: Requests sent to multiple natural language generation systems

[1054] Specific behavior:

[1055] The server sends the request "What year did Napoleon Bonaparte die?" to generation AI models A, B, and C in parallel.

[1056] Each generative AI model generates its own answer to this request.

[1057] Step 5:

[1058] The server receives the output returned from each natural language generation system. For example, it may return outputs such as "1821," "1821," and "1820."

[1059] Input: Output from each generating system

[1060] Output: The output received by the server from each generating system.

[1061] Specific behavior:

[1062] The server receives outputs such as "1821" from Generative AI Model A, "1821" from Generative AI Model B, and "1820" from Generative AI Model C.

[1063] The server temporarily stores these outputs.

[1064] Step 6:

[1065] Based on the recognition results of the emotion engine, the server applies emotion adjustment logic to the received output to select the best response, for example, "1821" with a detailed description that matches the emotion "interesting."

[1066] Input: Emotional information and output from each generative system

[1067] Output: Optimal output (e.g., a detailed description of "1821")

[1068] Specific behavior:

[1069] The server takes into account the sentiment of "interesting" and selects "1821" with a detailed description as the best response.

[1070] Emotion regulation logic selects outputs that are adapted to the user's emotional state.

[1071] Step 7:

[1072] The final selected output is sent to the user's terminal and displayed to the user.

[1073] Input: Optimal Output

[1074] Output: The output sent to the user's terminal.

[1075] Specific behavior:

[1076] The server sends the optimal output (a detailed description of "1821") to the user's terminal.

[1077] The terminal displays this to the user so that the user can check the results.

[1078] (Application example 2)

[1079] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1080] Conventional natural language generation systems have limitations in providing appropriate output in response to user requests. Furthermore, they lack a mechanism for selecting output that corresponds to the user's emotional state, preventing improvements in user experience. In particular, in the advertising field, there is a demand for providing advertisements that are customized according to the user's emotions, but there has been a problem in that no appropriate system exists to achieve this.

[1081] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a request from a user, means for sending a request to a plurality of natural language generation systems, means for receiving output from the plurality of natural language generation systems, means for comparing and evaluating the received output, emotion recognition means for recognizing the user's emotion, advertisement selection means for selecting an optimal advertisement based on the output of the emotion recognition means, and means for providing the optimal advertisement to the user terminal. This makes it possible to provide optimal output and advertisements adapted to the user's emotional state.

[1082] The "means for receiving requests from users" is a function for sending questions or requests entered by users to the system.

[1083] The "means for sending requests to a plurality of natural language generation systems" is a function for sending requests received from a user to a plurality of natural language generation models in parallel.

[1084] The "means for receiving output from a plurality of natural language generation systems" is a function for receiving responses and generated text from each natural language generation system.

[1085] "Means for comparing and evaluating received outputs" refers to a function for comparing outputs obtained from multiple generative AI models to evaluate their quality and suitability.

[1086] "Means for removing outliers based on evaluation results" refers to techniques for removing outputs that lack validity as a result of evaluation.

[1087] "Means for selecting the optimal output" is a function for selecting the most appropriate output from the results of comparison and evaluation.

[1088] The "means for providing the optimal output to the user" is a function for returning the selected optimal output to the user.

[1089] The "emotion recognition means for recognizing the user's emotions" is a function for analyzing and recognizing the user's emotional state from their statements and text.

[1090] The "advertisement selection means for selecting the most suitable advertisement based on the output of the emotion recognition means" is a function for selecting an advertisement that is suited to the emotional state of the user determined by the emotion recognition means.

[1091] The "means for providing optimal advertisements to user terminals" is a function for displaying selected advertisements on the user's device.

[1092] The present invention provides a system for providing optimal output and advertisements adapted to the emotional state of a user. Specific embodiments of the system are described in detail below.

[1093] This system mainly includes a server and a user terminal, and is composed of the following hardware and software.

[1094] Hardware

[1095] server

[1096] Computational resources that operate under multiple generative AI models

[1097] Computing resources to run the emotion recognition engine

[1098] User terminal

[1099] Smartphone (iOS or Android)

[1100] Input devices such as microphones and cameras

[1101] software

[1102] Python

[1103] Emotion recognition engine (e.g., a model built with Keras)

[1104] API communication (e.g., requests library)

[1105] Generative AI Models

[1106] Natural language generation engines (e.g., GPT-3)

[1107] Data processing and calculation

[1108] Server Processing

[1109] 1. Receiving a request from a user

[1110] The server receives requests from the user terminal in the form of voice or text, which are requests for information or advertisements that the user wishes to view.

[1111] 2. Emotion judgment

[1112] The received voice or text data is analyzed using an emotion recognition engine to recognize the user's emotional state. For voice data, the tone, pitch, and speed of the voice are analyzed, and for text data, keywords and context are analyzed.

[1113] 3. Sending a request

[1114] The server sends the user's request to multiple natural language generation systems (generative AI models) in parallel. Specifically, it sends requests to the API endpoints of the generative AI models.

[1115] 4. Receiving Output

[1116] The server receives the output returned from each generative AI system and temporarily stores it in storage.

[1117] 5. Output evaluation and outlier removal

[1118] Each received output is compared and evaluated using an evaluation model, and outliers are removed using statistical methods based on the evaluation results.

[1119] 6. Selecting the optimal output

[1120] From the evaluated outputs, the one that best suits the user's emotional state is selected, based on emotion regulation logic.

[1121] 7. Ad Selection

[1122] Based on the output of the emotion recognition engine, an algorithm is run to select the most suitable advertisement.

[1123] User terminal processing

[1124] 1. Submit a request

[1125] Users enter questions or requests through a dedicated application or web browser and send them to the server in voice or text format.

[1126] 2. Optimal output and display of ads

[1127] The optimal output and advertisement returned from the server are displayed on the user terminal.

[1128] Specific examples

[1129] Prompt Sentence Examples

[1130] The user speaks the following prompt into their smartphone:

[1131] "I'm in a mood to chill out. I want to see ads that help me relax."

[1132] Based on these prompts, the generative AI model generates and serves relaxing ads to users, such as ads for "relaxing scented candles" or "meditation apps."

[1133] As described above, the system of the present invention aims to improve the user experience by providing optimal information and advertisements to users based on the output and emotional state from multiple generative AI models.

[1134] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1135] Step 1:

[1136] The user inputs a request in voice or text format through a dedicated application or web browser using a device. The input request includes a specific question or request from the user. For example, a prompt such as "I feel like I want to relax. Please show me a relaxing advertisement" may be input.

[1137] Step 2:

[1138] The terminal receives a request from the user and sends the data to the server. Here, the request data is sent to the server via the network. The input is the user's request content, and the output is communication data to the server.

[1139] Step 3:

[1140] The server receives requests sent by users and passes them to the emotion recognition engine. The emotion recognition engine analyzes the input voice or text data to identify the user's emotional state. Specifically, it analyzes the tone, pitch, and speed of the voice, as well as keywords. The input is the user's request data, and the output is the user's emotional state.

[1141] Step 4:

[1142] Based on the emotional state output by the emotion recognition engine, the server sends requests to multiple generative AI models. Here, a request including a prompt sentence is sent to the API endpoint of the generative AI model. The input is the emotional state and the prompt sentence, and the output is the request data for the generative AI model.

[1143] Step 5:

[1144] The server receives output from multiple generative AI models. Each generative AI model returns the text it generates in response to the prompt. The input is the response data from the generative AI model, and the output is the generated output.

[1145] Step 6:

[1146] The server passes the received outputs to the evaluation model for comparison and evaluation. The evaluation model uses a machine learning algorithm to evaluate the quality and relevance of the outputs. The input is the generated multiple outputs, and the output is an evaluation score or ranking.

[1147] Step 7:

[1148] Based on the evaluation results, the server uses statistical methods to remove outliers, where significantly mismatched outputs are removed. The input is the output with the evaluation score, and the output is the list of outputs with the outliers removed.

[1149] Step 8:

[1150] The server selects the optimal output and advertisement based on the output of the emotion recognition engine. In this process, the output and advertisement that best fit the user's emotional state are selected based on the emotion adjustment logic. The input is the output list after outlier removal and the emotional state, and the output is the optimal output and advertisement.

[1151] Step 9:

[1152] The server sends the selected optimal output and advertisement to the user terminal. The input is the optimal output and advertisement, and the output is the communication data to the user terminal.

[1153] Step 10:

[1154] The terminal displays the optimal output and advertisement received from the server to the user. Here, the optimal response to the request entered by the user and the advertisement are displayed. The input is the communication data from the server, and the output is the display content of the user terminal.

[1155] Through the above steps, a system that adapts to the emotional state of the user and provides optimal output and advertisements is specifically realized.

[1156] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1157] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1158] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1159] [Fourth embodiment]

[1160] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1161] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1162] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1163] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1164] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1165] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1166] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1167] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1168] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1169] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1170] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1171] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1172] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1173] The present invention is a system for unifying the outputs from multiple natural language generation systems and providing the most appropriate response to the user. A specific form of this system is shown below.

[1174] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems. The user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[1175] User Action

[1176] Users access the system from their own devices using a dedicated application or a web browser. After accessing the system, the user enters a request, such as a question or a request to generate a sentence, and presses the send button. This operation sends the request to the server.

[1177] Server Processing

[1178] The server analyzes requests received from users and sends the request contents to multiple natural language generation systems in parallel. For example, requests are sent simultaneously to natural language generation systems A, B, and C. Each generation system generates an output based on the request and sends it back to the server.

[1179] The server collects the outputs returned by each generation system, compares and evaluates each output, and uses statistical methods and machine learning models as criteria for evaluation to eliminate outliers and inaccurate outputs. For example, it processes the output that receives the fewest votes overall. Through this process, the output deemed to be the most reliable is selected.

[1180] Specific examples

[1181] Consider a case where a user sends a request such as "What year did Napoleon Bonaparte die?" The server sends this request to natural language generation systems A, B, and C. Each generation system returns an answer. For example, if generation systems A and B return "1821" and generation system C returns "1820," the server evaluates these answers using statistical methods and machine learning models and selects the majority answer, "1821," as the optimal output. The server then returns this optimal output to the user.

[1182] In this way, the present invention is a system that can eliminate erroneous or biased information and provide users with accurate and reliable output.

[1183] The processing flow will be explained below.

[1184] Step 1:

[1185] The user uses a terminal to launch a dedicated application or a web browser, enters a question or a request for sentence generation in a request form, and clicks a send button.

[1186] Step 2:

[1187] The server receives a request from the user, which is sent to the server as an HTTP request and parsed.

[1188] Step 3:

[1189] The server analyzes the received request and sends it to multiple natural language generation systems in parallel, sending messages in the appropriate format to the API endpoints of each generation system.

[1190] Step 4:

[1191] Multiple natural language generation systems receive requests from the server, generate output through their own internal processing, and send it back to the server.

[1192] Step 5:

[1193] The server receives responses from each generation system. The received responses are obtained in a data format such as JSON.

[1194] Step 6:

[1195] The server aggregates the received multiple outputs, compares and evaluates the content of each output, and performs preprocessing of the data using statistical methods and machine learning models to evaluate its reliability.

[1196] Step 7:

[1197] Eliminate inaccurate outputs that are outliers based on the evaluation results, for example, eliminating the output that received the fewest votes.

[1198] Step 8:

[1199] The server selects the optimal output that is evaluated as the most reliable, i.e., the output with the highest score from the majority opinion or evaluation model.

[1200] Step 9:

[1201] The server returns the optimal output it has selected to the user, returning it to the user's device as an HTTP response and displaying it in the browser or application.

[1202] Step 10:

[1203] The user checks the optimal output returned on the device and receives the answer.

[1204] This system makes it possible to efficiently compare and evaluate the results of multiple generative AI services and provide users with highly reliable output.

[1205] Example 1

[1206] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1207] Current natural language generation systems rely on a single system, resulting in problems with the accuracy and consistency of responses. Furthermore, even when responses are obtained from multiple natural language generation systems, there is a lack of effective means for integrating these responses and selecting the optimal response. This makes it difficult to provide users with reliable information. Therefore, there is a need for a system that can compare and evaluate responses from multiple natural language generation systems, remove outliers, and provide the optimal response to the user.

[1208] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1209] In this invention, the server includes means for receiving a request from a user, means for transmitting a request to a plurality of natural language generation systems, means for receiving responses from the plurality of natural language generation systems, means for comparing and evaluating the received responses, means for removing outliers based on the evaluation result, means for selecting an optimal response from the set as a result of the evaluation, and means for providing the optimal response to the user. This makes it possible to effectively integrate responses obtained from the plurality of natural language generation systems and provide the user with reliable information.

[1210] A "user" is an entity that sends requests to the system and receives responses.

[1211] A "request" is information such as a question or a request for sentence generation that a user sends to the system.

[1212] "Multiple natural language generation systems" refers to a collection of multiple systems that generate natural language based on different algorithms or models.

[1213] A "response" is information that multiple natural language generation systems generate based on a request and return to the server.

[1214] "Comparing and evaluating means" refers to methods and techniques for comparing received responses with each other and evaluating their accuracy and reliability.

[1215] "Means for removing outliers" are methods or techniques that, based on the evaluation results, eliminate inaccurate responses that differ significantly from other responses.

[1216] "Means for selecting optimal responses" are methods or techniques for selecting the most appropriate and reliable responses based on the evaluation.

[1217] The "means for providing" refers to the method or technology for transmitting and displaying the selected optimal response to the user.

[1218] The present invention is a system that unifies responses from multiple natural language generation systems and provides the most appropriate response to the user. A specific form of this system is shown below.

[1219] composition

[1220] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems, and the user terminal provides an interface for users to input questions or requests and communicate with the server to obtain responses.

[1221] Hardware and software used

[1222] Hardware: Servers with high-performance processors and large amounts of memory, and devices for users to access the system (PCs, smartphones, tablets, etc.)

[1223] Software: Multiple natural language generation systems (generative AI models A, B, and C), statistical analysis software, and machine learning model evaluation software

[1224] Processing flow

[1225] User operations

[1226] Users access the system from their own devices using a dedicated application or web browser. After accessing the system, the user enters a question or request and presses the send button. This action sends the request to the server.

[1227] Specific examples

[1228] A user enters "What year did Napoleon Bonaparte die?" into the text input field of a web browser and clicks the submit button.

[1229] Server Processing

[1230] The server analyzes the request data received from the user and converts it into an appropriate format. It then sends the analyzed request to multiple natural language generation systems (generative AI models A, B, and C) in parallel and receives responses from each generation system.

[1231] The server collects the received responses, compares them with each other, and evaluates them. Statistical methods and machine learning models are used to filter out inaccurate responses and outliers. The server then selects the most reliable response and sends it back to the user.

[1232] Specific examples

[1233] The server sends the prompt "What year did Napoleon Bonaparte die?" to generative AI models A, B, and C. If generative AI models A and B respond with "1821" and generative AI model C responds with "1820," the server evaluates these responses using statistical methods and machine learning models, selects the majority response, "1821," as the optimal response, and returns it to the user device.

[1234] effect

[1235] This allows the system to effectively integrate information obtained from multiple natural language generation systems and provide users with highly reliable information, making the overall operation more efficient and significantly improving the user experience.

[1236] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1237] Step 1: User submits request

[1238] Specific operation: A user accesses the system using a dedicated application or a web browser from their own device (PC, smartphone, tablet, etc.), enters the request content (e.g., a question or a request to generate a sentence) in the text input field, and presses the send button.

[1239] Input: A user question or request for text generation (e.g., "What year did Napoleon Bonaparte die?")

[1240] Output: Request data sent to the server

[1241] Step 2: The server parses the request

[1242] Specific operation: The server analyzes the request data received from the user, understands the type and content of the request, and converts the request into an appropriate format based on this.

[1243] Input: Request data received from the user

[1244] Output: Parsed request data

[1245] Step 3: The server sends the request to the natural language generation system

[1246] Specific operation: The server sends parsed requests in parallel to multiple natural language generation systems (generative AI models A, B, and C).

[1247] Input: Parsed request data

[1248] Output: A prompt sent to each generative AI model (e.g., "What year did Napoleon Bonaparte die?")

[1249] Step 4: The natural language generation system generates a response

[1250] Specific operation: Each generative AI model generates a response based on the sent prompt sentence and sends it back to the server.

[1251] Input: Prompt sent to each generative AI model

[1252] Output: Response data returned by each generative AI model (e.g., Generative AI Models A and B respond "1821," while Generative AI Model C responds "1820")

[1253] Step 5: The server collects and compares the responses

[1254] What it does: The server collects the responses from each generation system, compares and evaluates them against each other, and uses statistical techniques and machine learning models to filter out inaccurate responses and outliers.

[1255] Input: Response data from each generative AI model

[1256] Output: Evaluated response data with incorrect responses and outliers removed

[1257] Step 6: The server picks the best response

[1258] Specific behavior: The server selects the single most authoritative response based on the evaluation results, e.g., the majority response.

[1259] Input: Evaluated response data

[1260] Output: Best response (e.g. "1821")

[1261] Step 7: The server provides the best response to the user

[1262] Specific operation: The server sends the selected best response to the user's terminal, where the user can receive and confirm it.

[1263] Input: Best response

[1264] Output: Response data sent to the user device

[1265] (Application example 1)

[1266] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1267] On conventional online shopping sites, it has been difficult to obtain a quick and accurate answer when users have questions about products. This problem can lead to a decrease in user satisfaction and a decrease in purchasing intent. Another issue is that there has been no effective method for integrating answers from multiple natural language generation systems and selecting the most appropriate answer.

[1268] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1269] In this invention, the server includes means for receiving requests from a user, means for transmitting requests to a plurality of natural language generation systems, means for receiving outputs from the plurality of natural language generation systems, means for comparing and evaluating the received outputs, means for removing outliers based on the evaluation results, means for selecting an optimal output, means for providing the optimal output to a user, means for communicating with an external content providing system, and means for responding with the optimal output in response to a product-related question from a user on an online shopping site. This makes it possible to provide a quick and accurate answer to a product-related question from a user, thereby increasing user satisfaction.

[1270] The "means for receiving requests from a user" is a function for sending questions or requests input by a user through an application to a server.

[1271] The "means for transmitting requests to a plurality of natural language generation systems" is a function that allows the server to simultaneously transmit requests received from users to a plurality of natural language generation systems.

[1272] The "means for receiving outputs from a plurality of natural language generation systems" refers to a function by which the server collects responses generated from each natural language generation system.

[1273] "Means for comparing and evaluating received outputs" refers to a function in which the server compares and evaluates multiple outputs using statistical methods or machine learning models.

[1274] The "means for removing outliers based on the evaluation results" is a function for removing unreliable outputs based on the evaluation results.

[1275] The "means for selecting the optimal output" is a function that selects the most reliable output through evaluation and removal of outliers.

[1276] The "means for providing the optimal output to the user" is a function that returns the selected optimal output to the user.

[1277] The "means for communicating with an external content providing system" is a communication function that enables the server to exchange information with an external service.

[1278] "Means for providing the optimal output in response to a question about a product from a user on an online shopping site" is a function that provides the optimal answer when a user asks a question about a product on an online shopping site.

[1279] The present invention provides a system for providing quick and accurate answers to user questions and requests. This system is particularly effective for online shopping sites and aims to improve the user experience. Specific embodiments of the system are described below.

[1280] The system consists of a server, a user terminal, and multiple natural language generation systems. The entire system operates with the following components:

[1281] Hardware and Software Configuration

[1282] Server: The server receives requests from users and has an interface for communicating with multiple natural language generation systems. Specifically, a cloud server such as AWS EC2 is used.

[1283] User terminal: The user terminal provides an interface for inputting user questions and requests through a smartphone application.

[1284] Natural language generation systems: External systems equipped with generative AI models (e.g., BERT, GPT-3) are used.

[1285] Data processing and calculation

[1286] 1. User input processing: The user enters a question or request through the application. This input data is sent from the user's device to the server.

[1287] 2. Request distribution: The server sends the received user request to multiple natural language generation systems in parallel, forming an appropriate prompt for each system.

[1288] For example: Question: "Is this T-shirt machine washable?"

[1289] 3. Receiving and evaluating the output: The server receives the answers from each generation system and evaluates and compares these answers using statistical methods and machine learning models.

[1290] 4. Outlier Removal: This process removes unreliable outputs based on the evaluation results. Specifically, it removes the output with the fewest votes.

[1291] 5. Selecting and providing the optimal output: Finally, the output with the highest rating is selected and returned to the user.

[1292] Specific examples

[1293] A user types a question into an online shopping site: "Is this T-shirt machine washable?" The system works as follows:

[1294] The server receives user requests and sends the requests to multiple natural language generation systems.

[1295] Each generating system returns the answer "Yes, this T-shirt is machine washable" to the server.

[1296] The server compares and evaluates these answers and provides the best answer to the user.

[1297] This system allows users to quickly obtain accurate and reliable answers, which is expected to improve the online shopping experience and increase user satisfaction.

[1298] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1299] Step 1:

[1300] A user accesses an online shopping site and inputs a question or request about a product through the application. The input information is sent from the user's terminal to the server.

[1301] Input: "Is this t-shirt machine washable?"

[1302] Output: The question is sent to the server.

[1303] Step 2:

[1304] The server receives the user's request, parses and understands it, and formats it in a format suitable for multiple natural language generation systems.

[1305] Input: User question

[1306] Output: Formatted prompt (e.g., "Question: Is this T-shirt machine washable?")

[1307] Step 3:

[1308] The server sends formatted prompt sentences to multiple natural language generation systems in parallel, sending requests to each generation system independently.

[1309] Input: Formatted prompt text

[1310] Output: The requests sent to each NLG system

[1311] Step 4:

[1312] Each natural language generation system generates a response based on the request and sends it back to the server.

[1313] Input: The request sent to each natural language generation system

[1314] Output: The answer from each generating system (e.g., "Yes, this T-shirt is machine washable.")

[1315] Step 5:

[1316] The server receives and collects responses from multiple natural language generation systems, storing all responses in a temporary database.

[1317] Input: Answers from each generating system

[1318] Output: Save all responses to a temporary database

[1319] Step 6:

[1320] The responses collected by the server are evaluated using statistical methods and machine learning models, and their reliability and relevance are assessed, with outlier responses being removed based on statistical methods.

[1321] Input: Response stored in temporary database

[1322] Output: Evaluation score and outlier removal results

[1323] Step 7:

[1324] Select the most reliable response based on the evaluation results. Sort by highest evaluation score to select the most appropriate response.

[1325] Input: Evaluation scores and outlier removal results

[1326] Output: Best response

[1327] Step 8:

[1328] The server formats the best response for return to the user and sends it to the user's terminal, which receives it and displays it on its screen.

[1329] Input: Best response

[1330] Output: The optimal response that will be displayed to the user (e.g., "Yes, this T-shirt is machine washable.")

[1331] The above are the detailed processing steps from when the user inputs a question about a product to when the most suitable response is obtained.

[1332] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1333] This invention is a system that unifies the outputs from multiple natural language generation systems and provides the most appropriate response to the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide an output that is adapted to the user's emotions. Specific forms of this system are shown below.

[1334] The system consists of a server and a user terminal. The server has an interface for communicating with multiple natural language generation systems and an emotion engine. The user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[1335] User Action

[1336] Users access the system from their own devices using a dedicated application or web browser. After accessing the system, they input requests such as questions or requests for sentence generation. At this time, the emotion engine recognizes the user's emotional state through voice or text input. The request and emotional state are then sent to the server.

[1337] Server Processing

[1338] The server analyzes the request content and emotional state received from the user. The request content is sent to multiple natural language generation systems in parallel, and the server receives the response output from each generation system. The server then takes into account the emotional state recognized by the emotion engine in its evaluation and applies emotion adjustment logic to ultimately select the optimal output.

[1339] Specifically, if a user requests "What year did Napoleon Bonaparte die?" and the emotion engine recognizes the emotional state as "interesting," the server sends this information in parallel to natural language generation systems A, B, and C. Each generation system returns answers such as "1821," "1821," and "1820," respectively. At this time, the emotional information recognized by the emotion engine is taken into account by the emotion adjustment logic, and an output that provides a detailed explanation of "1821" in a way that matches the emotion of "interesting" is selected.

[1340] Emotion Engine

[1341] The emotion engine uses speech and text analysis techniques to recognize the user's emotions. For voice input, it analyzes the tone, pitch, and speed of the voice to determine whether the user is excited or calm. For text input, it analyzes keywords and context in the content to identify emotions. The recognized emotion information is sent to the server along with the request and is used to evaluate and optimize the output.

[1342] Specific examples

[1343] If the emotion engine recognizes that the user is in an "uneasy" state when making a request, the system will select a generation system that is more likely to generate sentences that "provide a sense of security" to the user. Also, if the user makes a request in a "happy" state, the system will tend to select "fun and attractive" output.

[1344] This system can efficiently compare and evaluate the results of multiple generative AI services and provide output that is adapted to the user's emotional state, improving the user experience and providing information that better meets the user's needs.

[1345] The processing flow will be explained below.

[1346] Step 1:

[1347] The user uses a device to launch a dedicated application or a web browser, enters a question or a request for sentence generation into the request form, and clicks the send button. At this time, the device sends the user's emotion data to the emotion engine via voice or text input.

[1348] Step 2:

[1349] The server receives a request from the user, along with the request content and the user's emotion data analyzed by the emotion engine from the device.

[1350] Step 3:

[1351] The server analyzes the received request content and sends the request content and emotion data to multiple natural language generation systems in parallel. It sends messages in an appropriate format to the API endpoint of each generation system.

[1352] Step 4:

[1353] Multiple natural language generation systems receive requests from the server, generate output through their own internal processing, and send it back to the server. The response from each system is sent back to the server as generated text data.

[1354] Step 5:

[1355] The server receives responses from each generation system. The received responses are obtained in a data format such as JSON. For example, answers from natural language generation systems A, B, and C are obtained.

[1356] Step 6:

[1357] The server aggregates the received multiple outputs, compares and evaluates the content of each output, and performs preprocessing of the data using statistical methods and machine learning models to evaluate its reliability.

[1358] Step 7:

[1359] The server uses the emotion data from the emotion engine to filter out inaccurate outputs that are outliers based on the evaluation results, and then applies emotion adjustment logic to select outputs that reflect the emotion data.

[1360] Step 8:

[1361] The server selects the optimal output that is evaluated as the most reliable, specifically, the output with the highest score on majority opinion and emotion regulation logic.

[1362] Step 9:

[1363] The server returns the optimal output it has selected to the user, returning it to the user's device as an HTTP response and displaying it in the browser or application.

[1364] Step 10:

[1365] The user checks the best output on their device and receives the answer. For example, the response "Napoleon Bonaparte died in 1821" is displayed along with an additional explanation based on the emotion recognized by the emotion engine (such as "He was a great leader of his time").

[1366] This system can efficiently compare and evaluate the results of multiple generative AI services and provide output that is adapted to the user's emotional state, improving the user experience and providing information that better meets the user's needs.

[1367] Example 2

[1368] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1369] Current natural language generation systems have a mechanism for evaluating the outputs of multiple systems to obtain the optimal response to a user request, but because they do not take into account the user's emotional state, they are unable to provide the optimal response that meets the user's needs.In addition, there is no emotion-based evaluation in output selection, which results in a suboptimal user experience.

[1370] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1371] In this invention, the server includes means for receiving a user request, means for recognizing the user's emotion, means for transmitting the user's request content and emotional state, means for transmitting the request to a plurality of natural language generation systems, means for receiving outputs from the plurality of natural language generation systems, means for comparing and evaluating the received outputs based on the emotional state, means for selecting an optimal output based on the evaluation result, and means for providing the optimal output to the user. This makes it possible to select an optimal response from the outputs of the plurality of generation systems while taking the user's emotional state into consideration, thereby improving the user experience.

[1372] The "means for receiving requests" is a function that provides an interface for receiving questions or information requests from users and inputting them into the system.

[1373] "Means for recognizing emotions" is a function that analyzes the user's voice and text input to identify their emotional state at that time.

[1374] The "means for transmitting the request content and emotional state" is a function for transferring the request content received from the user and the recognized emotional state to the server.

[1375] The "means for sending a request to a natural language generation system" is a function that enables the server to send the request contents to multiple natural language generation engines in parallel.

[1376] The "means for receiving output" is a function for receiving response results generated from multiple natural language generation engines.

[1377] The "means for comparing and evaluating based on emotional state" is a function for analyzing each received output based on the user's emotional state and selecting the most suitable one.

[1378] The "means for selecting the optimal output" is a function for comparing and examining response results from multiple natural language generation systems and selecting the optimal response based on evaluation criteria (e.g., emotional state).

[1379] The "means for providing output to the user" is a function for sending the selected optimal response to the user's terminal and displaying the result so that the user can confirm it.

[1380] The present invention is a system that unifies the outputs of multiple natural language generation systems in response to a user request and provides the most appropriate response. This system is composed of a server and a user terminal, and operates as follows.

[1381] System Configuration

[1382] In this system, the server has an interface for communicating with multiple natural language generation systems and emotion engines, and the user terminal provides an interface for users to input questions or requests and communicates with the server to obtain responses.

[1383] Hardware and software used

[1384] The server requires a high-performance processor and a large amount of memory. In addition, a "generative AI model" (Model A, Model B, Model C) is used as the natural language generation system. The emotion engine requires an engine with voice analysis and text analysis technology. The user terminal is a smartphone or PC with an internet connection.

[1385] Process Overview

[1386] 1. Accepting user input

[1387] Users access the system from their own devices using a dedicated application or a web browser, and input requests such as questions and sentence generation requests by text or voice.

[1388] 2. User Emotion Recognition

[1389] The emotion engine analyzes the user's input (text or voice) to recognize the user's emotional state. For voice input, it analyzes the tone, pitch, and speed of the voice, and for text input, it analyzes keywords and context of the content to identify emotions.

[1390] 3. Sending request data

[1391] The device sends the user's input and emotional state to the server, which receives and stores them as log data.

[1392] 4. Processing of Natural Language Generation Systems

[1393] The server sends the request content to multiple natural language generation systems (generative AI models A, B, and C) in parallel. Each generation system independently generates an answer to this request.

[1394] 5. Receiving the output

[1395] The server receives the output returned from each natural language generation system. For example, it may return outputs such as "1821," "1821," and "1820."

[1396] 6. Applying emotion regulation logic

[1397] Based on the recognition results of the emotion engine, the server applies emotion adjustment logic to the received output to select the best response, for example, "1821" with a detailed description that matches the emotion "interesting."

[1398] 7. Providing a response to the user

[1399] The final selected output is sent to the user's terminal and displayed to the user.

[1400] Specific examples

[1401] For example, if the emotion engine recognizes that the user is in an "unsettled" state when making a request, the system will select a generation system that is likely to generate sentences that "provide a sense of security" to the user. Also, if the user makes a request while in a "happy" state, the system will tend to select "fun and attractive" output.

[1402] Prompt Sentence Examples

[1403] Below are some examples of prompt sentences to input to the generative AI model.

[1404] Please provide more details about the life of a historical figure that you find interesting.

[1405] "Users are worried, so please provide some comforting words."

[1406] This system allows users to quickly obtain the most appropriate information adapted to their emotional state, improving the user experience and increasing the accuracy and relevance of the information provided.

[1407] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1408] Step 1:

[1409] When a user opens a dedicated application or web browser on their device, an input form appears, where the user can enter their question or request via text or voice. Once the input is complete, the device sends the data to the next processing step.

[1410] Input: A user question or text generation request (e.g., "What year did Napoleon Bonaparte die?")

[1411] Output: Text or audio data is sent to the device.

[1412] Specific behavior:

[1413] The user can type "What year did Napoleon Bonaparte die?" into the text field or say "What year did Napoleon Bonaparte die?"

[1414] The terminal sends the entered data to a dedicated application.

[1415] Step 2:

[1416] The emotion engine analyzes text or voice data sent from the device to recognize the user's emotional state. For voice input, it analyzes the tone, pitch, and speed of the voice, and for text input, it analyzes keywords and context of the content to identify emotions.

[1417] Input: Text or audio data

[1418] Output: Analyzed user sentiment information (e.g., "interesting")

[1419] Specific behavior:

[1420] When voice input is used, the emotion engine analyzes the tone, pitch, and speed of the voice in real time to recognize emotions such as "interesting."

[1421] When text is entered, the emotion engine analyzes emotions from keywords and context in the entered text.

[1422] Step 3:

[1423] The device sends the user's input and emotional state to the server, which receives this data and stores it as log data.

[1424] Input: Input content (text and voice) and emotional information

[1425] Output: Input content and emotion information received by the server

[1426] Specific behavior:

[1427] The terminal sends the user's input data ("What year did Napoleon Bonaparte die?") and the emotion engine's analysis results ("Interesting") to the server.

[1428] The server receives this and stores it as log data.

[1429] Step 4:

[1430] The server sends the request content to multiple natural language generation systems (generative AI models A, B, and C) in parallel. Each generation system independently generates an answer to this request.

[1431] Input: User request

[1432] Output: Requests sent to multiple natural language generation systems

[1433] Specific behavior:

[1434] The server sends the request "What year did Napoleon Bonaparte die?" to generation AI models A, B, and C in parallel.

[1435] Each generative AI model generates its own answer to this request.

[1436] Step 5:

[1437] The server receives the output returned from each natural language generation system. For example, it may return outputs such as "1821," "1821," and "1820."

[1438] Input: Output from each generating system

[1439] Output: The output received by the server from each generating system.

[1440] Specific behavior:

[1441] The server receives outputs such as "1821" from Generative AI Model A, "1821" from Generative AI Model B, and "1820" from Generative AI Model C.

[1442] The server temporarily stores these outputs.

[1443] Step 6:

[1444] Based on the recognition results of the emotion engine, the server applies emotion adjustment logic to the received output to select the best response, for example, "1821" with a detailed description that matches the emotion "interesting."

[1445] Input: Emotional information and output from each generative system

[1446] Output: Optimal output (e.g., a detailed description of "1821")

[1447] Specific behavior:

[1448] The server takes into account the sentiment of "interesting" and selects "1821" with a detailed description as the best response.

[1449] Emotion regulation logic selects outputs that are adapted to the user's emotional state.

[1450] Step 7:

[1451] The final selected output is sent to the user's terminal and displayed to the user.

[1452] Input: Optimal Output

[1453] Output: The output sent to the user's terminal.

[1454] Specific behavior:

[1455] The server sends the optimal output (a detailed description of "1821") to the user's terminal.

[1456] The terminal displays this to the user so that the user can check the results.

[1457] (Application example 2)

[1458] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1459] Conventional natural language generation systems have limitations in providing appropriate output in response to user requests. Furthermore, they lack a mechanism for selecting output that corresponds to the user's emotional state, preventing improvements in user experience. In particular, in the advertising field, there is a demand for providing advertisements that are customized according to the user's emotions, but there has been a problem in that no appropriate system exists to achieve this.

[1460] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a request from a user, means for sending a request to a plurality of natural language generation systems, means for receiving output from the plurality of natural language generation systems, means for comparing and evaluating the received output, emotion recognition means for recognizing the user's emotion, advertisement selection means for selecting an optimal advertisement based on the output of the emotion recognition means, and means for providing the optimal advertisement to the user terminal. This makes it possible to provide optimal output and advertisements adapted to the user's emotional state.

[1461] The "means for receiving requests from users" is a function for sending questions or requests entered by users to the system.

[1462] The "means for sending requests to a plurality of natural language generation systems" is a function for sending requests received from a user to a plurality of natural language generation models in parallel.

[1463] The "means for receiving output from a plurality of natural language generation systems" is a function for receiving responses and generated text from each natural language generation system.

[1464] "Means for comparing and evaluating received outputs" refers to a function for comparing outputs obtained from multiple generative AI models to evaluate their quality and suitability.

[1465] "Means for removing outliers based on evaluation results" refers to techniques for removing outputs that lack validity as a result of evaluation.

[1466] "Means for selecting the optimal output" is a function for selecting the most appropriate output from the results of comparison and evaluation.

[1467] The "means for providing the optimal output to the user" is a function for returning the selected optimal output to the user.

[1468] The "emotion recognition means for recognizing the user's emotions" is a function for analyzing and recognizing the user's emotional state from their statements and text.

[1469] The "advertisement selection means for selecting the most suitable advertisement based on the output of the emotion recognition means" is a function for selecting an advertisement that is suited to the emotional state of the user determined by the emotion recognition means.

[1470] The "means for providing optimal advertisements to user terminals" is a function for displaying selected advertisements on the user's device.

[1471] The present invention provides a system for providing optimal output and advertisements adapted to the emotional state of a user. Specific embodiments of the system are described in detail below.

[1472] This system mainly includes a server and a user terminal, and is composed of the following hardware and software.

[1473] Hardware

[1474] server

[1475] Computational resources that operate under multiple generative AI models

[1476] Computing resources to run the emotion recognition engine

[1477] User terminal

[1478] Smartphone (iOS or Android)

[1479] Input devices such as microphones and cameras

[1480] software

[1481] Python

[1482] Emotion recognition engine (e.g., a model built with Keras)

[1483] API communication (e.g., requests library)

[1484] Generative AI Models

[1485] Natural language generation engines (e.g., GPT-3)

[1486] Data processing and calculation

[1487] Server Processing

[1488] 1. Receiving a request from a user

[1489] The server receives requests from the user terminal in the form of voice or text, which are requests for information or advertisements that the user wishes to view.

[1490] 2. Emotion judgment

[1491] The received voice or text data is analyzed using an emotion recognition engine to recognize the user's emotional state. For voice data, the tone, pitch, and speed of the voice are analyzed, and for text data, keywords and context are analyzed.

[1492] 3. Sending a request

[1493] The server sends the user's request to multiple natural language generation systems (generative AI models) in parallel. Specifically, it sends requests to the API endpoints of the generative AI models.

[1494] 4. Receiving Output

[1495] The server receives the output returned from each generative AI system and temporarily stores it in storage.

[1496] 5. Output evaluation and outlier removal

[1497] Each received output is compared and evaluated using an evaluation model, and outliers are removed using statistical methods based on the evaluation results.

[1498] 6. Selecting the optimal output

[1499] From the evaluated outputs, the one that best suits the user's emotional state is selected, based on emotion regulation logic.

[1500] 7. Ad Selection

[1501] Based on the output of the emotion recognition engine, an algorithm is run to select the most suitable advertisement.

[1502] User terminal processing

[1503] 1. Submit a request

[1504] Users enter questions or requests through a dedicated application or web browser and send them to the server in voice or text format.

[1505] 2. Optimal output and display of ads

[1506] The optimal output and advertisement returned from the server are displayed on the user terminal.

[1507] Specific examples

[1508] Prompt Sentence Examples

[1509] The user speaks the following prompt into their smartphone:

[1510] "I'm in a mood to chill out. I want to see ads that help me relax."

[1511] Based on these prompts, the generative AI model generates and serves relaxing ads to users, such as ads for "relaxing scented candles" or "meditation apps."

[1512] As described above, the system of the present invention aims to improve the user experience by providing optimal information and advertisements to users based on the output and emotional state from multiple generative AI models.

[1513] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1514] Step 1:

[1515] The user inputs a request in voice or text format through a dedicated application or web browser using a device. The input request includes a specific question or request from the user. For example, a prompt such as "I feel like I want to relax. Please show me a relaxing advertisement" may be input.

[1516] Step 2:

[1517] The terminal receives a request from the user and sends the data to the server. Here, the request data is sent to the server via the network. The input is the user's request content, and the output is communication data to the server.

[1518] Step 3:

[1519] The server receives requests sent by users and passes them to the emotion recognition engine. The emotion recognition engine analyzes the input voice or text data to identify the user's emotional state. Specifically, it analyzes the tone, pitch, and speed of the voice, as well as keywords. The input is the user's request data, and the output is the user's emotional state.

[1520] Step 4:

[1521] Based on the emotional state output by the emotion recognition engine, the server sends requests to multiple generative AI models. Here, a request including a prompt sentence is sent to the API endpoint of the generative AI model. The input is the emotional state and the prompt sentence, and the output is the request data for the generative AI model.

[1522] Step 5:

[1523] The server receives output from multiple generative AI models. Each generative AI model returns the text it generates in response to the prompt. The input is the response data from the generative AI model, and the output is the generated output.

[1524] Step 6:

[1525] The server passes the received outputs to the evaluation model for comparison and evaluation. The evaluation model uses a machine learning algorithm to evaluate the quality and relevance of the outputs. The input is the generated multiple outputs, and the output is an evaluation score or ranking.

[1526] Step 7:

[1527] Based on the evaluation results, the server uses statistical methods to remove outliers, where significantly mismatched outputs are removed. The input is the output with the evaluation score, and the output is the list of outputs with the outliers removed.

[1528] Step 8:

[1529] The server selects the optimal output and advertisement based on the output of the emotion recognition engine. In this process, the output and advertisement that best fit the user's emotional state are selected based on the emotion adjustment logic. The input is the output list after outlier removal and the emotional state, and the output is the optimal output and advertisement.

[1530] Step 9:

[1531] The server sends the selected optimal output and advertisement to the user terminal. The input is the optimal output and advertisement, and the output is the communication data to the user terminal.

[1532] Step 10:

[1533] The terminal displays the optimal output and advertisement received from the server to the user. Here, the optimal response to the request entered by the user and the advertisement are displayed. The input is the communication data from the server, and the output is the display content of the user terminal.

[1534] Through the above steps, a system that adapts to the emotional state of the user and provides optimal output and advertisements is specifically realized.

[1535] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1536] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1537] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1538] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1539] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1540] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1541] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1542] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1543] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1544] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1545] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1546] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1547] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1548] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1549] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1550] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1551] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1552] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1553] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1554] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1555] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1556] The following is further disclosed regarding the above embodiment.

[1557] (Claim 1)

[1558] means for receiving a request from a user;

[1559] means for sending requests to a plurality of natural language generation systems;

[1560] means for receiving output from a plurality of natural language generation systems;

[1561] a means of comparing and evaluating the received outputs;

[1562] means for removing outliers based on the evaluation results;

[1563] A means of selecting the optimal output;

[1564] A means of providing optimal output to users;

[1565] A system including:

[1566] (Claim 2)

[1567] 2. The system according to claim 1, wherein the means for removing outliers uses a statistical method.

[1568] (Claim 3)

[1569] 10. The system of claim 1, wherein the means for evaluating the received output uses a machine learning model.

[1570] "Example 1"

[1571] (Claim 1)

[1572] means for receiving a request from a user;

[1573] means for sending requests to a plurality of natural language generation systems;

[1574] means for receiving responses from a plurality of natural language generation systems;

[1575] a means for comparing and evaluating the responses received;

[1576] means for removing outliers based on the evaluation results;

[1577] means for selecting the best response in the set as a result of the evaluation;

[1578] a means for providing an optimal response to the user;

[1579] A system including:

[1580] (Claim 2)

[1581] 2. The system according to claim 1, wherein the means for removing outliers uses a statistical method.

[1582] (Claim 3)

[1583] 10. The system of claim 1, wherein the means for evaluating the received responses uses a machine learning model.

[1584] "Application Example 1"

[1585] (Claim 1)

[1586] means for receiving a request from a user;

[1587] means for sending requests to a plurality of natural language generation systems;

[1588] means for receiving output from a plurality of natural language generation systems;

[1589] a means of comparing and evaluating the received outputs;

[1590] means for removing outliers based on the evaluation results;

[1591] A means of selecting the optimal output;

[1592] A means of providing optimal output to users;

[1593] means for communicating with an external content providing system;

[1594] A means for responding with the optimal output in response to a question about a product from a user on an online shopping site;

[1595] A system including:

[1596] (Claim 2)

[1597] 2. The system according to claim 1, wherein the means for removing outliers uses a statistical method.

[1598] (Claim 3)

[1599] 10. The system of claim 1, wherein the means for evaluating the received output uses a machine learning model.

[1600] "Example 2: Combining Emotion Engines"

[1601] (Claim 1)

[1602] means for receiving a request from a user;

[1603] means for recognizing a user's emotion;

[1604] means for transmitting the user's request and emotional state;

[1605] means for sending requests to a plurality of natural language generation systems;

[1606] means for receiving output from a plurality of natural language generation systems;

[1607] a means for comparing and evaluating the received output based on emotional state;

[1608] A means for selecting the optimal output based on the evaluation results;

[1609] A means of providing optimal output to users;

[1610] A system including:

[1611] (Claim 2)

[1612] 2. The system according to claim 1, wherein the means for removing outliers uses a statistical method.

[1613] (Claim 3)

[1614] 10. The system of claim 1, wherein the means for evaluating the received output uses a machine learning model.

[1615] "Application example 2 when combining emotion engines"

[1616] (Claim 1)

[1617] means for receiving a request from a user;

[1618] means for sending requests to a plurality of natural language generation systems;

[1619] means for receiving output from a plurality of natural language generation systems;

[1620] a means of comparing and evaluating the received outputs;

[1621] means for removing outliers based on the evaluation results;

[1622] A means of selecting the optimal output;

[1623] A means of providing optimal output to users;

[1624] emotion recognition means for recognizing an emotion of a user;

[1625] an advertisement selection means for selecting an optimal advertisement based on the output of the emotion recognition means;

[1626] A means for providing an optimal advertisement to a user terminal;

[1627] A system including:

[1628] (Claim 2)

[1629] 2. The system according to claim 1, wherein the means for removing outliers uses a statistical method.

[1630] (Claim 3)

[1631] 10. The system of claim 1, wherein the means for evaluating the received output uses a machine learning model. [Explanation of symbols]

[1632] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving a request from a user; means for sending requests to a plurality of natural language generation systems; means for receiving output from a plurality of natural language generation systems; a means of comparing and evaluating the received outputs; means for removing outliers based on the evaluation results; A means of selecting the optimal output; A means of providing optimal output to users; A system including:

2. 2. The system of claim 1, wherein the means for removing outliers uses a statistical method.

3. 10. The system of claim 1, wherein the means for evaluating the received output uses a machine learning model.

Images