System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving input data from a user; means for sending a request to a plurality of generative models; means for receiving response data from the plurality of generative models; means for comparing, reviewing, and integrating the response data; means for generating an optimal output; and means for providing the optimal output to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, with so many generative models available, users have difficulty deciding which model to choose. While rankings and user reviews are helpful, they often fail to fully utilize the strengths of each generative model. Furthermore, combining generative models specialized in different domains is likely to produce higher-quality output, but this process is cumbersome for users. Therefore, there is a need for a system that can compare and examine the output of multiple generative models and automatically provide the optimal output. [Means for solving the problem]

[0005] The present invention relates to a system that includes a means for receiving input data from a user and sending requests to multiple generative models, a means for receiving response data from each generative model, a means for comparing, examining, and integrating the response data, a means for generating optimal output, and a means for providing this output to the user. This system allows users to easily obtain optimal output that combines the features of different generative models. Furthermore, by sending requests to specialized generative models and eliminating duplications and inconsistencies, it is possible to provide even more accurate results. For example, when creating travel plans or presentation materials, this system can provide high-quality suggestions that meet the user's needs.

[0006] "User" refers to an individual or organization that uses the system and provides input data.

[0007] "Input data" refers to the information provided by the user to the system, and is the data that forms the basis for processing the generative model.

[0008] A "generative model" refers to an algorithm or software that generates data specifically for a specific task, such as natural language generation or schedule optimization.

[0009] A "request" refers to a processing request that a system makes to a generative model, which is a request to generate output data based on specific input data.

[0010] "Response data" refers to the output data provided by a generative model in response to a request, and includes various information obtained from multiple generative models.

[0011] "Comparison and examination" refers to the process of analyzing multiple response data, evaluating the results, and selecting the most appropriate one.

[0012] "Synthesis" refers to the process of combining information obtained through comparison and scrutiny to produce a single, unified output.

[0013] "Output" refers to the resulting data that the system ultimately provides to the user, including the most appropriate information according to the user's requests.

[0014] "Serving" refers to the process of displaying or transmitting output from the system to the user.

[0015] "Travel planning" refers to detailed planning of travel destinations, dates, budget, etc., and is information that guides users when planning their trips.

[0016] "Presentation Materials" means the slides, documents, or visual materials used in delivering a presentation that contain information necessary for an effective presentation. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention relates to a system that helps a user utilize multiple generative models to obtain optimal output, and an embodiment of the system is described in detail below.

[0039] System configuration

[0040] The system consists of the following main components:

[0041] 1. A device that receives user input

[0042] 2. A server that sends requests to multiple generative models and receives their responses.

[0043] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[0044] 4. Devices that provide optimal output to users

[0045] Program Operation

[0046] Receiving user input

[0047] The device receives input data from the user. This data is the basis for the system to process. For example, a user might input a request such as, "I want to plan a trip next month with a budget of 100,000 yen."

[0048] Submitting a Request

[0049] The server receives user input data and sends requests to multiple generative models, including the following generative models:

[0050] Domestic travel plan generation model

[0051] Overseas travel plan generation model

[0052] Pricing Model

[0053] Schedule Optimization Model

[0054] The server creates and sends appropriate requests to these generative models.

[0055] Receiving response data

[0056] The server receives the response data from each generative model. For example, the response might look like this:

[0057] Domestic travel plan generation model: 3-night, 4-day Tokyo travel plan

[0058] Overseas travel plan generation model: 2-night, 3-day Seoul travel plan

[0059] Price calculation model: Tokyo trip costs 90,000 yen, Seoul trip costs 120,000 yen

[0060] Schedule optimization model: The best date is the second week of next month

[0061] Comparison, scrutiny and integration

[0062] The server uses an advanced generative model (Generation AI-4) to compare, examine, and integrate the output of each generative model. During this process, it eliminates duplicate information and inconsistencies to generate the optimal output. For example, if a 3-night, 4-day Tokyo trip plan is determined to be within budget, that plan will be selected.

[0063] Generate and deliver optimal output

[0064] Generation AI-4 generates the optimal output, which is received by the server, which then sends this output to the user's terminal, which finally displays the output to the user.

[0065] Specific examples

[0066] For example, if a user inputs "I want to plan a trip next month with a budget of 100,000 yen," the system will work as follows:

[0067] 1. The terminal receives input.

[0068] 2. The server sends requests to the domestic travel plan generation model, the international travel plan generation model, the fare calculation model, and the schedule optimization model.

[0069] 3. Each model returns a response: the domestic travel plan generation model suggests a "3-night, 4-day Tokyo trip plan," the overseas travel plan generation model suggests a "2-night, 3-day Seoul trip plan," the pricing calculation model suggests "A trip to Tokyo costs 90,000 yen, a trip to Seoul costs 120,000 yen," and the schedule optimization model suggests "The second week of next month is optimal."

[0070] 4. Generative AI-4 compares and examines these responses and generates the optimal output: "Plan a trip to Tokyo for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen."

[0071] 5. The server sends this output to the user's device.

[0072] 6. The device displays the best travel plan for the user.

[0073] In this way, users can easily obtain high-quality travel plans. Similar procedures can also be used to obtain high-quality output for other purposes, such as presentation materials.

[0074] The processing flow will be explained below.

[0075] Step 1:

[0076] The user inputs a request into the device. For example, the user inputs "I would like to plan a trip next month with a budget of 100,000 yen" into the device's input form.

[0077] Step 2:

[0078] The terminal receives input data from the user and sends it to the server.

[0079] Step 3:

[0080] The server analyzes the received input data and creates requests for multiple generation models: a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[0081] Step 4:

[0082] The server sends the created request to each generation model. For example, it sends a request such as "Generate a domestic travel plan with a budget of 100,000 yen next month" to the domestic travel plan generation model, and "Generate an overseas travel plan with a budget of 100,000 yen next month" to the overseas travel plan generation model.

[0083] Step 5:

[0084] The server receives responses from each generation model. Specifically, it receives responses such as "a four-day, three-night trip to Tokyo" from the domestic travel plan generation model, "a three-day, two-night trip to Seoul" from the overseas travel plan generation model, "a trip to Tokyo costs 90,000 yen, a trip to Seoul costs 120,000 yen" from the price calculation model, and "the second week of next month is optimal" from the schedule optimization model.

[0085] Step 6:

[0086] The server uses an advanced generative model (Generation AI-4) to compare and analyze the responses of each generative model and generate the optimal output. Specifically, it selects a plan that fits within the budget and optimizes the schedule.

[0087] Step 7:

[0088] Generation AI-4 generates the optimal output and provides the result to the server. For example, it might generate an output such as "A trip to Tokyo for three nights and four days in the second week of next month, with a budget of 90,000 yen."

[0089] Step 8:

[0090] The server generates the optimal output and sends it to the device.

[0091] Step 9:

[0092] The device displays the optimal output to the user, allowing the user to decide on specific actions to take. For example, it might say, "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen."

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] When using multiple generative models to obtain optimal output, it is necessary to appropriately process user input and examine and integrate the responses from each generative model to obtain high-quality results. However, these processes are complex to perform manually and require advanced processing to ensure the quality of the integrated output. Furthermore, there are insufficient means to automate the examination and integration process that takes into account the characteristics of multiple generative models, making it difficult to generate high-quality output in a short period of time.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes a means for a user to input a prompt sentence via an input form and for the server to send the prompt sentence to multiple generative models, a means for manipulating the received response data using numpy or pandas to generate optimal output, and a means for providing the optimal output to the user. This makes it possible to compare and examine the outputs of multiple generative models based on the user's input data and automatically generate high-quality output in a short period of time.

[0098] "User" means any person or entity that uses the System and provides Input Data.

[0099] A "terminal" is a computer system or mobile device used by a user to provide input data.

[0100] "Server" refers to a central processing unit for receiving input data from a user, sending requests to multiple generative models, receiving response data, and generating and providing optimal output.

[0101] A "generative model" is an algorithm or program that generates and responds to data based on a specific task.

[0102] A "prompt sentence" is text data that a user inputs to a system, and serves as the basic data for the system to process.

[0103] "Output" refers to the final result data generated by the server by comparing, examining, and integrating the response data of the generative models.

[0104] A "request" is an instruction or data that a server sends to a Generative Model.

[0105] "Response data" is data that a generative model returns in response to a request.

[0106] "Comparison and scrutiny" is the process of examining response data from multiple generative models and selecting the most appropriate data.

[0107] "Fusion" is the process by which the server combines appropriate response data from multiple generative models to produce a single output.

[0108] "numpy" is a Python library for manipulating large amounts of numerical data.

[0109] "pandas" is a Python library for data manipulation and analysis.

[0110] This invention relates to a system that allows a user to obtain optimal output using multiple generative models. The system aims to obtain optimal responses from multiple generative models by inputting a specific prompt sentence from the user, and then examine and integrate the responses to provide them to the user.

[0111] System configuration

[0112] The system mainly consists of the following components:

[0113] 1. A device that receives user input

[0114] 2. A server that sends requests to multiple generative models and receives their responses.

[0115] 3. A means to compare, examine, and integrate the outputs of generative models to generate the optimal output

[0116] 4. Devices that provide optimal output to users

[0117] Detailed processing

[0118] Receiving user input

[0119] The user uses a terminal to input a specific prompt sentence. The terminal is usually a computer device such as a PC or smartphone. The user inputs a prompt sentence such as "I would like to plan a trip next month with a budget of 100,000 yen" through an input form and submits it.

[0120] Submitting a Request

[0121] The terminal sends the received prompt to the server, which uses software such as Python's requests library or Java's HttpClient to send requests to multiple generative models. The generative models include a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[0122] Receiving response data

[0123] The server receives response data from each generation model. Each response data is usually received in JSON format and parsed using Python's json module or Java's Jackson library. For example, the server receives data such as a "3-night, 4-day Tokyo trip plan" from the domestic travel plan generation model, a "2-night, 3-day Seoul trip plan" from the overseas travel plan generation model, "Tokyo trip costs 90,000 yen, Seoul trip costs 120,000 yen" from the price calculation model, and "The second week of next month is optimal" from the schedule optimization model.

[0124] Comparison, scrutiny and integration

[0125] The server uses the Generative AI-4 to compare and refine the response data from each generative model and generate the optimal output. This process involves data manipulation using Python libraries such as numpy and pandas. For example, the response data is converted into a numpy array and the optimization algorithm is applied using a pandas data frame.

[0126] Generate and deliver optimal output

[0127] Generation AI-4 generates the optimal output: "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen." The server then sends this data to the user's device. The device then displays the received output using HTML and JavaScript and provides it to the user. Specifically, the received data is inserted into the display area, and the user is notified in a visually easy-to-understand format.

[0128] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0129] Step 1:

[0130] The user uses an input form on the terminal and enters the prompt statement "I would like to plan a trip next month with a budget of 100,000 yen." The input data is sent from the terminal to the server. The prompt statement is received as terminal input and an HTTP request is generated to send it to the server.

[0131] Step 2:

[0132] The server analyzes the prompt received from the user and sends requests to multiple generative models. Specifically, the server converts the prompt into an appropriate format and sends API requests to the generative models using the Python requests library. The input is the prompt, and the output is an API request to each generative model.

[0133] Step 3:

[0134] Each generative model receives a request and generates response data. For example, a domestic travel plan generation model generates a "Tokyo travel plan for 3 nights and 4 days" and sends it back to the server. The server receives the response data from the generative model, and the response is in JSON format.

[0135] Step 4:

[0136] The server receives and analyzes the response data from each generative model. Specifically, the server uses the Python json module to parse the response data and convert it into an internal data structure. The input is the response data in JSON format, and the output is an internal data structure (e.g., a Python dictionary or list).

[0137] Step 5:

[0138] The server uses numpy and pandas to compare and examine each response data and generate the optimal output. For example, the server compares the costs and dates of each travel plan and selects the optimal one. The input is the internal data structure, and the output is the optimal output (for example, "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen").

[0139] Step 6:

[0140] The server converts the optimal output into JSON format and sends it to the user's device. Specifically, the server generates an HTTP response using Flask or Django. The input is the optimal output data, and the output is the HTTP response sent to the user's device.

[0141] Step 7:

[0142] The terminal displays the output received from the server. Specifically, it displays the optimal travel plan on the screen using HTML and JavaScript. The input is the HTTP response from the server, and the output is the travel plan displayed to the user.

[0143] (Application example 1)

[0144] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0145] Conventional security incident response systems have difficulty in developing appropriate countermeasures and have experienced delays in response when a rapid response is required in seconds. Furthermore, it has been difficult to develop a system that can derive integrated and optimal countermeasures using multiple generative models specialized for individual elements. This has led to concerns about increased security risks and damage.

[0146] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0147] In this invention, the server includes means for receiving input data from a user, means for sending requests to multiple generative models, means for receiving response data from the multiple generative models, means for comparing, examining, and integrating the response data, means for generating an optimal output, means for providing the optimal output to the user, and means for receiving detailed information about a security incident and generating an incident response plan. This makes it possible to formulate quick and appropriate countermeasures for security incidents such as unauthorized access by utilizing multiple specialized generative models and provide them to users in real time.

[0148] "User-entered data" means information entered into a system by a user, including details and instructions regarding a security incident.

[0149] A "generative model" is an AI model with a specific role or function, such as analyzing unauthorized access patterns, assessing damage, generating emergency response procedures, and generating recovery procedures.

[0150] The "request sending means" is a means by which the system sends a request to the generative model based on user input data.

[0151] "Response data" is data that the generative model responds to a request, and includes the results of pattern analysis of unauthorized access and damage assessment results.

[0152] The "comparison and review means" refers to a means for comparing and reviewing the received response data and removing any duplication or inconsistency.

[0153] A "synthesis tool" is a tool that brings together the compared and reviewed response data into a single, integrated output.

[0154] The "output" is the optimal response or plan generated by the integration method.

[0155] "User-delivered means" refers to the means by which the integrated output is presented to the user.

[0156] A "security incident" is an event that threatens the security of a system or the protection of data, including unauthorized access or data breach.

[0157] An "incident response plan" is a plan that includes optimal responses and procedures for responding to security incidents.

[0158] This invention is a system that generates an optimal incident response plan through multiple generative models based on detailed information about a security incident entered by a user, and provides the plan to the user. This system can be run via a device such as a smartphone or smart glasses.

[0159] The system works as follows: First, the user enters detailed information about a security incident into a device such as a smartphone or smart glasses. This information includes unauthorized access logs and damage assessments. After receiving this information, the device sends it to the server.

[0160] Based on the input data from the user, the server sends requests to multiple generative models, such as a fraudulent access pattern analysis model, a damage assessment model, an emergency response procedure generation model, and a recovery procedure generation model. In response to this request, each generative model generates corresponding analysis results and proposals and returns them to the server.

[0161] The returned response data is compared and scrutinized by an advanced generative model (Generation AI-4) within the server. This advanced generative model has the ability to remove duplicates and inconsistencies from the response data and generate the optimal output. The output integrated by Generation AI-4 is further scrutinized within the server as the optimal incident response plan.

[0162] After generating an optimal incident response plan, the server sends this information back to the user's device and presents the user with appropriate countermeasures and procedures. The generative models and integration algorithms used in this process may utilize AI platforms such as Amazon SageMaker.

[0163] Below is a concrete example of how this system can be implemented, showing the prompt text that will be displayed when unauthorized access occurs.

[0164] Examples:

[0165] "Unauthorized access log ID: 12345 has occurred and the damage is high. Please generate the optimal countermeasures for this incident."

[0166] In this way, users can quickly receive and take appropriate measures to deal with security incidents. The system provides real-time information and helps minimize the damage caused by the incident.

[0167] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0168] Step 1:

[0169] The user enters details about the security incident into the device.

[0170] Input: Detailed information such as unauthorized access logs and damage status.

[0171] Output: Security incident information entered into the terminal.

[0172] Specific operation: The user enters "Unauthorized access log ID: 12345 has occurred and the damage is significant" into the device.

[0173] Step 2:

[0174] The terminal transmits the entered security incident information to the server.

[0175] Input: Information entered into the device.

[0176] Output: Information data sent to the server.

[0177] Specific operation: The device sends security incident information to a server via the Internet.

[0178] Step 3:

[0179] Based on the incident information received by the server, requests are sent to multiple generative models.

[0180] Input: Security incident information sent to the server.

[0181] Output: A request for each generative model.

[0182] Specific operation: The server sends requests to the unauthorized access pattern analysis model, the damage assessment model, the emergency response procedure generation model, and the recovery procedure generation model.

[0183] Step 4:

[0184] The generative model returns response data based on the request to the server.

[0185] Input: The requests sent to each generative model.

[0186] Output: Response data from each generative model.

[0187] Specific operation: The unauthorized access pattern analysis model generates the pattern analysis result, and the damage assessment model generates the damage assessment result, and these are returned to the server.

[0188] Step 5:

[0189] The server compares, examines, and integrates the response data received from each generative model using an advanced generative model (Generation AI-4).

[0190] Input: Response data from each generative model.

[0191] Output: A consolidated incident response plan.

[0192] Specific operation: The Generate AI-4 analyzes response data, removes duplicates and inconsistencies, and generates an optimal incident response plan.

[0193] Step 6:

[0194] The server sends the integrated incident response plan to the user's device.

[0195] Input: A consolidated incident response plan.

[0196] Output: The optimal response plan sent to the user's device.

[0197] Specific operation: The server sends the incident response plan to the user's device via the Internet.

[0198] Step 7:

[0199] The device displays an integrated incident response plan to the user.

[0200] Input: The corresponding plan sent to the user's device.

[0201] Output: The optimal incident response plan displayed to the user.

[0202] Specific operation: The device will display "Optimal countermeasure for unauthorized access log ID: 12345" on the screen.

[0203] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0204] The present invention relates to a system that helps a user obtain optimal output by utilizing multiple generative models and further recognizes the user's emotions to make optimal suggestions. An embodiment of this system will be described in detail below.

[0205] System configuration

[0206] The system consists of the following components:

[0207] 1. A device that receives user input

[0208] 2. A server that sends requests to multiple generative models and receives their responses.

[0209] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[0210] 4. Emotion engine that recognizes user emotions

[0211] 5. Devices that provide optimal output to users

[0212] Program Operation

[0213] Receiving user input

[0214] The terminal receives input data from the user. This data is the basis for the system to process. For example, a request might be entered such as, "I want to plan a trip next month with a budget of 100,000 yen."

[0215] Submitting a Request

[0216] Based on the user's input data, the server creates requests for multiple generation models: a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[0217] Recognizing and reflecting emotions

[0218] The server uses the emotion engine to analyze the emotion from the user's input data and reflects the results in the request to the generative model. For example, if a user inputs "I want a trip that will reduce stress," the emotion engine analyzes "relaxation" and sends a request to each generative model based on this emotion.

[0219] Receiving response data

[0220] The server receives response data from each generation model. For example, it receives a response such as "a 3-night, 4-day Okinawa trip plan" from the domestic travel plan generation model, a "2-night, 3-day Guam trip plan" from the overseas travel plan generation model, "a trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen" from the price calculation model, and "the best dates are the second week of next month" from the schedule optimization model.

[0221] Comparison, scrutiny and integration

[0222] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. In this example, a trip to Okinawa is selected, taking into consideration "relaxation." The server also references the user's previous emotional history to make more accurate suggestions.

[0223] Generate and deliver optimal output

[0224] Generation AI-4 generates the optimal output, which is received by the server. The server then sends this output to the user's device. If the optimal output is "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month with a budget of 90,000 yen," this will be provided to the user.

[0225] Displaying optimal output

[0226] The device displays the generated optimal output to the user, who can then decide on specific actions to take based on this information. For example, a detailed travel plan such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen" is displayed.

[0227] Specific examples

[0228] For example, if a user enters "I want to plan a trip next month with a budget of 100,000 yen, preferably somewhere relaxing," the system will work as follows:

[0229] 1. The terminal receives input.

[0230] 2. The server uses the emotion engine to analyze the emotion "relaxed" and reflects this in each generative model request.

[0231] 3. The server sends requests to the domestic travel plan generation model, the international travel plan generation model, the fare calculation model, and the schedule optimization model.

[0232] 4. Each model returns a response: the domestic travel plan generation model replies "A three-night, four-day trip to Okinawa," the overseas travel plan generation model replies "A two-night, three-day trip to Guam," the pricing calculation model replies "A trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen," and the schedule optimization model replies "The second week of next month is optimal."

[0233] 5. Generative AI-4 compares and refines these responses and generates the optimal output: "A relaxing, budget-friendly plan for a 3-night, 4-day trip to Okinawa in the second week of next month."

[0234] 6. The server sends this output to the user's device.

[0235] 7. The device displays the best travel plan for the user.

[0236] This system allows users to easily obtain high-quality travel plans based on emotions and requests. Similar procedures can also be used to obtain high-quality output for other purposes, such as presentation materials.

[0237] The processing flow will be explained below.

[0238] Step 1:

[0239] The user inputs a request into the device. For example, the user might input, "I want to plan a trip next month with a budget of 100,000 yen. I would like a relaxing place."

[0240] Step 2:

[0241] The terminal receives input data from the user and sends it to the server.

[0242] Step 3:

[0243] The server analyzes the received input data and uses an emotion engine to recognize the user's emotion, in this case "relaxed."

[0244] Step 4:

[0245] The server makes requests for multiple Generative Models, specifically the following Generative Models:

[0246] Domestic travel plan generation model: "Generate a relaxing domestic travel plan with a budget of 100,000 yen next month"

[0247] "Generate a relaxing overseas travel plan with a budget of 100,000 yen for next month"

[0248] "Calculate the cost of each plan" in the pricing model

[0249] "Generate optimal travel schedules" for schedule optimization models

[0250] Step 5:

[0251] The server sends the created request to each generative model.

[0252] Step 6:

[0253] The server receives a response from each generative model, which includes, for example, the following information:

[0254] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[0255] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[0256] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[0257] The schedule optimization model says, "The second week of next month is optimal."

[0258] Step 7:

[0259] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. Specifically, it selects a plan (Okinawa travel plan) that meets the "relaxation" criteria and fits within the budget.

[0260] Step 8:

[0261] The server then sends the generated optimal output to the terminal. For example, it generates a plan such as "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month with a budget of 90,000 yen."

[0262] Step 9:

[0263] The device displays the optimal output to the user, allowing the user to decide on specific actions to take. For example, a detailed travel plan such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen" is displayed.

[0264] Step 10:

[0265] The server stores the user's emotion history, which is then referenced the next time output is generated to provide more accurate suggestions.

[0266] Example 2

[0267] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0268] Conventional systems have struggled to generate optimal outputs that take into account specific user emotions. Furthermore, when integrating response data from multiple generative models, there was a lack of a way to incorporate user emotions, resulting in poor recommendation accuracy and reduced user satisfaction. To address this issue, there is a need for systems that can analyze user emotions and incorporate them into generative model requests to provide more accurate recommendations.

[0269] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0270] In this invention, the server includes means for receiving user input, means for sending requests to multiple generative models, means for analyzing emotions, means for receiving response data, means for comparing, examining, and integrating the response data, means for generating optimal output, means for providing the optimal output to the user, and means for reflecting the results of the emotion analysis in the requests of the generative models, thereby enabling the generation of highly accurate output that reflects the user's emotions.

[0271] "Means for receiving user input" refers to an input device or form through which a user enters information or requests into the system.

[0272] "Means for sending a request" refers to a device or program that has the function of creating and sending requests to multiple generative models based on information received from a user.

[0273] A "generative model" refers to an algorithm or program that generates a specific output based on specific conditions and input data.

[0274] "Means for analyzing emotions" means a program or device for analyzing and identifying a user's emotional state based on input data or other information from the user.

[0275] "Means for receiving response data" refers to a device or program for receiving results sent from multiple generative models.

[0276] "Means for comparing, examining, and integrating response data" refers to a program or device that compares data received from different generative models, eliminates inconsistencies and duplications, and integrates them in the most optimal way.

[0277] "Means for generating optimal outputs" refers to programs and algorithms that generate final recommendations or results based on integrated data and taking into account user requirements and emotions.

[0278] "Means for providing optimal output to a user" refers to devices or programs for displaying, transmitting, or providing the generated output to a user.

[0279] "Means for reflecting the results of emotion analysis in requests to a generative model" refers to a device or program that has the function of incorporating the analyzed user's emotion information into requests to a generative model and adjusting the content of the request.

[0280] MODE FOR CARRYING OUT THE INVENTION

[0281] The present invention relates to a system that helps a user obtain optimal output by utilizing multiple generative models and further recognizes the user's emotions to make optimal suggestions. An embodiment of this system will be described in detail below.

[0282] System configuration

[0283] The system consists of the following components:

[0284] 1. A device that receives user input

[0285] 2. A server that sends requests to multiple generative models and receives their responses.

[0286] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[0287] 4. Emotion engine that recognizes user emotions

[0288] 5. Devices that provide optimal output to users

[0289] Receiving user input

[0290] The specific hardware used for the device may be a PC, tablet, smartphone, etc. The software used to receive input data from the user and send it to the system may include a browser-based application or a dedicated app. For example, a user may enter "I'd like to plan a trip next month with a budget of 100,000 yen. I'd like a relaxing place" into the device's input field and press the send button.

[0291] Creating and Sending a Request

[0292] The server is a core system of hardware that creates and sends requests for the required generative models based on user input data. Specific generative models include:

[0293] Domestic travel plan generation model

[0294] Overseas travel plan generation model

[0295] Pricing Model

[0296] Schedule Optimization Model

[0297] The server sends a request to each generative model as follows:

[0298] A domestic travel plan generation model asks, "Generate a relaxing domestic travel plan with a budget of 100,000 yen."

[0299] For the overseas travel plan generation model, "Generate a relaxing overseas travel plan with a budget of 100,000 yen"

[0300] The pricing model asks, "Calculate the budget for domestic and international travel plans."

[0301] "Please provide me with the best travel dates for next month" to the schedule optimization model

[0302] Recognizing and reflecting emotions

[0303] The server uses an emotion engine to analyze emotions from the user's input data. Commonly available software tools such as Microsoft's Azure Emotion API and Google's Vision API can be used as emotion engines. The analyzed emotion data is reflected in the request. For example, if the emotion "relaxed" is analyzed from the user's input, the request is updated based on that element.

[0304] Receiving response data

[0305] The server receives response data from each generative model. For example, the following response is returned:

[0306] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[0307] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[0308] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[0309] The schedule optimization model says, "The second week of next month is optimal."

[0310] Comparison, scrutiny and integration

[0311] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. Generative AI-4 uses advanced algorithms built on OpenAI's GPT-4 and Google's BERT. This allows it to derive the most suitable suggestions.

[0312] Generate and deliver optimal output

[0313] Generation AI-4 generates the optimal output, which is received by the server. The server then sends this output to the user's device. For example, a plan might be generated for a relaxing, budget-friendly trip to Okinawa in the second week of next month for three nights and four days.

[0314] Displaying optimal output

[0315] The device displays the generated optimal output to the user, who can then decide on specific actions to take based on this information. For example, a detailed travel plan might be presented, such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen."

[0316] Specific prompt examples

[0317] For example, the prompt you would enter into a generative AI model might look like this:

[0318] "The user wants to relax and is planning a trip with a budget of 100,000 yen. Please suggest the best travel plan, whether domestic or international."

[0319] This procedure allows the system to provide optimal suggestions based on the user's requests and emotions.

[0320] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0321] Step 1:

[0322] The terminal receives user input.

[0323] Specific operation: The user enters "I want to plan a trip next month with a budget of 100,000 yen. I would like a relaxing place" into the input field on the device and presses the send button.

[0324] Input: Request data entered by the user (e.g., "I'd like to plan a trip next month with a budget of 100,000 yen. I'd like a relaxing place.")

[0325] Output: Request data sent from the terminal to the server

[0326] Step 2:

[0327] The server creates and sends requests for each generative model based on the user's input data.

[0328] Specific operation: The server analyzes the user's input data and sends the following request to each generative model:

[0329] A domestic travel plan generation model asks, "Generate a relaxing domestic travel plan with a budget of 100,000 yen."

[0330] For the overseas travel plan generation model, "Generate a relaxing overseas travel plan with a budget of 100,000 yen"

[0331] The pricing model asks, "Calculate the budget for domestic and international travel plans."

[0332] "Please provide me with the best travel dates for next month" to the schedule optimization model

[0333] Input: User request data

[0334] Output: Requests to each generative model

[0335] Step 3:

[0336] The server uses an emotion engine to analyze emotions from the user's input data.

[0337] Specific operation: The server uses an emotion engine (e.g., Microsoft's Azure Emotion API) to analyze the emotion "relaxed" and reflects this in the request to the generative model.

[0338] Input: User request data, emotion engine

[0339] Output: A request to the generative model that reflects the emotional elements

[0340] Step 4:

[0341] The generative model receives the server request and generates response data.

[0342] Specific operation: Each generative model generates the following response data based on the request and sends it to the server.

[0343] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[0344] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[0345] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[0346] The schedule optimization model says, "The second week of next month is optimal."

[0347] Input: The request sent by the server

[0348] Output: Response data from each generative model

[0349] Step 5:

[0350] The server receives the response data from each generative model.

[0351] Specific operation: The server receives and stores response data from each generative model.

[0352] Input: Response data from each generative model (e.g., "A trip to Okinawa for 3 nights and 4 days," "A trip to Guam for 2 nights and 3 days," "A trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen," "The second week of next month is best").

[0353] Output: Save the received response data

[0354] Step 6:

[0355] The server uses an advanced generative model (Generation AI-4) to compare, examine, and integrate the responses of each generative model to generate the optimal output.

[0356] Specific operation: The Generation AI-4 analyzes each response data and generates the optimal output while prioritizing the user's emotion of "relaxation." It also refers to the user's previous emotion history to select the most appropriate suggestion.

[0357] Input: Response data from each generative model, user emotion data, previous user emotion history

[0358] Output: Optimal output (e.g., "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen")

[0359] Step 7:

[0360] The server receives the optimal output from the generated AI-4 and sends it to the user's device.

[0361] Specific operation: The server receives the optimal output from generation AI-4 and sends it to the user's terminal.

[0362] Input: Optimal Output

[0363] Output: Data sent to the terminal

[0364] Step 8:

[0365] The device will display the generated optimal output to the user.

[0366] Specific operation: The device displays a detailed travel plan such as "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen." The user can then view this and make a specific reservation.

[0367] Input: The best output sent by the server

[0368] Output: Data displayed to the user

[0369] (Application example 2)

[0370] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0371] Conventional shopping assistant systems have the problem of only being able to make uniform product recommendations without fully considering the user's specific needs and emotions. Furthermore, it is difficult to make optimal product suggestions that reflect the user's emotions and circumstances, which change in real time, and they lack ingenuity to improve the user experience. Furthermore, it is practically difficult to provide interactive product information through smart glasses or other advanced user devices. To solve these issues, a system is needed that can analyze user input data and emotions in real time and provide optimal output accordingly.

[0372] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving input data from a user, means for sending requests to multiple generative models, means for receiving response data from the multiple generative models, means for recognizing the user's emotions and reflecting them in the request, means for comparing, examining, and integrating the response data, and means for providing optimal output to the user terminal. This enables high-quality, personalized product recommendations based on the user's specific needs and emotions.

[0373] "User input" refers to data or requirements provided by a user to a system.

[0374] A "generative model" is a machine learning algorithm that generates a specific output based on data.

[0375] A "request" refers to user input or an instruction or question sent by the system to a generative model.

[0376] "Response data" refers to the information or results returned by a generative model.

[0377] "Comparison and examination" refers to the act of comparing multiple response data with each other and eliminating any inconsistencies or discrepancies.

[0378] "Integration" is the act of combining the optimal outputs based on the results of comparison and scrutiny.

[0379] "Emotion recognition" refers to the techniques and processes used to analyze and recognize a user's emotional state.

[0380] "Smart glasses" are wearable devices that have built-in computer functions and allow users to obtain information visually.

[0381] "User terminal" refers to an electronic device used by a user to input or output information.

[0382] "Output" refers to the final results or recommendations that the system provides to the user.

[0383] This invention relates to a system that helps users obtain optimal output by utilizing multiple generative models and recognizes the user's emotions to provide optimal suggestions. This system can be realized as a shopping assistant application using smart glasses that enhances the shopping experience in brick-and-mortar stores.

[0384] System configuration

[0385] The system consists of the following components:

[0386] 1. A way to receive user input: Using the smart glasses' voice input or eye tracking capabilities, the glasses can receive requests about products the user is considering purchasing. For example, "I want a new smartphone, and I'm looking for a model with the latest features."

[0387] 2. A means for sending requests to multiple generative models: User input data is sent to the server, and requests are issued to various generative models. Specifically, requests are sent to the latest smartphone detail model, price comparison model, user review analysis model, and feature comparison model.

[0388] 3. Means for receiving response data from the generative model: The server receives response data from the generative model. For example, the latest smartphone detailed model might say "Latest Model A," the price comparison model might say "Model A is 90,000 yen," the user review analysis model might say "Model A's rating: 4.8," and the feature comparison model might say "Model A has the latest feature XX."

[0389] 4. Means for recognizing emotions and incorporating them into requests: The server uses emotion recognition APIs (e.g., Amazon Rekognition, Microsoft Azure Face API) to analyze the user's emotions from their tone of voice and facial expressions. This allows the server to analyze the user's emotions, such as "excitement" or "expectation," and incorporate them into the generative model's requests.

[0390] 5. Means of comparing, scrutinizing, and integrating response data: The server uses an advanced generative AI model (e.g., OpenAI GPT-4) to compare and scrutinize the responses from each generative model to generate the optimal output, eliminating duplication and inconsistency and making optimal suggestions taking into account user sentiment.

[0391] 6. Means for providing optimal output to the user device: The server sends the generated optimal output to the smart glasses. For example, the smart glasses display will show "Model A with the latest features is the best. Price is 90,000 yen, rating is 4.8."

[0392] Specific examples

[0393] If a user uses smart glasses to say, "I want a new smartphone, I'm looking for a model with the latest features," the system will:

[0394] 1. Receiving user input: The smart glasses receive requests via voice input or eye tracking.

[0395] 2. Sending a request: The received input data is sent to the server and a request is issued to the generative model.

[0396] 3. Emotion recognition and reflection: The emotion engine analyzes the user's emotions and reflects the emotions of "excitement and anticipation" in the generative model's requests.

[0397] 4. Receiving response data: The server receives the response data from the generative model.

[0398] 5. Compare, Refine and Integrate: The Generate AI-4 refines the response data and generates the optimal output.

[0399] 6. Providing optimal output: The generated output is sent to the smart glasses and displayed to the user.

[0400] Prompt Sentence Examples

[0401] "Please tell me the latest smartphone model."

[0402] "What is the average price of this smartphone?"

[0403] "What are the user reviews for this smartphone?"

[0404] "What are the main features of this smartphone?"

[0405] This system allows users to receive real-time personalized product information in physical stores, providing a better shopping experience.

[0406] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0407] Step 1: Receiving User Input

[0408] A user uses the smart glasses to input a request through voice input or eye tracking. This input data includes specific information about the product they are considering purchasing. For example, a user might say, "I want a new smartphone, and I'm looking for a model with the latest features." The device receives this input data and prepares it for processing.

[0409] Step 2: Create and submit a request

[0410] The device receives the user's input data and sends it to the server. The server analyzes this data and creates requests to multiple generative models. Specifically, based on the request received from the device, "I want a new smartphone," the server generates and sends the following prompts to the "latest smartphone detailed model," "price comparison model," "user review analysis model," and "feature comparison model," respectively:

[0411] "Please tell me the latest smartphone model."

[0412] "What is the average price of this smartphone?"

[0413] "What are the user reviews for this smartphone?"

[0414] "What are the main features of this smartphone?"

[0415] Step 3: Recognize the emotion and reflect it in the request

[0416] The server uses an emotion recognition API (e.g., Amazon Rekognition, Microsoft Azure Face API) to recognize emotions from the user's tone of voice and facial expressions. For example, if a user emphasizes "latest," the emotion engine analyzes the emotion as "excitement and anticipation." This emotion information is reflected in the request for the generative model, and a request with the emotion is sent to each generative model.

[0417] Step 4: Receiving response data

[0418] The server receives response data from each generative model. For example, the response from the "latest smartphone detailed model" may be "latest model A," the response from the "price comparison model" may be "Model A is 90,000 yen," the response from the "user review analysis model" may be "Model A's rating: 4.8," and the response from the "feature comparison model" may be "Model A is equipped with the latest feature XX." This response data is aggregated by the server.

[0419] Step 5: Compare, examine, and consolidate response data

[0420] The server uses an advanced generative AI model (e.g., OpenAI GPT-4) to compare, examine, and integrate the response data received from each generative model. During this process, it removes duplication and inconsistencies and selects the optimal output, taking into account the user's emotions. For example, it selects "Latest Model A" as the optimal output, reflecting emotions such as excitement and anticipation.

[0421] Step 6: Generate and deliver optimal output

[0422] The server generates the optimal output using Generation AI-4 and sends it to the user's device (smart glasses). For example, it generates content such as "Model A, equipped with the latest features, is the best. Price: 90,000 yen, rating: 4.8." The device receives this data and displays it on its screen.

[0423] Step 7: Displaying the optimal output

[0424] The device displays the optimal output to the user, who can then decide on a specific course of action based on this information. For example, the smart glasses display might say, "Model A, equipped with the latest features, is the best choice. It's priced at 90,000 yen and has a rating of 4.8." This allows the user to immediately consider purchasing the product in the store.

[0425] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0426] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0427] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0428] [Second embodiment]

[0429] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0430] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0431] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0432] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0433] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0434] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0435] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0436] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0437] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0438] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0439] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0440] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0441] The present invention relates to a system that helps a user utilize multiple generative models to obtain optimal output, and an embodiment of the system is described in detail below.

[0442] System configuration

[0443] The system consists of the following main components:

[0444] 1. A device that receives user input

[0445] 2. A server that sends requests to multiple generative models and receives their responses.

[0446] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[0447] 4. Devices that provide optimal output to users

[0448] Program Operation

[0449] Receiving user input

[0450] The device receives input data from the user. This data is the basis for the system to process. For example, a user might input a request such as, "I want to plan a trip next month with a budget of 100,000 yen."

[0451] Submitting a Request

[0452] The server receives user input data and sends requests to multiple generative models, including the following generative models:

[0453] Domestic travel plan generation model

[0454] Overseas travel plan generation model

[0455] Pricing Model

[0456] Schedule Optimization Model

[0457] The server creates and sends appropriate requests to these generative models.

[0458] Receiving response data

[0459] The server receives the response data from each generative model. For example, the response might look like this:

[0460] Domestic travel plan generation model: 3-night, 4-day Tokyo travel plan

[0461] Overseas travel plan generation model: 2-night, 3-day Seoul travel plan

[0462] Price calculation model: Tokyo trip costs 90,000 yen, Seoul trip costs 120,000 yen

[0463] Schedule optimization model: The best date is the second week of next month

[0464] Comparison, scrutiny and integration

[0465] The server uses an advanced generative model (Generation AI-4) to compare, examine, and integrate the output of each generative model. During this process, it eliminates duplicate information and inconsistencies to generate the optimal output. For example, if a 3-night, 4-day Tokyo trip plan is determined to be within budget, that plan will be selected.

[0466] Generate and deliver optimal output

[0467] Generation AI-4 generates the optimal output, which is received by the server, which then sends this output to the user's terminal, which finally displays the output to the user.

[0468] Specific examples

[0469] For example, if a user inputs "I want to plan a trip next month with a budget of 100,000 yen," the system will work as follows:

[0470] 1. The terminal receives input.

[0471] 2. The server sends requests to the domestic travel plan generation model, the international travel plan generation model, the fare calculation model, and the schedule optimization model.

[0472] 3. Each model returns a response: the domestic travel plan generation model suggests a "3-night, 4-day Tokyo trip plan," the overseas travel plan generation model suggests a "2-night, 3-day Seoul trip plan," the pricing calculation model suggests "A trip to Tokyo costs 90,000 yen, a trip to Seoul costs 120,000 yen," and the schedule optimization model suggests "The second week of next month is optimal."

[0473] 4. Generative AI-4 compares and examines these responses and generates the optimal output: "Plan a trip to Tokyo for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen."

[0474] 5. The server sends this output to the user's device.

[0475] 6. The device displays the best travel plan for the user.

[0476] In this way, users can easily obtain high-quality travel plans. Similar procedures can also be used to obtain high-quality output for other purposes, such as presentation materials.

[0477] The processing flow will be explained below.

[0478] Step 1:

[0479] The user inputs a request into the device. For example, the user inputs "I would like to plan a trip next month with a budget of 100,000 yen" into the device's input form.

[0480] Step 2:

[0481] The terminal receives input data from the user and sends it to the server.

[0482] Step 3:

[0483] The server analyzes the received input data and creates requests for multiple generation models: a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[0484] Step 4:

[0485] The server sends the created request to each generation model. For example, it sends a request such as "Generate a domestic travel plan with a budget of 100,000 yen next month" to the domestic travel plan generation model, and "Generate an overseas travel plan with a budget of 100,000 yen next month" to the overseas travel plan generation model.

[0486] Step 5:

[0487] The server receives responses from each generation model. Specifically, it receives responses such as "a four-day, three-night trip to Tokyo" from the domestic travel plan generation model, "a three-day, two-night trip to Seoul" from the overseas travel plan generation model, "a trip to Tokyo costs 90,000 yen, a trip to Seoul costs 120,000 yen" from the price calculation model, and "the second week of next month is optimal" from the schedule optimization model.

[0488] Step 6:

[0489] The server uses an advanced generative model (Generation AI-4) to compare and analyze the responses of each generative model and generate the optimal output. Specifically, it selects a plan that fits within the budget and optimizes the schedule.

[0490] Step 7:

[0491] Generation AI-4 generates the optimal output and provides the result to the server. For example, it might generate an output such as "A trip to Tokyo for three nights and four days in the second week of next month, with a budget of 90,000 yen."

[0492] Step 8:

[0493] The server generates the optimal output and sends it to the device.

[0494] Step 9:

[0495] The device displays the optimal output to the user, allowing the user to decide on specific actions to take. For example, it might say, "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen."

[0496] Example 1

[0497] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0498] When using multiple generative models to obtain optimal output, it is necessary to appropriately process user input and examine and integrate the responses from each generative model to obtain high-quality results. However, these processes are complex to perform manually and require advanced processing to ensure the quality of the integrated output. Furthermore, there are insufficient means to automate the examination and integration process that takes into account the characteristics of multiple generative models, making it difficult to generate high-quality output in a short period of time.

[0499] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0500] In this invention, the server includes a means for a user to input a prompt sentence via an input form and for the server to send the prompt sentence to multiple generative models, a means for manipulating the received response data using numpy or pandas to generate optimal output, and a means for providing the optimal output to the user. This makes it possible to compare and examine the outputs of multiple generative models based on the user's input data and automatically generate high-quality output in a short period of time.

[0501] "User" means any person or entity that uses the System and provides Input Data.

[0502] A "terminal" is a computer system or mobile device used by a user to provide input data.

[0503] "Server" refers to a central processing unit for receiving input data from a user, sending requests to multiple generative models, receiving response data, and generating and providing optimal output.

[0504] A "generative model" is an algorithm or program that generates and responds to data based on a specific task.

[0505] A "prompt sentence" is text data that a user inputs to a system, and serves as the basic data for the system to process.

[0506] "Output" refers to the final result data generated by the server by comparing, examining, and integrating the response data of the generative models.

[0507] A "request" is an instruction or data that a server sends to a Generative Model.

[0508] "Response data" is data that a generative model returns in response to a request.

[0509] "Comparison and scrutiny" is the process of examining response data from multiple generative models and selecting the most appropriate data.

[0510] "Fusion" is the process by which the server combines appropriate response data from multiple generative models to produce a single output.

[0511] "numpy" is a Python library for manipulating large amounts of numerical data.

[0512] "pandas" is a Python library for data manipulation and analysis.

[0513] This invention relates to a system that allows a user to obtain optimal output using multiple generative models. The system aims to obtain optimal responses from multiple generative models by inputting a specific prompt sentence from the user, and then examine and integrate the responses to provide them to the user.

[0514] System configuration

[0515] The system mainly consists of the following components:

[0516] 1. A device that receives user input

[0517] 2. A server that sends requests to multiple generative models and receives their responses.

[0518] 3. A means to compare, examine, and integrate the outputs of generative models to generate the optimal output

[0519] 4. Devices that provide optimal output to users

[0520] Detailed processing

[0521] Receiving user input

[0522] The user uses a terminal to input a specific prompt sentence. The terminal is usually a computer device such as a PC or smartphone. The user inputs a prompt sentence such as "I would like to plan a trip next month with a budget of 100,000 yen" through an input form and submits it.

[0523] Submitting a Request

[0524] The terminal sends the received prompt to the server, which uses software such as Python's requests library or Java's HttpClient to send requests to multiple generative models. The generative models include a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[0525] Receiving response data

[0526] The server receives response data from each generation model. Each response data is usually received in JSON format and parsed using Python's json module or Java's Jackson library. For example, the server receives data such as a "3-night, 4-day Tokyo trip plan" from the domestic travel plan generation model, a "2-night, 3-day Seoul trip plan" from the overseas travel plan generation model, "Tokyo trip costs 90,000 yen, Seoul trip costs 120,000 yen" from the price calculation model, and "The second week of next month is optimal" from the schedule optimization model.

[0527] Comparison, scrutiny and integration

[0528] The server uses the Generative AI-4 to compare and refine the response data from each generative model and generate the optimal output. This process involves data manipulation using Python libraries such as numpy and pandas. For example, the response data is converted into a numpy array and the optimization algorithm is applied using a pandas data frame.

[0529] Generate and deliver optimal output

[0530] Generation AI-4 generates the optimal output: "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen." The server then sends this data to the user's device. The device then displays the received output using HTML and JavaScript and provides it to the user. Specifically, the received data is inserted into the display area, and the user is notified in a visually easy-to-understand format.

[0531] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0532] Step 1:

[0533] The user uses an input form on the terminal and enters the prompt statement "I would like to plan a trip next month with a budget of 100,000 yen." The input data is sent from the terminal to the server. The prompt statement is received as terminal input and an HTTP request is generated to send it to the server.

[0534] Step 2:

[0535] The server analyzes the prompt received from the user and sends requests to multiple generative models. Specifically, the server converts the prompt into an appropriate format and sends API requests to the generative models using the Python requests library. The input is the prompt, and the output is an API request to each generative model.

[0536] Step 3:

[0537] Each generative model receives a request and generates response data. For example, a domestic travel plan generation model generates a "Tokyo travel plan for 3 nights and 4 days" and sends it back to the server. The server receives the response data from the generative model, and the response is in JSON format.

[0538] Step 4:

[0539] The server receives and analyzes the response data from each generative model. Specifically, the server uses the Python json module to parse the response data and convert it into an internal data structure. The input is the response data in JSON format, and the output is an internal data structure (e.g., a Python dictionary or list).

[0540] Step 5:

[0541] The server uses numpy and pandas to compare and examine each response data and generate the optimal output. For example, the server compares the costs and dates of each travel plan and selects the optimal one. The input is the internal data structure, and the output is the optimal output (for example, "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen").

[0542] Step 6:

[0543] The server converts the optimal output into JSON format and sends it to the user's device. Specifically, the server generates an HTTP response using Flask or Django. The input is the optimal output data, and the output is the HTTP response sent to the user's device.

[0544] Step 7:

[0545] The terminal displays the output received from the server. Specifically, it displays the optimal travel plan on the screen using HTML and JavaScript. The input is the HTTP response from the server, and the output is the travel plan displayed to the user.

[0546] (Application example 1)

[0547] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0548] Conventional security incident response systems have difficulty in developing appropriate countermeasures and have experienced delays in response when a rapid response is required in seconds. Furthermore, it has been difficult to develop a system that can derive integrated and optimal countermeasures using multiple generative models specialized for individual elements. This has led to concerns about increased security risks and damage.

[0549] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0550] In this invention, the server includes means for receiving input data from a user, means for sending requests to multiple generative models, means for receiving response data from the multiple generative models, means for comparing, examining, and integrating the response data, means for generating an optimal output, means for providing the optimal output to the user, and means for receiving detailed information about a security incident and generating an incident response plan. This makes it possible to formulate quick and appropriate countermeasures for security incidents such as unauthorized access by utilizing multiple specialized generative models and provide them to users in real time.

[0551] "User-entered data" means information entered into a system by a user, including details and instructions regarding a security incident.

[0552] A "generative model" is an AI model with a specific role or function, such as analyzing unauthorized access patterns, assessing damage, generating emergency response procedures, and generating recovery procedures.

[0553] The "request sending means" is a means by which the system sends a request to the generative model based on user input data.

[0554] "Response data" is data that the generative model responds to a request, and includes the results of pattern analysis of unauthorized access and damage assessment results.

[0555] The "comparison and review means" refers to a means for comparing and reviewing the received response data and removing any duplication or inconsistency.

[0556] A "synthesis tool" is a tool that brings together the compared and reviewed response data into a single, integrated output.

[0557] The "output" is the optimal response or plan generated by the integration method.

[0558] "User-delivered means" refers to the means by which the integrated output is presented to the user.

[0559] A "security incident" is an event that threatens the security of a system or the protection of data, including unauthorized access or data breach.

[0560] An "incident response plan" is a plan that includes optimal responses and procedures for responding to security incidents.

[0561] This invention is a system that generates an optimal incident response plan through multiple generative models based on detailed information about a security incident entered by a user, and provides the plan to the user. This system can be run via a device such as a smartphone or smart glasses.

[0562] The system works as follows: First, the user enters detailed information about a security incident into a device such as a smartphone or smart glasses. This information includes unauthorized access logs and damage assessments. After receiving this information, the device sends it to the server.

[0563] Based on the input data from the user, the server sends requests to multiple generative models, such as a fraudulent access pattern analysis model, a damage assessment model, an emergency response procedure generation model, and a recovery procedure generation model. In response to this request, each generative model generates corresponding analysis results and proposals and returns them to the server.

[0564] The returned response data is compared and scrutinized by an advanced generative model (Generation AI-4) within the server. This advanced generative model has the ability to remove duplicates and inconsistencies from the response data and generate the optimal output. The output integrated by Generation AI-4 is further scrutinized within the server as the optimal incident response plan.

[0565] After generating an optimal incident response plan, the server sends this information back to the user's device and presents the user with appropriate countermeasures and procedures. The generative models and integration algorithms used in this process may utilize AI platforms such as Amazon SageMaker.

[0566] Below is a concrete example of how this system can be implemented, showing the prompt text that will be displayed when unauthorized access occurs.

[0567] Examples:

[0568] "Unauthorized access log ID: 12345 has occurred and the damage is high. Please generate the optimal countermeasures for this incident."

[0569] In this way, users can quickly receive and take appropriate measures to deal with security incidents. The system provides real-time information and helps minimize the damage caused by the incident.

[0570] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0571] Step 1:

[0572] The user enters details about the security incident into the device.

[0573] Input: Detailed information such as unauthorized access logs and damage status.

[0574] Output: Security incident information entered into the terminal.

[0575] Specific operation: The user enters "Unauthorized access log ID: 12345 has occurred and the damage is significant" into the device.

[0576] Step 2:

[0577] The terminal transmits the entered security incident information to the server.

[0578] Input: Information entered into the device.

[0579] Output: Information data sent to the server.

[0580] Specific operation: The device sends security incident information to a server via the Internet.

[0581] Step 3:

[0582] Based on the incident information received by the server, requests are sent to multiple generative models.

[0583] Input: Security incident information sent to the server.

[0584] Output: A request for each generative model.

[0585] Specific operation: The server sends requests to the unauthorized access pattern analysis model, the damage assessment model, the emergency response procedure generation model, and the recovery procedure generation model.

[0586] Step 4:

[0587] The generative model returns response data based on the request to the server.

[0588] Input: The requests sent to each generative model.

[0589] Output: Response data from each generative model.

[0590] Specific operation: The unauthorized access pattern analysis model generates the pattern analysis result, and the damage assessment model generates the damage assessment result, and these are returned to the server.

[0591] Step 5:

[0592] The server compares, examines, and integrates the response data received from each generative model using an advanced generative model (Generation AI-4).

[0593] Input: Response data from each generative model.

[0594] Output: A consolidated incident response plan.

[0595] Specific operation: The Generate AI-4 analyzes response data, removes duplicates and inconsistencies, and generates an optimal incident response plan.

[0596] Step 6:

[0597] The server sends the integrated incident response plan to the user's device.

[0598] Input: A consolidated incident response plan.

[0599] Output: The optimal response plan sent to the user's device.

[0600] Specific operation: The server sends the incident response plan to the user's device via the Internet.

[0601] Step 7:

[0602] The device displays an integrated incident response plan to the user.

[0603] Input: The corresponding plan sent to the user's device.

[0604] Output: The optimal incident response plan displayed to the user.

[0605] Specific operation: The device will display "Optimal countermeasure for unauthorized access log ID: 12345" on the screen.

[0606] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0607] The present invention relates to a system that helps a user obtain optimal output by utilizing multiple generative models and further recognizes the user's emotions to make optimal suggestions. An embodiment of this system will be described in detail below.

[0608] System configuration

[0609] The system consists of the following components:

[0610] 1. A device that receives user input

[0611] 2. A server that sends requests to multiple generative models and receives their responses.

[0612] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[0613] 4. Emotion engine that recognizes user emotions

[0614] 5. Devices that provide optimal output to users

[0615] Program Operation

[0616] Receiving user input

[0617] The terminal receives input data from the user. This data is the basis for the system to process. For example, a request might be entered such as, "I want to plan a trip next month with a budget of 100,000 yen."

[0618] Submitting a Request

[0619] Based on the user's input data, the server creates requests for multiple generation models: a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[0620] Recognizing and reflecting emotions

[0621] The server uses the emotion engine to analyze the emotion from the user's input data and reflects the results in the request to the generative model. For example, if a user inputs "I want a trip that will reduce stress," the emotion engine analyzes "relaxation" and sends a request to each generative model based on this emotion.

[0622] Receiving response data

[0623] The server receives response data from each generation model. For example, it receives a response such as "a 3-night, 4-day Okinawa trip plan" from the domestic travel plan generation model, a "2-night, 3-day Guam trip plan" from the overseas travel plan generation model, "a trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen" from the price calculation model, and "the best dates are the second week of next month" from the schedule optimization model.

[0624] Comparison, scrutiny and integration

[0625] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. In this example, a trip to Okinawa is selected, taking into consideration "relaxation." The server also references the user's previous emotional history to make more accurate suggestions.

[0626] Generate and deliver optimal output

[0627] Generation AI-4 generates the optimal output, which is received by the server. The server then sends this output to the user's device. If the optimal output is "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month with a budget of 90,000 yen," this will be provided to the user.

[0628] Displaying optimal output

[0629] The device displays the generated optimal output to the user, who can then decide on specific actions to take based on this information. For example, a detailed travel plan such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen" is displayed.

[0630] Specific examples

[0631] For example, if a user enters "I want to plan a trip next month with a budget of 100,000 yen, preferably somewhere relaxing," the system will work as follows:

[0632] 1. The terminal receives input.

[0633] 2. The server uses the emotion engine to analyze the emotion "relaxed" and reflects this in each generative model request.

[0634] 3. The server sends requests to the domestic travel plan generation model, the international travel plan generation model, the fare calculation model, and the schedule optimization model.

[0635] 4. Each model returns a response: the domestic travel plan generation model replies "A three-night, four-day trip to Okinawa," the overseas travel plan generation model replies "A two-night, three-day trip to Guam," the pricing calculation model replies "A trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen," and the schedule optimization model replies "The second week of next month is optimal."

[0636] 5. Generative AI-4 compares and refines these responses and generates the optimal output: "A relaxing, budget-friendly plan for a 3-night, 4-day trip to Okinawa in the second week of next month."

[0637] 6. The server sends this output to the user's device.

[0638] 7. The device displays the best travel plan for the user.

[0639] This system allows users to easily obtain high-quality travel plans based on emotions and requests. Similar procedures can also be used to obtain high-quality output for other purposes, such as presentation materials.

[0640] The processing flow will be explained below.

[0641] Step 1:

[0642] The user inputs a request into the device. For example, the user might input, "I want to plan a trip next month with a budget of 100,000 yen. I would like a relaxing place."

[0643] Step 2:

[0644] The terminal receives input data from the user and sends it to the server.

[0645] Step 3:

[0646] The server analyzes the received input data and uses an emotion engine to recognize the user's emotion, in this case "relaxed."

[0647] Step 4:

[0648] The server makes requests for multiple Generative Models, specifically the following Generative Models:

[0649] Domestic travel plan generation model: "Generate a relaxing domestic travel plan with a budget of 100,000 yen next month"

[0650] "Generate a relaxing overseas travel plan with a budget of 100,000 yen for next month"

[0651] "Calculate the cost of each plan" in the pricing model

[0652] "Generate optimal travel schedules" for schedule optimization models

[0653] Step 5:

[0654] The server sends the created request to each generative model.

[0655] Step 6:

[0656] The server receives a response from each generative model, which includes, for example, the following information:

[0657] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[0658] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[0659] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[0660] The schedule optimization model says, "The second week of next month is optimal."

[0661] Step 7:

[0662] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. Specifically, it selects a plan (Okinawa travel plan) that meets the "relaxation" criteria and fits within the budget.

[0663] Step 8:

[0664] The server then sends the generated optimal output to the terminal. For example, it generates a plan such as "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month with a budget of 90,000 yen."

[0665] Step 9:

[0666] The device displays the optimal output to the user, allowing the user to decide on specific actions to take. For example, a detailed travel plan such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen" is displayed.

[0667] Step 10:

[0668] The server stores the user's emotion history, which is then referenced the next time output is generated to provide more accurate suggestions.

[0669] Example 2

[0670] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0671] Conventional systems have struggled to generate optimal outputs that take into account specific user emotions. Furthermore, when integrating response data from multiple generative models, there was a lack of a way to incorporate user emotions, resulting in poor recommendation accuracy and reduced user satisfaction. To address this issue, there is a need for systems that can analyze user emotions and incorporate them into generative model requests to provide more accurate recommendations.

[0672] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0673] In this invention, the server includes means for receiving user input, means for sending requests to multiple generative models, means for analyzing emotions, means for receiving response data, means for comparing, examining, and integrating the response data, means for generating optimal output, means for providing the optimal output to the user, and means for reflecting the results of the emotion analysis in the requests of the generative models, thereby enabling the generation of highly accurate output that reflects the user's emotions.

[0674] "Means for receiving user input" refers to an input device or form through which a user enters information or requests into the system.

[0675] "Means for sending a request" refers to a device or program that has the function of creating and sending requests to multiple generative models based on information received from a user.

[0676] A "generative model" refers to an algorithm or program that generates a specific output based on specific conditions and input data.

[0677] "Means for analyzing emotions" means a program or device for analyzing and identifying a user's emotional state based on input data or other information from the user.

[0678] "Means for receiving response data" refers to a device or program for receiving results sent from multiple generative models.

[0679] "Means for comparing, examining, and integrating response data" refers to a program or device that compares data received from different generative models, eliminates inconsistencies and duplications, and integrates them in the most optimal way.

[0680] "Means for generating optimal outputs" refers to programs and algorithms that generate final recommendations or results based on integrated data and taking into account user requirements and emotions.

[0681] "Means for providing optimal output to a user" refers to devices or programs for displaying, transmitting, or providing the generated output to a user.

[0682] "Means for reflecting the results of emotion analysis in requests to a generative model" refers to a device or program that has the function of incorporating the analyzed user's emotion information into requests to a generative model and adjusting the content of the request.

[0683] MODE FOR CARRYING OUT THE INVENTION

[0684] The present invention relates to a system that helps a user obtain optimal output by utilizing multiple generative models and further recognizes the user's emotions to make optimal suggestions. An embodiment of this system will be described in detail below.

[0685] System configuration

[0686] The system consists of the following components:

[0687] 1. A device that receives user input

[0688] 2. A server that sends requests to multiple generative models and receives their responses.

[0689] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[0690] 4. Emotion engine that recognizes user emotions

[0691] 5. Devices that provide optimal output to users

[0692] Receiving user input

[0693] The specific hardware used for the device may be a PC, tablet, smartphone, etc. The software used to receive input data from the user and send it to the system may include a browser-based application or a dedicated app. For example, a user may enter "I'd like to plan a trip next month with a budget of 100,000 yen. I'd like a relaxing place" into the device's input field and press the send button.

[0694] Creating and Sending a Request

[0695] The server is a core system of hardware that creates and sends requests for the required generative models based on user input data. Specific generative models include:

[0696] Domestic travel plan generation model

[0697] Overseas travel plan generation model

[0698] Pricing Model

[0699] Schedule Optimization Model

[0700] The server sends a request to each generative model as follows:

[0701] A domestic travel plan generation model asks, "Generate a relaxing domestic travel plan with a budget of 100,000 yen."

[0702] For the overseas travel plan generation model, "Generate a relaxing overseas travel plan with a budget of 100,000 yen"

[0703] The pricing model asks, "Calculate the budget for domestic and international travel plans."

[0704] "Please provide me with the best travel dates for next month" to the schedule optimization model

[0705] Recognizing and reflecting emotions

[0706] The server uses an emotion engine to analyze emotions from the user's input data. Commonly available software tools such as Microsoft's Azure Emotion API and Google's Vision API can be used as emotion engines. The analyzed emotion data is reflected in the request. For example, if the emotion "relaxed" is analyzed from the user's input, the request is updated based on that element.

[0707] Receiving response data

[0708] The server receives response data from each generative model. For example, the following response is returned:

[0709] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[0710] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[0711] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[0712] The schedule optimization model says, "The second week of next month is optimal."

[0713] Comparison, scrutiny and integration

[0714] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. Generative AI-4 uses advanced algorithms built on OpenAI's GPT-4 and Google's BERT. This allows it to derive the most suitable suggestions.

[0715] Generate and deliver optimal output

[0716] Generation AI-4 generates the optimal output, which is received by the server. The server then sends this output to the user's device. For example, a plan might be generated for a relaxing, budget-friendly trip to Okinawa in the second week of next month for three nights and four days.

[0717] Displaying optimal output

[0718] The device displays the generated optimal output to the user, who can then decide on specific actions to take based on this information. For example, a detailed travel plan might be presented, such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen."

[0719] Specific prompt examples

[0720] For example, the prompt you would enter into a generative AI model might look like this:

[0721] "The user wants to relax and is planning a trip with a budget of 100,000 yen. Please suggest the best travel plan, whether domestic or international."

[0722] This procedure allows the system to provide optimal suggestions based on the user's requests and emotions.

[0723] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0724] Step 1:

[0725] The terminal receives user input.

[0726] Specific operation: The user enters "I want to plan a trip next month with a budget of 100,000 yen. I would like a relaxing place" into the input field on the device and presses the send button.

[0727] Input: Request data entered by the user (e.g., "I'd like to plan a trip next month with a budget of 100,000 yen. I'd like a relaxing place.")

[0728] Output: Request data sent from the terminal to the server

[0729] Step 2:

[0730] The server creates and sends requests for each generative model based on the user's input data.

[0731] Specific operation: The server analyzes the user's input data and sends the following request to each generative model:

[0732] A domestic travel plan generation model asks, "Generate a relaxing domestic travel plan with a budget of 100,000 yen."

[0733] For the overseas travel plan generation model, "Generate a relaxing overseas travel plan with a budget of 100,000 yen"

[0734] The pricing model asks, "Calculate the budget for domestic and international travel plans."

[0735] "Please provide me with the best travel dates for next month" to the schedule optimization model

[0736] Input: User request data

[0737] Output: Requests to each generative model

[0738] Step 3:

[0739] The server uses an emotion engine to analyze emotions from the user's input data.

[0740] Specific operation: The server uses an emotion engine (e.g., Microsoft's Azure Emotion API) to analyze the emotion "relaxed" and reflects this in the request to the generative model.

[0741] Input: User request data, emotion engine

[0742] Output: A request to the generative model that reflects the emotional elements

[0743] Step 4:

[0744] The generative model receives the server request and generates response data.

[0745] Specific operation: Each generative model generates the following response data based on the request and sends it to the server.

[0746] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[0747] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[0748] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[0749] The schedule optimization model says, "The second week of next month is optimal."

[0750] Input: The request sent by the server

[0751] Output: Response data from each generative model

[0752] Step 5:

[0753] The server receives the response data from each generative model.

[0754] Specific operation: The server receives and stores response data from each generative model.

[0755] Input: Response data from each generative model (e.g., "A trip to Okinawa for 3 nights and 4 days," "A trip to Guam for 2 nights and 3 days," "A trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen," "The second week of next month is best").

[0756] Output: Save the received response data

[0757] Step 6:

[0758] The server uses an advanced generative model (Generation AI-4) to compare, examine, and integrate the responses of each generative model to generate the optimal output.

[0759] Specific operation: The Generation AI-4 analyzes each response data and generates the optimal output while prioritizing the user's emotion of "relaxation." It also refers to the user's previous emotion history to select the most appropriate suggestion.

[0760] Input: Response data from each generative model, user emotion data, previous user emotion history

[0761] Output: Optimal output (e.g., "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen")

[0762] Step 7:

[0763] The server receives the optimal output from the generated AI-4 and sends it to the user's device.

[0764] Specific operation: The server receives the optimal output from generation AI-4 and sends it to the user's terminal.

[0765] Input: Optimal Output

[0766] Output: Data sent to the terminal

[0767] Step 8:

[0768] The device will display the generated optimal output to the user.

[0769] Specific operation: The device displays a detailed travel plan such as "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen." The user can then view this and make a specific reservation.

[0770] Input: The best output sent by the server

[0771] Output: Data displayed to the user

[0772] (Application example 2)

[0773] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0774] Conventional shopping assistant systems have the problem of only being able to make uniform product recommendations without fully considering the user's specific needs and emotions. Furthermore, it is difficult to make optimal product suggestions that reflect the user's emotions and circumstances, which change in real time, and they lack ingenuity to improve the user experience. Furthermore, it is practically difficult to provide interactive product information through smart glasses or other advanced user devices. To solve these issues, a system is needed that can analyze user input data and emotions in real time and provide optimal output accordingly.

[0775] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving input data from a user, means for sending requests to multiple generative models, means for receiving response data from the multiple generative models, means for recognizing the user's emotions and reflecting them in the request, means for comparing, examining, and integrating the response data, and means for providing optimal output to the user terminal. This enables high-quality, personalized product recommendations based on the user's specific needs and emotions.

[0776] "User input" refers to data or requirements provided by a user to a system.

[0777] A "generative model" is a machine learning algorithm that generates a specific output based on data.

[0778] A "request" refers to user input or an instruction or question sent by the system to a generative model.

[0779] "Response data" refers to the information or results returned by a generative model.

[0780] "Comparison and examination" refers to the act of comparing multiple response data with each other and eliminating any inconsistencies or discrepancies.

[0781] "Integration" is the act of combining the optimal outputs based on the results of comparison and scrutiny.

[0782] "Emotion recognition" refers to the techniques and processes used to analyze and recognize a user's emotional state.

[0783] "Smart glasses" are wearable devices that have built-in computer functions and allow users to obtain information visually.

[0784] "User terminal" refers to an electronic device used by a user to input or output information.

[0785] "Output" refers to the final results or recommendations that the system provides to the user.

[0786] This invention relates to a system that helps users obtain optimal output by utilizing multiple generative models and recognizes the user's emotions to provide optimal suggestions. This system can be realized as a shopping assistant application using smart glasses that enhances the shopping experience in brick-and-mortar stores.

[0787] System configuration

[0788] The system consists of the following components:

[0789] 1. A way to receive user input: Using the smart glasses' voice input or eye tracking capabilities, the glasses can receive requests about products the user is considering purchasing. For example, "I want a new smartphone, and I'm looking for a model with the latest features."

[0790] 2. A means for sending requests to multiple generative models: User input data is sent to the server, and requests are issued to various generative models. Specifically, requests are sent to the latest smartphone detail model, price comparison model, user review analysis model, and feature comparison model.

[0791] 3. Means for receiving response data from the generative model: The server receives response data from the generative model. For example, the latest smartphone detailed model might say "Latest Model A," the price comparison model might say "Model A is 90,000 yen," the user review analysis model might say "Model A's rating: 4.8," and the feature comparison model might say "Model A has the latest feature XX."

[0792] 4. Means for recognizing emotions and incorporating them into requests: The server uses emotion recognition APIs (e.g., Amazon Rekognition, Microsoft Azure Face API) to analyze the user's emotions from their tone of voice and facial expressions. This allows the server to analyze the user's emotions, such as "excitement" or "expectation," and incorporate them into the generative model's requests.

[0793] 5. Means of comparing, scrutinizing, and integrating response data: The server uses an advanced generative AI model (e.g., OpenAI GPT-4) to compare and scrutinize the responses from each generative model to generate the optimal output, eliminating duplication and inconsistency and making optimal suggestions taking into account user sentiment.

[0794] 6. Means for providing optimal output to the user device: The server sends the generated optimal output to the smart glasses. For example, the smart glasses display will show "Model A with the latest features is the best. Price is 90,000 yen, rating is 4.8."

[0795] Specific examples

[0796] If a user uses smart glasses to say, "I want a new smartphone, I'm looking for a model with the latest features," the system will:

[0797] 1. Receiving user input: The smart glasses receive requests via voice input or eye tracking.

[0798] 2. Sending a request: The received input data is sent to the server and a request is issued to the generative model.

[0799] 3. Emotion recognition and reflection: The emotion engine analyzes the user's emotions and reflects the emotions of "excitement and anticipation" in the generative model's requests.

[0800] 4. Receiving response data: The server receives the response data from the generative model.

[0801] 5. Compare, Refine and Integrate: The Generate AI-4 refines the response data and generates the optimal output.

[0802] 6. Providing optimal output: The generated output is sent to the smart glasses and displayed to the user.

[0803] Prompt Sentence Examples

[0804] "Please tell me the latest smartphone model."

[0805] "What is the average price of this smartphone?"

[0806] "What are the user reviews for this smartphone?"

[0807] "What are the main features of this smartphone?"

[0808] This system allows users to receive real-time personalized product information in physical stores, providing a better shopping experience.

[0809] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0810] Step 1: Receiving User Input

[0811] A user uses the smart glasses to input a request through voice input or eye tracking. This input data includes specific information about the product they are considering purchasing. For example, a user might say, "I want a new smartphone, and I'm looking for a model with the latest features." The device receives this input data and prepares it for processing.

[0812] Step 2: Create and submit a request

[0813] The device receives the user's input data and sends it to the server. The server analyzes this data and creates requests to multiple generative models. Specifically, based on the request received from the device, "I want a new smartphone," the server generates and sends the following prompts to the "latest smartphone detailed model," "price comparison model," "user review analysis model," and "feature comparison model," respectively:

[0814] "Please tell me the latest smartphone model."

[0815] "What is the average price of this smartphone?"

[0816] "What are the user reviews for this smartphone?"

[0817] "What are the main features of this smartphone?"

[0818] Step 3: Recognize the emotion and reflect it in the request

[0819] The server uses an emotion recognition API (e.g., Amazon Rekognition, Microsoft Azure Face API) to recognize emotions from the user's tone of voice and facial expressions. For example, if a user emphasizes "latest," the emotion engine analyzes the emotion as "excitement and anticipation." This emotion information is reflected in the request for the generative model, and a request with the emotion is sent to each generative model.

[0820] Step 4: Receiving response data

[0821] The server receives response data from each generative model. For example, the response from the "latest smartphone detailed model" may be "latest model A," the response from the "price comparison model" may be "Model A is 90,000 yen," the response from the "user review analysis model" may be "Model A's rating: 4.8," and the response from the "feature comparison model" may be "Model A is equipped with the latest feature XX." This response data is aggregated by the server.

[0822] Step 5: Compare, examine, and consolidate response data

[0823] The server uses an advanced generative AI model (e.g., OpenAI GPT-4) to compare, examine, and integrate the response data received from each generative model. During this process, it removes duplication and inconsistencies and selects the optimal output, taking into account the user's emotions. For example, it selects "Latest Model A" as the optimal output, reflecting emotions such as excitement and anticipation.

[0824] Step 6: Generate and deliver optimal output

[0825] The server generates the optimal output using Generation AI-4 and sends it to the user's device (smart glasses). For example, it generates content such as "Model A, equipped with the latest features, is the best. Price: 90,000 yen, rating: 4.8." The device receives this data and displays it on its screen.

[0826] Step 7: Displaying the optimal output

[0827] The device displays the optimal output to the user, who can then decide on a specific course of action based on this information. For example, the smart glasses display might say, "Model A, equipped with the latest features, is the best choice. It's priced at 90,000 yen and has a rating of 4.8." This allows the user to immediately consider purchasing the product in the store.

[0828] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0829] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0830] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0831] [Third embodiment]

[0832] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0833] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0834] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0835] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0836] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0837] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0838] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0839] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0840] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0841] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0842] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0843] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0844] The present invention relates to a system that helps a user utilize multiple generative models to obtain optimal output, and an embodiment of the system is described in detail below.

[0845] System configuration

[0846] The system consists of the following main components:

[0847] 1. A device that receives user input

[0848] 2. A server that sends requests to multiple generative models and receives their responses.

[0849] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[0850] 4. Devices that provide optimal output to users

[0851] Program Operation

[0852] Receiving user input

[0853] The device receives input data from the user. This data is the basis for the system to process. For example, a user might input a request such as, "I want to plan a trip next month with a budget of 100,000 yen."

[0854] Submitting a Request

[0855] The server receives user input data and sends requests to multiple generative models, including the following generative models:

[0856] Domestic travel plan generation model

[0857] Overseas travel plan generation model

[0858] Pricing Model

[0859] Schedule Optimization Model

[0860] The server creates and sends appropriate requests to these generative models.

[0861] Receiving response data

[0862] The server receives the response data from each generative model. For example, the response might look like this:

[0863] Domestic travel plan generation model: 3-night, 4-day Tokyo travel plan

[0864] Overseas travel plan generation model: 2-night, 3-day Seoul travel plan

[0865] Price calculation model: Tokyo trip costs 90,000 yen, Seoul trip costs 120,000 yen

[0866] Schedule optimization model: The best date is the second week of next month

[0867] Comparison, scrutiny and integration

[0868] The server uses an advanced generative model (Generation AI-4) to compare, examine, and integrate the output of each generative model. During this process, it eliminates duplicate information and inconsistencies to generate the optimal output. For example, if a 3-night, 4-day Tokyo trip plan is determined to be within budget, that plan will be selected.

[0869] Generate and deliver optimal output

[0870] Generation AI-4 generates the optimal output, which is received by the server, which then sends this output to the user's terminal, which finally displays the output to the user.

[0871] Specific examples

[0872] For example, if a user inputs "I want to plan a trip next month with a budget of 100,000 yen," the system will work as follows:

[0873] 1. The terminal receives input.

[0874] 2. The server sends requests to the domestic travel plan generation model, the international travel plan generation model, the fare calculation model, and the schedule optimization model.

[0875] 3. Each model returns a response: the domestic travel plan generation model suggests a "3-night, 4-day Tokyo trip plan," the overseas travel plan generation model suggests a "2-night, 3-day Seoul trip plan," the pricing calculation model suggests "A trip to Tokyo costs 90,000 yen, a trip to Seoul costs 120,000 yen," and the schedule optimization model suggests "The second week of next month is optimal."

[0876] 4. Generative AI-4 compares and examines these responses and generates the optimal output: "Plan a trip to Tokyo for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen."

[0877] 5. The server sends this output to the user's device.

[0878] 6. The device displays the best travel plan for the user.

[0879] In this way, users can easily obtain high-quality travel plans. Similar procedures can also be used to obtain high-quality output for other purposes, such as presentation materials.

[0880] The processing flow will be explained below.

[0881] Step 1:

[0882] The user inputs a request into the device. For example, the user inputs "I would like to plan a trip next month with a budget of 100,000 yen" into the device's input form.

[0883] Step 2:

[0884] The terminal receives input data from the user and sends it to the server.

[0885] Step 3:

[0886] The server analyzes the received input data and creates requests for multiple generation models: a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[0887] Step 4:

[0888] The server sends the created request to each generation model. For example, it sends a request such as "Generate a domestic travel plan with a budget of 100,000 yen next month" to the domestic travel plan generation model, and "Generate an overseas travel plan with a budget of 100,000 yen next month" to the overseas travel plan generation model.

[0889] Step 5:

[0890] The server receives responses from each generation model. Specifically, it receives responses such as "a four-day, three-night trip to Tokyo" from the domestic travel plan generation model, "a three-day, two-night trip to Seoul" from the overseas travel plan generation model, "a trip to Tokyo costs 90,000 yen, a trip to Seoul costs 120,000 yen" from the price calculation model, and "the second week of next month is optimal" from the schedule optimization model.

[0891] Step 6:

[0892] The server uses an advanced generative model (Generation AI-4) to compare and analyze the responses of each generative model and generate the optimal output. Specifically, it selects a plan that fits within the budget and optimizes the schedule.

[0893] Step 7:

[0894] Generation AI-4 generates the optimal output and provides the result to the server. For example, it might generate an output such as "A trip to Tokyo for three nights and four days in the second week of next month, with a budget of 90,000 yen."

[0895] Step 8:

[0896] The server generates the optimal output and sends it to the device.

[0897] Step 9:

[0898] The device displays the optimal output to the user, allowing the user to decide on specific actions to take. For example, it might say, "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen."

[0899] Example 1

[0900] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0901] When using multiple generative models to obtain optimal output, it is necessary to appropriately process user input and examine and integrate the responses from each generative model to obtain high-quality results. However, these processes are complex to perform manually and require advanced processing to ensure the quality of the integrated output. Furthermore, there are insufficient means to automate the examination and integration process that takes into account the characteristics of multiple generative models, making it difficult to generate high-quality output in a short period of time.

[0902] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0903] In this invention, the server includes a means for a user to input a prompt sentence via an input form and for the server to send the prompt sentence to multiple generative models, a means for manipulating the received response data using numpy or pandas to generate optimal output, and a means for providing the optimal output to the user. This makes it possible to compare and examine the outputs of multiple generative models based on the user's input data and automatically generate high-quality output in a short period of time.

[0904] "User" means any person or entity that uses the System and provides Input Data.

[0905] A "terminal" is a computer system or mobile device used by a user to provide input data.

[0906] "Server" refers to a central processing unit for receiving input data from a user, sending requests to multiple generative models, receiving response data, and generating and providing optimal output.

[0907] A "generative model" is an algorithm or program that generates and responds to data based on a specific task.

[0908] A "prompt sentence" is text data that a user inputs to a system, and serves as the basic data for the system to process.

[0909] "Output" refers to the final result data generated by the server by comparing, examining, and integrating the response data of the generative models.

[0910] A "request" is an instruction or data that a server sends to a Generative Model.

[0911] "Response data" is data that a generative model returns in response to a request.

[0912] "Comparison and scrutiny" is the process of examining response data from multiple generative models and selecting the most appropriate data.

[0913] "Fusion" is the process by which the server combines appropriate response data from multiple generative models to produce a single output.

[0914] "numpy" is a Python library for manipulating large amounts of numerical data.

[0915] "pandas" is a Python library for data manipulation and analysis.

[0916] This invention relates to a system that allows a user to obtain optimal output using multiple generative models. The system aims to obtain optimal responses from multiple generative models by inputting a specific prompt sentence from the user, and then examine and integrate the responses to provide them to the user.

[0917] System configuration

[0918] The system mainly consists of the following components:

[0919] 1. A device that receives user input

[0920] 2. A server that sends requests to multiple generative models and receives their responses.

[0921] 3. A means to compare, examine, and integrate the outputs of generative models to generate the optimal output

[0922] 4. Devices that provide optimal output to users

[0923] Detailed processing

[0924] Receiving user input

[0925] The user uses a terminal to input a specific prompt sentence. The terminal is usually a computer device such as a PC or smartphone. The user inputs a prompt sentence such as "I would like to plan a trip next month with a budget of 100,000 yen" through an input form and submits it.

[0926] Submitting a Request

[0927] The terminal sends the received prompt to the server, which uses software such as Python's requests library or Java's HttpClient to send requests to multiple generative models. The generative models include a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[0928] Receiving response data

[0929] The server receives response data from each generation model. Each response data is usually received in JSON format and parsed using Python's json module or Java's Jackson library. For example, the server receives data such as a "3-night, 4-day Tokyo trip plan" from the domestic travel plan generation model, a "2-night, 3-day Seoul trip plan" from the overseas travel plan generation model, "Tokyo trip costs 90,000 yen, Seoul trip costs 120,000 yen" from the price calculation model, and "The second week of next month is optimal" from the schedule optimization model.

[0930] Comparison, scrutiny and integration

[0931] The server uses the Generative AI-4 to compare and refine the response data from each generative model and generate the optimal output. This process involves data manipulation using Python libraries such as numpy and pandas. For example, the response data is converted into a numpy array and the optimization algorithm is applied using a pandas data frame.

[0932] Generate and deliver optimal output

[0933] Generation AI-4 generates the optimal output: "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen." The server then sends this data to the user's device. The device then displays the received output using HTML and JavaScript and provides it to the user. Specifically, the received data is inserted into the display area, and the user is notified in a visually easy-to-understand format.

[0934] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0935] Step 1:

[0936] The user uses an input form on the terminal and enters the prompt statement "I would like to plan a trip next month with a budget of 100,000 yen." The input data is sent from the terminal to the server. The prompt statement is received as terminal input and an HTTP request is generated to send it to the server.

[0937] Step 2:

[0938] The server analyzes the prompt received from the user and sends requests to multiple generative models. Specifically, the server converts the prompt into an appropriate format and sends API requests to the generative models using the Python requests library. The input is the prompt, and the output is an API request to each generative model.

[0939] Step 3:

[0940] Each generative model receives a request and generates response data. For example, a domestic travel plan generation model generates a "Tokyo travel plan for 3 nights and 4 days" and sends it back to the server. The server receives the response data from the generative model, and the response is in JSON format.

[0941] Step 4:

[0942] The server receives and analyzes the response data from each generative model. Specifically, the server uses the Python json module to parse the response data and convert it into an internal data structure. The input is the response data in JSON format, and the output is an internal data structure (e.g., a Python dictionary or list).

[0943] Step 5:

[0944] The server uses numpy and pandas to compare and examine each response data and generate the optimal output. For example, the server compares the costs and dates of each travel plan and selects the optimal one. The input is the internal data structure, and the output is the optimal output (for example, "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen").

[0945] Step 6:

[0946] The server converts the optimal output into JSON format and sends it to the user's device. Specifically, the server generates an HTTP response using Flask or Django. The input is the optimal output data, and the output is the HTTP response sent to the user's device.

[0947] Step 7:

[0948] The terminal displays the output received from the server. Specifically, it displays the optimal travel plan on the screen using HTML and JavaScript. The input is the HTTP response from the server, and the output is the travel plan displayed to the user.

[0949] (Application example 1)

[0950] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0951] Conventional security incident response systems have difficulty in developing appropriate countermeasures and have experienced delays in response when a rapid response is required in seconds. Furthermore, it has been difficult to develop a system that can derive integrated and optimal countermeasures using multiple generative models specialized for individual elements. This has led to concerns about increased security risks and damage.

[0952] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0953] In this invention, the server includes means for receiving input data from a user, means for sending requests to multiple generative models, means for receiving response data from the multiple generative models, means for comparing, examining, and integrating the response data, means for generating an optimal output, means for providing the optimal output to the user, and means for receiving detailed information about a security incident and generating an incident response plan. This makes it possible to formulate quick and appropriate countermeasures for security incidents such as unauthorized access by utilizing multiple specialized generative models and provide them to users in real time.

[0954] "User-entered data" means information entered into a system by a user, including details and instructions regarding a security incident.

[0955] A "generative model" is an AI model with a specific role or function, such as analyzing unauthorized access patterns, assessing damage, generating emergency response procedures, and generating recovery procedures.

[0956] The "request sending means" is a means by which the system sends a request to the generative model based on user input data.

[0957] "Response data" is data that the generative model responds to a request, and includes the results of pattern analysis of unauthorized access and damage assessment results.

[0958] The "comparison and review means" refers to a means for comparing and reviewing the received response data and removing any duplication or inconsistency.

[0959] A "synthesis tool" is a tool that brings together the compared and reviewed response data into a single, integrated output.

[0960] The "output" is the optimal response or plan generated by the integration method.

[0961] "User-delivered means" refers to the means by which the integrated output is presented to the user.

[0962] A "security incident" is an event that threatens the security of a system or the protection of data, including unauthorized access or data breach.

[0963] An "incident response plan" is a plan that includes optimal responses and procedures for responding to security incidents.

[0964] This invention is a system that generates an optimal incident response plan through multiple generative models based on detailed information about a security incident entered by a user, and provides the plan to the user. This system can be run via a device such as a smartphone or smart glasses.

[0965] The system works as follows: First, the user enters detailed information about a security incident into a device such as a smartphone or smart glasses. This information includes unauthorized access logs and damage assessments. After receiving this information, the device sends it to the server.

[0966] Based on the input data from the user, the server sends requests to multiple generative models, such as a fraudulent access pattern analysis model, a damage assessment model, an emergency response procedure generation model, and a recovery procedure generation model. In response to this request, each generative model generates corresponding analysis results and proposals and returns them to the server.

[0967] The returned response data is compared and scrutinized by an advanced generative model (Generation AI-4) within the server. This advanced generative model has the ability to remove duplicates and inconsistencies from the response data and generate the optimal output. The output integrated by Generation AI-4 is further scrutinized within the server as the optimal incident response plan.

[0968] After generating an optimal incident response plan, the server sends this information back to the user's device and presents the user with appropriate countermeasures and procedures. The generative models and integration algorithms used in this process may utilize AI platforms such as Amazon SageMaker.

[0969] Below is a concrete example of how this system can be implemented, showing the prompt text that will be displayed when unauthorized access occurs.

[0970] Examples:

[0971] "Unauthorized access log ID: 12345 has occurred and the damage is high. Please generate the optimal countermeasures for this incident."

[0972] In this way, users can quickly receive and take appropriate measures to deal with security incidents. The system provides real-time information and helps minimize the damage caused by the incident.

[0973] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0974] Step 1:

[0975] The user enters details about the security incident into the device.

[0976] Input: Detailed information such as unauthorized access logs and damage status.

[0977] Output: Security incident information entered into the terminal.

[0978] Specific operation: The user enters "Unauthorized access log ID: 12345 has occurred and the damage is significant" into the device.

[0979] Step 2:

[0980] The terminal transmits the entered security incident information to the server.

[0981] Input: Information entered into the device.

[0982] Output: Information data sent to the server.

[0983] Specific operation: The device sends security incident information to a server via the Internet.

[0984] Step 3:

[0985] Based on the incident information received by the server, requests are sent to multiple generative models.

[0986] Input: Security incident information sent to the server.

[0987] Output: A request for each generative model.

[0988] Specific operation: The server sends requests to the unauthorized access pattern analysis model, the damage assessment model, the emergency response procedure generation model, and the recovery procedure generation model.

[0989] Step 4:

[0990] The generative model returns response data based on the request to the server.

[0991] Input: The requests sent to each generative model.

[0992] Output: Response data from each generative model.

[0993] Specific operation: The unauthorized access pattern analysis model generates the pattern analysis result, and the damage assessment model generates the damage assessment result, and these are returned to the server.

[0994] Step 5:

[0995] The server compares, examines, and integrates the response data received from each generative model using an advanced generative model (Generation AI-4).

[0996] Input: Response data from each generative model.

[0997] Output: A consolidated incident response plan.

[0998] Specific operation: The Generate AI-4 analyzes response data, removes duplicates and inconsistencies, and generates an optimal incident response plan.

[0999] Step 6:

[1000] The server sends the integrated incident response plan to the user's device.

[1001] Input: A consolidated incident response plan.

[1002] Output: The optimal response plan sent to the user's device.

[1003] Specific operation: The server sends the incident response plan to the user's device via the Internet.

[1004] Step 7:

[1005] The device displays an integrated incident response plan to the user.

[1006] Input: The corresponding plan sent to the user's device.

[1007] Output: The optimal incident response plan displayed to the user.

[1008] Specific operation: The device will display "Optimal countermeasure for unauthorized access log ID: 12345" on the screen.

[1009] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1010] The present invention relates to a system that helps a user obtain optimal output by utilizing multiple generative models and further recognizes the user's emotions to make optimal suggestions. An embodiment of this system will be described in detail below.

[1011] System configuration

[1012] The system consists of the following components:

[1013] 1. A device that receives user input

[1014] 2. A server that sends requests to multiple generative models and receives their responses.

[1015] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[1016] 4. Emotion engine that recognizes user emotions

[1017] 5. Devices that provide optimal output to users

[1018] Program Operation

[1019] Receiving user input

[1020] The terminal receives input data from the user. This data is the basis for the system to process. For example, a request might be entered such as, "I want to plan a trip next month with a budget of 100,000 yen."

[1021] Submitting a Request

[1022] Based on the user's input data, the server creates requests for multiple generation models: a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[1023] Recognizing and reflecting emotions

[1024] The server uses the emotion engine to analyze the emotion from the user's input data and reflects the results in the request to the generative model. For example, if a user inputs "I want a trip that will reduce stress," the emotion engine analyzes "relaxation" and sends a request to each generative model based on this emotion.

[1025] Receiving response data

[1026] The server receives response data from each generation model. For example, it receives a response such as "a 3-night, 4-day Okinawa trip plan" from the domestic travel plan generation model, a "2-night, 3-day Guam trip plan" from the overseas travel plan generation model, "a trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen" from the price calculation model, and "the best dates are the second week of next month" from the schedule optimization model.

[1027] Comparison, scrutiny and integration

[1028] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. In this example, a trip to Okinawa is selected, taking into consideration "relaxation." The server also references the user's previous emotional history to make more accurate suggestions.

[1029] Generate and deliver optimal output

[1030] Generation AI-4 generates the optimal output, which is received by the server. The server then sends this output to the user's device. If the optimal output is "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month with a budget of 90,000 yen," this will be provided to the user.

[1031] Displaying optimal output

[1032] The device displays the generated optimal output to the user, who can then decide on specific actions to take based on this information. For example, a detailed travel plan such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen" is displayed.

[1033] Specific examples

[1034] For example, if a user enters "I want to plan a trip next month with a budget of 100,000 yen, preferably somewhere relaxing," the system will work as follows:

[1035] 1. The terminal receives input.

[1036] 2. The server uses the emotion engine to analyze the emotion "relaxed" and reflects this in each generative model request.

[1037] 3. The server sends requests to the domestic travel plan generation model, the international travel plan generation model, the fare calculation model, and the schedule optimization model.

[1038] 4. Each model returns a response: the domestic travel plan generation model replies "A three-night, four-day trip to Okinawa," the overseas travel plan generation model replies "A two-night, three-day trip to Guam," the pricing calculation model replies "A trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen," and the schedule optimization model replies "The second week of next month is optimal."

[1039] 5. Generative AI-4 compares and refines these responses and generates the optimal output: "A relaxing, budget-friendly plan for a 3-night, 4-day trip to Okinawa in the second week of next month."

[1040] 6. The server sends this output to the user's device.

[1041] 7. The device displays the best travel plan for the user.

[1042] This system allows users to easily obtain high-quality travel plans based on emotions and requests. Similar procedures can also be used to obtain high-quality output for other purposes, such as presentation materials.

[1043] The processing flow will be explained below.

[1044] Step 1:

[1045] The user inputs a request into the device. For example, the user might input, "I want to plan a trip next month with a budget of 100,000 yen. I would like a relaxing place."

[1046] Step 2:

[1047] The terminal receives input data from the user and sends it to the server.

[1048] Step 3:

[1049] The server analyzes the received input data and uses an emotion engine to recognize the user's emotion, in this case "relaxed."

[1050] Step 4:

[1051] The server makes requests for multiple Generative Models, specifically the following Generative Models:

[1052] Domestic travel plan generation model: "Generate a relaxing domestic travel plan with a budget of 100,000 yen next month"

[1053] "Generate a relaxing overseas travel plan with a budget of 100,000 yen for next month"

[1054] "Calculate the cost of each plan" in the pricing model

[1055] "Generate optimal travel schedules" for schedule optimization models

[1056] Step 5:

[1057] The server sends the created request to each generative model.

[1058] Step 6:

[1059] The server receives a response from each generative model, which includes, for example, the following information:

[1060] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[1061] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[1062] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[1063] The schedule optimization model says, "The second week of next month is optimal."

[1064] Step 7:

[1065] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. Specifically, it selects a plan (Okinawa travel plan) that meets the "relaxation" criteria and fits within the budget.

[1066] Step 8:

[1067] The server then sends the generated optimal output to the terminal. For example, it generates a plan such as "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month with a budget of 90,000 yen."

[1068] Step 9:

[1069] The device displays the optimal output to the user, allowing the user to decide on specific actions to take. For example, a detailed travel plan such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen" is displayed.

[1070] Step 10:

[1071] The server stores the user's emotion history, which is then referenced the next time output is generated to provide more accurate suggestions.

[1072] Example 2

[1073] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1074] Conventional systems have struggled to generate optimal outputs that take into account specific user emotions. Furthermore, when integrating response data from multiple generative models, there was a lack of a way to incorporate user emotions, resulting in poor recommendation accuracy and reduced user satisfaction. To address this issue, there is a need for systems that can analyze user emotions and incorporate them into generative model requests to provide more accurate recommendations.

[1075] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1076] In this invention, the server includes means for receiving user input, means for sending requests to multiple generative models, means for analyzing emotions, means for receiving response data, means for comparing, examining, and integrating the response data, means for generating optimal output, means for providing the optimal output to the user, and means for reflecting the results of the emotion analysis in the requests of the generative models, thereby enabling the generation of highly accurate output that reflects the user's emotions.

[1077] "Means for receiving user input" refers to an input device or form through which a user enters information or requests into the system.

[1078] "Means for sending a request" refers to a device or program that has the function of creating and sending requests to multiple generative models based on information received from a user.

[1079] A "generative model" refers to an algorithm or program that generates a specific output based on specific conditions and input data.

[1080] "Means for analyzing emotions" means a program or device for analyzing and identifying a user's emotional state based on input data or other information from the user.

[1081] "Means for receiving response data" refers to a device or program for receiving results sent from multiple generative models.

[1082] "Means for comparing, examining, and integrating response data" refers to a program or device that compares data received from different generative models, eliminates inconsistencies and duplications, and integrates them in the most optimal way.

[1083] "Means for generating optimal outputs" refers to programs and algorithms that generate final recommendations or results based on integrated data and taking into account user requirements and emotions.

[1084] "Means for providing optimal output to a user" refers to devices or programs for displaying, transmitting, or providing the generated output to a user.

[1085] "Means for reflecting the results of emotion analysis in requests to a generative model" refers to a device or program that has the function of incorporating the analyzed user's emotion information into requests to a generative model and adjusting the content of the request.

[1086] MODE FOR CARRYING OUT THE INVENTION

[1087] The present invention relates to a system that helps a user obtain optimal output by utilizing multiple generative models and further recognizes the user's emotions to make optimal suggestions. An embodiment of this system will be described in detail below.

[1088] System configuration

[1089] The system consists of the following components:

[1090] 1. A device that receives user input

[1091] 2. A server that sends requests to multiple generative models and receives their responses.

[1092] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[1093] 4. Emotion engine that recognizes user emotions

[1094] 5. Devices that provide optimal output to users

[1095] Receiving user input

[1096] The specific hardware used for the device may be a PC, tablet, smartphone, etc. The software used to receive input data from the user and send it to the system may include a browser-based application or a dedicated app. For example, a user may enter "I'd like to plan a trip next month with a budget of 100,000 yen. I'd like a relaxing place" into the device's input field and press the send button.

[1097] Creating and Sending a Request

[1098] The server is a core system of hardware that creates and sends requests for the required generative models based on user input data. Specific generative models include:

[1099] Domestic travel plan generation model

[1100] Overseas travel plan generation model

[1101] Pricing Model

[1102] Schedule Optimization Model

[1103] The server sends a request to each generative model as follows:

[1104] A domestic travel plan generation model asks, "Generate a relaxing domestic travel plan with a budget of 100,000 yen."

[1105] For the overseas travel plan generation model, "Generate a relaxing overseas travel plan with a budget of 100,000 yen"

[1106] The pricing model asks, "Calculate the budget for domestic and international travel plans."

[1107] "Please provide me with the best travel dates for next month" to the schedule optimization model

[1108] Recognizing and reflecting emotions

[1109] The server uses an emotion engine to analyze emotions from the user's input data. Commonly available software tools such as Microsoft's Azure Emotion API and Google's Vision API can be used as emotion engines. The analyzed emotion data is reflected in the request. For example, if the emotion "relaxed" is analyzed from the user's input, the request is updated based on that element.

[1110] Receiving response data

[1111] The server receives response data from each generative model. For example, the following response is returned:

[1112] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[1113] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[1114] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[1115] The schedule optimization model says, "The second week of next month is optimal."

[1116] Comparison, scrutiny and integration

[1117] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. Generative AI-4 uses advanced algorithms built on OpenAI's GPT-4 and Google's BERT. This allows it to derive the most suitable suggestions.

[1118] Generate and deliver optimal output

[1119] Generation AI-4 generates the optimal output, which is received by the server. The server then sends this output to the user's device. For example, a plan might be generated for a relaxing, budget-friendly trip to Okinawa in the second week of next month for three nights and four days.

[1120] Displaying optimal output

[1121] The device displays the generated optimal output to the user, who can then decide on specific actions to take based on this information. For example, a detailed travel plan might be presented, such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen."

[1122] Specific prompt examples

[1123] For example, the prompt you would enter into a generative AI model might look like this:

[1124] "The user wants to relax and is planning a trip with a budget of 100,000 yen. Please suggest the best travel plan, whether domestic or international."

[1125] This procedure allows the system to provide optimal suggestions based on the user's requests and emotions.

[1126] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1127] Step 1:

[1128] The terminal receives user input.

[1129] Specific operation: The user enters "I want to plan a trip next month with a budget of 100,000 yen. I would like a relaxing place" into the input field on the device and presses the send button.

[1130] Input: Request data entered by the user (e.g., "I'd like to plan a trip next month with a budget of 100,000 yen. I'd like a relaxing place.")

[1131] Output: Request data sent from the terminal to the server

[1132] Step 2:

[1133] The server creates and sends requests for each generative model based on the user's input data.

[1134] Specific operation: The server analyzes the user's input data and sends the following request to each generative model:

[1135] A domestic travel plan generation model asks, "Generate a relaxing domestic travel plan with a budget of 100,000 yen."

[1136] For the overseas travel plan generation model, "Generate a relaxing overseas travel plan with a budget of 100,000 yen"

[1137] The pricing model asks, "Calculate the budget for domestic and international travel plans."

[1138] "Please provide me with the best travel dates for next month" to the schedule optimization model

[1139] Input: User request data

[1140] Output: Requests to each generative model

[1141] Step 3:

[1142] The server uses an emotion engine to analyze emotions from the user's input data.

[1143] Specific operation: The server uses an emotion engine (e.g., Microsoft's Azure Emotion API) to analyze the emotion "relaxed" and reflects this in the request to the generative model.

[1144] Input: User request data, emotion engine

[1145] Output: A request to the generative model that reflects the emotional elements

[1146] Step 4:

[1147] The generative model receives the server request and generates response data.

[1148] Specific operation: Each generative model generates the following response data based on the request and sends it to the server.

[1149] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[1150] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[1151] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[1152] The schedule optimization model says, "The second week of next month is optimal."

[1153] Input: The request sent by the server

[1154] Output: Response data from each generative model

[1155] Step 5:

[1156] The server receives the response data from each generative model.

[1157] Specific operation: The server receives and stores response data from each generative model.

[1158] Input: Response data from each generative model (e.g., "A trip to Okinawa for 3 nights and 4 days," "A trip to Guam for 2 nights and 3 days," "A trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen," "The second week of next month is best").

[1159] Output: Save the received response data

[1160] Step 6:

[1161] The server uses an advanced generative model (Generation AI-4) to compare, examine, and integrate the responses of each generative model to generate the optimal output.

[1162] Specific operation: The Generation AI-4 analyzes each response data and generates the optimal output while prioritizing the user's emotion of "relaxation." It also refers to the user's previous emotion history to select the most appropriate suggestion.

[1163] Input: Response data from each generative model, user emotion data, previous user emotion history

[1164] Output: Optimal output (e.g., "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen")

[1165] Step 7:

[1166] The server receives the optimal output from the generated AI-4 and sends it to the user's device.

[1167] Specific operation: The server receives the optimal output from generation AI-4 and sends it to the user's terminal.

[1168] Input: Optimal Output

[1169] Output: Data sent to the terminal

[1170] Step 8:

[1171] The device will display the generated optimal output to the user.

[1172] Specific operation: The device displays a detailed travel plan such as "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen." The user can then view this and make a specific reservation.

[1173] Input: The best output sent by the server

[1174] Output: Data displayed to the user

[1175] (Application example 2)

[1176] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1177] Conventional shopping assistant systems have the problem of only being able to make uniform product recommendations without fully considering the user's specific needs and emotions. Furthermore, it is difficult to make optimal product suggestions that reflect the user's emotions and circumstances, which change in real time, and they lack ingenuity to improve the user experience. Furthermore, it is practically difficult to provide interactive product information through smart glasses or other advanced user devices. To solve these issues, a system is needed that can analyze user input data and emotions in real time and provide optimal output accordingly.

[1178] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving input data from a user, means for sending requests to multiple generative models, means for receiving response data from the multiple generative models, means for recognizing the user's emotions and reflecting them in the request, means for comparing, examining, and integrating the response data, and means for providing optimal output to the user terminal. This enables high-quality, personalized product recommendations based on the user's specific needs and emotions.

[1179] "User input" refers to data or requirements provided by a user to a system.

[1180] A "generative model" is a machine learning algorithm that generates a specific output based on data.

[1181] A "request" refers to user input or an instruction or question sent by the system to a generative model.

[1182] "Response data" refers to the information or results returned by a generative model.

[1183] "Comparison and examination" refers to the act of comparing multiple response data with each other and eliminating any inconsistencies or discrepancies.

[1184] "Integration" is the act of combining the optimal outputs based on the results of comparison and scrutiny.

[1185] "Emotion recognition" refers to the techniques and processes used to analyze and recognize a user's emotional state.

[1186] "Smart glasses" are wearable devices that have built-in computer functions and allow users to obtain information visually.

[1187] "User terminal" refers to an electronic device used by a user to input or output information.

[1188] "Output" refers to the final results or recommendations that the system provides to the user.

[1189] This invention relates to a system that helps users obtain optimal output by utilizing multiple generative models and recognizes the user's emotions to provide optimal suggestions. This system can be realized as a shopping assistant application using smart glasses that enhances the shopping experience in brick-and-mortar stores.

[1190] System configuration

[1191] The system consists of the following components:

[1192] 1. A way to receive user input: Using the smart glasses' voice input or eye tracking capabilities, the glasses can receive requests about products the user is considering purchasing. For example, "I want a new smartphone, and I'm looking for a model with the latest features."

[1193] 2. A means for sending requests to multiple generative models: User input data is sent to the server, and requests are issued to various generative models. Specifically, requests are sent to the latest smartphone detail model, price comparison model, user review analysis model, and feature comparison model.

[1194] 3. Means for receiving response data from the generative model: The server receives response data from the generative model. For example, the latest smartphone detailed model might say "Latest Model A," the price comparison model might say "Model A is 90,000 yen," the user review analysis model might say "Model A's rating: 4.8," and the feature comparison model might say "Model A has the latest feature XX."

[1195] 4. Means for recognizing emotions and incorporating them into requests: The server uses emotion recognition APIs (e.g., Amazon Rekognition, Microsoft Azure Face API) to analyze the user's emotions from their tone of voice and facial expressions. This allows the server to analyze the user's emotions, such as "excitement" or "expectation," and incorporate them into the generative model's requests.

[1196] 5. Means of comparing, scrutinizing, and integrating response data: The server uses an advanced generative AI model (e.g., OpenAI GPT-4) to compare and scrutinize the responses from each generative model to generate the optimal output, eliminating duplication and inconsistency and making optimal suggestions taking into account user sentiment.

[1197] 6. Means for providing optimal output to the user device: The server sends the generated optimal output to the smart glasses. For example, the smart glasses display will show "Model A with the latest features is the best. Price is 90,000 yen, rating is 4.8."

[1198] Specific examples

[1199] If a user uses smart glasses to say, "I want a new smartphone, I'm looking for a model with the latest features," the system will:

[1200] 1. Receiving user input: The smart glasses receive requests via voice input or eye tracking.

[1201] 2. Sending a request: The received input data is sent to the server and a request is issued to the generative model.

[1202] 3. Emotion recognition and reflection: The emotion engine analyzes the user's emotions and reflects the emotions of "excitement and anticipation" in the generative model's requests.

[1203] 4. Receiving response data: The server receives the response data from the generative model.

[1204] 5. Compare, Refine and Integrate: The Generate AI-4 refines the response data and generates the optimal output.

[1205] 6. Providing optimal output: The generated output is sent to the smart glasses and displayed to the user.

[1206] Prompt Sentence Examples

[1207] "Please tell me the latest smartphone model."

[1208] "What is the average price of this smartphone?"

[1209] "What are the user reviews for this smartphone?"

[1210] "What are the main features of this smartphone?"

[1211] This system allows users to receive real-time personalized product information in physical stores, providing a better shopping experience.

[1212] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1213] Step 1: Receiving User Input

[1214] A user uses the smart glasses to input a request through voice input or eye tracking. This input data includes specific information about the product they are considering purchasing. For example, a user might say, "I want a new smartphone, and I'm looking for a model with the latest features." The device receives this input data and prepares it for processing.

[1215] Step 2: Create and submit a request

[1216] The device receives the user's input data and sends it to the server. The server analyzes this data and creates requests to multiple generative models. Specifically, based on the request received from the device, "I want a new smartphone," the server generates and sends the following prompts to the "latest smartphone detailed model," "price comparison model," "user review analysis model," and "feature comparison model," respectively:

[1217] "Please tell me the latest smartphone model."

[1218] "What is the average price of this smartphone?"

[1219] "What are the user reviews for this smartphone?"

[1220] "What are the main features of this smartphone?"

[1221] Step 3: Recognize the emotion and reflect it in the request

[1222] The server uses an emotion recognition API (e.g., Amazon Rekognition, Microsoft Azure Face API) to recognize emotions from the user's tone of voice and facial expressions. For example, if a user emphasizes "latest," the emotion engine analyzes the emotion as "excitement and anticipation." This emotion information is reflected in the request for the generative model, and a request with the emotion is sent to each generative model.

[1223] Step 4: Receiving response data

[1224] The server receives response data from each generative model. For example, the response from the "latest smartphone detailed model" may be "latest model A," the response from the "price comparison model" may be "Model A is 90,000 yen," the response from the "user review analysis model" may be "Model A's rating: 4.8," and the response from the "feature comparison model" may be "Model A is equipped with the latest feature XX." This response data is aggregated by the server.

[1225] Step 5: Compare, examine, and consolidate response data

[1226] The server uses an advanced generative AI model (e.g., OpenAI GPT-4) to compare, examine, and integrate the response data received from each generative model. During this process, it removes duplication and inconsistencies and selects the optimal output, taking into account the user's emotions. For example, it selects "Latest Model A" as the optimal output, reflecting emotions such as excitement and anticipation.

[1227] Step 6: Generate and deliver optimal output

[1228] The server generates the optimal output using Generation AI-4 and sends it to the user's device (smart glasses). For example, it generates content such as "Model A, equipped with the latest features, is the best. Price: 90,000 yen, rating: 4.8." The device receives this data and displays it on its screen.

[1229] Step 7: Displaying the optimal output

[1230] The device displays the optimal output to the user, who can then decide on a specific course of action based on this information. For example, the smart glasses display might say, "Model A, equipped with the latest features, is the best choice. It's priced at 90,000 yen and has a rating of 4.8." This allows the user to immediately consider purchasing the product in the store.

[1231] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1232] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1233] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1234] [Fourth embodiment]

[1235] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1236] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1237] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1238] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1239] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1240] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1241] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1242] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1243] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1244] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1245] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1246] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1247] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1248] The present invention relates to a system that helps a user utilize multiple generative models to obtain optimal output, and an embodiment of the system is described in detail below.

[1249] System configuration

[1250] The system consists of the following main components:

[1251] 1. A device that receives user input

[1252] 2. A server that sends requests to multiple generative models and receives their responses.

[1253] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[1254] 4. Devices that provide optimal output to users

[1255] Program Operation

[1256] Receiving user input

[1257] The device receives input data from the user. This data is the basis for the system to process. For example, a user might input a request such as, "I want to plan a trip next month with a budget of 100,000 yen."

[1258] Submitting a Request

[1259] The server receives user input data and sends requests to multiple generative models, including the following generative models:

[1260] Domestic travel plan generation model

[1261] Overseas travel plan generation model

[1262] Pricing Model

[1263] Schedule Optimization Model

[1264] The server creates and sends appropriate requests to these generative models.

[1265] Receiving response data

[1266] The server receives the response data from each generative model. For example, the response might look like this:

[1267] Domestic travel plan generation model: 3-night, 4-day Tokyo travel plan

[1268] Overseas travel plan generation model: 2-night, 3-day Seoul travel plan

[1269] Price calculation model: Tokyo trip costs 90,000 yen, Seoul trip costs 120,000 yen

[1270] Schedule optimization model: The best date is the second week of next month

[1271] Comparison, scrutiny and integration

[1272] The server uses an advanced generative model (Generation AI-4) to compare, examine, and integrate the output of each generative model. During this process, it eliminates duplicate information and inconsistencies to generate the optimal output. For example, if a 3-night, 4-day Tokyo trip plan is determined to be within budget, that plan will be selected.

[1273] Generate and deliver optimal output

[1274] Generation AI-4 generates the optimal output, which is received by the server, which then sends this output to the user's terminal, which finally displays the output to the user.

[1275] Specific examples

[1276] For example, if a user inputs "I want to plan a trip next month with a budget of 100,000 yen," the system will work as follows:

[1277] 1. The terminal receives input.

[1278] 2. The server sends requests to the domestic travel plan generation model, the international travel plan generation model, the fare calculation model, and the schedule optimization model.

[1279] 3. Each model returns a response: the domestic travel plan generation model suggests a "3-night, 4-day Tokyo trip plan," the overseas travel plan generation model suggests a "2-night, 3-day Seoul trip plan," the pricing calculation model suggests "A trip to Tokyo costs 90,000 yen, a trip to Seoul costs 120,000 yen," and the schedule optimization model suggests "The second week of next month is optimal."

[1280] 4. Generative AI-4 compares and examines these responses and generates the optimal output: "Plan a trip to Tokyo for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen."

[1281] 5. The server sends this output to the user's device.

[1282] 6. The device displays the best travel plan for the user.

[1283] In this way, users can easily obtain high-quality travel plans. Similar procedures can also be used to obtain high-quality output for other purposes, such as presentation materials.

[1284] The processing flow will be explained below.

[1285] Step 1:

[1286] The user inputs a request into the device. For example, the user inputs "I would like to plan a trip next month with a budget of 100,000 yen" into the device's input form.

[1287] Step 2:

[1288] The terminal receives input data from the user and sends it to the server.

[1289] Step 3:

[1290] The server analyzes the received input data and creates requests for multiple generation models: a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[1291] Step 4:

[1292] The server sends the created request to each generation model. For example, it sends a request such as "Generate a domestic travel plan with a budget of 100,000 yen next month" to the domestic travel plan generation model, and "Generate an overseas travel plan with a budget of 100,000 yen next month" to the overseas travel plan generation model.

[1293] Step 5:

[1294] The server receives responses from each generation model. Specifically, it receives responses such as "a four-day, three-night trip to Tokyo" from the domestic travel plan generation model, "a three-day, two-night trip to Seoul" from the overseas travel plan generation model, "a trip to Tokyo costs 90,000 yen, a trip to Seoul costs 120,000 yen" from the price calculation model, and "the second week of next month is optimal" from the schedule optimization model.

[1295] Step 6:

[1296] The server uses an advanced generative model (Generation AI-4) to compare and analyze the responses of each generative model and generate the optimal output. Specifically, it selects a plan that fits within the budget and optimizes the schedule.

[1297] Step 7:

[1298] Generation AI-4 generates the optimal output and provides the result to the server. For example, it might generate an output such as "A trip to Tokyo for three nights and four days in the second week of next month, with a budget of 90,000 yen."

[1299] Step 8:

[1300] The server generates the optimal output and sends it to the device.

[1301] Step 9:

[1302] The device displays the optimal output to the user, allowing the user to decide on specific actions to take. For example, it might say, "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen."

[1303] Example 1

[1304] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1305] When using multiple generative models to obtain optimal output, it is necessary to appropriately process user input and examine and integrate the responses from each generative model to obtain high-quality results. However, these processes are complex to perform manually and require advanced processing to ensure the quality of the integrated output. Furthermore, there are insufficient means to automate the examination and integration process that takes into account the characteristics of multiple generative models, making it difficult to generate high-quality output in a short period of time.

[1306] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1307] In this invention, the server includes a means for a user to input a prompt sentence via an input form and for the server to send the prompt sentence to multiple generative models, a means for manipulating the received response data using numpy or pandas to generate optimal output, and a means for providing the optimal output to the user. This makes it possible to compare and examine the outputs of multiple generative models based on the user's input data and automatically generate high-quality output in a short period of time.

[1308] "User" means any person or entity that uses the System and provides Input Data.

[1309] A "terminal" is a computer system or mobile device used by a user to provide input data.

[1310] "Server" refers to a central processing unit for receiving input data from a user, sending requests to multiple generative models, receiving response data, and generating and providing optimal output.

[1311] A "generative model" is an algorithm or program that generates and responds to data based on a specific task.

[1312] A "prompt sentence" is text data that a user inputs to a system, and serves as the basic data for the system to process.

[1313] "Output" refers to the final result data generated by the server by comparing, examining, and integrating the response data of the generative models.

[1314] A "request" is an instruction or data that a server sends to a Generative Model.

[1315] "Response data" is data that a generative model returns in response to a request.

[1316] "Comparison and scrutiny" is the process of examining response data from multiple generative models and selecting the most appropriate data.

[1317] "Fusion" is the process by which the server combines appropriate response data from multiple generative models to produce a single output.

[1318] "numpy" is a Python library for manipulating large amounts of numerical data.

[1319] "pandas" is a Python library for data manipulation and analysis.

[1320] This invention relates to a system that allows a user to obtain optimal output using multiple generative models. The system aims to obtain optimal responses from multiple generative models by inputting a specific prompt sentence from the user, and then examine and integrate the responses to provide them to the user.

[1321] System configuration

[1322] The system mainly consists of the following components:

[1323] 1. A device that receives user input

[1324] 2. A server that sends requests to multiple generative models and receives their responses.

[1325] 3. A means to compare, examine, and integrate the outputs of generative models to generate the optimal output

[1326] 4. Devices that provide optimal output to users

[1327] Detailed processing

[1328] Receiving user input

[1329] The user uses a terminal to input a specific prompt sentence. The terminal is usually a computer device such as a PC or smartphone. The user inputs a prompt sentence such as "I would like to plan a trip next month with a budget of 100,000 yen" through an input form and submits it.

[1330] Submitting a Request

[1331] The terminal sends the received prompt to the server, which uses software such as Python's requests library or Java's HttpClient to send requests to multiple generative models. The generative models include a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[1332] Receiving response data

[1333] The server receives response data from each generation model. Each response data is usually received in JSON format and parsed using Python's json module or Java's Jackson library. For example, the server receives data such as a "3-night, 4-day Tokyo trip plan" from the domestic travel plan generation model, a "2-night, 3-day Seoul trip plan" from the overseas travel plan generation model, "Tokyo trip costs 90,000 yen, Seoul trip costs 120,000 yen" from the price calculation model, and "The second week of next month is optimal" from the schedule optimization model.

[1334] Comparison, scrutiny and integration

[1335] The server uses the Generative AI-4 to compare and refine the response data from each generative model and generate the optimal output. This process involves data manipulation using Python libraries such as numpy and pandas. For example, the response data is converted into a numpy array and the optimization algorithm is applied using a pandas data frame.

[1336] Generate and deliver optimal output

[1337] Generation AI-4 generates the optimal output: "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen." The server then sends this data to the user's device. The device then displays the received output using HTML and JavaScript and provides it to the user. Specifically, the received data is inserted into the display area, and the user is notified in a visually easy-to-understand format.

[1338] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1339] Step 1:

[1340] The user uses an input form on the terminal and enters the prompt statement "I would like to plan a trip next month with a budget of 100,000 yen." The input data is sent from the terminal to the server. The prompt statement is received as terminal input and an HTTP request is generated to send it to the server.

[1341] Step 2:

[1342] The server analyzes the prompt received from the user and sends requests to multiple generative models. Specifically, the server converts the prompt into an appropriate format and sends API requests to the generative models using the Python requests library. The input is the prompt, and the output is an API request to each generative model.

[1343] Step 3:

[1344] Each generative model receives a request and generates response data. For example, a domestic travel plan generation model generates a "Tokyo travel plan for 3 nights and 4 days" and sends it back to the server. The server receives the response data from the generative model, and the response is in JSON format.

[1345] Step 4:

[1346] The server receives and analyzes the response data from each generative model. Specifically, the server uses the Python json module to parse the response data and convert it into an internal data structure. The input is the response data in JSON format, and the output is an internal data structure (e.g., a Python dictionary or list).

[1347] Step 5:

[1348] The server uses numpy and pandas to compare and examine each response data and generate the optimal output. For example, the server compares the costs and dates of each travel plan and selects the optimal one. The input is the internal data structure, and the output is the optimal output (for example, "Plan a 3-night, 4-day trip to Tokyo in the second week of next month, with a budget of 90,000 yen").

[1349] Step 6:

[1350] The server converts the optimal output into JSON format and sends it to the user's device. Specifically, the server generates an HTTP response using Flask or Django. The input is the optimal output data, and the output is the HTTP response sent to the user's device.

[1351] Step 7:

[1352] The terminal displays the output received from the server. Specifically, it displays the optimal travel plan on the screen using HTML and JavaScript. The input is the HTTP response from the server, and the output is the travel plan displayed to the user.

[1353] (Application example 1)

[1354] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1355] Conventional security incident response systems have difficulty in developing appropriate countermeasures and have experienced delays in response when a rapid response is required in seconds. Furthermore, it has been difficult to develop a system that can derive integrated and optimal countermeasures using multiple generative models specialized for individual elements. This has led to concerns about increased security risks and damage.

[1356] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1357] In this invention, the server includes means for receiving input data from a user, means for sending requests to multiple generative models, means for receiving response data from the multiple generative models, means for comparing, examining, and integrating the response data, means for generating an optimal output, means for providing the optimal output to the user, and means for receiving detailed information about a security incident and generating an incident response plan. This makes it possible to formulate quick and appropriate countermeasures for security incidents such as unauthorized access by utilizing multiple specialized generative models and provide them to users in real time.

[1358] "User-entered data" means information entered into a system by a user, including details and instructions regarding a security incident.

[1359] A "generative model" is an AI model with a specific role or function, such as analyzing unauthorized access patterns, assessing damage, generating emergency response procedures, and generating recovery procedures.

[1360] The "request sending means" is a means by which the system sends a request to the generative model based on user input data.

[1361] "Response data" is data that the generative model responds to a request, and includes the results of pattern analysis of unauthorized access and damage assessment results.

[1362] The "comparison and review means" refers to a means for comparing and reviewing the received response data and removing any duplication or inconsistency.

[1363] A "synthesis tool" is a tool that brings together the compared and reviewed response data into a single, integrated output.

[1364] The "output" is the optimal response or plan generated by the integration method.

[1365] "User-delivered means" refers to the means by which the integrated output is presented to the user.

[1366] A "security incident" is an event that threatens the security of a system or the protection of data, including unauthorized access or data breach.

[1367] An "incident response plan" is a plan that includes optimal responses and procedures for responding to security incidents.

[1368] This invention is a system that generates an optimal incident response plan through multiple generative models based on detailed information about a security incident entered by a user, and provides the plan to the user. This system can be run via a device such as a smartphone or smart glasses.

[1369] The system works as follows: First, the user enters detailed information about a security incident into a device such as a smartphone or smart glasses. This information includes unauthorized access logs and damage assessments. After receiving this information, the device sends it to the server.

[1370] Based on the input data from the user, the server sends requests to multiple generative models, such as a fraudulent access pattern analysis model, a damage assessment model, an emergency response procedure generation model, and a recovery procedure generation model. In response to this request, each generative model generates corresponding analysis results and proposals and returns them to the server.

[1371] The returned response data is compared and scrutinized by an advanced generative model (Generation AI-4) within the server. This advanced generative model has the ability to remove duplicates and inconsistencies from the response data and generate the optimal output. The output integrated by Generation AI-4 is further scrutinized within the server as the optimal incident response plan.

[1372] After generating an optimal incident response plan, the server sends this information back to the user's device and presents the user with appropriate countermeasures and procedures. The generative models and integration algorithms used in this process may utilize AI platforms such as Amazon SageMaker.

[1373] Below is a concrete example of how this system can be implemented, showing the prompt text that will be displayed when unauthorized access occurs.

[1374] Examples:

[1375] "Unauthorized access log ID: 12345 has occurred and the damage is high. Please generate the optimal countermeasures for this incident."

[1376] In this way, users can quickly receive and take appropriate measures to deal with security incidents. The system provides real-time information and helps minimize the damage caused by the incident.

[1377] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1378] Step 1:

[1379] The user enters details about the security incident into the device.

[1380] Input: Detailed information such as unauthorized access logs and damage status.

[1381] Output: Security incident information entered into the terminal.

[1382] Specific operation: The user enters "Unauthorized access log ID: 12345 has occurred and the damage is significant" into the device.

[1383] Step 2:

[1384] The terminal transmits the entered security incident information to the server.

[1385] Input: Information entered into the device.

[1386] Output: Information data sent to the server.

[1387] Specific operation: The device sends security incident information to a server via the Internet.

[1388] Step 3:

[1389] Based on the incident information received by the server, requests are sent to multiple generative models.

[1390] Input: Security incident information sent to the server.

[1391] Output: A request for each generative model.

[1392] Specific operation: The server sends requests to the unauthorized access pattern analysis model, the damage assessment model, the emergency response procedure generation model, and the recovery procedure generation model.

[1393] Step 4:

[1394] The generative model returns response data based on the request to the server.

[1395] Input: The requests sent to each generative model.

[1396] Output: Response data from each generative model.

[1397] Specific operation: The unauthorized access pattern analysis model generates the pattern analysis result, and the damage assessment model generates the damage assessment result, and these are returned to the server.

[1398] Step 5:

[1399] The server compares, examines, and integrates the response data received from each generative model using an advanced generative model (Generation AI-4).

[1400] Input: Response data from each generative model.

[1401] Output: A consolidated incident response plan.

[1402] Specific operation: The Generate AI-4 analyzes response data, removes duplicates and inconsistencies, and generates an optimal incident response plan.

[1403] Step 6:

[1404] The server sends the integrated incident response plan to the user's device.

[1405] Input: A consolidated incident response plan.

[1406] Output: The optimal response plan sent to the user's device.

[1407] Specific operation: The server sends the incident response plan to the user's device via the Internet.

[1408] Step 7:

[1409] The device displays an integrated incident response plan to the user.

[1410] Input: The corresponding plan sent to the user's device.

[1411] Output: The optimal incident response plan displayed to the user.

[1412] Specific operation: The device will display "Optimal countermeasure for unauthorized access log ID: 12345" on the screen.

[1413] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1414] The present invention relates to a system that helps a user obtain optimal output by utilizing multiple generative models and further recognizes the user's emotions to make optimal suggestions. An embodiment of this system will be described in detail below.

[1415] System configuration

[1416] The system consists of the following components:

[1417] 1. A device that receives user input

[1418] 2. A server that sends requests to multiple generative models and receives their responses.

[1419] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[1420] 4. Emotion engine that recognizes user emotions

[1421] 5. Devices that provide optimal output to users

[1422] Program Operation

[1423] Receiving user input

[1424] The terminal receives input data from the user. This data is the basis for the system to process. For example, a request might be entered such as, "I want to plan a trip next month with a budget of 100,000 yen."

[1425] Submitting a Request

[1426] Based on the user's input data, the server creates requests for multiple generation models: a domestic travel plan generation model, an international travel plan generation model, a fare calculation model, and a schedule optimization model.

[1427] Recognizing and reflecting emotions

[1428] The server uses the emotion engine to analyze the emotion from the user's input data and reflects the results in the request to the generative model. For example, if a user inputs "I want a trip that will reduce stress," the emotion engine analyzes "relaxation" and sends a request to each generative model based on this emotion.

[1429] Receiving response data

[1430] The server receives response data from each generation model. For example, it receives a response such as "a 3-night, 4-day Okinawa trip plan" from the domestic travel plan generation model, a "2-night, 3-day Guam trip plan" from the overseas travel plan generation model, "a trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen" from the price calculation model, and "the best dates are the second week of next month" from the schedule optimization model.

[1431] Comparison, scrutiny and integration

[1432] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. In this example, a trip to Okinawa is selected, taking into consideration "relaxation." The server also references the user's previous emotional history to make more accurate suggestions.

[1433] Generate and deliver optimal output

[1434] Generation AI-4 generates the optimal output, which is received by the server. The server then sends this output to the user's device. If the optimal output is "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month with a budget of 90,000 yen," this will be provided to the user.

[1435] Displaying optimal output

[1436] The device displays the generated optimal output to the user, who can then decide on specific actions to take based on this information. For example, a detailed travel plan such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen" is displayed.

[1437] Specific examples

[1438] For example, if a user enters "I want to plan a trip next month with a budget of 100,000 yen, preferably somewhere relaxing," the system will work as follows:

[1439] 1. The terminal receives input.

[1440] 2. The server uses the emotion engine to analyze the emotion "relaxed" and reflects this in each generative model request.

[1441] 3. The server sends requests to the domestic travel plan generation model, the international travel plan generation model, the fare calculation model, and the schedule optimization model.

[1442] 4. Each model returns a response: the domestic travel plan generation model replies "A three-night, four-day trip to Okinawa," the overseas travel plan generation model replies "A two-night, three-day trip to Guam," the pricing calculation model replies "A trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen," and the schedule optimization model replies "The second week of next month is optimal."

[1443] 5. Generative AI-4 compares and refines these responses and generates the optimal output: "A relaxing, budget-friendly plan for a 3-night, 4-day trip to Okinawa in the second week of next month."

[1444] 6. The server sends this output to the user's device.

[1445] 7. The device displays the best travel plan for the user.

[1446] This system allows users to easily obtain high-quality travel plans based on emotions and requests. Similar procedures can also be used to obtain high-quality output for other purposes, such as presentation materials.

[1447] The processing flow will be explained below.

[1448] Step 1:

[1449] The user inputs a request into the device. For example, the user might input, "I want to plan a trip next month with a budget of 100,000 yen. I would like a relaxing place."

[1450] Step 2:

[1451] The terminal receives input data from the user and sends it to the server.

[1452] Step 3:

[1453] The server analyzes the received input data and uses an emotion engine to recognize the user's emotion, in this case "relaxed."

[1454] Step 4:

[1455] The server makes requests for multiple Generative Models, specifically the following Generative Models:

[1456] Domestic travel plan generation model: "Generate a relaxing domestic travel plan with a budget of 100,000 yen next month"

[1457] "Generate a relaxing overseas travel plan with a budget of 100,000 yen for next month"

[1458] "Calculate the cost of each plan" in the pricing model

[1459] "Generate optimal travel schedules" for schedule optimization models

[1460] Step 5:

[1461] The server sends the created request to each generative model.

[1462] Step 6:

[1463] The server receives a response from each generative model, which includes, for example, the following information:

[1464] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[1465] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[1466] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[1467] The schedule optimization model says, "The second week of next month is optimal."

[1468] Step 7:

[1469] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. Specifically, it selects a plan (Okinawa travel plan) that meets the "relaxation" criteria and fits within the budget.

[1470] Step 8:

[1471] The server then sends the generated optimal output to the terminal. For example, it generates a plan such as "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month with a budget of 90,000 yen."

[1472] Step 9:

[1473] The device displays the optimal output to the user, allowing the user to decide on specific actions to take. For example, a detailed travel plan such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen" is displayed.

[1474] Step 10:

[1475] The server stores the user's emotion history, which is then referenced the next time output is generated to provide more accurate suggestions.

[1476] Example 2

[1477] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1478] Conventional systems have struggled to generate optimal outputs that take into account specific user emotions. Furthermore, when integrating response data from multiple generative models, there was a lack of a way to incorporate user emotions, resulting in poor recommendation accuracy and reduced user satisfaction. To address this issue, there is a need for systems that can analyze user emotions and incorporate them into generative model requests to provide more accurate recommendations.

[1479] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1480] In this invention, the server includes means for receiving user input, means for sending requests to multiple generative models, means for analyzing emotions, means for receiving response data, means for comparing, examining, and integrating the response data, means for generating optimal output, means for providing the optimal output to the user, and means for reflecting the results of the emotion analysis in the requests of the generative models, thereby enabling the generation of highly accurate output that reflects the user's emotions.

[1481] "Means for receiving user input" refers to an input device or form through which a user enters information or requests into the system.

[1482] "Means for sending a request" refers to a device or program that has the function of creating and sending requests to multiple generative models based on information received from a user.

[1483] A "generative model" refers to an algorithm or program that generates a specific output based on specific conditions and input data.

[1484] "Means for analyzing emotions" means a program or device for analyzing and identifying a user's emotional state based on input data or other information from the user.

[1485] "Means for receiving response data" refers to a device or program for receiving results sent from multiple generative models.

[1486] "Means for comparing, examining, and integrating response data" refers to a program or device that compares data received from different generative models, eliminates inconsistencies and duplications, and integrates them in the most optimal way.

[1487] "Means for generating optimal outputs" refers to programs and algorithms that generate final recommendations or results based on integrated data and taking into account user requirements and emotions.

[1488] "Means for providing optimal output to a user" refers to devices or programs for displaying, transmitting, or providing the generated output to a user.

[1489] "Means for reflecting the results of emotion analysis in requests to a generative model" refers to a device or program that has the function of incorporating the analyzed user's emotion information into requests to a generative model and adjusting the content of the request.

[1490] MODE FOR CARRYING OUT THE INVENTION

[1491] The present invention relates to a system that helps a user obtain optimal output by utilizing multiple generative models and further recognizes the user's emotions to make optimal suggestions. An embodiment of this system will be described in detail below.

[1492] System configuration

[1493] The system consists of the following components:

[1494] 1. A device that receives user input

[1495] 2. A server that sends requests to multiple generative models and receives their responses.

[1496] 3. An advanced generative model (Generation AI-4) that compares, refines, and integrates the outputs of generative models to generate the optimal output.

[1497] 4. Emotion engine that recognizes user emotions

[1498] 5. Devices that provide optimal output to users

[1499] Receiving user input

[1500] The specific hardware used for the device may be a PC, tablet, smartphone, etc. The software used to receive input data from the user and send it to the system may include a browser-based application or a dedicated app. For example, a user may enter "I'd like to plan a trip next month with a budget of 100,000 yen. I'd like a relaxing place" into the device's input field and press the send button.

[1501] Creating and Sending a Request

[1502] The server is a core system of hardware that creates and sends requests for the required generative models based on user input data. Specific generative models include:

[1503] Domestic travel plan generation model

[1504] Overseas travel plan generation model

[1505] Pricing Model

[1506] Schedule Optimization Model

[1507] The server sends a request to each generative model as follows:

[1508] A domestic travel plan generation model asks, "Generate a relaxing domestic travel plan with a budget of 100,000 yen."

[1509] For the overseas travel plan generation model, "Generate a relaxing overseas travel plan with a budget of 100,000 yen"

[1510] The pricing model asks, "Calculate the budget for domestic and international travel plans."

[1511] "Please provide me with the best travel dates for next month" to the schedule optimization model

[1512] Recognizing and reflecting emotions

[1513] The server uses an emotion engine to analyze emotions from the user's input data. Commonly available software tools such as Microsoft's Azure Emotion API and Google's Vision API can be used as emotion engines. The analyzed emotion data is reflected in the request. For example, if the emotion "relaxed" is analyzed from the user's input, the request is updated based on that element.

[1514] Receiving response data

[1515] The server receives response data from each generative model. For example, the following response is returned:

[1516] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[1517] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[1518] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[1519] The schedule optimization model says, "The second week of next month is optimal."

[1520] Comparison, scrutiny and integration

[1521] The server uses an advanced generative model (Generation AI-4) to compare and examine the responses of each generative model and generate the optimal output. Generative AI-4 uses advanced algorithms built on OpenAI's GPT-4 and Google's BERT. This allows it to derive the most suitable suggestions.

[1522] Generate and deliver optimal output

[1523] Generation AI-4 generates the optimal output, which is received by the server. The server then sends this output to the user's device. For example, a plan might be generated for a relaxing, budget-friendly trip to Okinawa in the second week of next month for three nights and four days.

[1524] Displaying optimal output

[1525] The device displays the generated optimal output to the user, who can then decide on specific actions to take based on this information. For example, a detailed travel plan might be presented, such as "Plan a relaxing trip to Okinawa for three nights and four days in the second week of next month, with a budget of 90,000 yen."

[1526] Specific prompt examples

[1527] For example, the prompt you would enter into a generative AI model might look like this:

[1528] "The user wants to relax and is planning a trip with a budget of 100,000 yen. Please suggest the best travel plan, whether domestic or international."

[1529] This procedure allows the system to provide optimal suggestions based on the user's requests and emotions.

[1530] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1531] Step 1:

[1532] The terminal receives user input.

[1533] Specific operation: The user enters "I want to plan a trip next month with a budget of 100,000 yen. I would like a relaxing place" into the input field on the device and presses the send button.

[1534] Input: Request data entered by the user (e.g., "I'd like to plan a trip next month with a budget of 100,000 yen. I'd like a relaxing place.")

[1535] Output: Request data sent from the terminal to the server

[1536] Step 2:

[1537] The server creates and sends requests for each generative model based on the user's input data.

[1538] Specific operation: The server analyzes the user's input data and sends the following request to each generative model:

[1539] A domestic travel plan generation model asks, "Generate a relaxing domestic travel plan with a budget of 100,000 yen."

[1540] For the overseas travel plan generation model, "Generate a relaxing overseas travel plan with a budget of 100,000 yen"

[1541] The pricing model asks, "Calculate the budget for domestic and international travel plans."

[1542] "Please provide me with the best travel dates for next month" to the schedule optimization model

[1543] Input: User request data

[1544] Output: Requests to each generative model

[1545] Step 3:

[1546] The server uses an emotion engine to analyze emotions from the user's input data.

[1547] Specific operation: The server uses an emotion engine (e.g., Microsoft's Azure Emotion API) to analyze the emotion "relaxed" and reflects this in the request to the generative model.

[1548] Input: User request data, emotion engine

[1549] Output: A request to the generative model that reflects the emotional elements

[1550] Step 4:

[1551] The generative model receives the server request and generates response data.

[1552] Specific operation: Each generative model generates the following response data based on the request and sends it to the server.

[1553] "Okinawa travel plan for 3 nights and 4 days" from the domestic travel plan generation model

[1554] "Guam travel plan for 2 nights and 3 days" from the overseas travel plan generation model

[1555] From the pricing calculation model, "A trip to Okinawa costs 90,000 yen, and a trip to Guam costs 120,000 yen"

[1556] The schedule optimization model says, "The second week of next month is optimal."

[1557] Input: The request sent by the server

[1558] Output: Response data from each generative model

[1559] Step 5:

[1560] The server receives the response data from each generative model.

[1561] Specific operation: The server receives and stores response data from each generative model.

[1562] Input: Response data from each generative model (e.g., "A trip to Okinawa for 3 nights and 4 days," "A trip to Guam for 2 nights and 3 days," "A trip to Okinawa costs 90,000 yen, a trip to Guam costs 120,000 yen," "The second week of next month is best").

[1563] Output: Save the received response data

[1564] Step 6:

[1565] The server uses an advanced generative model (Generation AI-4) to compare, examine, and integrate the responses of each generative model to generate the optimal output.

[1566] Specific operation: The Generation AI-4 analyzes each response data and generates the optimal output while prioritizing the user's emotion of "relaxation." It also refers to the user's previous emotion history to select the most appropriate suggestion.

[1567] Input: Response data from each generative model, user emotion data, previous user emotion history

[1568] Output: Optimal output (e.g., "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen")

[1569] Step 7:

[1570] The server receives the optimal output from the generated AI-4 and sends it to the user's device.

[1571] Specific operation: The server receives the optimal output from generation AI-4 and sends it to the user's terminal.

[1572] Input: Optimal Output

[1573] Output: Data sent to the terminal

[1574] Step 8:

[1575] The device will display the generated optimal output to the user.

[1576] Specific operation: The device displays a detailed travel plan such as "Plan a relaxing trip to Okinawa for 3 nights and 4 days in the second week of next month, with a budget of 90,000 yen." The user can then view this and make a specific reservation.

[1577] Input: The best output sent by the server

[1578] Output: Data displayed to the user

[1579] (Application example 2)

[1580] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1581] Conventional shopping assistant systems have the problem of only being able to make uniform product recommendations without fully considering the user's specific needs and emotions. Furthermore, it is difficult to make optimal product suggestions that reflect the user's emotions and circumstances, which change in real time, and they lack ingenuity to improve the user experience. Furthermore, it is practically difficult to provide interactive product information through smart glasses or other advanced user devices. To solve these issues, a system is needed that can analyze user input data and emotions in real time and provide optimal output accordingly.

[1582] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving input data from a user, means for sending requests to multiple generative models, means for receiving response data from the multiple generative models, means for recognizing the user's emotions and reflecting them in the request, means for comparing, examining, and integrating the response data, and means for providing optimal output to the user terminal. This enables high-quality, personalized product recommendations based on the user's specific needs and emotions.

[1583] "User input" refers to data or requirements provided by a user to a system.

[1584] A "generative model" is a machine learning algorithm that generates a specific output based on data.

[1585] A "request" refers to user input or an instruction or question sent by the system to a generative model.

[1586] "Response data" refers to the information or results returned by a generative model.

[1587] "Comparison and examination" refers to the act of comparing multiple response data with each other and eliminating any inconsistencies or discrepancies.

[1588] "Integration" is the act of combining the optimal outputs based on the results of comparison and scrutiny.

[1589] "Emotion recognition" refers to the techniques and processes used to analyze and recognize a user's emotional state.

[1590] "Smart glasses" are wearable devices that have built-in computer functions and allow users to obtain information visually.

[1591] "User terminal" refers to an electronic device used by a user to input or output information.

[1592] "Output" refers to the final results or recommendations that the system provides to the user.

[1593] This invention relates to a system that helps users obtain optimal output by utilizing multiple generative models and recognizes the user's emotions to provide optimal suggestions. This system can be realized as a shopping assistant application using smart glasses that enhances the shopping experience in brick-and-mortar stores.

[1594] System configuration

[1595] The system consists of the following components:

[1596] 1. A way to receive user input: Using the smart glasses' voice input or eye tracking capabilities, the glasses can receive requests about products the user is considering purchasing. For example, "I want a new smartphone, and I'm looking for a model with the latest features."

[1597] 2. A means for sending requests to multiple generative models: User input data is sent to the server, and requests are issued to various generative models. Specifically, requests are sent to the latest smartphone detail model, price comparison model, user review analysis model, and feature comparison model.

[1598] 3. Means for receiving response data from the generative model: The server receives response data from the generative model. For example, the latest smartphone detailed model might say "Latest Model A," the price comparison model might say "Model A is 90,000 yen," the user review analysis model might say "Model A's rating: 4.8," and the feature comparison model might say "Model A has the latest feature XX."

[1599] 4. Means for recognizing emotions and incorporating them into requests: The server uses emotion recognition APIs (e.g., Amazon Rekognition, Microsoft Azure Face API) to analyze the user's emotions from their tone of voice and facial expressions. This allows the server to analyze the user's emotions, such as "excitement" or "expectation," and incorporate them into the generative model's requests.

[1600] 5. Means of comparing, scrutinizing, and integrating response data: The server uses an advanced generative AI model (e.g., OpenAI GPT-4) to compare and scrutinize the responses from each generative model to generate the optimal output, eliminating duplication and inconsistency and making optimal suggestions taking into account user sentiment.

[1601] 6. Means for providing optimal output to the user device: The server sends the generated optimal output to the smart glasses. For example, the smart glasses display will show "Model A with the latest features is the best. Price is 90,000 yen, rating is 4.8."

[1602] Specific examples

[1603] If a user uses smart glasses to say, "I want a new smartphone, I'm looking for a model with the latest features," the system will:

[1604] 1. Receiving user input: The smart glasses receive requests via voice input or eye tracking.

[1605] 2. Sending a request: The received input data is sent to the server and a request is issued to the generative model.

[1606] 3. Emotion recognition and reflection: The emotion engine analyzes the user's emotions and reflects the emotions of "excitement and anticipation" in the generative model's requests.

[1607] 4. Receiving response data: The server receives the response data from the generative model.

[1608] 5. Compare, Refine and Integrate: The Generate AI-4 refines the response data and generates the optimal output.

[1609] 6. Providing optimal output: The generated output is sent to the smart glasses and displayed to the user.

[1610] Prompt Sentence Examples

[1611] "Please tell me the latest smartphone model."

[1612] "What is the average price of this smartphone?"

[1613] "What are the user reviews for this smartphone?"

[1614] "What are the main features of this smartphone?"

[1615] This system allows users to receive real-time personalized product information in physical stores, providing a better shopping experience.

[1616] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1617] Step 1: Receiving User Input

[1618] A user uses the smart glasses to input a request through voice input or eye tracking. This input data includes specific information about the product they are considering purchasing. For example, a user might say, "I want a new smartphone, and I'm looking for a model with the latest features." The device receives this input data and prepares it for processing.

[1619] Step 2: Create and submit a request

[1620] The device receives the user's input data and sends it to the server. The server analyzes this data and creates requests to multiple generative models. Specifically, based on the request received from the device, "I want a new smartphone," the server generates and sends the following prompts to the "latest smartphone detailed model," "price comparison model," "user review analysis model," and "feature comparison model," respectively:

[1621] "Please tell me the latest smartphone model."

[1622] "What is the average price of this smartphone?"

[1623] "What are the user reviews for this smartphone?"

[1624] "What are the main features of this smartphone?"

[1625] Step 3: Recognize the emotion and reflect it in the request

[1626] The server uses an emotion recognition API (e.g., Amazon Rekognition, Microsoft Azure Face API) to recognize emotions from the user's tone of voice and facial expressions. For example, if a user emphasizes "latest," the emotion engine analyzes the emotion as "excitement and anticipation." This emotion information is reflected in the request for the generative model, and a request with the emotion is sent to each generative model.

[1627] Step 4: Receiving response data

[1628] The server receives response data from each generative model. For example, the response from the "latest smartphone detailed model" may be "latest model A," the response from the "price comparison model" may be "Model A is 90,000 yen," the response from the "user review analysis model" may be "Model A's rating: 4.8," and the response from the "feature comparison model" may be "Model A is equipped with the latest feature XX." This response data is aggregated by the server.

[1629] Step 5: Compare, examine, and consolidate response data

[1630] The server uses an advanced generative AI model (e.g., OpenAI GPT-4) to compare, examine, and integrate the response data received from each generative model. During this process, it removes duplication and inconsistencies and selects the optimal output, taking into account the user's emotions. For example, it selects "Latest Model A" as the optimal output, reflecting emotions such as excitement and anticipation.

[1631] Step 6: Generate and deliver optimal output

[1632] The server generates the optimal output using Generation AI-4 and sends it to the user's device (smart glasses). For example, it generates content such as "Model A, equipped with the latest features, is the best. Price: 90,000 yen, rating: 4.8." The device receives this data and displays it on its screen.

[1633] Step 7: Displaying the optimal output

[1634] The device displays the optimal output to the user, who can then decide on a specific course of action based on this information. For example, the smart glasses display might say, "Model A, equipped with the latest features, is the best choice. It's priced at 90,000 yen and has a rating of 4.8." This allows the user to immediately consider purchasing the product in the store.

[1635] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1636] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1637] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1638] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1639] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1640] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1641] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1642] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1643] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1644] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1645] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1646] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1647] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1648] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1649] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1650] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1651] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1652] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1653] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1654] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1655] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1656] The following is further disclosed regarding the above embodiment.

[1657] (Claim 1)

[1658] a means for receiving input data from a user;

[1659] a means for sending requests to a plurality of generative models;

[1660] means for receiving response data from the plurality of generative models;

[1661] means for comparing, examining, and integrating said response data;

[1662] a means of generating optimal output;

[1663] means for providing said optimal output to a user;

[1664] A system including:

[1665] (Claim 2)

[1666] 2. The system of claim 1, wherein the request sending means sends a request to a specialized generative model.

[1667] (Claim 3)

[1668] 2. The system according to claim 1, wherein the comparison and review means removes duplicates and inconsistencies from the response data.

[1669] (Claim 4)

[1670] The system of claim 1 , wherein the optimal output relates to a travel plan.

[1671] (Claim 5)

[1672] 2. The system of claim 1, wherein the optimal output relates to presentation materials.

[1673] "Example 1"

[1674] (Claim 1)

[1675] a means for receiving input data from a user;

[1676] a means for sending requests to a plurality of generative models;

[1677] means for receiving response data from the plurality of generative models;

[1678] means for comparing, examining, and integrating said response data;

[1679] a means of generating optimal output;

[1680] means for providing said optimal output to a user;

[1681] A system including:

[1682] (Claim 2)

[1683] 2. The system according to claim 1, wherein the system is a means for a user to input a prompt sentence via an input form, and the server transmits the prompt sentence to a plurality of generative models.

[1684] (Claim 3)

[1685] The system according to claim 1, wherein the comparison and examination means manipulates the received response data using numpy or pandas to generate an optimal output.

[1686] "Application Example 1"

[1687] (Claim 1)

[1688] a means for receiving input data from a user;

[1689] a means for sending requests to a plurality of generative models;

[1690] means for receiving response data from the plurality of generative models;

[1691] means for comparing, examining, and integrating said response data;

[1692] a means of generating optimal output;

[1693] means for providing said optimal output to a user;

[1694] a means of receiving detailed information about a security incident and generating an incident response plan;

[1695] A system including:

[1696] (Claim 2)

[1697] The system of claim 1, wherein the request sending means sends requests to specialized generation models, including a fraudulent access pattern analysis model, a damage assessment model, an emergency response procedure generation model, and a recovery procedure generation model.

[1698] (Claim 3)

[1699] 2. The system of claim 1, wherein the comparison and review means removes duplicates and inconsistencies from the response data and generates an integrated incident response plan.

[1700] "Example 2: Combining Emotion Engines"

[1701] (Claim 1)

[1702] a means for receiving input data from a user;

[1703] a means for sending requests to a plurality of generative models;

[1704] means for receiving response data from the plurality of generative models;

[1705] means for comparing, examining, and integrating said response data;

[1706] a means of generating optimal output;

[1707] means for providing said optimal output to a user;

[1708] A means of analyzing user emotions,

[1709] means for reflecting the result of the sentiment analysis in a request for a generative model;

[1710] A system including:

[1711] (Claim 2)

[1712] 2. The system of claim 1, wherein the request sending means sends a request to a specialized generative model.

[1713] (Claim 3)

[1714] 2. The system according to claim 1, wherein the comparison and review means removes duplicates and inconsistencies from the response data.

[1715] "Application example 2 when combining emotion engines"

[1716] (Claim 1)

[1717] a means for receiving input data from a user;

[1718] a means for sending requests to a plurality of generative models;

[1719] means for receiving response data from the plurality of generative models;

[1720] means for comparing, examining, and integrating said response data;

[1721] A way to recognize user emotions and reflect them in requests;

[1722] A means to provide optimal output to user devices such as smart glasses, and

[1723] A system including:

[1724] (Claim 2)

[1725] 2. The system of claim 1, wherein the request sending means sends a request to a specialized generative model.

[1726] (Claim 3)

[1727] 2. The system according to claim 1, wherein the comparison and scrutiny means removes duplications and inconsistencies from the response data and selects the optimal output based on user sentiment. [Explanation of symbols]

[1728] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for receiving input data from a user; a means for sending requests to a plurality of generative models; means for receiving response data from the plurality of generative models; means for comparing, examining, and integrating said response data; a means of generating optimal output; means for providing said optimal output to a user; A system including:

2. 2. The system of claim 1, wherein the request sending means sends a request to a specialized generative model.

3. 2. The system according to claim 1, wherein the comparison and review means removes duplicates and inconsistencies from the response data.

4. The system of claim 1 , wherein the optimal output relates to a travel plan.

5. The system of claim 1 , wherein the optimal output relates to presentation materials.

Images