System

Abstract

A system is provided.SOLUTION: A system including input means for a user to input an instruction to a generative model, database means for storing data of past instructions and results thereof, analysis means for analyzing the stored past instructions and results thereof and classifying successful cases and failed cases, proposal means for proposing a new instruction to the user based on the classified successful cases, and interface means for transmitting the instruction input by the user to a server in real time and displaying the proposed instruction on a screen.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] It shows the "problem that the invention aims to solve" and the "means for solving the problem."

[0005] In order for users to obtain the desired results in generative models, they must input appropriate instructions. However, particularly when giving complex instructions, this often requires repeated revisions, which is time-consuming. Furthermore, some users do not know how to create instructions, which is a barrier to using generative models. In this situation, there is a need for a system that can assist users in efficiently creating appropriate instructions and facilitate the use of generative models. [Means for solving the problem]

[0006] The present invention solves the above-mentioned problems with a system including an input means for a user to input instructions to a generative model, a database means for storing data on past instructions and their results, an analysis means for analyzing the stored past instructions and their results and classifying them into success cases and failure cases, a proposal means for suggesting new instructions to the user based on the classified success cases, and an interface means for transmitting the instructions input by the user to a server in real time and displaying the proposed instructions on a screen. Specifically, the analysis means extracts common instruction patterns from the stored past instructions and trains a natural language processing model, allowing the proposal means to suggest appropriate instructions to the user. The proposal means can also suggest similar success cases in real time based on partial instructions input by the user. This allows the user to efficiently input appropriate instructions to the generative model and obtain desired results.

[0007] A "user" is a person or entity that inputs instructions to a generative model and receives the results.

[0008] A "generative model" is an artificial intelligence system that generates text, images, etc. based on instructions from a user.

[0009] An "input means" is a device or software that provides an interface for a user to input instructions to a generative model.

[0010] The "database means" is a recording medium or system for storing data on past user instructions and their results.

[0011] An "analysis tool" is an algorithm or system that analyzes stored past instructions and results and classifies successes and failures.

[0012] "Success stories" are past instructions that have produced results that meet the user's expectations.

[0013] A "failure" is a past instruction that produced results that did not meet the user's expectations.

[0014] The "suggestion means" is an algorithm or system that suggests new instructions to the user based on the success cases classified by the analysis means.

[0015] The "interface means" is a device or software that transmits instructions input by the user to the server in real time and displays suggested instructions on the screen.

[0016] A "natural language processing model" is an algorithm or system that parses, understands, and generates human language.

[0017] "Real-time" means that input information is processed and results are returned almost immediately.

[0018] "Suggestions" are potential instructions that may be useful, based on partial instructions entered by the user. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] ---

[0041] The present invention provides a system that supports users in inputting appropriate instructions to a generative model. The system is implemented through a series of procedures involving interactions between a server, a terminal, and a user.

[0042] Server Processing

[0043] 1. Data Collection

[0044] The server collects the instructions that users have previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data is stored in a database.

[0045] 2. Data analysis

[0046] The server analyzes the instructions and results collected from the database using text analysis algorithms to extract characteristics of the instructions and results, and then classifies the instructions as successful or unsuccessful.

[0047] 3. Training the prompt generation model

[0048] The server trains a natural language processing (NLP) model to learn common patterns among success prompts. Specifically, it converts success prompts into numerical vectors using TF-IDF vectorization and applies a K-means clustering algorithm to group similar prompts.

[0049] 4. Real-time suggestions

[0050] As users enter partial instructions, the server generates appropriate prompt suggestions in real time, leveraging trained NLP models.

[0051] Terminal handling

[0052] 1. User Interface (UI)

[0053] The terminal provides an interface for the user to input instructions to the generative model. This UI includes input fields, a submit button, and a list of past inputs and suggested prompts.

[0054] 2. Real-time assistance

[0055] As the user enters prompts, the terminal incrementally sends requests to the server and displays suggestions returned by the server in real time, allowing the user to select or modify the suggested prompts.

[0056] User Action

[0057] 1. Prompt Input

[0058] The user inputs a prompt to the generative model, specifying the task they want it to perform, such as "Show me a picture of a cat."

[0059] 2. Proposal acceptance

[0060] The user reviews the suggestions displayed on the device, selects the appropriate one, and optionally customizes the suggested prompts before sending them to the generative model.

[0061] Specific examples

[0062] 1. Success stories

[0063] When a user types "I want to see pictures of animals," the device sends this partial prompt to the server. The server analyzes past successes and suggests successful prompts, such as "Show me pictures of cats" or "Show me pictures of dogs." The device displays these suggestions to the user, who then selects "Show me pictures of cats."

[0064] 2. Failure example

[0065] If the user only types "animals," the server may not be able to properly handle this ambiguous prompt. However, the system's suggestion feature can suggest more specific prompts so the user can provide specific instructions.

[0066] In this way, the system of the present invention helps the user input appropriate instructions into the generative model, enabling efficient use.

[0067] ---

[0068] The processing flow will be explained below.

[0069] ---

[0070] Step 1:

[0071] The server collects and stores in a database the instructions that users have previously input to the generative model and the results of those inputs. This data also includes user feedback information (evaluation of success / failure).

[0072] Step 2:

[0073] The server periodically analyzes the stored data, first retrieving past instructions and their results, then using a text analysis algorithm to extract the characteristics of each instruction, and then classifying them into success cases and failure cases.

[0074] Step 3:

[0075] The server trains a natural language processing model (NLP model) to extract common patterns from success stories, converting success stories into numerical vectors using TF-IDF vectorization and grouping similar prompts using the K-means clustering algorithm.

[0076] Step 4:

[0077] The user inputs instructions to the generative model through the terminal. When the user starts inputting, the terminal sends partial instructions to the server in real time.

[0078] Step 5:

[0079] The server uses existing NLP models to generate appropriate prompts based on the partial instructions received, and searches for similar success stories in real time to generate appropriate suggestions.

[0080] Step 6:

[0081] The server sends the generated proposals back to the terminal, which receives them and displays them in its user interface.

[0082] Step 7:

[0083] The user reviews the suggested prompts, selects appropriate ones, or customizes them if necessary, after which the final selected instructions are sent to the generative model.

[0084] Step 8:

[0085] The generative model generates results based on the user's specifications and returns the results to the device, which then displays the generated results to the user.

[0086] Step 9:

[0087] The user checks the displayed results and provides feedback to the server via the terminal, which stores this feedback in a database for subsequent analysis.

[0088] This series of steps allows the system of the present invention to allow the user to efficiently input appropriate instructions to the generative model and obtain the desired results.

[0089] Example 1

[0090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0091] In recent years, generative AI models have been used in a wide range of applications, but it is not easy for users to input appropriate instructions into generative models. In particular, when users input ambiguous instructions, it is difficult for the generative model to provide the expected results. This leads to a mismatch between the user's intention and the results, resulting in a poor user experience. In addition, there is a need for a system that can efficiently complement instructions by utilizing past success stories and suggest appropriate instructions to the user.

[0092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0093] In this invention, the server includes a data storage means, a data analysis means, a proposal generation means, and a user interface means, which allows a user to input appropriate instructions to the generative model, enabling efficient use of the generative AI model.

[0094] "Input means" refers to a device or software that allows a user to input instructions to a generative model.

[0095] "Data storage means" refers to a device or software for recording and storing instructions that a user has previously input into a generative model and the results thereof.

[0096] "Data analysis means" refers to a device or software that analyzes stored past instructions and their results and classifies successful cases and unsuccessful cases.

[0097] The "suggestion generation means" refers to a device or software for suggesting new instructions to the user based on the success cases classified by the data analysis means.

[0098] "User interface means" refers to a device or software for transmitting instructions entered by a user to a server in real time and displaying suggested instructions on the user's screen.

[0099] A "generative AI model" refers to an artificial intelligence system that generates text, images, audio, etc. based on user input.

[0100] A "prompt sentence" refers to an input sentence that instructs a generative AI model to generate something specific.

[0101] A "success case" refers to an example in which a generative AI model produced the expected results in response to user instructions.

[0102] A "failure case" refers to an instance in which the generative AI model did not produce the expected result in response to a user's instructions.

[0103] The present invention is a system that supports users in inputting appropriate instructions to a generative AI model. This system is composed of a server, a terminal, and a series of processes including user interaction.

[0104] Server Processing

[0105] The server uses the data storage means, data analysis means, and suggestion generation means to analyze the prompts entered by the user and make appropriate suggestions, as follows:

[0106] 1. Data storage method

[0107] The server collects the instructions that users have previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data is stored in a database.

[0108] 2. Data analysis methods

[0109] The server analyzes the instructions and results collected from the database. For the analysis, it uses text analysis algorithms (e.g., morphological analysis, TF-IDF, etc.) to extract characteristics of the instruction content and results. The server classifies the instruction as successful or unsuccessful.

[0110] 3. Proposal generation means

[0111] The server trains a natural language processing (NLP) model to learn common patterns among success prompts. Specifically, it converts success prompts into numerical vectors using TF-IDF vectorization and applies a K-means clustering algorithm to group similar prompts. For partial instructions entered by the user, the server generates appropriate prompt suggestions in real time.

[0112] Terminal handling

[0113] The terminal provides an interface for a user to input instructions to the generative model using a user interface means, specifically as follows.

[0114] 1. User Interface Methods

[0115] The terminal provides an interface for users to input instructions to the generative model. The UI includes an input field, a submit button, and a list of past inputs and suggested prompts. As the user begins to input a prompt, the terminal incrementally sends requests to the server and displays the suggestions returned by the server in real time.

[0116] User Action

[0117] The user inputs the task they want the generative AI model to perform as a prompt. The specific operations are as follows:

[0118] 1. Prompt Input

[0119] The user inputs specific instructions to the generative model, such as "Show me a picture of a cat."

[0120] 2. Proposal acceptance

[0121] The user reviews the suggestions displayed on the device, selects the appropriate one, optionally customizes the suggested prompts, and sends them to the generative model.

[0122] Specific examples

[0123] Success stories

[0124] When a user types "I want to see pictures of animals," the device sends this partial prompt to the server. The server analyzes past successes and suggests successful prompts, such as "Show me pictures of cats" or "Show me pictures of dogs." The device displays these suggestions to the user, who then selects "Show me pictures of cats."

[0125] Failure example

[0126] If the user only types "animals," the server may not be able to properly handle this vague prompt. However, the system's suggestion feature can suggest more specific prompts, such as "Show me a picture of a cat," allowing the user to provide specific instructions.

[0127] In this way, the system of the present invention helps users input appropriate instructions into the generative model, enabling efficient use of the generative AI model.

[0128] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0129] Step 1: The user inputs instructions into the generative model

[0130] The user inputs the task they want to perform as a prompt to the generative model through the device's user interface. For example, the input might be "Show me pictures of cats." This input is reflected in real time on the device.

[0131] Step 2: The terminal sends partial input to the server

[0132] As the user begins to type the prompt, the terminal incrementally sends the partial prompt to the server. For example, as soon as the user types "cat's", the partial prompt is sent to the server. The terminal sends partial input (e.g., "cat's") to the server and waits for the result after each input.

[0133] Step 3: The server performs data analysis

[0134] Based on the partial input received, the server analyzes the previous prompts and their results in the database using text analysis algorithms such as morphological analysis and TF-IDF. Specifically, it extracts previous prompts (e.g., "Show me pictures of cats") and classifies them as successful or unsuccessful.

[0135] Step 4: The server uses the proposed generative model

[0136] Based on the results of the data analysis, the server uses an NLP model that has learned common patterns in successful cases to generate appropriate suggested prompts. Specifically, it converts successful prompts into numerical vectors using TF-IDF vectorization and K-means clustering, and groups similar prompts. For this partial input, it generates suggestions such as "Show me a photo of a cat" or "Show me an image of a cat."

[0137] Step 5: The server sends the proposal to the device

[0138] The generated suggestion prompt is sent from the server to the device. The server sends the suggestion prompt (e.g., "Show me pictures of cats") to the device.

[0139] Step 6: The device displays the suggestion to the user

[0140] The device displays the received suggested prompts in the user interface, and the user can review the suggested prompts (e.g., "Show me pictures of cats") and select or modify them as needed.

[0141] Step 7: User selects / modifies final prompt

[0142] The user selects or modifies the suggested prompts and finalizes the instructions to be sent to the generative model, for example, "Show me pictures of cats."

[0143] Step 8: The device sends the final prompt to the generative model

[0144] The final prompt is sent from the device to the generative AI model. The device then sends the final prompt (e.g., "Show me a picture of a cat") to the generative model.

[0145] Step 9: The generative AI model generates and outputs results

[0146] The generative AI model processes the prompts it receives and generates an appropriate result (e.g., a photo of a cat), which is then served to the user.

[0147] The above is the specific processing flow of this system.

[0148] (Application example 1)

[0149] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0150] In online shopping, it is important for users to enter specific and appropriate search instructions to smoothly search for and purchase products. However, many users enter vague instructions or incomplete prompts, which results in a long time to find the desired product and a poor user experience. In addition, existing systems lack the functionality to provide appropriate suggestions to users based on past success stories. This leads to issues such as reduced product search efficiency and lower satisfaction.

[0151] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0152] In this invention, the server includes input means for a user to input instructions to the generative model, database means for saving data on past instructions and their results, analysis means for analyzing the saved past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, interface means for sending a request to the server in real time when the user inputs partial instructions and displaying the suggested instructions on the screen, and product suggestion means for generating specific product suggestions based on past search data when the user searches for products. This allows the user to receive appropriate suggestions even if they only input vague instructions, enabling them to quickly find the product they are looking for.

[0153] The "input means" is an interface function that allows the user to input instructions to the generative model. The user can use this means to specify any prompt.

[0154] The "database means" is a function that records and saves past user instructions and their results. This allows past search history and success / failure cases to be accumulated.

[0155] The "analysis means" is a function that analyzes saved past instructions and their results, and classifies success cases and failure cases. This allows for the extraction of patterns in instruction content and data analysis.

[0156] The "suggestion means" is a function that suggests new instructions to the user based on the classified success cases, thereby enabling the user to easily input appropriate instructions.

[0157] The "interface means" is a function that, when a user inputs partial instructions, sends a request to the server in real time and displays suggested instructions on the screen, allowing the user to receive appropriate suggestions in real time.

[0158] The "product suggestion means" is a function that generates specific product suggestions based on past search data when a user searches for a product, allowing the user to efficiently find the product they are looking for.

[0159] MODE FOR CARRYING OUT THE INVENTION

[0160] The present invention provides a system that provides appropriate suggestions when a user inputs instructions to a generative model. The system mainly includes a server, a terminal, and a user.

[0161] Server Processing

[0162] The server provides the following functionality:

[0163] 1. Data Collection

[0164] The server collects the instructions that the user has previously input into the generative model, the results, and the user's feedback. This data is stored in a database. The server often uses cloud infrastructure (e.g., AWS EC2 or Google Cloud Compute Engine).

[0165] 2. Data analysis

[0166] The server analyzes the stored past commands and their results. A text analysis algorithm (natural language processing model) is used to extract the characteristics of the command content and results. The command is then classified as successful or unsuccessful. This process is carried out using Python and the Scikit-learn library.

[0167] 3. Training the prompt generation model

[0168] The server trains a natural language processing (NLP) model to learn common patterns among successful cases, converting successful prompts into numerical vectors using TF-IDF vectorization and applying a K-means clustering algorithm to group similar prompts.

[0169] 4. Real-time suggestions

[0170] As users enter partial instructions, the server generates appropriate prompt suggestions in real time, leveraging trained NLP models.

[0171] Terminal handling

[0172] The terminal provides the following features:

[0173] 1. User Interface (UI)

[0174] The terminal provides an interface for the user to input instructions to the generative model. The UI includes input fields, a submit button, and a list of past inputs and suggested prompts.

[0175] 2. Real-time assistance

[0176] As the user enters prompts, the terminal incrementally sends requests to the server and displays suggestions returned by the server in real time, allowing the user to select or modify the suggested prompts.

[0177] User Action

[0178] The user performs the following steps:

[0179] 1. Prompt Input

[0180] The user inputs a prompt to the generative model, describing the task they want it to perform, such as a specific instruction like "a red shirt."

[0181] 2. Proposal acceptance

[0182] The user reviews the suggestions displayed on the device, selects the appropriate one, and optionally customizes the suggested prompts before sending them to the generative model.

[0183] Specific examples

[0184] As a concrete example, here is the steps a user takes to search for a "red shirt":

[0185] 1. User types "red shirt"

[0186] Using the device's UI, the user enters the prompt "red shirt." Once the input is complete, the device sends this partial instruction to the server.

[0187] 2. The server parses the prompt and generates suggestions

[0188] The server generates specific suggestions such as "men's red shirt, size large" or "red casual shirt" based on past success stories.

[0189] 3. A suggested prompt will appear on the screen

[0190] The device displays suggested prompts to the user, who can then select or customize the appropriate suggestion and send it to the generative model.

[0191] In this way, the user can receive specific suggestions even when the user gives vague instructions, and can efficiently find the desired product.

[0192] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0193] Step 1:

[0194] User prompt input

[0195] The user uses the terminal's user interface to enter a prompt to search for a specific product (e.g., "red shirt"), and the input field allows the user to enter any text. The entered prompt is transmitted to the server in real time.

[0196] Step 2:

[0197] Data collection by the server

[0198] The server receives the input prompt and simultaneously collects from the database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those instructions. For example, it collects examples of success and failure when the prompt "red shirt" was input in the past.

[0199] Step 3:

[0200] Data analysis by server

[0201] The server analyzes the collected data. It uses a text analysis algorithm to extract features of the instructions and results and classify them into success cases and failure cases. Specifically, it converts the prompts into numerical vectors using TF-IDF vectorization and applies the K-means clustering algorithm to group similar prompts. The input is the collected historical data, and the output is the classified clusters and their features.

[0202] Step 4:

[0203] Training a prompt generation model

[0204] The server trains a natural language processing (NLP) model to learn common patterns in successful cases. It uses past successful prompts to improve the model, thereby increasing the accuracy of its suggestions. The input is the analyzed successful prompts, and the output is the trained model.

[0205] Step 5:

[0206] Real-time proposal generation

[0207] The server generates appropriate prompt suggestions in real time based on the partial instructions entered by the user. It uses a trained NLP model to generate specific suggestions (e.g., "men's red shirt, size large," "red casual shirt") for the user's "red shirt" input. The input is the partial instructions from the user, and the output is the suggested specific prompt.

[0208] Step 6:

[0209] Display suggestions on the device

[0210] The terminal displays the suggestions returned by the server in real time. The user can review the suggested prompts and select an appropriate one or customize them as needed. Based on the displayed suggestions, the user can modify the input fields and send the request to the server again. The input is the suggestion from the server, and the output is the specific prompt presented to the user.

[0211] Step 7:

[0212] User's last input and search execution

[0213] The user finally confirms the selected or modified prompts and sends them to the generative model, which then returns specific search results and displays them to the user. The input is the prompts confirmed by the user, and the output is the search results from the generative model.

[0214] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0215] ---

[0216] The present invention is a system that supports users in inputting appropriate instructions to a generative model, and further combines it with an emotion engine that recognizes the user's emotions. This system is implemented through a series of procedures including interactions between a server, a terminal, and a user.

[0217] Server Processing

[0218] 1. Data Collection

[0219] The server collects and stores in a database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data also includes the user's emotional information recognized by the emotion engine.

[0220] 2. Data analysis

[0221] The server analyzes the instructions and results collected from the database. It uses a text analysis algorithm to extract characteristics of the instructions and results. It then categorizes them into success and failure cases and evaluates the effectiveness of the suggestions using the user's emotional information.

[0222] 3. Training the prompt generation model

[0223] The server trains a natural language processing (NLP) model to extract common patterns from success stories. It converts success stories into numerical vectors using TF-IDF vectorization and groups similar prompts using the K-means clustering algorithm. Sentiment data provided by the emotion engine is also fed into this training model.

[0224] 4. Real-time suggestions

[0225] For partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion engine.

[0226] Terminal handling

[0227] 1. User Interface (UI)

[0228] The device provides an interface for the user to input instructions to the generative model, including input fields, a submit button, a list of past inputs and suggested prompts, and even a feedback display showing the user's emotional state.

[0229] 2. Real-time assistance

[0230] When the user inputs the prompt, the device sends partial instructions to the server in real time. At the same time, the emotion engine recognizes emotions from the user's voice or text input and sends the emotion information to the server. The server then receives suggestions and displays them on the interface.

[0231] User Action

[0232] 1. Prompt Input

[0233] The user inputs a prompt to the generative model, describing the task they want to accomplish, such as "Show me a picture of a cat."

[0234] 2. Proposal acceptance

[0235] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotional information recognized by the emotion engine is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[0236] Specific examples

[0237] 1. Success stories

[0238] If a user inputs "I want to see pictures of animals" and the emotion engine recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[0239] 2. Failure example

[0240] If the user only types "animals," and the emotion engine recognizes this as a "confused" state, the server will suggest more specific instructions to the user to properly handle this ambiguous prompt. For example, it will help the user to provide more specific instructions using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?"

[0241] In this way, the system of the present invention allows users to efficiently input appropriate instructions to the generative model to achieve the desired results, and by taking into account the user's emotional state, the system can provide more personalized suggestions and improve the user experience.

[0242] The processing flow will be explained below.

[0243] ---

[0244] Step 1:

[0245] The server collects and stores in a database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those inputs, including the user's emotional information.

[0246] Step 2:

[0247] The server periodically analyzes the stored data. First, it uses a text analysis algorithm to analyze past instructions and their results, extracting characteristics of the instruction content and results. It then classifies cases into successes and failures.

[0248] Step 3:

[0249] The server trains a natural language processing (NLP) model to extract common patterns from success stories, converts success stories into numerical vectors using TF-IDF vectorization, and groups similar prompts using the K-means clustering algorithm.

[0250] Step 4:

[0251] The user inputs instructions to the generative model through a terminal, for example, "Show me a picture of a cat."

[0252] Step 5:

[0253] The emotion engine recognizes emotions from the user's input voice or text, and obtains emotion information such as whether the user is excited or confused.

[0254] Step 6:

[0255] The terminal transmits the partial instructions and emotion information input by the user to the server in real time.

[0256] Step 7:

[0257] The server generates appropriate prompt suggestions using existing NLP models based on partial instructions and emotional information, searches for similar success stories in real time, and customizes the suggestions taking into account the user's emotions.

[0258] Step 8:

[0259] The server sends the generated suggestions back to the device, which receives them and displays them as suggestions in the user interface. For example, suggestions might be "Show me pictures of cats" or "Show me pictures of dogs."

[0260] Step 9:

[0261] The user reviews the suggested prompts, selects the appropriate one, and customizes the prompt if necessary. Emotional information provided by the emotion engine is also displayed, and the user makes their selection based on that feedback.

[0262] Step 10:

[0263] The final instructions selected by the user are sent to the generative model, which generates results based on the user's specifications and returns the results to the user via the terminal.

[0264] Step 11:

[0265] The user reviews the generated results and provides feedback via their device to the server, which stores this feedback in a database for subsequent analysis and training.

[0266] Through this series of steps, the system of the present invention allows users to efficiently input appropriate instructions to the generative model and provides personalized suggestions that reflect the user's feelings, thereby enabling users to achieve the results they desire and improving their experience using the generative model.

[0267] Example 2

[0268] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0269] Conventional systems for inputting instructions to generative models lack support for users to input accurate instructions, making it difficult to obtain appropriate results. Furthermore, suggestions do not take into account the user's emotional state, which results in a lack of improvement in the user experience. The present invention aims to recognize the user's emotions and reflect them in instruction suggestions, thereby enabling users to efficiently input appropriate instructions to generative models and obtain desired results.

[0270] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0271] In this invention, the server includes input means for the user to input instructions to the generative model, database means for saving data on past instructions and their results, analysis means for analyzing the saved past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, interface means for sending the instructions input by the user to the server in real time and displaying the suggested instructions on a screen, and an emotion engine for collecting emotional information from the user and reflecting it in the analysis means and suggestion means. This not only enables the user to efficiently input appropriate instructions and obtain the desired results, but also enables personalized suggestions that take emotional information into account.

[0272] An "input means" is a device or interface that allows a user to input instructions to a generative model.

[0273] "Database means" refers to a system or storage device for storing data on past instructions and their results.

[0274] An "analysis tool" is an algorithm or system that analyzes stored past instructions and their results and classifies successes and failures.

[0275] The "suggestion means" is a device or system for suggesting new instructions to the user based on the classified success cases.

[0276] "Interface means" refers to a device or interface for transmitting instructions entered by a user to a server in real time and displaying suggested instructions.

[0277] An "emotion engine" is a system or algorithm that collects and analyzes user emotional information and reflects it in analytical and suggestion means.

[0278] "Emotional information" is data indicative of a user's emotional state, obtained from speech, text, or other forms of input.

[0279] The present invention is a system that supports users in inputting appropriate instructions to a generative model, and further combines it with an emotion engine that recognizes the user's emotions. This system is implemented through a series of procedures including interactions between a server, a terminal, and the user.

[0280] Server Processing

[0281] 1. Data Collection

[0282] The server stores the instructions that the user has previously input into the generative model, the results, and the user's feedback in a database. This data also includes the user's emotional information recognized by the emotion engine. The database uses a commercial database engine (e.g., MySQL, PostgreSQL, etc.).

[0283] 2. Data analysis

[0284] The server analyzes the instructions collected from the database and their results. It uses text analysis algorithms to extract features of the instructions and results. Natural language processing (NLP) libraries (e.g., spaCy, NLTK, etc.) are used for the analysis. Furthermore, it classifies success and failure cases and evaluates the effectiveness of suggestions using user sentiment information.

[0285] 3. Training the prompt generation model

[0286] The server trains a natural language processing model to extract common patterns from success stories. It converts success stories into numerical vectors using Term Frequency-Inverse Document Frequency (TF-IDF) vectorization and groups similar prompts using the K-means clustering algorithm. Emotion data provided by the emotion engine is also fed into this training model.

[0287] 4. Real-time suggestions

[0288] For partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion engine.

[0289] Terminal handling

[0290] 1. User Interface (UI)

[0291] The device provides an interface for the user to input instructions to the generative model. The UI includes input fields, a submit button, a history of past inputs, a list of suggested prompts, and a feedback display showing the user's emotional state. The interface is built using web-based technologies (e.g., HTML, CSS, JavaScript).

[0292] 2. Real-time assistance

[0293] As the user enters the prompts, the device sends partial instructions to the server in real time. At the same time, the emotion engine recognizes emotions from the user's voice and text input and sends the emotion information to the server. Suggestions from the server are sent to the device and displayed in the interface. This process is realized using technologies that enable two-way communication (e.g., WebSocket, HTTP / 2).

[0294] User Action

[0295] 1. Prompt Input

[0296] The user inputs a prompt to the generative model, specifying the task they want to accomplish, such as "Show me a picture of a cat."

[0297] 2. Proposal acceptance

[0298] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotional information recognized by the emotion engine is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[0299] Specific examples

[0300] 1. Success stories

[0301] If a user inputs "I want to see pictures of animals" and the emotion engine recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[0302] 2. Failure example

[0303] If the user only types "animals" and the emotion engine recognizes the "confused" state, the server will suggest more specific instructions to the user to handle this ambiguous prompt appropriately, for example, using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?" to help the user provide more specific instructions.

[0304] Prompt Sentence Examples

[0305] "Can you recommend some day trips I can take next weekend?"

[0306] I want to know some delicious pasta recipes.

[0307] "Tell me about the latest technology trends"

[0308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0309] Step 1: User enters prompt text

[0310] Specific operation: The user inputs instructions for the generative model into the device's user interface (UI). For example, "Tell me recommended places for my next vacation."

[0311] Input: The user types the prompt text into the input field.

[0312] Output: The entered prompt is sent to the terminal as a string of data.

[0313] Step 2: Acquiring emotional information

[0314] Specific operation: The emotion engine installed in the device analyzes the emotion from the user's voice and text input. Example: The user's input is determined to be "excited."

[0315] Input: User voice and text data.

[0316] Output: Analyzed emotion information (e.g., "excited") is obtained.

[0317] Step 3: Sending data

[0318] Specific operation: The terminal sends the input prompt sentence and emotion information to the server.

[0319] Input: prompt text and emotion information.

[0320] Output: The prompt sentence and emotion information are sent to the server as a data packet.

[0321] Step 4: Save your data

[0322] Specific operation: The server stores the received prompt sentence and emotion information in the database. Example: A new record is added to the database.

[0323] Input: prompt sentence and emotion information.

[0324] Output: The new record saved in the database.

[0325] Step 5: Data acquisition and analysis

[0326] Specific operation: The server retrieves past instructions and their results from the database and applies a text analysis algorithm, e.g., converting past prompt sentences into TF-IDF vectors and extracting their features.

[0327] Input: Past instructions and result data retrieved from the database.

[0328] Output: The parsed feature vector.

[0329] Step 6: Categorize success stories and failure stories

[0330] Specific operation: The server classifies cases into success and failure based on the analyzed instructions. Example: User feedback is classified as "success" or "failure."

[0331] Input: Parsed feature vectors and feedback data.

[0332] Output: A dataset categorized into success and failure cases.

[0333] Step 7: Train the prompt generation model

[0334] What it does: The server trains a natural language processing (NLP) model to extract common patterns from success stories, e.g., grouping similar prompt sentences using TF-IDF vectors and K-means clustering.

[0335] Input: A dataset of success stories.

[0336] Output: A trained NLP model.

[0337] Step 8: Integrating Emotional Data

[0338] Specific operation: The server also incorporates the emotion data provided by the emotion engine into the training model. For example, it integrates the prompt sentence for the "excited" state and its success pattern into the NLP model.

[0339] Input: Sentiment data and a trained NLP model.

[0340] Output: An integrated model reflecting the emotion data.

[0341] Step 9: Generate real-time suggestions

[0342] What it does: Based on the user's partial input, the server generates appropriate prompt suggestions in real time. For example, if you enter "travel," it generates a complete prompt such as "Tell me some recommended places."

[0343] Input: Partial prompt sentence and integrated model.

[0344] Output: A suggested prompt.

[0345] Step 10: Submit and view your proposal

[0346] Specific operation: The server sends the generated proposal to the terminal, and the terminal displays the proposal to the user on the interface.

[0347] Input: A suggested prompt sentence.

[0348] Output: Proposals displayed on the screen.

[0349] Step 11: Accepting the proposal

[0350] What happens: The user reviews the suggestions displayed on the device and selects the appropriate one or customizes it as needed.

[0351] Input: On-screen suggestions.

[0352] Output: The prompt statement that the user finally selected.

[0353] Step 12: Sending final instructions

[0354] Specific behavior: The final prompt sentence selected by the user is sent to the generative model.

[0355] Input: The final prompt statement selected by the user.

[0356] Output: The prompt sentence sent to the generative model.

[0357] (Application example 2)

[0358] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0359] Conventional instruction input systems for generative models have had problems such as users being unable to input appropriate prompts and not taking into account emotional states, resulting in a poor user experience. In particular, when dealing with customers in brick-and-mortar stores, it has been difficult to improve customer satisfaction with customer service methods that ignore the emotional states of customers.

[0360] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0361] In this invention, the server includes input means for a user to input instructions to the generative model, database means for storing data on past instructions and their results, analysis means for analyzing the stored past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, emotion recognition means for recognizing the emotional state of the user, and interface means for transmitting the instructions input by the user to the server in real time and displaying the suggested instructions on the screen. This not only enables the user to input appropriate prompts to the generative model, but also enables personalized suggestions based on the user's emotional state.

[0362] The "input means" is an interface through which the user inputs instructions to the generative model.

[0363] The "database means" is a system or device for storing instructions previously input by the user and the results thereof.

[0364] The "analysis means" is a device or program for analyzing stored past instructions and their results, and classifying success cases and failure cases.

[0365] The "suggestion means" is a device or program for suggesting new instructions to the user based on the success cases classified by the analysis means.

[0366] The "emotion recognition means" is a device or program for detecting and recognizing the emotional state of a user from their voice and facial expression.

[0367] The "interface means" is a device or program for transmitting instructions input by the user to the server in real time and displaying suggestions from the server on the screen.

[0368] A "server" is a central processing unit that stores instructions and result data, performs analysis, and provides suggestions to users.

[0369] A "generative model" is an artificial intelligence or machine learning model that generates output based on user instructions.

[0370] The "emotional state" is information that represents the user's emotions, and is recognized via emotion recognition means.

[0371] A "prompt" is text in the form of an instruction or question to be posed to the generative model.

[0372] MODE FOR CARRYING OUT THE INVENTION

[0373] The present invention provides a system that assists users in inputting appropriate instructions to a generative model, and further combines an emotion recognition function that recognizes the user's emotional state. This system is implemented through a series of procedures including interactions between a server, a terminal, and a user.

[0374] Server Processing

[0375] The server has the following main processes:

[0376] 1. Data Collection

[0377] The server collects and stores in a database the instructions that users have previously input to the generative model, the results of those inputs, and feedback on those instructions. This data also includes emotion information recognized by the emotion recognition means.

[0378] 2. Data analysis

[0379] The server analyzes the instructions and their results collected from the database. It uses text analysis algorithms and machine learning models (e.g., TensorFlow) to extract features of the instructions and results. It also classifies success and failure cases and evaluates the effectiveness of the suggestions using user sentiment.

[0380] 3. Training the prompt generation model

[0381] The server trains a natural language processing (NLP) model to extract common patterns from success stories. It converts success stories into numerical vectors using TF-IDF vectorization and groups similar prompts using the K-means clustering algorithm. Emotion data provided by the emotion recognizer is also fed into this training model.

[0382] 4. Real-time suggestions

[0383] In response to partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion recognition means.

[0384] Terminal handling

[0385] The terminal has the following main processes:

[0386] 1. User Interface (UI)

[0387] The device provides an interface for the user to input instructions to the generative model, including input fields, a submit button, a list of past inputs and suggested prompts, and even a feedback display showing the user's emotional state.

[0388] 2. Real-time assistance

[0389] When the user inputs a prompt, the device sends partial instructions to the server in real time. At the same time, the emotion recognition means recognizes emotions from the user's voice or text input, and sends the emotion information to the server. The device receives suggestions from the server and displays them on the interface.

[0390] User Action

[0391] The user has the following main processes:

[0392] 1. Prompt Input

[0393] The user inputs a prompt to the generative model, describing the task they want to accomplish, such as "Show me a picture of a cat."

[0394] 2. Proposal acceptance

[0395] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotion information recognized by the emotion recognition means is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[0396] Specific examples

[0397] 1. Success stories

[0398] If the user inputs "I want to see pictures of animals" and the emotion recognition means recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[0399] 2. Failure example

[0400] If the user only types "animals" and the emotion recognizer recognizes the "confused" state, the server will suggest more specific instructions to the user to handle this ambiguous prompt appropriately. For example, it will help the user to provide more specific instructions using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?"

[0401] Prompt Sentence Examples

[0402] 1. Success prompt

[0403] "A customer seems confused. Would you like to ask about a specific item?"

[0404] 2. Failure prompt

[0405] "The client seems anxious. Would you like me to check in and reassure them about any concerns?"

[0406] In this way, the system of the present invention allows users to efficiently input appropriate instructions to the generative model to achieve the desired results, and by taking into account the user's emotional state, the system can provide more personalized suggestions and improve the user experience.

[0407] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0408] Step 1:

[0409] The terminal provides an interface where the user can input the task they want to perform on the generative model as a prompt. For example, the user can enter "I want to see pictures of animals" in the input field. The input data is a string, which is sent to the server in real time.

[0410] Step 2:

[0411] The server receives the character string data sent from the terminal. At the same time, it uses emotion recognition means to analyze and recognize the user's emotional state from their voice input and facial expression data, and sends this emotional information together to the server. The server then stores the character string and the emotional state in a database.

[0412] Step 3:

[0413] The server retrieves and analyzes the stored past instruction data and user sentiment information. Using text analysis algorithms and machine learning models (e.g., TensorFlow), it extracts features of the instruction content and results and classifies them into success cases and failure cases. The output of this analysis is a list of similar success cases.

[0414] Step 4:

[0415] The server generates candidate prompts based on the success cases. The generated prompts also reflect the user's emotional information obtained from the emotion recognition means. For example, if the user is in an "excited" state, suggested prompts include "Show me a picture of a cat" or "Show me an image of a dog." The candidate prompts are sent to the terminal in real time.

[0416] Step 5:

[0417] The device displays the candidate prompts sent from the server on its interface. The user checks the displayed candidate prompts and selects the appropriate one. If the user selects "Show me pictures of cats," the selection data is sent from the device to the server and used as the final instruction for the generative model.

[0418] Step 6:

[0419] The server sends the final prompt selected by the user to the generative model and obtains the output from the generative model. This output is the result data (e.g., a list of cat images) generated based on the specified conditions. This result data is then sent to the terminal to be presented to the user.

[0420] Step 7:

[0421] The terminal displays the output of the generative model received from the server on its interface, allowing the user to check the displayed results and provide further operations or instructions as necessary.

[0422] The above processing steps enable users to efficiently input appropriate instructions to the generative model and receive personalized suggestions based on their emotional state.

[0423] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0424] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0425] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0426] [Second embodiment]

[0427] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0428] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0429] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0430] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0431] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0432] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0433] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0434] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0435] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0436] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0437] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0438] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0439] ---

[0440] The present invention provides a system that supports users in inputting appropriate instructions to a generative model. The system is implemented through a series of procedures involving interactions between a server, a terminal, and a user.

[0441] Server Processing

[0442] 1. Data Collection

[0443] The server collects the instructions that users have previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data is stored in a database.

[0444] 2. Data analysis

[0445] The server analyzes the instructions and results collected from the database using text analysis algorithms to extract characteristics of the instructions and results, and then classifies the instructions as successful or unsuccessful.

[0446] 3. Training the prompt generation model

[0447] The server trains a natural language processing (NLP) model to learn common patterns among success prompts. Specifically, it converts success prompts into numerical vectors using TF-IDF vectorization and applies a K-means clustering algorithm to group similar prompts.

[0448] 4. Real-time suggestions

[0449] As users enter partial instructions, the server generates appropriate prompt suggestions in real time, leveraging trained NLP models.

[0450] Terminal handling

[0451] 1. User Interface (UI)

[0452] The terminal provides an interface for the user to input instructions to the generative model. This UI includes input fields, a submit button, and a list of past inputs and suggested prompts.

[0453] 2. Real-time assistance

[0454] As the user enters prompts, the terminal incrementally sends requests to the server and displays suggestions returned by the server in real time, allowing the user to select or modify the suggested prompts.

[0455] User Action

[0456] 1. Prompt Input

[0457] The user inputs a prompt to the generative model, specifying the task they want it to perform, such as "Show me a picture of a cat."

[0458] 2. Proposal acceptance

[0459] The user reviews the suggestions displayed on the device, selects the appropriate one, and optionally customizes the suggested prompts before sending them to the generative model.

[0460] Specific examples

[0461] 1. Success stories

[0462] When a user types "I want to see pictures of animals," the device sends this partial prompt to the server. The server analyzes past successes and suggests successful prompts, such as "Show me pictures of cats" or "Show me pictures of dogs." The device displays these suggestions to the user, who then selects "Show me pictures of cats."

[0463] 2. Failure example

[0464] If the user only types "animals," the server may not be able to properly handle this ambiguous prompt. However, the system's suggestion feature can suggest more specific prompts so the user can provide specific instructions.

[0465] In this way, the system of the present invention helps the user input appropriate instructions into the generative model, enabling efficient use.

[0466] ---

[0467] The processing flow will be explained below.

[0468] ---

[0469] Step 1:

[0470] The server collects and stores in a database the instructions that users have previously input to the generative model and the results of those inputs. This data also includes user feedback information (evaluation of success / failure).

[0471] Step 2:

[0472] The server periodically analyzes the stored data, first retrieving past instructions and their results, then using a text analysis algorithm to extract the characteristics of each instruction, and then classifying them into success cases and failure cases.

[0473] Step 3:

[0474] The server trains a natural language processing model (NLP model) to extract common patterns from success stories, converting success stories into numerical vectors using TF-IDF vectorization and grouping similar prompts using the K-means clustering algorithm.

[0475] Step 4:

[0476] The user inputs instructions to the generative model through the terminal. When the user starts inputting, the terminal sends partial instructions to the server in real time.

[0477] Step 5:

[0478] The server uses existing NLP models to generate appropriate prompts based on the partial instructions received, and searches for similar success stories in real time to generate appropriate suggestions.

[0479] Step 6:

[0480] The server sends the generated proposals back to the terminal, which receives them and displays them in its user interface.

[0481] Step 7:

[0482] The user reviews the suggested prompts, selects appropriate ones, or customizes them if necessary, after which the final selected instructions are sent to the generative model.

[0483] Step 8:

[0484] The generative model generates results based on the user's specifications and returns the results to the device, which then displays the generated results to the user.

[0485] Step 9:

[0486] The user checks the displayed results and provides feedback to the server via the terminal, which stores this feedback in a database for subsequent analysis.

[0487] This series of steps allows the system of the present invention to allow the user to efficiently input appropriate instructions to the generative model and obtain the desired results.

[0488] Example 1

[0489] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0490] In recent years, generative AI models have been used in a wide range of applications, but it is not easy for users to input appropriate instructions into generative models. In particular, when users input ambiguous instructions, it is difficult for the generative model to provide the expected results. This leads to a mismatch between the user's intention and the results, resulting in a poor user experience. In addition, there is a need for a system that can efficiently complement instructions by utilizing past success stories and suggest appropriate instructions to the user.

[0491] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0492] In this invention, the server includes a data storage means, a data analysis means, a proposal generation means, and a user interface means, which allows a user to input appropriate instructions to the generative model, enabling efficient use of the generative AI model.

[0493] "Input means" refers to a device or software that allows a user to input instructions to a generative model.

[0494] "Data storage means" refers to a device or software for recording and storing instructions that a user has previously input into a generative model and the results thereof.

[0495] "Data analysis means" refers to a device or software that analyzes stored past instructions and their results and classifies successful cases and unsuccessful cases.

[0496] The "suggestion generation means" refers to a device or software for suggesting new instructions to the user based on the success cases classified by the data analysis means.

[0497] "User interface means" refers to a device or software for transmitting instructions entered by a user to a server in real time and displaying suggested instructions on the user's screen.

[0498] A "generative AI model" refers to an artificial intelligence system that generates text, images, audio, etc. based on user input.

[0499] A "prompt sentence" refers to an input sentence that instructs a generative AI model to generate something specific.

[0500] A "success case" refers to an example in which a generative AI model produced the expected results in response to user instructions.

[0501] A "failure case" refers to an instance in which the generative AI model did not produce the expected result in response to a user's instructions.

[0502] The present invention is a system that supports users in inputting appropriate instructions to a generative AI model. This system is composed of a server, a terminal, and a series of processes including user interaction.

[0503] Server Processing

[0504] The server uses the data storage means, data analysis means, and suggestion generation means to analyze the prompts entered by the user and make appropriate suggestions, as follows:

[0505] 1. Data storage method

[0506] The server collects the instructions that users have previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data is stored in a database.

[0507] 2. Data analysis methods

[0508] The server analyzes the instructions and results collected from the database. For the analysis, it uses text analysis algorithms (e.g., morphological analysis, TF-IDF, etc.) to extract characteristics of the instruction content and results. The server classifies the instruction as successful or unsuccessful.

[0509] 3. Proposal generation means

[0510] The server trains a natural language processing (NLP) model to learn common patterns among success prompts. Specifically, it converts success prompts into numerical vectors using TF-IDF vectorization and applies a K-means clustering algorithm to group similar prompts. For partial instructions entered by the user, the server generates appropriate prompt suggestions in real time.

[0511] Terminal handling

[0512] The terminal provides an interface for a user to input instructions to the generative model using a user interface means, specifically as follows.

[0513] 1. User Interface Methods

[0514] The terminal provides an interface for users to input instructions to the generative model. The UI includes an input field, a submit button, and a list of past inputs and suggested prompts. As the user begins to input a prompt, the terminal incrementally sends requests to the server and displays the suggestions returned by the server in real time.

[0515] User Action

[0516] The user inputs the task they want the generative AI model to perform as a prompt. The specific operations are as follows:

[0517] 1. Prompt Input

[0518] The user inputs specific instructions to the generative model, such as "Show me a picture of a cat."

[0519] 2. Proposal acceptance

[0520] The user reviews the suggestions displayed on the device, selects the appropriate one, optionally customizes the suggested prompts, and sends them to the generative model.

[0521] Specific examples

[0522] Success stories

[0523] When a user types "I want to see pictures of animals," the device sends this partial prompt to the server. The server analyzes past successes and suggests successful prompts, such as "Show me pictures of cats" or "Show me pictures of dogs." The device displays these suggestions to the user, who then selects "Show me pictures of cats."

[0524] Failure example

[0525] If the user only types "animals," the server may not be able to properly handle this vague prompt. However, the system's suggestion feature can suggest more specific prompts, such as "Show me a picture of a cat," allowing the user to provide specific instructions.

[0526] In this way, the system of the present invention helps users input appropriate instructions into the generative model, enabling efficient use of the generative AI model.

[0527] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0528] Step 1: The user inputs instructions into the generative model

[0529] The user inputs the task they want to perform as a prompt to the generative model through the device's user interface. For example, the input might be "Show me pictures of cats." This input is reflected in real time on the device.

[0530] Step 2: The terminal sends partial input to the server

[0531] As the user begins to type the prompt, the terminal incrementally sends the partial prompt to the server. For example, as soon as the user types "cat's", the partial prompt is sent to the server. The terminal sends partial input (e.g., "cat's") to the server and waits for the result each time.

[0532] Step 3: The server performs data analysis

[0533] Based on the partial input received, the server analyzes the previous prompts and their results in the database using text analysis algorithms such as morphological analysis and TF-IDF. Specifically, it extracts previous prompts (e.g., "Show me pictures of cats") and classifies them as successful or unsuccessful.

[0534] Step 4: The server uses the proposed generative model

[0535] Based on the results of the data analysis, the server uses an NLP model that has learned common patterns in successful cases to generate appropriate suggested prompts. Specifically, it converts successful prompts into numerical vectors using TF-IDF vectorization and K-means clustering, and groups similar prompts. For this partial input, it generates suggestions such as "Show me a photo of a cat" or "Show me an image of a cat."

[0536] Step 5: The server sends the proposal to the device

[0537] The generated suggestion prompt is sent from the server to the device. The server sends the suggestion prompt (e.g., "Show me pictures of cats") to the device.

[0538] Step 6: The device displays the suggestion to the user

[0539] The device displays the received suggested prompts in the user interface, and the user can review the suggested prompts (e.g., "Show me pictures of cats") and select or modify them as needed.

[0540] Step 7: User selects / modifies final prompt

[0541] The user selects or modifies the suggested prompts and finalizes the instructions to be sent to the generative model, for example, "Show me pictures of cats."

[0542] Step 8: The device sends the final prompt to the generative model

[0543] The final prompt is sent from the device to the generative AI model. The device then sends the final prompt (e.g., "Show me a picture of a cat") to the generative model.

[0544] Step 9: The generative AI model generates and outputs results

[0545] The generative AI model processes the prompts it receives and generates an appropriate result (e.g., a photo of a cat), which is then served to the user.

[0546] The above is the specific processing flow of this system.

[0547] (Application example 1)

[0548] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0549] In online shopping, it is important for users to enter specific and appropriate search instructions to smoothly search for and purchase products. However, many users enter vague instructions or incomplete prompts, which results in a long time to find the desired product and a poor user experience. In addition, existing systems lack the functionality to provide appropriate suggestions to users based on past success stories. This leads to issues such as reduced product search efficiency and lower satisfaction.

[0550] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0551] In this invention, the server includes input means for a user to input instructions to the generative model, database means for saving data on past instructions and their results, analysis means for analyzing the saved past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, interface means for sending a request to the server in real time when the user inputs partial instructions and displaying the suggested instructions on the screen, and product suggestion means for generating specific product suggestions based on past search data when the user searches for products. This allows the user to receive appropriate suggestions even if they only input vague instructions, enabling them to quickly find the product they are looking for.

[0552] The "input means" is an interface function that allows the user to input instructions to the generative model. The user can use this means to specify any prompt.

[0553] The "database means" is a function that records and saves past user instructions and their results. This allows past search history and success / failure cases to be accumulated.

[0554] The "analysis means" is a function that analyzes saved past instructions and their results, and classifies success cases and failure cases. This allows for the extraction of patterns in instruction content and data analysis.

[0555] The "suggestion means" is a function that suggests new instructions to the user based on the classified success cases, thereby enabling the user to easily input appropriate instructions.

[0556] The "interface means" is a function that, when a user inputs partial instructions, sends a request to the server in real time and displays suggested instructions on the screen, allowing the user to receive appropriate suggestions in real time.

[0557] The "product suggestion means" is a function that generates specific product suggestions based on past search data when a user searches for a product, allowing the user to efficiently find the product they are looking for.

[0558] MODE FOR CARRYING OUT THE INVENTION

[0559] The present invention provides a system that provides appropriate suggestions when a user inputs instructions to a generative model. The system mainly includes a server, a terminal, and a user.

[0560] Server Processing

[0561] The server provides the following functionality:

[0562] 1. Data Collection

[0563] The server collects the instructions that the user has previously input into the generative model, the results, and the user's feedback. This data is stored in a database. The server often uses cloud infrastructure (e.g., AWS EC2 or Google Cloud Compute Engine).

[0564] 2. Data analysis

[0565] The server analyzes the stored past commands and their results. A text analysis algorithm (natural language processing model) is used to extract the characteristics of the command content and results. The command is then classified as successful or unsuccessful. This process is carried out using Python and the Scikit-learn library.

[0566] 3. Training the prompt generation model

[0567] The server trains a natural language processing (NLP) model to learn common patterns among successful cases, converting successful prompts into numerical vectors using TF-IDF vectorization and applying a K-means clustering algorithm to group similar prompts.

[0568] 4. Real-time suggestions

[0569] As users enter partial instructions, the server generates appropriate prompt suggestions in real time, leveraging trained NLP models.

[0570] Terminal handling

[0571] The terminal provides the following features:

[0572] 1. User Interface (UI)

[0573] The terminal provides an interface for the user to input instructions to the generative model. The UI includes input fields, a submit button, and a list of past inputs and suggested prompts.

[0574] 2. Real-time assistance

[0575] As the user enters prompts, the terminal incrementally sends requests to the server and displays suggestions returned by the server in real time, allowing the user to select or modify the suggested prompts.

[0576] User Action

[0577] The user performs the following steps:

[0578] 1. Prompt Input

[0579] The user inputs a prompt to the generative model, describing the task they want it to perform, such as a specific instruction like "a red shirt."

[0580] 2. Proposal acceptance

[0581] The user reviews the suggestions displayed on the device, selects the appropriate one, and optionally customizes the suggested prompts before sending them to the generative model.

[0582] Specific examples

[0583] As a concrete example, here is the steps a user takes to search for a "red shirt":

[0584] 1. User types "red shirt"

[0585] Using the device's UI, the user enters the prompt "red shirt." Once the input is complete, the device sends this partial instruction to the server.

[0586] 2. The server parses the prompt and generates suggestions

[0587] The server generates specific suggestions such as "men's red shirt, size large" or "red casual shirt" based on past success stories.

[0588] 3. A suggested prompt will appear on the screen

[0589] The device displays suggested prompts to the user, who can then select or customize the appropriate suggestion and send it to the generative model.

[0590] In this way, the user can receive specific suggestions even when the user gives vague instructions, and can efficiently find the desired product.

[0591] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0592] Step 1:

[0593] User prompt input

[0594] The user uses the terminal's user interface to enter a prompt to search for a specific product (e.g., "red shirt"), and the input field allows the user to enter any text. The entered prompt is transmitted to the server in real time.

[0595] Step 2:

[0596] Data collection by the server

[0597] The server receives the input prompt and simultaneously collects from the database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those instructions. For example, it collects examples of success and failure when the prompt "red shirt" was input in the past.

[0598] Step 3:

[0599] Data analysis by server

[0600] The server analyzes the collected data. It uses a text analysis algorithm to extract features of the instructions and results and classify them into success cases and failure cases. Specifically, it converts the prompts into numerical vectors using TF-IDF vectorization and applies the K-means clustering algorithm to group similar prompts. The input is the collected historical data, and the output is the classified clusters and their features.

[0601] Step 4:

[0602] Training a prompt generation model

[0603] The server trains a natural language processing (NLP) model to learn common patterns in successful cases. It uses past successful prompts to improve the model, thereby increasing the accuracy of its suggestions. The input is the analyzed successful prompts, and the output is the trained model.

[0604] Step 5:

[0605] Real-time proposal generation

[0606] The server generates appropriate prompt suggestions in real time based on the partial instructions entered by the user. It uses a trained NLP model to generate specific suggestions (e.g., "men's red shirt, size large," "red casual shirt") for the user's "red shirt" input. The input is the partial instructions from the user, and the output is the suggested specific prompt.

[0607] Step 6:

[0608] Display suggestions on the device

[0609] The terminal displays the suggestions returned by the server in real time. The user can review the suggested prompts and select an appropriate one or customize them as needed. Based on the displayed suggestions, the user can modify the input fields and send the request to the server again. The input is the suggestion from the server, and the output is the specific prompt presented to the user.

[0610] Step 7:

[0611] User's last input and search execution

[0612] The user finally confirms the selected or modified prompts and sends them to the generative model, which then returns specific search results and displays them to the user. The input is the prompts confirmed by the user, and the output is the search results from the generative model.

[0613] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0614] ---

[0615] The present invention is a system that supports users in inputting appropriate instructions to a generative model, and further combines it with an emotion engine that recognizes the user's emotions. This system is implemented through a series of procedures including interactions between a server, a terminal, and a user.

[0616] Server Processing

[0617] 1. Data Collection

[0618] The server collects and stores in a database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data also includes the user's emotional information recognized by the emotion engine.

[0619] 2. Data analysis

[0620] The server analyzes the instructions and results collected from the database. It uses a text analysis algorithm to extract characteristics of the instructions and results. It then categorizes them into success and failure cases and evaluates the effectiveness of the suggestions using the user's emotional information.

[0621] 3. Training the prompt generation model

[0622] The server trains a natural language processing (NLP) model to extract common patterns from success stories. It converts success stories into numerical vectors using TF-IDF vectorization and groups similar prompts using the K-means clustering algorithm. Sentiment data provided by the emotion engine is also fed into this training model.

[0623] 4. Real-time suggestions

[0624] For partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion engine.

[0625] Terminal handling

[0626] 1. User Interface (UI)

[0627] The device provides an interface for the user to input instructions to the generative model, including input fields, a submit button, a list of past inputs and suggested prompts, and even a feedback display showing the user's emotional state.

[0628] 2. Real-time assistance

[0629] When the user inputs the prompt, the device sends partial instructions to the server in real time. At the same time, the emotion engine recognizes emotions from the user's voice or text input and sends the emotion information to the server. The server then receives suggestions and displays them on the interface.

[0630] User Action

[0631] 1. Prompt Input

[0632] The user inputs a prompt to the generative model, describing the task they want to accomplish, such as "Show me a picture of a cat."

[0633] 2. Proposal acceptance

[0634] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotional information recognized by the emotion engine is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[0635] Specific examples

[0636] 1. Success stories

[0637] If a user inputs "I want to see pictures of animals" and the emotion engine recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[0638] 2. Failure example

[0639] If the user only types "animals," and the emotion engine recognizes this as a "confused" state, the server will suggest more specific instructions to the user to properly handle this ambiguous prompt. For example, it will help the user to provide more specific instructions using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?"

[0640] In this way, the system of the present invention allows users to efficiently input appropriate instructions to the generative model to achieve the desired results, and by taking into account the user's emotional state, the system can provide more personalized suggestions and improve the user experience.

[0641] The processing flow will be explained below.

[0642] ---

[0643] Step 1:

[0644] The server collects and stores in a database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those inputs, including the user's emotional information.

[0645] Step 2:

[0646] The server periodically analyzes the stored data. First, it uses a text analysis algorithm to analyze past instructions and their results, extracting characteristics of the instruction content and results. It then classifies cases into successes and failures.

[0647] Step 3:

[0648] The server trains a natural language processing (NLP) model to extract common patterns from success stories, converts success stories into numerical vectors using TF-IDF vectorization, and groups similar prompts using the K-means clustering algorithm.

[0649] Step 4:

[0650] The user inputs instructions to the generative model through a terminal, for example, "Show me a picture of a cat."

[0651] Step 5:

[0652] The emotion engine recognizes emotions from the user's input voice or text, and obtains emotion information such as whether the user is excited or confused.

[0653] Step 6:

[0654] The terminal transmits the partial instructions and emotion information input by the user to the server in real time.

[0655] Step 7:

[0656] The server generates appropriate prompt suggestions using existing NLP models based on partial instructions and emotional information, searches for similar success stories in real time, and customizes the suggestions taking into account the user's emotions.

[0657] Step 8:

[0658] The server sends the generated suggestions back to the device, which receives them and displays them as suggestions in the user interface. For example, suggestions might be "Show me pictures of cats" or "Show me pictures of dogs."

[0659] Step 9:

[0660] The user reviews the suggested prompts, selects the appropriate one, and customizes the prompt if necessary. Emotional information provided by the emotion engine is also displayed, and the user makes their selection based on that feedback.

[0661] Step 10:

[0662] The final instructions selected by the user are sent to the generative model, which generates results based on the user's specifications and returns the results to the user via the terminal.

[0663] Step 11:

[0664] The user reviews the generated results and provides feedback via their device to the server, which stores this feedback in a database for subsequent analysis and training.

[0665] Through this series of steps, the system of the present invention allows users to efficiently input appropriate instructions to the generative model and provides personalized suggestions that reflect the user's feelings, thereby enabling users to achieve the results they desire and improving their experience using the generative model.

[0666] Example 2

[0667] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0668] Conventional systems for inputting instructions to generative models lack support for users to input accurate instructions, making it difficult to obtain appropriate results. Furthermore, suggestions do not take into account the user's emotional state, which results in a lack of improvement in the user experience. The present invention aims to recognize the user's emotions and reflect them in instruction suggestions, thereby enabling users to efficiently input appropriate instructions to generative models and obtain desired results.

[0669] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0670] In this invention, the server includes input means for the user to input instructions to the generative model, database means for saving data on past instructions and their results, analysis means for analyzing the saved past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, interface means for sending the instructions input by the user to the server in real time and displaying the suggested instructions on a screen, and an emotion engine for collecting emotional information from the user and reflecting it in the analysis means and suggestion means. This not only enables the user to efficiently input appropriate instructions and obtain the desired results, but also enables personalized suggestions that take emotional information into account.

[0671] An "input means" is a device or interface that allows a user to input instructions to a generative model.

[0672] "Database means" refers to a system or storage device for storing data on past instructions and their results.

[0673] An "analysis tool" is an algorithm or system that analyzes stored past instructions and their results and classifies successes and failures.

[0674] The "suggestion means" is a device or system for suggesting new instructions to the user based on the classified success cases.

[0675] "Interface means" refers to a device or interface for transmitting instructions entered by a user to a server in real time and displaying suggested instructions.

[0676] An "emotion engine" is a system or algorithm that collects and analyzes user emotional information and reflects it in analytical and suggestion means.

[0677] "Emotional information" is data indicative of a user's emotional state, obtained from speech, text, or other forms of input.

[0678] The present invention is a system that supports users in inputting appropriate instructions to a generative model, and further combines it with an emotion engine that recognizes the user's emotions. This system is implemented through a series of procedures including interactions between a server, a terminal, and the user.

[0679] Server Processing

[0680] 1. Data Collection

[0681] The server stores the instructions that the user has previously input into the generative model, the results, and the user's feedback in a database. This data also includes the user's emotional information recognized by the emotion engine. The database uses a commercial database engine (e.g., MySQL, PostgreSQL, etc.).

[0682] 2. Data analysis

[0683] The server analyzes the instructions collected from the database and their results. It uses text analysis algorithms to extract features of the instructions and results. Natural language processing (NLP) libraries (e.g., spaCy, NLTK, etc.) are used for the analysis. Furthermore, it classifies success and failure cases and evaluates the effectiveness of suggestions using user sentiment information.

[0684] 3. Training the prompt generation model

[0685] The server trains a natural language processing model to extract common patterns from success stories. It converts success stories into numerical vectors using Term Frequency-Inverse Document Frequency (TF-IDF) vectorization and groups similar prompts using the K-means clustering algorithm. Emotion data provided by the emotion engine is also fed into this training model.

[0686] 4. Real-time suggestions

[0687] For partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion engine.

[0688] Terminal handling

[0689] 1. User Interface (UI)

[0690] The device provides an interface for the user to input instructions to the generative model. The UI includes input fields, a submit button, a history of past inputs, a list of suggested prompts, and a feedback display showing the user's emotional state. The interface is built using web-based technologies (e.g., HTML, CSS, JavaScript).

[0691] 2. Real-time assistance

[0692] As the user enters the prompts, the device sends partial instructions to the server in real time. At the same time, the emotion engine recognizes emotions from the user's voice and text input and sends the emotion information to the server. Suggestions from the server are sent to the device and displayed in the interface. This process is realized using technologies that enable two-way communication (e.g., WebSocket, HTTP / 2).

[0693] User Action

[0694] 1. Prompt Input

[0695] The user inputs a prompt to the generative model, specifying the task they want to accomplish, such as "Show me a picture of a cat."

[0696] 2. Proposal acceptance

[0697] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotional information recognized by the emotion engine is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[0698] Specific examples

[0699] 1. Success stories

[0700] If a user inputs "I want to see pictures of animals" and the emotion engine recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[0701] 2. Failure example

[0702] If the user only types "animals" and the emotion engine recognizes the "confused" state, the server will suggest more specific instructions to the user to handle this ambiguous prompt appropriately, for example, using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?" to help the user provide more specific instructions.

[0703] Prompt Sentence Examples

[0704] "Can you recommend some day trips I can take next weekend?"

[0705] I want to know some delicious pasta recipes.

[0706] "Tell me about the latest technology trends"

[0707] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0708] Step 1: User enters prompt text

[0709] Specific operation: The user inputs instructions for the generative model into the device's user interface (UI). For example, "Tell me recommended places for my next vacation."

[0710] Input: The user types the prompt text into the input field.

[0711] Output: The entered prompt is sent to the terminal as a string of data.

[0712] Step 2: Acquiring emotional information

[0713] Specific operation: The emotion engine installed in the device analyzes the emotion from the user's voice and text input. Example: The user's input is determined to be "excited."

[0714] Input: User voice and text data.

[0715] Output: Analyzed emotion information (e.g., "excited") is obtained.

[0716] Step 3: Sending data

[0717] Specific operation: The terminal sends the input prompt sentence and emotion information to the server.

[0718] Input: prompt text and emotion information.

[0719] Output: The prompt sentence and emotion information are sent to the server as a data packet.

[0720] Step 4: Save your data

[0721] Specific operation: The server stores the received prompt sentence and emotion information in the database. Example: A new record is added to the database.

[0722] Input: prompt sentence and emotion information.

[0723] Output: The new record saved in the database.

[0724] Step 5: Data acquisition and analysis

[0725] Specific operation: The server retrieves past instructions and their results from the database and applies a text analysis algorithm, e.g., converting past prompt sentences into TF-IDF vectors and extracting their features.

[0726] Input: Past instructions and result data retrieved from the database.

[0727] Output: The parsed feature vector.

[0728] Step 6: Categorize success stories and failure stories

[0729] Specific operation: The server classifies cases into success and failure based on the analyzed instructions. Example: User feedback is classified as "success" or "failure."

[0730] Input: Parsed feature vectors and feedback data.

[0731] Output: A dataset categorized into success and failure cases.

[0732] Step 7: Train the prompt generation model

[0733] What it does: The server trains a natural language processing (NLP) model to extract common patterns from success stories, e.g., grouping similar prompt sentences using TF-IDF vectors and K-means clustering.

[0734] Input: A dataset of success stories.

[0735] Output: A trained NLP model.

[0736] Step 8: Integrating Emotional Data

[0737] Specific operation: The server also incorporates the emotion data provided by the emotion engine into the training model. For example, it integrates the prompt sentence for the "excited" state and its success pattern into the NLP model.

[0738] Input: Sentiment data and a trained NLP model.

[0739] Output: An integrated model reflecting the emotion data.

[0740] Step 9: Generate real-time suggestions

[0741] What it does: Based on the user's partial input, the server generates appropriate prompt suggestions in real time. For example, if you enter "travel," it generates a complete prompt such as "Tell me some recommended places."

[0742] Input: Partial prompt sentence and integrated model.

[0743] Output: A suggested prompt.

[0744] Step 10: Submit and view your proposal

[0745] Specific operation: The server sends the generated proposal to the terminal, and the terminal displays the proposal to the user on the interface.

[0746] Input: A suggested prompt sentence.

[0747] Output: Proposals displayed on the screen.

[0748] Step 11: Accepting the proposal

[0749] What happens: The user reviews the suggestions displayed on the device and selects the appropriate one or customizes it as needed.

[0750] Input: On-screen suggestions.

[0751] Output: The prompt statement that the user finally selected.

[0752] Step 12: Sending final instructions

[0753] Specific behavior: The final prompt sentence selected by the user is sent to the generative model.

[0754] Input: The final prompt statement selected by the user.

[0755] Output: The prompt sentence sent to the generative model.

[0756] (Application example 2)

[0757] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0758] Conventional instruction input systems for generative models have had problems such as users being unable to input appropriate prompts and not taking into account emotional states, resulting in a poor user experience. In particular, when dealing with customers in brick-and-mortar stores, it has been difficult to improve customer satisfaction with customer service methods that ignore the emotional states of customers.

[0759] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0760] In this invention, the server includes input means for a user to input instructions to the generative model, database means for storing data on past instructions and their results, analysis means for analyzing the stored past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, emotion recognition means for recognizing the emotional state of the user, and interface means for transmitting the instructions input by the user to the server in real time and displaying the suggested instructions on the screen. This not only enables the user to input appropriate prompts to the generative model, but also enables personalized suggestions based on the user's emotional state.

[0761] The "input means" is an interface through which the user inputs instructions to the generative model.

[0762] The "database means" is a system or device for storing instructions previously input by the user and the results thereof.

[0763] The "analysis means" is a device or program for analyzing stored past instructions and their results, and classifying success cases and failure cases.

[0764] The "suggestion means" is a device or program for suggesting new instructions to the user based on the success cases classified by the analysis means.

[0765] The "emotion recognition means" is a device or program for detecting and recognizing the emotional state of a user from their voice and facial expression.

[0766] The "interface means" is a device or program for transmitting instructions input by the user to the server in real time and displaying suggestions from the server on the screen.

[0767] A "server" is a central processing unit that stores instructions and result data, performs analysis, and provides suggestions to users.

[0768] A "generative model" is an artificial intelligence or machine learning model that generates output based on user instructions.

[0769] The "emotional state" is information that represents the user's emotions, and is recognized via emotion recognition means.

[0770] A "prompt" is text in the form of an instruction or question to be posed to the generative model.

[0771] MODE FOR CARRYING OUT THE INVENTION

[0772] The present invention provides a system that assists users in inputting appropriate instructions to a generative model, and further combines an emotion recognition function that recognizes the user's emotional state. This system is implemented through a series of procedures including interactions between a server, a terminal, and a user.

[0773] Server Processing

[0774] The server has the following main processes:

[0775] 1. Data Collection

[0776] The server collects and stores in a database the instructions that users have previously input to the generative model, the results of those inputs, and feedback on those instructions. This data also includes emotion information recognized by the emotion recognition means.

[0777] 2. Data analysis

[0778] The server analyzes the instructions and their results collected from the database. It uses text analysis algorithms and machine learning models (e.g., TensorFlow) to extract features of the instructions and results. It also classifies success and failure cases and evaluates the effectiveness of the suggestions using user sentiment.

[0779] 3. Training the prompt generation model

[0780] The server trains a natural language processing (NLP) model to extract common patterns from success stories. It converts success stories into numerical vectors using TF-IDF vectorization and groups similar prompts using the K-means clustering algorithm. Emotion data provided by the emotion recognizer is also fed into this training model.

[0781] 4. Real-time suggestions

[0782] In response to partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion recognition means.

[0783] Terminal handling

[0784] The terminal has the following main processes:

[0785] 1. User Interface (UI)

[0786] The device provides an interface for the user to input instructions to the generative model, including input fields, a submit button, a list of past inputs and suggested prompts, and even a feedback display showing the user's emotional state.

[0787] 2. Real-time assistance

[0788] When the user inputs a prompt, the device sends partial instructions to the server in real time. At the same time, the emotion recognition means recognizes emotions from the user's voice or text input, and sends the emotion information to the server. The device receives suggestions from the server and displays them on the interface.

[0789] User Action

[0790] The user has the following main processes:

[0791] 1. Prompt Input

[0792] The user inputs a prompt to the generative model, describing the task they want to accomplish, such as "Show me a picture of a cat."

[0793] 2. Proposal acceptance

[0794] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotion information recognized by the emotion recognition means is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[0795] Specific examples

[0796] 1. Success stories

[0797] If the user inputs "I want to see pictures of animals" and the emotion recognition means recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[0798] 2. Failure example

[0799] If the user only types "animals" and the emotion recognizer recognizes the "confused" state, the server will suggest more specific instructions to the user to handle this ambiguous prompt appropriately. For example, it will help the user to provide more specific instructions using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?"

[0800] Prompt Sentence Examples

[0801] 1. Success prompt

[0802] "A customer seems confused. Would you like to ask about a specific item?"

[0803] 2. Failure prompt

[0804] "The client seems anxious. Would you like me to check in and reassure them about any concerns?"

[0805] In this way, the system of the present invention allows users to efficiently input appropriate instructions to the generative model to achieve the desired results, and by taking into account the user's emotional state, the system can provide more personalized suggestions and improve the user experience.

[0806] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0807] Step 1:

[0808] The terminal provides an interface where the user can input the task they want to perform on the generative model as a prompt. For example, the user can enter "I want to see pictures of animals" in the input field. The input data is a string, which is sent to the server in real time.

[0809] Step 2:

[0810] The server receives the character string data sent from the terminal. At the same time, it uses emotion recognition means to analyze and recognize the user's emotional state from their voice input and facial expression data, and sends this emotional information together to the server. The server then stores the character string and the emotional state in a database.

[0811] Step 3:

[0812] The server retrieves and analyzes the stored past instruction data and user sentiment information. Using text analysis algorithms and machine learning models (e.g., TensorFlow), it extracts features of the instruction content and results and classifies them into success cases and failure cases. The output of this analysis is a list of similar success cases.

[0813] Step 4:

[0814] The server generates candidate prompts based on the success cases. The generated prompts also reflect the user's emotional information obtained from the emotion recognition means. For example, if the user is in an "excited" state, suggested prompts include "Show me a picture of a cat" or "Show me an image of a dog." The candidate prompts are sent to the terminal in real time.

[0815] Step 5:

[0816] The device displays the candidate prompts sent from the server on its interface. The user checks the displayed candidate prompts and selects the appropriate one. If the user selects "Show me pictures of cats," the selection data is sent from the device to the server and used as the final instruction for the generative model.

[0817] Step 6:

[0818] The server sends the final prompt selected by the user to the generative model and obtains the output from the generative model. This output is the result data (e.g., a list of cat images) generated based on the specified conditions. This result data is then sent to the terminal to be presented to the user.

[0819] Step 7:

[0820] The terminal displays the output of the generative model received from the server on its interface, allowing the user to check the displayed results and provide further operations or instructions as necessary.

[0821] The above processing steps enable users to efficiently input appropriate instructions to the generative model and receive personalized suggestions based on their emotional state.

[0822] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0823] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0824] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0825] [Third embodiment]

[0826] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0827] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0828] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0829] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0830] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0831] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0832] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0833] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0834] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0835] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0836] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0837] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0838] ---

[0839] The present invention provides a system that supports users in inputting appropriate instructions to a generative model. The system is implemented through a series of procedures involving interactions between a server, a terminal, and a user.

[0840] Server Processing

[0841] 1. Data Collection

[0842] The server collects the instructions that users have previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data is stored in a database.

[0843] 2. Data analysis

[0844] The server analyzes the instructions and results collected from the database using text analysis algorithms to extract characteristics of the instructions and results, and then classifies the instructions as successful or unsuccessful.

[0845] 3. Training the prompt generation model

[0846] The server trains a natural language processing (NLP) model to learn common patterns among success prompts. Specifically, it converts success prompts into numerical vectors using TF-IDF vectorization and applies a K-means clustering algorithm to group similar prompts.

[0847] 4. Real-time suggestions

[0848] As users enter partial instructions, the server generates appropriate prompt suggestions in real time, leveraging trained NLP models.

[0849] Terminal handling

[0850] 1. User Interface (UI)

[0851] The terminal provides an interface for the user to input instructions to the generative model. This UI includes input fields, a submit button, and a list of past inputs and suggested prompts.

[0852] 2. Real-time assistance

[0853] As the user enters prompts, the terminal incrementally sends requests to the server and displays suggestions returned by the server in real time, allowing the user to select or modify the suggested prompts.

[0854] User Action

[0855] 1. Prompt Input

[0856] The user inputs a prompt to the generative model, specifying the task they want it to perform, such as "Show me a picture of a cat."

[0857] 2. Proposal acceptance

[0858] The user reviews the suggestions displayed on the device, selects the appropriate one, and optionally customizes the suggested prompts before sending them to the generative model.

[0859] Specific examples

[0860] 1. Success stories

[0861] When a user types "I want to see pictures of animals," the device sends this partial prompt to the server. The server analyzes past successes and suggests successful prompts, such as "Show me pictures of cats" or "Show me pictures of dogs." The device displays these suggestions to the user, who then selects "Show me pictures of cats."

[0862] 2. Failure example

[0863] If the user only types "animals," the server may not be able to properly handle this ambiguous prompt. However, the system's suggestion feature can suggest more specific prompts so the user can provide specific instructions.

[0864] In this way, the system of the present invention helps the user input appropriate instructions into the generative model, enabling efficient use.

[0865] ---

[0866] The processing flow will be explained below.

[0867] ---

[0868] Step 1:

[0869] The server collects and stores in a database the instructions that users have previously input to the generative model and the results of those inputs. This data also includes user feedback information (evaluation of success / failure).

[0870] Step 2:

[0871] The server periodically analyzes the stored data, first retrieving past instructions and their results, then using a text analysis algorithm to extract the characteristics of each instruction, and then classifying them into success cases and failure cases.

[0872] Step 3:

[0873] The server trains a natural language processing model (NLP model) to extract common patterns from success stories, converting success stories into numerical vectors using TF-IDF vectorization and grouping similar prompts using the K-means clustering algorithm.

[0874] Step 4:

[0875] The user inputs instructions to the generative model through the terminal. When the user starts inputting, the terminal sends partial instructions to the server in real time.

[0876] Step 5:

[0877] The server uses existing NLP models to generate appropriate prompts based on the partial instructions received, and searches for similar success stories in real time to generate appropriate suggestions.

[0878] Step 6:

[0879] The server sends the generated proposals back to the terminal, which receives them and displays them in its user interface.

[0880] Step 7:

[0881] The user reviews the suggested prompts, selects appropriate ones, or customizes them if necessary, after which the final selected instructions are sent to the generative model.

[0882] Step 8:

[0883] The generative model generates results based on the user's specifications and returns the results to the device, which then displays the generated results to the user.

[0884] Step 9:

[0885] The user checks the displayed results and provides feedback to the server via the terminal, which stores this feedback in a database for subsequent analysis.

[0886] This series of steps allows the system of the present invention to allow the user to efficiently input appropriate instructions to the generative model and obtain the desired results.

[0887] Example 1

[0888] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0889] In recent years, generative AI models have been used in a wide range of applications, but it is not easy for users to input appropriate instructions into generative models. In particular, when users input ambiguous instructions, it is difficult for the generative model to provide the expected results. This leads to a mismatch between the user's intention and the results, resulting in a poor user experience. In addition, there is a need for a system that can efficiently complement instructions by utilizing past success stories and suggest appropriate instructions to the user.

[0890] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0891] In this invention, the server includes a data storage means, a data analysis means, a proposal generation means, and a user interface means, which allows a user to input appropriate instructions to the generative model, enabling efficient use of the generative AI model.

[0892] "Input means" refers to a device or software that allows a user to input instructions to a generative model.

[0893] "Data storage means" refers to a device or software for recording and storing instructions that a user has previously input into a generative model and the results thereof.

[0894] "Data analysis means" refers to a device or software that analyzes stored past instructions and their results and classifies successful cases and unsuccessful cases.

[0895] The "suggestion generation means" refers to a device or software for suggesting new instructions to the user based on the success cases classified by the data analysis means.

[0896] "User interface means" refers to a device or software for transmitting instructions entered by a user to a server in real time and displaying suggested instructions on the user's screen.

[0897] A "generative AI model" refers to an artificial intelligence system that generates text, images, audio, etc. based on user input.

[0898] A "prompt sentence" refers to an input sentence that instructs a generative AI model to generate something specific.

[0899] A "success case" refers to an example in which a generative AI model produced the expected results in response to user instructions.

[0900] A "failure case" refers to an instance in which the generative AI model did not produce the expected result in response to a user's instructions.

[0901] The present invention is a system that supports users in inputting appropriate instructions to a generative AI model. This system is composed of a server, a terminal, and a series of processes including user interaction.

[0902] Server Processing

[0903] The server uses the data storage means, data analysis means, and suggestion generation means to analyze the prompts entered by the user and make appropriate suggestions, as follows:

[0904] 1. Data storage method

[0905] The server collects the instructions that users have previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data is stored in a database.

[0906] 2. Data analysis methods

[0907] The server analyzes the instructions and results collected from the database. For the analysis, it uses text analysis algorithms (e.g., morphological analysis, TF-IDF, etc.) to extract characteristics of the instruction content and results. The server classifies the instruction as successful or unsuccessful.

[0908] 3. Proposal generation means

[0909] The server trains a natural language processing (NLP) model to learn common patterns among success prompts. Specifically, it converts success prompts into numerical vectors using TF-IDF vectorization and applies a K-means clustering algorithm to group similar prompts. For partial instructions entered by the user, the server generates appropriate prompt suggestions in real time.

[0910] Terminal handling

[0911] The terminal provides an interface for a user to input instructions to the generative model using a user interface means, specifically as follows.

[0912] 1. User Interface Methods

[0913] The terminal provides an interface for users to input instructions to the generative model. The UI includes an input field, a submit button, and a list of past inputs and suggested prompts. As the user begins to input a prompt, the terminal incrementally sends requests to the server and displays the suggestions returned by the server in real time.

[0914] User Action

[0915] The user inputs the task they want the generative AI model to perform as a prompt. The specific operations are as follows:

[0916] 1. Prompt Input

[0917] The user inputs specific instructions to the generative model, such as "Show me a picture of a cat."

[0918] 2. Proposal acceptance

[0919] The user reviews the suggestions displayed on the device, selects the appropriate one, optionally customizes the suggested prompts, and sends them to the generative model.

[0920] Specific examples

[0921] Success stories

[0922] When a user types "I want to see pictures of animals," the device sends this partial prompt to the server. The server analyzes past successes and suggests successful prompts, such as "Show me pictures of cats" or "Show me pictures of dogs." The device displays these suggestions to the user, who then selects "Show me pictures of cats."

[0923] Failure example

[0924] If the user only types "animals," the server may not be able to properly handle this vague prompt. However, the system's suggestion feature can suggest more specific prompts, such as "Show me a picture of a cat," allowing the user to provide specific instructions.

[0925] In this way, the system of the present invention helps users input appropriate instructions into the generative model, enabling efficient use of the generative AI model.

[0926] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0927] Step 1: The user inputs instructions into the generative model

[0928] The user inputs the task they want to perform as a prompt to the generative model through the device's user interface. For example, the input might be "Show me pictures of cats." This input is reflected in real time on the device.

[0929] Step 2: The terminal sends partial input to the server

[0930] As the user begins to type the prompt, the terminal incrementally sends the partial prompt to the server. For example, as soon as the user types "cat's", the partial prompt is sent to the server. The terminal sends partial input (e.g., "cat's") to the server and waits for the result each time.

[0931] Step 3: The server performs data analysis

[0932] Based on the partial input received, the server analyzes the previous prompts and their results in the database using text analysis algorithms such as morphological analysis and TF-IDF. Specifically, it extracts previous prompts (e.g., "Show me pictures of cats") and classifies them as successful or unsuccessful.

[0933] Step 4: The server uses the proposed generative model

[0934] Based on the results of the data analysis, the server uses an NLP model that has learned common patterns in successful cases to generate appropriate suggested prompts. Specifically, it converts successful prompts into numerical vectors using TF-IDF vectorization and K-means clustering, and groups similar prompts. For this partial input, it generates suggestions such as "Show me a photo of a cat" or "Show me an image of a cat."

[0935] Step 5: The server sends the proposal to the device

[0936] The generated suggestion prompt is sent from the server to the device. The server sends the suggestion prompt (e.g., "Show me pictures of cats") to the device.

[0937] Step 6: The device displays the suggestion to the user

[0938] The device displays the received suggested prompts in the user interface, and the user can review the suggested prompts (e.g., "Show me pictures of cats") and select or modify them as needed.

[0939] Step 7: User selects / modifies final prompt

[0940] The user selects or modifies the suggested prompts and finalizes the instructions to be sent to the generative model, for example, "Show me pictures of cats."

[0941] Step 8: The device sends the final prompt to the generative model

[0942] The final prompt is sent from the device to the generative AI model. The device then sends the final prompt (e.g., "Show me a picture of a cat") to the generative model.

[0943] Step 9: The generative AI model generates and outputs results

[0944] The generative AI model processes the prompts it receives and generates an appropriate result (e.g., a photo of a cat), which is then served to the user.

[0945] The above is the specific processing flow of this system.

[0946] (Application example 1)

[0947] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0948] In online shopping, it is important for users to enter specific and appropriate search instructions to smoothly search for and purchase products. However, many users enter vague instructions or incomplete prompts, which results in a long time to find the desired product and a poor user experience. In addition, existing systems lack the functionality to provide appropriate suggestions to users based on past success stories. This leads to issues such as reduced product search efficiency and lower satisfaction.

[0949] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0950] In this invention, the server includes input means for a user to input instructions to the generative model, database means for saving data on past instructions and their results, analysis means for analyzing the saved past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, interface means for sending a request to the server in real time when the user inputs partial instructions and displaying the suggested instructions on the screen, and product suggestion means for generating specific product suggestions based on past search data when the user searches for products. This allows the user to receive appropriate suggestions even if they only input vague instructions, enabling them to quickly find the product they are looking for.

[0951] The "input means" is an interface function that allows the user to input instructions to the generative model. The user can use this means to specify any prompt.

[0952] The "database means" is a function that records and saves past user instructions and their results. This allows past search history and success / failure cases to be accumulated.

[0953] The "analysis means" is a function that analyzes saved past instructions and their results, and classifies success cases and failure cases. This allows for the extraction of patterns in instruction content and data analysis.

[0954] The "suggestion means" is a function that suggests new instructions to the user based on the classified success cases, thereby enabling the user to easily input appropriate instructions.

[0955] The "interface means" is a function that, when a user inputs partial instructions, sends a request to the server in real time and displays suggested instructions on the screen, allowing the user to receive appropriate suggestions in real time.

[0956] The "product suggestion means" is a function that generates specific product suggestions based on past search data when a user searches for a product, allowing the user to efficiently find the product they are looking for.

[0957] MODE FOR CARRYING OUT THE INVENTION

[0958] The present invention provides a system that provides appropriate suggestions when a user inputs instructions to a generative model. The system mainly includes a server, a terminal, and a user.

[0959] Server Processing

[0960] The server provides the following functionality:

[0961] 1. Data Collection

[0962] The server collects the instructions that the user has previously input into the generative model, the results, and the user's feedback. This data is stored in a database. The server often uses cloud infrastructure (e.g., AWS EC2 or Google Cloud Compute Engine).

[0963] 2. Data analysis

[0964] The server analyzes the stored past commands and their results. A text analysis algorithm (natural language processing model) is used to extract the characteristics of the command content and results. The command is then classified as successful or unsuccessful. This process is carried out using Python and the Scikit-learn library.

[0965] 3. Training the prompt generation model

[0966] The server trains a natural language processing (NLP) model to learn common patterns among successful cases, converting successful prompts into numerical vectors using TF-IDF vectorization and applying a K-means clustering algorithm to group similar prompts.

[0967] 4. Real-time suggestions

[0968] As users enter partial instructions, the server generates appropriate prompt suggestions in real time, leveraging trained NLP models.

[0969] Terminal handling

[0970] The terminal provides the following features:

[0971] 1. User Interface (UI)

[0972] The terminal provides an interface for the user to input instructions to the generative model. The UI includes input fields, a submit button, and a list of past inputs and suggested prompts.

[0973] 2. Real-time assistance

[0974] As the user enters prompts, the terminal incrementally sends requests to the server and displays suggestions returned by the server in real time, allowing the user to select or modify the suggested prompts.

[0975] User Action

[0976] The user performs the following steps:

[0977] 1. Prompt Input

[0978] The user inputs a prompt to the generative model, describing the task they want it to perform, such as a specific instruction like "a red shirt."

[0979] 2. Proposal acceptance

[0980] The user reviews the suggestions displayed on the device, selects the appropriate one, and optionally customizes the suggested prompts before sending them to the generative model.

[0981] Specific examples

[0982] As a concrete example, here is the steps a user takes to search for a "red shirt":

[0983] 1. User types "red shirt"

[0984] Using the device's UI, the user enters the prompt "red shirt." Once the input is complete, the device sends this partial instruction to the server.

[0985] 2. The server parses the prompt and generates suggestions

[0986] The server generates specific suggestions such as "men's red shirt, size large" or "red casual shirt" based on past success stories.

[0987] 3. A suggested prompt will appear on the screen

[0988] The device displays suggested prompts to the user, who can then select or customize the appropriate suggestion and send it to the generative model.

[0989] In this way, the user can receive specific suggestions even when the user gives vague instructions, and can efficiently find the desired product.

[0990] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0991] Step 1:

[0992] User prompt input

[0993] The user uses the terminal's user interface to enter a prompt to search for a specific product (e.g., "red shirt"), and the input field allows the user to enter any text. The entered prompt is transmitted to the server in real time.

[0994] Step 2:

[0995] Data collection by the server

[0996] The server receives the input prompt and simultaneously collects from the database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those instructions. For example, it collects examples of success and failure when the prompt "red shirt" was input in the past.

[0997] Step 3:

[0998] Data analysis by server

[0999] The server analyzes the collected data. It uses a text analysis algorithm to extract features of the instructions and results and classify them into success cases and failure cases. Specifically, it converts the prompts into numerical vectors using TF-IDF vectorization and applies the K-means clustering algorithm to group similar prompts. The input is the collected historical data, and the output is the classified clusters and their features.

[1000] Step 4:

[1001] Training a prompt generation model

[1002] The server trains a natural language processing (NLP) model to learn common patterns in successful cases. It uses past successful prompts to improve the model, thereby increasing the accuracy of its suggestions. The input is the analyzed successful prompts, and the output is the trained model.

[1003] Step 5:

[1004] Real-time proposal generation

[1005] The server generates appropriate prompt suggestions in real time based on the partial instructions entered by the user. It uses a trained NLP model to generate specific suggestions (e.g., "men's red shirt, size large," "red casual shirt") for the user's "red shirt" input. The input is the partial instructions from the user, and the output is the suggested specific prompt.

[1006] Step 6:

[1007] Display suggestions on the device

[1008] The terminal displays the suggestions returned by the server in real time. The user can review the suggested prompts and select an appropriate one or customize them as needed. Based on the displayed suggestions, the user can modify the input fields and send the request to the server again. The input is the suggestion from the server, and the output is the specific prompt presented to the user.

[1009] Step 7:

[1010] User's last input and search execution

[1011] The user finally confirms the selected or modified prompts and sends them to the generative model, which then returns specific search results and displays them to the user. The input is the prompts confirmed by the user, and the output is the search results from the generative model.

[1012] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1013] ---

[1014] The present invention is a system that supports users in inputting appropriate instructions to a generative model, and further combines it with an emotion engine that recognizes the user's emotions. This system is implemented through a series of procedures including interactions between a server, a terminal, and a user.

[1015] Server Processing

[1016] 1. Data Collection

[1017] The server collects and stores in a database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data also includes the user's emotional information recognized by the emotion engine.

[1018] 2. Data analysis

[1019] The server analyzes the instructions and results collected from the database. It uses a text analysis algorithm to extract characteristics of the instructions and results. It then categorizes them into success and failure cases and evaluates the effectiveness of the suggestions using the user's emotional information.

[1020] 3. Training the prompt generation model

[1021] The server trains a natural language processing (NLP) model to extract common patterns from success stories. It converts success stories into numerical vectors using TF-IDF vectorization and groups similar prompts using the K-means clustering algorithm. Sentiment data provided by the emotion engine is also fed into this training model.

[1022] 4. Real-time suggestions

[1023] For partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion engine.

[1024] Terminal handling

[1025] 1. User Interface (UI)

[1026] The device provides an interface for the user to input instructions to the generative model, including input fields, a submit button, a list of past inputs and suggested prompts, and even a feedback display showing the user's emotional state.

[1027] 2. Real-time assistance

[1028] When the user inputs the prompt, the device sends partial instructions to the server in real time. At the same time, the emotion engine recognizes emotions from the user's voice or text input and sends the emotion information to the server. The server then receives suggestions and displays them on the interface.

[1029] User Action

[1030] 1. Prompt Input

[1031] The user inputs a prompt to the generative model, describing the task they want to accomplish, such as "Show me a picture of a cat."

[1032] 2. Proposal acceptance

[1033] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotional information recognized by the emotion engine is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[1034] Specific examples

[1035] 1. Success stories

[1036] If a user inputs "I want to see pictures of animals" and the emotion engine recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[1037] 2. Failure example

[1038] If the user only types "animals," and the emotion engine recognizes this as a "confused" state, the server will suggest more specific instructions to the user to properly handle this ambiguous prompt. For example, it will help the user to provide more specific instructions using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?"

[1039] In this way, the system of the present invention allows users to efficiently input appropriate instructions to the generative model to achieve the desired results, and by taking into account the user's emotional state, the system can provide more personalized suggestions and improve the user experience.

[1040] The processing flow will be explained below.

[1041] ---

[1042] Step 1:

[1043] The server collects and stores in a database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those inputs, including the user's emotional information.

[1044] Step 2:

[1045] The server periodically analyzes the stored data. First, it uses a text analysis algorithm to analyze past instructions and their results, extracting characteristics of the instruction content and results. It then classifies cases into successes and failures.

[1046] Step 3:

[1047] The server trains a natural language processing (NLP) model to extract common patterns from success stories, converts success stories into numerical vectors using TF-IDF vectorization, and groups similar prompts using the K-means clustering algorithm.

[1048] Step 4:

[1049] The user inputs instructions to the generative model through a terminal, for example, "Show me a picture of a cat."

[1050] Step 5:

[1051] The emotion engine recognizes emotions from the user's input voice or text, and obtains emotion information such as whether the user is excited or confused.

[1052] Step 6:

[1053] The terminal transmits the partial instructions and emotion information input by the user to the server in real time.

[1054] Step 7:

[1055] The server generates appropriate prompt suggestions using existing NLP models based on partial instructions and emotional information, searches for similar success stories in real time, and customizes the suggestions taking into account the user's emotions.

[1056] Step 8:

[1057] The server sends the generated suggestions back to the device, which receives them and displays them as suggestions in the user interface. For example, suggestions might be "Show me pictures of cats" or "Show me pictures of dogs."

[1058] Step 9:

[1059] The user reviews the suggested prompts, selects the appropriate one, and customizes the prompt if necessary. Emotional information provided by the emotion engine is also displayed, and the user makes their selection based on that feedback.

[1060] Step 10:

[1061] The final instructions selected by the user are sent to the generative model, which generates results based on the user's specifications and returns the results to the user via the terminal.

[1062] Step 11:

[1063] The user reviews the generated results and provides feedback via their device to the server, which stores this feedback in a database for subsequent analysis and training.

[1064] Through this series of steps, the system of the present invention allows users to efficiently input appropriate instructions to the generative model and provides personalized suggestions that reflect the user's feelings, thereby enabling users to achieve the results they desire and improving their experience using the generative model.

[1065] Example 2

[1066] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1067] Conventional systems for inputting instructions to generative models lack support for users to input accurate instructions, making it difficult to obtain appropriate results. Furthermore, suggestions do not take into account the user's emotional state, which results in a lack of improvement in the user experience. The present invention aims to recognize the user's emotions and reflect them in instruction suggestions, thereby enabling users to efficiently input appropriate instructions to generative models and obtain desired results.

[1068] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1069] In this invention, the server includes input means for the user to input instructions to the generative model, database means for saving data on past instructions and their results, analysis means for analyzing the saved past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, interface means for sending the instructions input by the user to the server in real time and displaying the suggested instructions on a screen, and an emotion engine for collecting emotional information from the user and reflecting it in the analysis means and suggestion means. This not only enables the user to efficiently input appropriate instructions and obtain the desired results, but also enables personalized suggestions that take emotional information into account.

[1070] An "input means" is a device or interface that allows a user to input instructions to a generative model.

[1071] "Database means" refers to a system or storage device for storing data on past instructions and their results.

[1072] An "analysis tool" is an algorithm or system that analyzes stored past instructions and their results and classifies successes and failures.

[1073] The "suggestion means" is a device or system for suggesting new instructions to the user based on the classified success cases.

[1074] "Interface means" refers to a device or interface for transmitting instructions entered by a user to a server in real time and displaying suggested instructions.

[1075] An "emotion engine" is a system or algorithm that collects and analyzes user emotional information and reflects it in analytical and suggestion means.

[1076] "Emotional information" is data indicative of a user's emotional state, obtained from speech, text, or other forms of input.

[1077] The present invention is a system that supports users in inputting appropriate instructions to a generative model, and further combines it with an emotion engine that recognizes the user's emotions. This system is implemented through a series of procedures including interactions between a server, a terminal, and the user.

[1078] Server Processing

[1079] 1. Data Collection

[1080] The server stores the instructions that the user has previously input into the generative model, the results, and the user's feedback in a database. This data also includes the user's emotional information recognized by the emotion engine. The database uses a commercial database engine (e.g., MySQL, PostgreSQL, etc.).

[1081] 2. Data analysis

[1082] The server analyzes the instructions collected from the database and their results. It uses text analysis algorithms to extract features of the instructions and results. Natural language processing (NLP) libraries (e.g., spaCy, NLTK, etc.) are used for the analysis. Furthermore, it classifies success and failure cases and evaluates the effectiveness of suggestions using user sentiment information.

[1083] 3. Training the prompt generation model

[1084] The server trains a natural language processing model to extract common patterns from success stories. It converts success stories into numerical vectors using Term Frequency-Inverse Document Frequency (TF-IDF) vectorization and groups similar prompts using the K-means clustering algorithm. Emotion data provided by the emotion engine is also fed into this training model.

[1085] 4. Real-time suggestions

[1086] For partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion engine.

[1087] Terminal handling

[1088] 1. User Interface (UI)

[1089] The device provides an interface for the user to input instructions to the generative model. The UI includes input fields, a submit button, a history of past inputs, a list of suggested prompts, and a feedback display showing the user's emotional state. The interface is built using web-based technologies (e.g., HTML, CSS, JavaScript).

[1090] 2. Real-time assistance

[1091] As the user enters the prompts, the device sends partial instructions to the server in real time. At the same time, the emotion engine recognizes emotions from the user's voice and text input and sends the emotion information to the server. Suggestions from the server are sent to the device and displayed in the interface. This process is realized using technologies that enable two-way communication (e.g., WebSocket, HTTP / 2).

[1092] User Action

[1093] 1. Prompt Input

[1094] The user inputs a prompt to the generative model, specifying the task they want to accomplish, such as "Show me a picture of a cat."

[1095] 2. Proposal acceptance

[1096] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotional information recognized by the emotion engine is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[1097] Specific examples

[1098] 1. Success stories

[1099] If a user inputs "I want to see pictures of animals" and the emotion engine recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[1100] 2. Failure example

[1101] If the user only types "animals" and the emotion engine recognizes the "confused" state, the server will suggest more specific instructions to the user to handle this ambiguous prompt appropriately, for example, using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?" to help the user provide more specific instructions.

[1102] Prompt Sentence Examples

[1103] "Can you recommend some day trips I can take next weekend?"

[1104] I want to know some delicious pasta recipes.

[1105] "Tell me about the latest technology trends"

[1106] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1107] Step 1: User enters prompt text

[1108] Specific operation: The user inputs instructions for the generative model into the device's user interface (UI). For example, "Tell me recommended places for my next vacation."

[1109] Input: The user types the prompt text into the input field.

[1110] Output: The entered prompt is sent to the terminal as a string of data.

[1111] Step 2: Acquiring emotional information

[1112] Specific operation: The emotion engine installed in the device analyzes the emotion from the user's voice and text input. Example: The user's input is determined to be "excited."

[1113] Input: User voice and text data.

[1114] Output: Analyzed emotion information (e.g., "excited") is obtained.

[1115] Step 3: Sending data

[1116] Specific operation: The terminal sends the input prompt sentence and emotion information to the server.

[1117] Input: prompt text and emotion information.

[1118] Output: The prompt sentence and emotion information are sent to the server as a data packet.

[1119] Step 4: Save your data

[1120] Specific operation: The server stores the received prompt sentence and emotion information in the database. Example: A new record is added to the database.

[1121] Input: prompt sentence and emotion information.

[1122] Output: The new record saved in the database.

[1123] Step 5: Data acquisition and analysis

[1124] Specific operation: The server retrieves past instructions and their results from the database and applies a text analysis algorithm, e.g., converting past prompt sentences into TF-IDF vectors and extracting their features.

[1125] Input: Past instructions and result data retrieved from the database.

[1126] Output: The parsed feature vector.

[1127] Step 6: Categorize success stories and failure stories

[1128] Specific operation: The server classifies cases into success and failure based on the analyzed instructions. Example: User feedback is classified as "success" or "failure."

[1129] Input: Parsed feature vectors and feedback data.

[1130] Output: A dataset categorized into success and failure cases.

[1131] Step 7: Train the prompt generation model

[1132] What it does: The server trains a natural language processing (NLP) model to extract common patterns from success stories, e.g., grouping similar prompt sentences using TF-IDF vectors and K-means clustering.

[1133] Input: A dataset of success stories.

[1134] Output: A trained NLP model.

[1135] Step 8: Integrating Emotional Data

[1136] Specific operation: The server also incorporates the emotion data provided by the emotion engine into the training model. For example, it integrates the prompt sentence for the "excited" state and its success pattern into the NLP model.

[1137] Input: Sentiment data and a trained NLP model.

[1138] Output: An integrated model reflecting the emotion data.

[1139] Step 9: Generate real-time proposals

[1140] What it does: Based on the user's partial input, the server generates appropriate prompt suggestions in real time. For example, if you enter "travel," it generates a complete prompt such as "Tell me some recommended places."

[1141] Input: Partial prompt sentence and integrated model.

[1142] Output: A suggested prompt.

[1143] Step 10: Submit and view your proposal

[1144] Specific operation: The server sends the generated proposal to the terminal, and the terminal displays the proposal to the user on the interface.

[1145] Input: A suggested prompt sentence.

[1146] Output: Proposals displayed on the screen.

[1147] Step 11: Accepting the proposal

[1148] What happens: The user reviews the suggestions displayed on the device and selects the appropriate one or customizes it as needed.

[1149] Input: On-screen suggestions.

[1150] Output: The prompt statement that the user finally selected.

[1151] Step 12: Sending final instructions

[1152] Specific behavior: The final prompt sentence selected by the user is sent to the generative model.

[1153] Input: The final prompt statement selected by the user.

[1154] Output: The prompt sentence sent to the generative model.

[1155] (Application example 2)

[1156] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1157] Conventional instruction input systems for generative models have had problems such as users being unable to input appropriate prompts and not taking into account emotional states, resulting in a poor user experience. In particular, when dealing with customers in brick-and-mortar stores, it has been difficult to improve customer satisfaction with customer service methods that ignore the emotional states of customers.

[1158] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1159] In this invention, the server includes input means for a user to input instructions to the generative model, database means for storing data on past instructions and their results, analysis means for analyzing the stored past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, emotion recognition means for recognizing the emotional state of the user, and interface means for transmitting the instructions input by the user to the server in real time and displaying the suggested instructions on the screen. This not only enables the user to input appropriate prompts to the generative model, but also enables personalized suggestions based on the user's emotional state.

[1160] The "input means" is an interface through which the user inputs instructions to the generative model.

[1161] The "database means" is a system or device for storing instructions previously input by the user and the results thereof.

[1162] The "analysis means" is a device or program for analyzing stored past instructions and their results, and classifying success cases and failure cases.

[1163] The "suggestion means" is a device or program for suggesting new instructions to the user based on the success cases classified by the analysis means.

[1164] The "emotion recognition means" is a device or program for detecting and recognizing the emotional state of a user from their voice and facial expression.

[1165] The "interface means" is a device or program for transmitting instructions input by the user to the server in real time and displaying suggestions from the server on the screen.

[1166] A "server" is a central processing unit that stores instructions and result data, performs analysis, and provides suggestions to users.

[1167] A "generative model" is an artificial intelligence or machine learning model that generates output based on user instructions.

[1168] The "emotional state" is information that represents the user's emotions, and is recognized via emotion recognition means.

[1169] A "prompt" is text in the form of an instruction or question to be posed to the generative model.

[1170] MODE FOR CARRYING OUT THE INVENTION

[1171] The present invention provides a system that assists users in inputting appropriate instructions to a generative model, and further combines an emotion recognition function that recognizes the user's emotional state. This system is implemented through a series of procedures including interactions between a server, a terminal, and a user.

[1172] Server Processing

[1173] The server has the following main processes:

[1174] 1. Data Collection

[1175] The server collects and stores in a database the instructions that users have previously input to the generative model, the results of those inputs, and feedback on those instructions. This data also includes emotion information recognized by the emotion recognition means.

[1176] 2. Data analysis

[1177] The server analyzes the instructions and their results collected from the database. It uses text analysis algorithms and machine learning models (e.g., TensorFlow) to extract features of the instructions and results. It also classifies success and failure cases and evaluates the effectiveness of the suggestions using user sentiment.

[1178] 3. Training the prompt generation model

[1179] The server trains a natural language processing (NLP) model to extract common patterns from success stories. It converts success stories into numerical vectors using TF-IDF vectorization and groups similar prompts using the K-means clustering algorithm. Emotion data provided by the emotion recognizer is also fed into this training model.

[1180] 4. Real-time suggestions

[1181] In response to partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion recognition means.

[1182] Terminal handling

[1183] The terminal has the following main processes:

[1184] 1. User Interface (UI)

[1185] The device provides an interface for the user to input instructions to the generative model, including input fields, a submit button, a list of past inputs and suggested prompts, and even a feedback display showing the user's emotional state.

[1186] 2. Real-time assistance

[1187] When the user inputs a prompt, the device sends partial instructions to the server in real time. At the same time, the emotion recognition means recognizes emotions from the user's voice or text input, and sends the emotion information to the server. The device receives suggestions from the server and displays them on the interface.

[1188] User Action

[1189] The user has the following main processes:

[1190] 1. Prompt Input

[1191] The user inputs a prompt to the generative model, describing the task they want to accomplish, such as "Show me a picture of a cat."

[1192] 2. Proposal acceptance

[1193] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotion information recognized by the emotion recognition means is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[1194] Specific examples

[1195] 1. Success stories

[1196] If the user inputs "I want to see pictures of animals" and the emotion recognition means recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[1197] 2. Failure example

[1198] If the user only types "animals" and the emotion recognizer recognizes the "confused" state, the server will suggest more specific instructions to the user to handle this ambiguous prompt appropriately. For example, it will help the user to provide more specific instructions using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?"

[1199] Prompt Sentence Examples

[1200] 1. Success prompt

[1201] "A customer seems confused. Would you like to ask about a specific item?"

[1202] 2. Failure prompt

[1203] "The client seems anxious. Would you like me to check in and reassure them about any concerns?"

[1204] In this way, the system of the present invention allows users to efficiently input appropriate instructions to the generative model to achieve the desired results, and by taking into account the user's emotional state, the system can provide more personalized suggestions and improve the user experience.

[1205] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1206] Step 1:

[1207] The terminal provides an interface where the user can input the task they want to perform on the generative model as a prompt. For example, the user can enter "I want to see pictures of animals" in the input field. The input data is a string, which is sent to the server in real time.

[1208] Step 2:

[1209] The server receives the character string data sent from the terminal. At the same time, it uses emotion recognition means to analyze and recognize the user's emotional state from their voice input and facial expression data, and sends this emotional information together to the server. The server then stores the character string and the emotional state in a database.

[1210] Step 3:

[1211] The server retrieves and analyzes the stored past instruction data and user sentiment information. Using text analysis algorithms and machine learning models (e.g., TensorFlow), it extracts features of the instruction content and results and classifies them into success cases and failure cases. The output of this analysis is a list of similar success cases.

[1212] Step 4:

[1213] The server generates candidate prompts based on the success cases. The generated prompts also reflect the user's emotional information obtained from the emotion recognition means. For example, if the user is in an "excited" state, suggested prompts include "Show me a picture of a cat" or "Show me an image of a dog." The candidate prompts are sent to the terminal in real time.

[1214] Step 5:

[1215] The device displays the candidate prompts sent from the server on its interface. The user checks the displayed candidate prompts and selects the appropriate one. If the user selects "Show me pictures of cats," the selection data is sent from the device to the server and used as the final instruction for the generative model.

[1216] Step 6:

[1217] The server sends the final prompt selected by the user to the generative model and obtains the output from the generative model. This output is the result data (e.g., a list of cat images) generated based on the specified conditions. This result data is then sent to the terminal to be presented to the user.

[1218] Step 7:

[1219] The terminal displays the output of the generative model received from the server on its interface, allowing the user to check the displayed results and provide further operations or instructions as necessary.

[1220] The above processing steps enable users to efficiently input appropriate instructions to the generative model and receive personalized suggestions based on their emotional state.

[1221] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1222] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1223] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1224] [Fourth embodiment]

[1225] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1226] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1227] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1228] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1229] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1230] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1231] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1232] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1233] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1234] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1235] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1236] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1237] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1238] ---

[1239] The present invention provides a system that supports users in inputting appropriate instructions to a generative model. The system is implemented through a series of procedures involving interactions between a server, a terminal, and a user.

[1240] Server Processing

[1241] 1. Data Collection

[1242] The server collects the instructions that users have previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data is stored in a database.

[1243] 2. Data analysis

[1244] The server analyzes the instructions and results collected from the database using text analysis algorithms to extract characteristics of the instructions and results, and then classifies the instructions as successful or unsuccessful.

[1245] 3. Training the prompt generation model

[1246] The server trains a natural language processing (NLP) model to learn common patterns among success prompts. Specifically, it converts success prompts into numerical vectors using TF-IDF vectorization and applies a K-means clustering algorithm to group similar prompts.

[1247] 4. Real-time suggestions

[1248] As users enter partial instructions, the server generates appropriate prompt suggestions in real time, leveraging trained NLP models.

[1249] Terminal handling

[1250] 1. User Interface (UI)

[1251] The terminal provides an interface for the user to input instructions to the generative model. This UI includes input fields, a submit button, and a list of past inputs and suggested prompts.

[1252] 2. Real-time assistance

[1253] As the user enters prompts, the terminal incrementally sends requests to the server and displays suggestions returned by the server in real time, allowing the user to select or modify the suggested prompts.

[1254] User Action

[1255] 1. Prompt Input

[1256] The user inputs a prompt to the generative model, specifying the task they want it to perform, such as "Show me a picture of a cat."

[1257] 2. Proposal acceptance

[1258] The user reviews the suggestions displayed on the device, selects the appropriate one, and optionally customizes the suggested prompts before sending them to the generative model.

[1259] Specific examples

[1260] 1. Success stories

[1261] When a user types "I want to see pictures of animals," the device sends this partial prompt to the server. The server analyzes past successes and suggests successful prompts, such as "Show me pictures of cats" or "Show me pictures of dogs." The device displays these suggestions to the user, who then selects "Show me pictures of cats."

[1262] 2. Failure example

[1263] If the user only types "animals," the server may not be able to properly handle this ambiguous prompt. However, the system's suggestion feature can suggest more specific prompts so the user can provide specific instructions.

[1264] In this way, the system of the present invention helps the user input appropriate instructions into the generative model, enabling efficient use.

[1265] ---

[1266] The processing flow will be explained below.

[1267] ---

[1268] Step 1:

[1269] The server collects and stores in a database the instructions that users have previously input to the generative model and the results of those inputs. This data also includes user feedback information (evaluation of success / failure).

[1270] Step 2:

[1271] The server periodically analyzes the stored data, first retrieving past instructions and their results, then using a text analysis algorithm to extract the characteristics of each instruction, and then classifying them into success cases and failure cases.

[1272] Step 3:

[1273] The server trains a natural language processing model (NLP model) to extract common patterns from success stories, converting success stories into numerical vectors using TF-IDF vectorization and grouping similar prompts using the K-means clustering algorithm.

[1274] Step 4:

[1275] The user inputs instructions to the generative model through the terminal. When the user starts inputting, the terminal sends partial instructions to the server in real time.

[1276] Step 5:

[1277] The server uses existing NLP models to generate appropriate prompts based on the partial instructions received, and searches for similar success stories in real time to generate appropriate suggestions.

[1278] Step 6:

[1279] The server sends the generated proposals back to the terminal, which receives them and displays them in its user interface.

[1280] Step 7:

[1281] The user reviews the suggested prompts, selects appropriate ones, or customizes them if necessary, after which the final selected instructions are sent to the generative model.

[1282] Step 8:

[1283] The generative model generates results based on the user's specifications and returns the results to the device, which then displays the generated results to the user.

[1284] Step 9:

[1285] The user checks the displayed results and provides feedback to the server via the terminal, which stores this feedback in a database for subsequent analysis.

[1286] This series of steps allows the system of the present invention to allow the user to efficiently input appropriate instructions to the generative model and obtain the desired results.

[1287] Example 1

[1288] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1289] In recent years, generative AI models have been used in a wide range of applications, but it is not easy for users to input appropriate instructions into generative models. In particular, when users input ambiguous instructions, it is difficult for the generative model to provide the expected results. This leads to a mismatch between the user's intention and the results, resulting in a poor user experience. In addition, there is a need for a system that can efficiently complement instructions by utilizing past success stories and suggest appropriate instructions to the user.

[1290] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1291] In this invention, the server includes a data storage means, a data analysis means, a proposal generation means, and a user interface means, which allows a user to input appropriate instructions to the generative model, enabling efficient use of the generative AI model.

[1292] "Input means" refers to a device or software that allows a user to input instructions to a generative model.

[1293] "Data storage means" refers to a device or software for recording and storing instructions that a user has previously input into a generative model and the results thereof.

[1294] "Data analysis means" refers to a device or software that analyzes stored past instructions and their results and classifies successful cases and unsuccessful cases.

[1295] The "suggestion generation means" refers to a device or software for suggesting new instructions to the user based on the success cases classified by the data analysis means.

[1296] "User interface means" refers to a device or software for transmitting instructions entered by a user to a server in real time and displaying suggested instructions on the user's screen.

[1297] A "generative AI model" refers to an artificial intelligence system that generates text, images, audio, etc. based on user input.

[1298] A "prompt sentence" refers to an input sentence that instructs a generative AI model to generate something specific.

[1299] A "success case" refers to an example in which a generative AI model produced the expected results in response to user instructions.

[1300] A "failure case" refers to an instance in which the generative AI model did not produce the expected result in response to a user's instructions.

[1301] The present invention is a system that supports users in inputting appropriate instructions to a generative AI model. This system is composed of a server, a terminal, and a series of processes including user interaction.

[1302] Server Processing

[1303] The server uses the data storage means, data analysis means, and suggestion generation means to analyze the prompts entered by the user and make appropriate suggestions, as follows:

[1304] 1. Data storage method

[1305] The server collects the instructions that users have previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data is stored in a database.

[1306] 2. Data analysis methods

[1307] The server analyzes the instructions and results collected from the database. For the analysis, it uses text analysis algorithms (e.g., morphological analysis, TF-IDF, etc.) to extract characteristics of the instruction content and results. The server classifies the instruction as successful or unsuccessful.

[1308] 3. Proposal generation means

[1309] The server trains a natural language processing (NLP) model to learn common patterns among success prompts. Specifically, it converts success prompts into numerical vectors using TF-IDF vectorization and applies a K-means clustering algorithm to group similar prompts. For partial instructions entered by the user, the server generates appropriate prompt suggestions in real time.

[1310] Terminal handling

[1311] The terminal provides an interface for a user to input instructions to the generative model using a user interface means, specifically as follows.

[1312] 1. User Interface Methods

[1313] The terminal provides an interface for users to input instructions to the generative model. The UI includes an input field, a submit button, and a list of past inputs and suggested prompts. As the user begins to input a prompt, the terminal incrementally sends requests to the server and displays the suggestions returned by the server in real time.

[1314] User Action

[1315] The user inputs the task they want the generative AI model to perform as a prompt. The specific operations are as follows:

[1316] 1. Prompt Input

[1317] The user inputs specific instructions to the generative model, such as "Show me a picture of a cat."

[1318] 2. Proposal acceptance

[1319] The user reviews the suggestions displayed on the device, selects the appropriate one, optionally customizes the suggested prompts, and sends them to the generative model.

[1320] Specific examples

[1321] Success stories

[1322] When a user types "I want to see pictures of animals," the device sends this partial prompt to the server. The server analyzes past successes and suggests successful prompts, such as "Show me pictures of cats" or "Show me pictures of dogs." The device displays these suggestions to the user, who then selects "Show me pictures of cats."

[1323] Failure example

[1324] If the user only types "animals," the server may not be able to properly handle this vague prompt. However, the system's suggestion feature can suggest more specific prompts, such as "Show me a picture of a cat," allowing the user to provide specific instructions.

[1325] In this way, the system of the present invention helps users input appropriate instructions into the generative model, enabling efficient use of the generative AI model.

[1326] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1327] Step 1: The user inputs instructions into the generative model

[1328] The user inputs the task they want to perform as a prompt to the generative model through the device's user interface. For example, the input might be "Show me pictures of cats." This input is reflected in real time on the device.

[1329] Step 2: The terminal sends partial input to the server

[1330] As the user begins to type the prompt, the terminal incrementally sends the partial prompt to the server. For example, as soon as the user types "cat's", the partial prompt is sent to the server. The terminal sends partial input (e.g., "cat's") to the server and waits for the result each time.

[1331] Step 3: The server performs data analysis

[1332] Based on the partial input received, the server analyzes the previous prompts and their results in the database using text analysis algorithms such as morphological analysis and TF-IDF. Specifically, it extracts previous prompts (e.g., "Show me pictures of cats") and classifies them as successful or unsuccessful.

[1333] Step 4: The server uses the proposed generative model

[1334] Based on the results of the data analysis, the server uses an NLP model that has learned common patterns in successful cases to generate appropriate suggested prompts. Specifically, it converts successful prompts into numerical vectors using TF-IDF vectorization and K-means clustering, and groups similar prompts. For this partial input, it generates suggestions such as "Show me a photo of a cat" or "Show me an image of a cat."

[1335] Step 5: The server sends the proposal to the device

[1336] The generated suggestion prompt is sent from the server to the device. The server sends the suggestion prompt (e.g., "Show me pictures of cats") to the device.

[1337] Step 6: The device displays the suggestion to the user

[1338] The device displays the received suggested prompts in the user interface, and the user can review the suggested prompts (e.g., "Show me pictures of cats") and select or modify them as needed.

[1339] Step 7: User selects / modifies final prompt

[1340] The user selects or modifies the suggested prompts and finalizes the instructions to be sent to the generative model, for example, "Show me pictures of cats."

[1341] Step 8: The device sends the final prompt to the generative model

[1342] The final prompt is sent from the device to the generative AI model. The device then sends the final prompt (e.g., "Show me a picture of a cat") to the generative model.

[1343] Step 9: The generative AI model generates and outputs results

[1344] The generative AI model processes the prompts it receives and generates an appropriate result (e.g., a photo of a cat), which is then served to the user.

[1345] The above is the specific processing flow of this system.

[1346] (Application example 1)

[1347] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1348] In online shopping, it is important for users to enter specific and appropriate search instructions to smoothly search for and purchase products. However, many users enter vague instructions or incomplete prompts, which results in a long time to find the desired product and a poor user experience. In addition, existing systems lack the functionality to provide appropriate suggestions to users based on past success stories. This leads to issues such as reduced product search efficiency and lower satisfaction.

[1349] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1350] In this invention, the server includes input means for a user to input instructions to the generative model, database means for saving data on past instructions and their results, analysis means for analyzing the saved past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, interface means for sending a request to the server in real time when the user inputs partial instructions and displaying the suggested instructions on the screen, and product suggestion means for generating specific product suggestions based on past search data when the user searches for products. This allows the user to receive appropriate suggestions even if they only input vague instructions, enabling them to quickly find the product they are looking for.

[1351] The "input means" is an interface function that allows the user to input instructions to the generative model. The user can use this means to specify any prompt.

[1352] The "database means" is a function that records and saves past user instructions and their results. This allows past search history and success / failure cases to be accumulated.

[1353] The "analysis means" is a function that analyzes saved past instructions and their results, and classifies success cases and failure cases. This allows for the extraction of patterns in instruction content and data analysis.

[1354] The "suggestion means" is a function that suggests new instructions to the user based on the classified success cases, thereby enabling the user to easily input appropriate instructions.

[1355] The "interface means" is a function that, when a user inputs partial instructions, sends a request to the server in real time and displays suggested instructions on the screen, allowing the user to receive appropriate suggestions in real time.

[1356] The "product suggestion means" is a function that generates specific product suggestions based on past search data when a user searches for a product, allowing the user to efficiently find the product they are looking for.

[1357] MODE FOR CARRYING OUT THE INVENTION

[1358] The present invention provides a system that provides appropriate suggestions when a user inputs instructions to a generative model. The system mainly includes a server, a terminal, and a user.

[1359] Server Processing

[1360] The server provides the following functionality:

[1361] 1. Data Collection

[1362] The server collects the instructions that the user has previously input into the generative model, the results, and the user's feedback. This data is stored in a database. The server often uses cloud infrastructure (e.g., AWS EC2 or Google Cloud Compute Engine).

[1363] 2. Data analysis

[1364] The server analyzes the stored past commands and their results. A text analysis algorithm (natural language processing model) is used to extract the characteristics of the command content and results. The command is then classified as successful or unsuccessful. This process is carried out using Python and the Scikit-learn library.

[1365] 3. Training the prompt generation model

[1366] The server trains a natural language processing (NLP) model to learn common patterns among successful cases, converting successful prompts into numerical vectors using TF-IDF vectorization and applying a K-means clustering algorithm to group similar prompts.

[1367] 4. Real-time suggestions

[1368] As users enter partial instructions, the server generates appropriate prompt suggestions in real time, leveraging trained NLP models.

[1369] Terminal handling

[1370] The terminal provides the following features:

[1371] 1. User Interface (UI)

[1372] The terminal provides an interface for the user to input instructions to the generative model. The UI includes input fields, a submit button, and a list of past inputs and suggested prompts.

[1373] 2. Real-time assistance

[1374] As the user enters prompts, the terminal incrementally sends requests to the server and displays suggestions returned by the server in real time, allowing the user to select or modify the suggested prompts.

[1375] User Action

[1376] The user performs the following steps:

[1377] 1. Prompt Input

[1378] The user inputs a prompt to the generative model, describing the task they want it to perform, such as a specific instruction like "a red shirt."

[1379] 2. Proposal acceptance

[1380] The user reviews the suggestions displayed on the device, selects the appropriate one, and optionally customizes the suggested prompts before sending them to the generative model.

[1381] Specific examples

[1382] As a concrete example, here is the steps a user takes to search for a "red shirt":

[1383] 1. User types "red shirt"

[1384] Using the device's UI, the user enters the prompt "red shirt." Once the input is complete, the device sends this partial instruction to the server.

[1385] 2. The server parses the prompt and generates suggestions

[1386] The server generates specific suggestions such as "men's red shirt, size large" or "red casual shirt" based on past success stories.

[1387] 3. A suggested prompt will appear on the screen

[1388] The device displays suggested prompts to the user, who can then select or customize the appropriate suggestion and send it to the generative model.

[1389] In this way, the user can receive specific suggestions even when the user gives vague instructions, and can efficiently find the desired product.

[1390] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1391] Step 1:

[1392] User prompt input

[1393] The user uses the terminal's user interface to enter a prompt to search for a specific product (e.g., "red shirt"), and the input field allows the user to enter any text. The entered prompt is transmitted to the server in real time.

[1394] Step 2:

[1395] Data collection by the server

[1396] The server receives the input prompt and simultaneously collects from the database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those instructions. For example, it collects examples of success and failure when the prompt "red shirt" was input in the past.

[1397] Step 3:

[1398] Data analysis by server

[1399] The server analyzes the collected data. It uses a text analysis algorithm to extract features of the instructions and results and classify them into success cases and failure cases. Specifically, it converts the prompts into numerical vectors using TF-IDF vectorization and applies the K-means clustering algorithm to group similar prompts. The input is the collected historical data, and the output is the classified clusters and their features.

[1400] Step 4:

[1401] Training a prompt generation model

[1402] The server trains a natural language processing (NLP) model to learn common patterns in successful cases. It uses past successful prompts to improve the model, thereby increasing the accuracy of its suggestions. The input is the analyzed successful prompts, and the output is the trained model.

[1403] Step 5:

[1404] Real-time proposal generation

[1405] The server generates appropriate prompt suggestions in real time based on the partial instructions entered by the user. It uses a trained NLP model to generate specific suggestions (e.g., "men's red shirt, size large," "red casual shirt") for the user's "red shirt" input. The input is the partial instructions from the user, and the output is the suggested specific prompt.

[1406] Step 6:

[1407] Display suggestions on the device

[1408] The terminal displays the suggestions returned by the server in real time. The user can review the suggested prompts and select an appropriate one or customize them as needed. Based on the displayed suggestions, the user can modify the input fields and send the request to the server again. The input is the suggestion from the server, and the output is the specific prompt presented to the user.

[1409] Step 7:

[1410] User's last input and search execution

[1411] The user finally confirms the selected or modified prompts and sends them to the generative model, which then returns specific search results and displays them to the user. The input is the prompts confirmed by the user, and the output is the search results from the generative model.

[1412] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1413] ---

[1414] The present invention is a system that supports users in inputting appropriate instructions to a generative model, and further combines it with an emotion engine that recognizes the user's emotions. This system is implemented through a series of procedures including interactions between a server, a terminal, and a user.

[1415] Server Processing

[1416] 1. Data Collection

[1417] The server collects and stores in a database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those inputs. This data also includes the user's emotional information recognized by the emotion engine.

[1418] 2. Data analysis

[1419] The server analyzes the instructions and results collected from the database. It uses a text analysis algorithm to extract characteristics of the instructions and results. It then categorizes them into success and failure cases and evaluates the effectiveness of the suggestions using the user's emotional information.

[1420] 3. Training the prompt generation model

[1421] The server trains a natural language processing (NLP) model to extract common patterns from success stories. It converts success stories into numerical vectors using TF-IDF vectorization and groups similar prompts using the K-means clustering algorithm. Sentiment data provided by the emotion engine is also fed into this training model.

[1422] 4. Real-time suggestions

[1423] For partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion engine.

[1424] Terminal handling

[1425] 1. User Interface (UI)

[1426] The device provides an interface for the user to input instructions to the generative model, including input fields, a submit button, a list of past inputs and suggested prompts, and even a feedback display showing the user's emotional state.

[1427] 2. Real-time assistance

[1428] When the user inputs the prompt, the device sends partial instructions to the server in real time. At the same time, the emotion engine recognizes emotions from the user's voice or text input and sends the emotion information to the server. The server then receives suggestions and displays them on the interface.

[1429] User Action

[1430] 1. Prompt Input

[1431] The user inputs a prompt to the generative model, describing the task they want to accomplish, such as "Show me a picture of a cat."

[1432] 2. Proposal acceptance

[1433] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotional information recognized by the emotion engine is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[1434] Specific examples

[1435] 1. Success stories

[1436] If a user inputs "I want to see pictures of animals" and the emotion engine recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[1437] 2. Failure example

[1438] If the user only types "animals," and the emotion engine recognizes this as a "confused" state, the server will suggest more specific instructions to the user to properly handle this ambiguous prompt. For example, it will help the user to provide more specific instructions using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?"

[1439] In this way, the system of the present invention allows users to efficiently input appropriate instructions to the generative model to achieve the desired results, and by taking into account the user's emotional state, the system can provide more personalized suggestions and improve the user experience.

[1440] The processing flow will be explained below.

[1441] ---

[1442] Step 1:

[1443] The server collects and stores in a database the instructions that the user has previously input to the generative model, the results of those inputs, and the user's feedback on those inputs, including the user's emotional information.

[1444] Step 2:

[1445] The server periodically analyzes the stored data. First, it uses a text analysis algorithm to analyze past instructions and their results, extracting characteristics of the instruction content and results. It then classifies cases into successes and failures.

[1446] Step 3:

[1447] The server trains a natural language processing (NLP) model to extract common patterns from success stories, converts success stories into numerical vectors using TF-IDF vectorization, and groups similar prompts using the K-means clustering algorithm.

[1448] Step 4:

[1449] The user inputs instructions to the generative model through a terminal, for example, "Show me a picture of a cat."

[1450] Step 5:

[1451] The emotion engine recognizes emotions from the user's input voice or text, and obtains emotion information such as whether the user is excited or confused.

[1452] Step 6:

[1453] The terminal transmits the partial instructions and emotion information input by the user to the server in real time.

[1454] Step 7:

[1455] The server generates appropriate prompt suggestions using existing NLP models based on partial instructions and emotional information, searches for similar success stories in real time, and customizes the suggestions taking into account the user's emotions.

[1456] Step 8:

[1457] The server sends the generated suggestions back to the device, which receives them and displays them as suggestions in the user interface. For example, suggestions might be "Show me pictures of cats" or "Show me pictures of dogs."

[1458] Step 9:

[1459] The user reviews the suggested prompts, selects the appropriate one, and customizes the prompt if necessary. Emotional information provided by the emotion engine is also displayed, and the user makes their selection based on that feedback.

[1460] Step 10:

[1461] The final instructions selected by the user are sent to the generative model, which generates results based on the user's specifications and returns the results to the user via the terminal.

[1462] Step 11:

[1463] The user reviews the generated results and provides feedback via their device to the server, which stores this feedback in a database for subsequent analysis and training.

[1464] Through this series of steps, the system of the present invention allows users to efficiently input appropriate instructions to the generative model and provides personalized suggestions that reflect the user's feelings, thereby enabling users to achieve the results they desire and improving their experience using the generative model.

[1465] Example 2

[1466] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1467] Conventional systems for inputting instructions to generative models lack support for users to input accurate instructions, making it difficult to obtain appropriate results. Furthermore, suggestions do not take into account the user's emotional state, which results in a lack of improvement in the user experience. The present invention aims to recognize the user's emotions and reflect them in instruction suggestions, thereby enabling users to efficiently input appropriate instructions to generative models and obtain desired results.

[1468] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1469] In this invention, the server includes input means for the user to input instructions to the generative model, database means for saving data on past instructions and their results, analysis means for analyzing the saved past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, interface means for sending the instructions input by the user to the server in real time and displaying the suggested instructions on a screen, and an emotion engine for collecting emotional information from the user and reflecting it in the analysis means and suggestion means. This not only enables the user to efficiently input appropriate instructions and obtain the desired results, but also enables personalized suggestions that take emotional information into account.

[1470] An "input means" is a device or interface that allows a user to input instructions to a generative model.

[1471] "Database means" refers to a system or storage device for storing data on past instructions and their results.

[1472] An "analysis tool" is an algorithm or system that analyzes stored past instructions and their results and classifies successes and failures.

[1473] The "suggestion means" is a device or system for suggesting new instructions to the user based on the classified success cases.

[1474] "Interface means" refers to a device or interface for transmitting instructions entered by a user to a server in real time and displaying suggested instructions.

[1475] An "emotion engine" is a system or algorithm that collects and analyzes user emotional information and reflects it in analytical and suggestion means.

[1476] "Emotional information" is data indicative of a user's emotional state, obtained from speech, text, or other forms of input.

[1477] The present invention is a system that supports users in inputting appropriate instructions to a generative model, and further combines it with an emotion engine that recognizes the user's emotions. This system is implemented through a series of procedures including interactions between a server, a terminal, and the user.

[1478] Server Processing

[1479] 1. Data Collection

[1480] The server stores the instructions that the user has previously input into the generative model, the results, and the user's feedback in a database. This data also includes the user's emotional information recognized by the emotion engine. The database uses a commercial database engine (e.g., MySQL, PostgreSQL, etc.).

[1481] 2. Data analysis

[1482] The server analyzes the instructions collected from the database and their results. It uses text analysis algorithms to extract features of the instructions and results. Natural language processing (NLP) libraries (e.g., spaCy, NLTK, etc.) are used for the analysis. Furthermore, it classifies success and failure cases and evaluates the effectiveness of suggestions using user sentiment information.

[1483] 3. Training the prompt generation model

[1484] The server trains a natural language processing model to extract common patterns from success stories. It converts success stories into numerical vectors using Term Frequency-Inverse Document Frequency (TF-IDF) vectorization and groups similar prompts using the K-means clustering algorithm. Emotion data provided by the emotion engine is also fed into this training model.

[1485] 4. Real-time suggestions

[1486] For partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion engine.

[1487] Terminal handling

[1488] 1. User Interface (UI)

[1489] The device provides an interface for the user to input instructions to the generative model. The UI includes input fields, a submit button, a history of past inputs, a list of suggested prompts, and a feedback display showing the user's emotional state. The interface is built using web-based technologies (e.g., HTML, CSS, JavaScript).

[1490] 2. Real-time assistance

[1491] As the user enters the prompts, the device sends partial instructions to the server in real time. At the same time, the emotion engine recognizes emotions from the user's voice and text input and sends the emotion information to the server. Suggestions from the server are sent to the device and displayed in the interface. This process is realized using technologies that enable two-way communication (e.g., WebSocket, HTTP / 2).

[1492] User Action

[1493] 1. Prompt Input

[1494] The user inputs a prompt to the generative model, specifying the task they want to accomplish, such as "Show me a picture of a cat."

[1495] 2. Proposal acceptance

[1496] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotional information recognized by the emotion engine is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[1497] Specific examples

[1498] 1. Success stories

[1499] If a user inputs "I want to see pictures of animals" and the emotion engine recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[1500] 2. Failure example

[1501] If the user only types "animals" and the emotion engine recognizes the "confused" state, the server will suggest more specific instructions to the user to handle this ambiguous prompt appropriately, for example, using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?" to help the user provide more specific instructions.

[1502] Prompt Sentence Examples

[1503] "Can you recommend some day trips I can take next weekend?"

[1504] I want to know some delicious pasta recipes.

[1505] "Tell me about the latest technology trends"

[1506] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1507] Step 1: User enters prompt text

[1508] Specific operation: The user inputs instructions for the generative model into the device's user interface (UI). For example, "Tell me recommended places for my next vacation."

[1509] Input: The user types the prompt text into the input field.

[1510] Output: The entered prompt is sent to the terminal as a string of data.

[1511] Step 2: Acquiring emotional information

[1512] Specific operation: The emotion engine installed in the device analyzes the emotion from the user's voice and text input. Example: The user's input is determined to be "excited."

[1513] Input: User voice and text data.

[1514] Output: Analyzed emotion information (e.g., "excited") is obtained.

[1515] Step 3: Sending data

[1516] Specific operation: The terminal sends the input prompt sentence and emotion information to the server.

[1517] Input: prompt text and emotion information.

[1518] Output: The prompt sentence and emotion information are sent to the server as a data packet.

[1519] Step 4: Save your data

[1520] Specific operation: The server stores the received prompt sentence and emotion information in the database. Example: A new record is added to the database.

[1521] Input: prompt sentence and emotion information.

[1522] Output: The new record saved in the database.

[1523] Step 5: Data acquisition and analysis

[1524] Specific operation: The server retrieves past instructions and their results from the database and applies a text analysis algorithm, e.g., converting past prompt sentences into TF-IDF vectors and extracting their features.

[1525] Input: Past instructions and result data retrieved from the database.

[1526] Output: The parsed feature vector.

[1527] Step 6: Categorize success stories and failure stories

[1528] Specific operation: The server classifies cases into success and failure based on the analyzed instructions. Example: User feedback is classified as "success" or "failure."

[1529] Input: Parsed feature vectors and feedback data.

[1530] Output: A dataset categorized into success and failure cases.

[1531] Step 7: Train the prompt generation model

[1532] What it does: The server trains a natural language processing (NLP) model to extract common patterns from success stories, e.g., grouping similar prompt sentences using TF-IDF vectors and K-means clustering.

[1533] Input: A dataset of success stories.

[1534] Output: A trained NLP model.

[1535] Step 8: Integrating Emotional Data

[1536] Specific operation: The server also incorporates the emotion data provided by the emotion engine into the training model. For example, it integrates the prompt sentence for the "excited" state and its success pattern into the NLP model.

[1537] Input: Sentiment data and a trained NLP model.

[1538] Output: An integrated model reflecting the emotion data.

[1539] Step 9: Generate real-time proposals

[1540] What it does: Based on the user's partial input, the server generates appropriate prompt suggestions in real time. For example, if you enter "travel," it generates a complete prompt such as "Tell me some recommended places."

[1541] Input: Partial prompt sentence and integrated model.

[1542] Output: A suggested prompt.

[1543] Step 10: Submit and view your proposal

[1544] Specific operation: The server sends the generated proposal to the terminal, and the terminal displays the proposal to the user on the interface.

[1545] Input: A suggested prompt sentence.

[1546] Output: Proposals displayed on the screen.

[1547] Step 11: Accepting the proposal

[1548] What happens: The user reviews the suggestions displayed on the device and selects the appropriate one or customizes it as needed.

[1549] Input: On-screen suggestions.

[1550] Output: The prompt statement that the user finally selected.

[1551] Step 12: Sending final instructions

[1552] Specific behavior: The final prompt sentence selected by the user is sent to the generative model.

[1553] Input: The final prompt statement selected by the user.

[1554] Output: The prompt sentence sent to the generative model.

[1555] (Application example 2)

[1556] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1557] Conventional instruction input systems for generative models have had problems such as users being unable to input appropriate prompts and not taking into account emotional states, resulting in a poor user experience. In particular, when dealing with customers in brick-and-mortar stores, it has been difficult to improve customer satisfaction with customer service methods that ignore the emotional states of customers.

[1558] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1559] In this invention, the server includes input means for a user to input instructions to the generative model, database means for storing data on past instructions and their results, analysis means for analyzing the stored past instructions and their results and classifying them into success cases and failure cases, suggestion means for suggesting new instructions to the user based on the classified success cases, emotion recognition means for recognizing the emotional state of the user, and interface means for transmitting the instructions input by the user to the server in real time and displaying the suggested instructions on the screen. This not only enables the user to input appropriate prompts to the generative model, but also enables personalized suggestions based on the user's emotional state.

[1560] The "input means" is an interface through which the user inputs instructions to the generative model.

[1561] The "database means" is a system or device for storing instructions previously input by the user and the results thereof.

[1562] The "analysis means" is a device or program for analyzing stored past instructions and their results, and classifying success cases and failure cases.

[1563] The "suggestion means" is a device or program for suggesting new instructions to the user based on the success cases classified by the analysis means.

[1564] The "emotion recognition means" is a device or program for detecting and recognizing the emotional state of a user from their voice and facial expression.

[1565] The "interface means" is a device or program for transmitting instructions input by the user to the server in real time and displaying suggestions from the server on the screen.

[1566] A "server" is a central processing unit that stores instructions and result data, performs analysis, and provides suggestions to users.

[1567] A "generative model" is an artificial intelligence or machine learning model that generates output based on user instructions.

[1568] The "emotional state" is information that represents the user's emotions, and is recognized via emotion recognition means.

[1569] A "prompt" is text in the form of an instruction or question to be posed to the generative model.

[1570] MODE FOR CARRYING OUT THE INVENTION

[1571] The present invention provides a system that assists users in inputting appropriate instructions to a generative model, and further combines an emotion recognition function that recognizes the user's emotional state. This system is implemented through a series of procedures including interactions between a server, a terminal, and a user.

[1572] Server Processing

[1573] The server has the following main processes:

[1574] 1. Data Collection

[1575] The server collects and stores in a database the instructions that users have previously input to the generative model, the results of those inputs, and feedback on those instructions. This data also includes emotion information recognized by the emotion recognition means.

[1576] 2. Data analysis

[1577] The server analyzes the instructions and their results collected from the database. It uses text analysis algorithms and machine learning models (e.g., TensorFlow) to extract features of the instructions and results. It also classifies success and failure cases and evaluates the effectiveness of the suggestions using user sentiment.

[1578] 3. Training the prompt generation model

[1579] The server trains a natural language processing (NLP) model to extract common patterns from success stories. It converts success stories into numerical vectors using TF-IDF vectorization and groups similar prompts using the K-means clustering algorithm. Emotion data provided by the emotion recognizer is also fed into this training model.

[1580] 4. Real-time suggestions

[1581] In response to partial instructions input by the user, the server generates appropriate prompt suggestions in real time, taking into account the user's emotional information recognized by the emotion recognition means.

[1582] Terminal handling

[1583] The terminal has the following main processes:

[1584] 1. User Interface (UI)

[1585] The device provides an interface for the user to input instructions to the generative model, including input fields, a submit button, a list of past inputs and suggested prompts, and even a feedback display showing the user's emotional state.

[1586] 2. Real-time assistance

[1587] When the user inputs a prompt, the device sends partial instructions to the server in real time. At the same time, the emotion recognition means recognizes emotions from the user's voice or text input, and sends the emotion information to the server. The device receives suggestions from the server and displays them on the interface.

[1588] User Action

[1589] The user has the following main processes:

[1590] 1. Prompt Input

[1591] The user inputs a prompt to the generative model, describing the task they want to accomplish, such as "Show me a picture of a cat."

[1592] 2. Proposal acceptance

[1593] The user reviews the suggestions displayed on the device, selects the appropriate one, or customizes it as needed. Emotion information recognized by the emotion recognition means is also displayed to help the user make a selection. The final selected instruction is sent to the generative model.

[1594] Specific examples

[1595] 1. Success stories

[1596] If the user inputs "I want to see pictures of animals" and the emotion recognition means recognizes the "excited" state, the device sends this partial prompt to the server. The server analyzes past success cases and generates successful prompts such as "Show me pictures of cats" or "Show me pictures of dogs," making the most appropriate suggestion based on the user's excitement state. The device displays these suggestions to the user, and the user selects "Show me pictures of cats."

[1597] 2. Failure example

[1598] If the user only types "animals" and the emotion recognizer recognizes the "confused" state, the server will suggest more specific instructions to the user to handle this ambiguous prompt appropriately. For example, it will help the user to provide more specific instructions using questions such as "Do you want to see pictures of animals?" or "Are you looking for pictures of a specific animal?"

[1599] Prompt Sentence Examples

[1600] 1. Success prompt

[1601] "A customer seems confused. Would you like to ask about a specific item?"

[1602] 2. Failure prompt

[1603] "The client seems anxious. Would you like me to check in and reassure them about any concerns?"

[1604] In this way, the system of the present invention allows users to efficiently input appropriate instructions to the generative model to achieve the desired results, and by taking into account the user's emotional state, the system can provide more personalized suggestions and improve the user experience.

[1605] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1606] Step 1:

[1607] The terminal provides an interface where the user can input the task they want to perform on the generative model as a prompt. For example, the user can enter "I want to see pictures of animals" in the input field. The input data is a string, which is sent to the server in real time.

[1608] Step 2:

[1609] The server receives the character string data sent from the terminal. At the same time, it uses emotion recognition means to analyze and recognize the user's emotional state from their voice input and facial expression data, and sends this emotional information together to the server. The server then stores the character string and the emotional state in a database.

[1610] Step 3:

[1611] The server retrieves and analyzes the stored past instruction data and user sentiment information. Using text analysis algorithms and machine learning models (e.g., TensorFlow), it extracts features of the instruction content and results and classifies them into success cases and failure cases. The output of this analysis is a list of similar success cases.

[1612] Step 4:

[1613] The server generates candidate prompts based on the success cases. The generated prompts also reflect the user's emotional information obtained from the emotion recognition means. For example, if the user is in an "excited" state, suggested prompts include "Show me a picture of a cat" or "Show me an image of a dog." The candidate prompts are sent to the terminal in real time.

[1614] Step 5:

[1615] The device displays the candidate prompts sent from the server on its interface. The user checks the displayed candidate prompts and selects the appropriate one. If the user selects "Show me pictures of cats," the selection data is sent from the device to the server and used as the final instruction for the generative model.

[1616] Step 6:

[1617] The server sends the final prompt selected by the user to the generative model and obtains the output from the generative model. This output is the result data (e.g., a list of cat images) generated based on the specified conditions. This result data is then sent to the terminal to be presented to the user.

[1618] Step 7:

[1619] The terminal displays the output of the generative model received from the server on its interface, allowing the user to check the displayed results and provide further operations or instructions as necessary.

[1620] The above processing steps enable users to efficiently input appropriate instructions to the generative model and receive personalized suggestions based on their emotional state.

[1621] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1622] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1623] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1624] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1625] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1626] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1627] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1628] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1629] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1630] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1631] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1632] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1633] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1634] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1635] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1636] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1637] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1638] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1639] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1640] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1641] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1642] The following is further disclosed regarding the above embodiment.

[1643] (Claim 1)

[1644] an input means for a user to input instructions to the generative model;

[1645] a database means for storing data on past instructions and their results;

[1646] an analysis means for analyzing the stored past instructions and their results and classifying success cases and failure cases;

[1647] a suggestion means for suggesting new instructions to a user based on the classified success cases;

[1648] The system includes an interface means for transmitting instructions input by a user to a server in real time and displaying suggested instructions on a screen.

[1649] (Claim 2)

[1650] 2. The system of claim 1, wherein the analyzing means includes means for extracting common instruction patterns from stored past instructions and training a natural language processing model.

[1651] (Claim 3)

[1652] 2. The system according to claim 1, wherein the suggesting means includes means for suggesting similar success stories in real time based on partial instructions input by the user.

[1653] "Example 1"

[1654] (Claim 1)

[1655] an input means for a user to input instructions to the generative model;

[1656] a data storage means for storing data on past instructions and their results;

[1657] a data analysis means for analyzing the stored past instructions and their results and classifying success cases and failure cases;

[1658] a suggestion generating means for suggesting new instructions to a user based on the classified success cases;

[1659] The system includes a user interface means for transmitting instructions input by a user to a server in real time and displaying suggested instructions on a screen.

[1660] (Claim 2)

[1661] 2. The system of claim 1, wherein the data analysis means includes means for extracting common instruction patterns from stored past instructions and training a natural language processing model.

[1662] (Claim 3)

[1663] 2. The system of claim 1, wherein the suggestion generating means includes means for suggesting similar success stories in real time based on partial instructions input by the user.

[1664] "Application Example 1"

[1665] Rewriting of original claims

[1666] (Claim 1)

[1667] an input means for a user to input instructions to the generative model;

[1668] a database means for storing data on past instructions and their results;

[1669] an analysis means for analyzing the stored past instructions and their results and classifying success cases and failure cases;

[1670] a suggestion means for suggesting new instructions to a user based on the classified success cases;

[1671] interface means for sending requests to the server in real time and displaying suggested instructions on the screen when the user inputs partial instructions;

[1672] A system including a product suggestion means for generating specific product suggestions based on past search data when a user searches for a product.

[1673] (Claim 2)

[1674] 2. The system of claim 1, wherein the analyzing means includes means for extracting common instruction patterns from stored past instructions and training a natural language processing model.

[1675] (Claim 3)

[1676] 2. The system according to claim 1, wherein the suggesting means includes means for suggesting similar success stories in real time based on partial instructions input by the user.

[1677] "Example 2: Combining Emotion Engines"

[1678] (Claim 1)

[1679] an input means for a user to input instructions to the generative model;

[1680] a database means for storing data on past instructions and their results;

[1681] an analysis means for analyzing the stored past instructions and their results and classifying success cases and failure cases;

[1682] a suggestion means for suggesting new instructions to a user based on the classified success cases;

[1683] an interface means for transmitting instructions input by a user to a server in real time and displaying suggested instructions on a screen;

[1684] A system including an emotion engine that collects user emotion information and reflects it in analysis and proposal means.

[1685] (Claim 2)

[1686] 2. The system of claim 1, wherein the analyzing means includes means for extracting common instruction patterns from stored past instructions and training a natural language processing model.

[1687] (Claim 3)

[1688] 2. The system according to claim 1, wherein the suggesting means includes means for suggesting similar success stories in real time based on partial instructions input by the user and emotional information of the user.

[1689] (Claim 4)

[1690] 10. The system of claim 1, wherein the emotion engine comprises means for analyzing a user's voice and text input to obtain the user's emotion information in real time.

[1691] "Application example 2 when combining emotion engines"

[1692] Rewritten claims:

[1693] (Claim 1)

[1694] an input means for a user to input instructions to the generative model;

[1695] a database means for storing data on past instructions and their results;

[1696] an analysis means for analyzing the stored past instructions and their results and classifying success cases and failure cases;

[1697] a suggestion means for suggesting new instructions to a user based on the classified success cases;

[1698] emotion recognition means for recognizing an emotional state of a user;

[1699] The system includes an interface means for transmitting instructions input by a user to a server in real time and displaying suggested instructions on a screen.

[1700] (Claim 2)

[1701] 2. The system of claim 1, wherein the analyzing means includes means for extracting common instruction patterns from stored past instructions and training a natural language processing model.

[1702] (Claim 3)

[1703] 2. The system of claim 1, wherein the suggestion means includes means for suggesting similar success stories in real time based on partial instructions input by the user, and means for suggesting optimal prompts based on the user's emotional state. [Explanation of symbols]

[1704] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. an input means for a user to input instructions to the generative model; a database means for storing data on past instructions and their results; an analysis means for analyzing the stored past instructions and their results and classifying success cases and failure cases; a suggestion means for suggesting new instructions to a user based on the classified success cases; The system includes an interface means for transmitting instructions input by a user to a server in real time and displaying suggested instructions on a screen.

2. 2. The system of claim 1, wherein the analyzing means includes means for extracting common instruction patterns from stored past instructions and training a natural language processing model.

3. The system according to claim 1 , wherein the suggesting means includes means for suggesting similar success stories in real time based on partial instructions input by the user.

Images