system

Abstract

We provide the system. [Solution] A selection method for users to individually configure the visual and voice characteristics of an artificial intelligence agent, A generation means for generating an artificial intelligence agent based on selected visuals and audio, A distribution method for delivering the generated artificial intelligence agent to the user's device, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Current artificial intelligence agents have the problem that it is difficult for users to customize their visuals and voices individually, and they do not sufficiently meet the needs of personalization. Also, data used by users in the past is not carried over to newly set agents, lacking unity. Furthermore, there is also a problem that there is a shortage of platforms for easily selling artificial intelligence agents independently developed by creators.

Means for Solving the Problems

[0005] This invention addresses users' personalization needs by providing a generation means that allows users to select the visual and voice characteristics of an artificial intelligence agent that can be individually configured, and then generates an agent based on that selection. Furthermore, it provides a consistent user experience by linking the agent generated based on the selected customization information with the user's past data. In addition, this invention provides a system that facilitates the distribution of creators' works by providing a means for creators to post and sell artificial intelligence agents they have developed through a marketplace.

[0006] A "user-configurable artificial intelligence agent" is an agent whose appearance, voice, and other attributes can be selected and adjusted by the user.

[0007] "Visual" refers to elements related to the appearance of an artificial intelligence agent, and includes the form of images or avatars.

[0008] "Speech" refers to the characteristics of the sound output by an artificial intelligence agent, including the manner of speaking and voice quality, as well as other acoustic characteristics.

[0009] "Generation means" refers to the process of constructing an artificial intelligence agent based on selected visual and audio information, and the system that executes it.

[0010] "Distribution method" refers to the process and system for delivering the generated artificial intelligence agent to the user's device and making it available for use.

[0011] "Means of inheriting users' past data" refers to the process and system for incorporating existing data into newly generated artificial intelligence agents to provide a consistent experience.

[0012] A "marketplace" refers to an online platform where creators can post and sell artificial intelligence agents they have developed.

[0013] "Posting method" refers to the process and system by which creators list their artificial intelligence agents on the marketplace. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] The system of this invention allows users to select the visual and voice characteristics of an artificial intelligence agent according to their preferences, thereby generating and utilizing a personalized agent. The system also provides a function for creators to sell their independently developed agents on a marketplace.

[0036] The entire system relies on interaction between the server, terminal, and user to reflect individual user settings. The following describes the processing flow in natural language.

[0037] 1. User customization

[0038] The user uses a device to select customization options for the artificial intelligence agent, including its visuals, voice, and skills. The device retrieves this selection information and sends it to the server.

[0039] 2. Agent generation

[0040] The server generates an artificial intelligence agent based on the selection information received from the user. Here, pre-prepared visual materials and audio data are combined, and natural language processing capabilities are integrated into the agent.

[0041] 3. Agent distribution and use

[0042] The server delivers the generated artificial intelligence agent to the user's device and configures it for immediate use. The user can then activate the agent on their device and have it perform various tasks.

[0043] 4. Using the Marketplace

[0044] The system provides a means for creators to post artificial intelligence agents on the marketplace, making them available for purchase by other users. The server manages the agent information posted by creators and displays it in a list on the platform.

[0045] Specific example

[0046] Personal assistant customization

[0047] User X wanted to customize a personal assistant to improve his work efficiency. He selected a friendly male voice and an anime-style character with business-specific skills on the platform. Based on these selections, the server generated an agent tailored to User X's needs and delivered it to his device.

[0048] Agent sales by creators

[0049] Creator Y developed an agent specifically for language learning. This agent supports multiple languages ​​and includes a pronunciation practice function. He posted this product on a marketplace, and it was managed on a server so that other users could purchase it.

[0050] This invention provides a new platform that explores personalization and creativity for both users and creators, enabling the generation and use of advanced AI agents.

[0051] The following describes the processing flow.

[0052] Step 1:

[0053] The user logs into the platform using their device. The device retrieves the user's login information and sends it to the server for authentication.

[0054] Step 2:

[0055] The server checks the received login information against its database and returns the authentication result to the terminal. If authentication is successful, the user is shown a customized interface.

[0056] Step 3:

[0057] Users access a customizable interface to determine the AI ​​agent's visuals, voice, and skills. The device collects the user's selected options and sends them to the server.

[0058] Step 4:

[0059] The server initiates the AI ​​agent generation process based on the user's selection data. It retrieves the selected visual and audio data and integrates it with a natural language processing model to generate a custom agent.

[0060] Step 5:

[0061] The generated AI agent is stored on the server and delivered to the user's device. The device then provides the user with an interface that allows them to launch the agent.

[0062] Step 6:

[0063] The user initiates interaction with the AI ​​agent through their device. The agent performs various tasks in response to the user's requests and updates information by communicating with the server as needed.

[0064] Step 7:

[0065] Creators access the marketplace and register their own AI agent. The information posted by the creator is sent from the device to the server, which adds it to the market list.

[0066] Step 8:

[0067] The server manages AI agents listed on the marketplace, making them available for other users to purchase. When a user attempts to purchase an agent, the server manages the purchase process and delivers the provided agent to the buyer's device.

[0068] (Example 1)

[0069] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0070] In modern information processing, there is a need for users to utilize customized information processing agents tailored to their individual needs and to maximize their functionality. However, existing systems make the process of generating personalized agents by allowing users to easily select visual representations and sounds according to their preferences cumbersome, and there are no established means for sharing or selling these agents to other users. It is necessary to solve this problem and provide an effective platform for the sale and sharing of agents among users.

[0071] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0072] In this invention, the server includes selection means for selecting a visual representation and sound of an information processing agent that can be individually configured by the user, generation means for generating an information processing agent based on the selected visual representation and sound, and transmission means for distributing the generated information processing agent to the user's information processing device. This provides a platform that allows users to easily generate and use personalized agents, and also facilitates the sale and sharing of agents.

[0073] A "selection mechanism" is an interface or mechanism that allows users to individually configure the visual and auditory settings of an information processing agent.

[0074] "Generation means" refers to a process or system for creating an information processing agent based on selected visual and auditory information.

[0075] "Transmission means" refers to a method or apparatus for transferring the generated information processing agent to the user's information processing device.

[0076] "Operation means" refers to means of interaction that allow the user to experience the functions of the generated information processing agent.

[0077] "Registration method" refers to the procedure or mechanism for making the created information processing agent accessible to other users at the sales location.

[0078] A "data integration method" is a way to incorporate historical user data into an information processing agent to provide a consistent user experience.

[0079] "Commercial transaction management measures" refer to measures or platforms for managing the publication and commercial transactions of information processing agents.

[0080] The system of this invention allows users to individually configure, generate, and utilize information processing agents. The system is primarily realized through the interaction between a server, a terminal, and a user.

[0081] Users customize the agent's visual representation and voice via a terminal. The terminal is equipped with a dedicated application or web interface, which is used to collect the selected customization information. This selected information is securely transmitted to a server. Secure communication protocols such as SSL can be used for transmission.

[0082] The server generates an information processing agent based on the customized information received from the user. Specifically, the server has a database where visual materials, audio data, and skill sets are stored. Necessary components are retrieved from this database and integrated with the artificial intelligence engine using a generation method. Programming languages ​​such as Python and natural language processing libraries are often used for this process.

[0083] The generated information processing agent is packaged and delivered to the user's terminal. Once the agent is operational on the terminal, the user can activate it and utilize its various functions. For example, considering a personal assistant agent used by a user to improve work efficiency, an agent with a friendly voice and business-specific functions can be selected, allowing for immediate task execution.

[0084] Furthermore, creators can create their own information processing agents and sell them on a dedicated marketplace. The server provides a means of managing commercial transactions along with the registration of creators, facilitating the sharing and trading of agents among users.

[0085] Examples of prompt messages include the following:

[0086] "How can we create a personal assistant agent with a friendly male voice and business-specific skills?"

[0087] This system allows users to create and use their own agents, and enables creators to open up new markets.

[0088] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0089] Step 1:

[0090] The user selects the visual and audio settings for the information processing agent using a terminal application or web interface. As input, the user reviews and selects from a list of visual design options and audio samples. The output is a set of selected visual representation IDs and audio IDs, which are then constructed as the foundational data for subsequent processing.

[0091] Step 2:

[0092] The terminal sends the user-selected customization information to the server as a data packet. Specifically, the selected information is packaged as JSON data and securely transmitted to the server over the internet. The input is the user's selected information, and the output is the received data record on the server.

[0093] Step 3:

[0094] The server analyzes the received customization information and extracts relevant visual materials, audio data, and skill sets from the database. The input is user selection information, and the output is agent data integrating these materials. Specifically, an SQL query is generated based on the selected ID, and the corresponding materials are retrieved from the database.

[0095] Step 4:

[0096] The server generates an information processing agent using the acquired material. Python scripts and natural language processing libraries are used for this generation. The input is a set of material, and the output is a working agent package. At this stage, natural language processing capabilities are incorporated, and the agent's conversational skills are formed.

[0097] Step 5:

[0098] The server packages the generated information processing agent in a compressed format and sends a download link to the user's device. The input is the generated agent and user information, and the output is the downloadable link information. Specifically, the link is notified to the device via email or app notification.

[0099] Step 6:

[0100] The user downloads and installs the agent on their terminal. The terminal then performs the necessary installation process to make the agent ready to run. The input is the downloaded agent package, and the output is the agent as a usable application.

[0101] Step 7:

[0102] The user activates an information processing agent and utilizes its customized functions. Specifically, the agent receives voice input and text commands and returns responses in natural language. The input is the user's operation instructions, and the output is the agent's corresponding action.

[0103] Step 8:

[0104] Creators use an agent development tool to create new agents and post them to the marketplace. The input is agent data provided by the creator, and the output is agent registration to the marketplace. The server manages this using commercial transaction management tools.

[0105] (Application Example 1)

[0106] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0107] When consumers shop online or in virtual environments, they need an environment where they can quickly and efficiently find products that match their individual preferences. However, conventional systems do not adequately provide personalized product suggestions that suit consumers' preferences, resulting in a time-consuming process for consumers to find what they want.

[0108] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0109] In this invention, the server includes a selection means for selecting the visual and auditory aspects of an artificial intelligence agent that can be individually configured by the user; a generation means for generating an artificial intelligence agent based on the selected visual and auditory aspects; a transmission means for sending the generated artificial intelligence agent to the user's information terminal; and a support means for suggesting products tailored to the user's preferences in a virtual store. This enables consumers to receive a personalized shopping experience and efficiently find products.

[0110] A "user" is the consumer themselves who uses the system to select and customize products.

[0111] "Individually configurable" refers to a state where users can freely select and change specific elements of the system based on their own preferences.

[0112] An "artificial intelligence agent" is a type of software that automatically performs various tasks based on user instructions.

[0113] "Visual" refers to the appearance or visual representation of the agent.

[0114] "Voice" refers to the expression of the agent's voice and sounds, and is a means of communication with the user through voice.

[0115] "Selection method" refers to an interface or technique that allows users to choose specific attributes of an agent according to their preferences.

[0116] "Generation means" refers to the function that manages the process of creating artificial intelligence agents based on selected visual and audio data.

[0117] "Transmission means" refers to the methods and functions for distributing the generated agent to the user's information terminal.

[0118] A "virtual store" is a virtual sales environment operated on the internet, where users can conduct online shopping.

[0119] "Support methods" refer to the technologies and processes used to suggest products tailored to the individual preferences of users.

[0120] To realize this invention, the following system configuration is used.

[0121] First, the server generates an artificial intelligence agent based on user input. The user sets their visual and auditory preferences through the terminal, and this information is sent to the server. The server receives the selection information and, using pre-prepared materials, generates a customized artificial intelligence agent based on the selection. This generation process uses advanced software such as Unity or Unreal Engine to construct realistic visual representations.

[0122] Next, the generated agent is sent to the user's device. The user can activate the agent within the virtual store and receive assistance in selecting products. On the device, a speech recognition API such as Google® Cloud Speech-to-Text processes the user's voice commands and queries the product database.

[0123] Cloud services such as Amazon Web Services and Microsoft Azure® are used for data processing and computation. This allows product recommendations to be generated by leveraging consumers' past purchase history and current trend information through machine learning libraries such as TENSORFLOW® and PyTorch.

[0124] As a concrete example illustrating the mechanism, consider a scenario where a user visits a virtual store through smart glasses. When the user gives a voice command such as "Show me recommended electronic products," an agent provides a list of recommended products based on past purchase history and trends.

[0125] Furthermore, the following are specific examples of prompt statements that can be used in generative AI models:

[0126] Based on the user's past purchase history and current trends, please suggest the following products: categories of the user's interests. Filter by appearance and price range according to the following specifications.

[0127] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0128] Step 1:

[0129] The user selects the visual, auditory, and skill-based features of the artificial intelligence agent through their device. The input includes the user's preferences and required functions, and this information is sent to the server via the selection mechanism. The output generates customized information for the user.

[0130] Step 2:

[0131] The server generates an artificial intelligence agent using the user's customization information received. The input consists of user selections and pre-prepared material data. Unity or Unreal Engine is used for data processing to assemble personalized visuals. The AI ​​model generated during this process is configured to function based on specific prompt statements.

[0132] Step 3:

[0133] The generated artificial intelligence agent is sent from the server to the user's terminal. The server sends customized agent information to the terminal via a transmission method, preparing it for immediate use by the user. The output is the agent installed on the user's terminal.

[0134] Step 4:

[0135] On the device, the user accesses a virtual store and activates an agent. Using speech recognition, the user can give voice commands to the agent. The entered voice commands are converted into text data via the Google Cloud Speech-to-Text API.

[0136] Step 5:

[0137] The server queries a product database based on voice commands and provides the user with recommended product information. Inputs include converted text commands and the user's purchase history and trend data. TensorFlow or PyTorch is used to run the recommendation engine, and the output is a list of recommended products displayed on the user's device.

[0138] Step 6:

[0139] The user views the details of suggested products on the device and selects or purchases products using voice or touch commands. The input commands trigger the next step, displaying detailed information about the selected product.

[0140] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0141] This invention provides a system that can recognize a user's emotions and dynamically adjust the response of an artificial intelligence agent based on those emotions. In addition to customizing visuals and voices based on the user's individual settings, the system can analyze the user's emotional state using an emotion engine. This enables a more personalized user experience.

[0142] The system's implementation is based on interaction between servers, terminals, and users. The following describes the processing flow in natural language.

[0143] 1. User customization of agents

[0144] The user uses their device to select the visual and voice characteristics of the AI ​​agent on the platform. After customization, this information is sent from the device to the server, and a user-specific agent is prepared.

[0145] 2. Integration of the Emotional Engine

[0146] The server integrates an emotion engine with the AI ​​agent and analyzes user speech and facial expression data. This engine identifies emotions from the user's voice tone, language patterns, and facial expressions.

[0147] 3. Generating dynamic responses

[0148] After the emotion engine identifies the emotion, the server generates a response from the AI ​​agent based on the data. This enables conversations that are tailored to the user's emotional state. The agent further adjusts visual elements and voice tone to provide an appropriate response.

[0149] 4. User Interaction

[0150] The device displays the interface through which the optimized AI agent interacts with the user. Each time the user interacts with the agent, the emotion engine repeats the process in real time.

[0151] Specific example

[0152] A conversational agent for stress relief

[0153] User A wanted to relax when he felt stressed at work. He began interacting with an AI agent he had previously set up. The emotion engine sensed User A's stress level from his tone of voice and brief greetings, and the agent, with a calm voice and soothing visuals, presented topics and advice to help him relax.

[0154] Emotional response agents for entertainment

[0155] User B wanted to use the agent during a family meal. The agent analyzes the laughter and cheerful tone of User B's family and provides jokes and interesting trivia to liven up the conversation.

[0156] This system allows users to enjoy emotionally appropriate interactions tailored to their individual situations, creating richer experiences.

[0157] The following describes the processing flow.

[0158] Step 1:

[0159] The user accesses the platform using their device and logs into their account. The device sends the login information to the server for authentication.

[0160] Step 2:

[0161] The server confirms that authentication was successful and displays the user interface on the terminal. The user selects the visual and voice of the AI ​​agent on the interface.

[0162] Step 3:

[0163] Based on the user's selection, the device sends visual and audio customization information to the server. The server then uses this information to begin generating the specified AI agent.

[0164] Step 4:

[0165] The server integrates an emotion engine with the AI ​​agent and performs data processing to analyze the user's emotions. The emotion engine analyzes voice tone and facial expression data to identify the user's emotional state.

[0166] Step 5:

[0167] The server dynamically adjusts the AI ​​agent's response based on the analysis results of the emotion engine. This includes the agent's utterances, voice tone, and visuals.

[0168] Step 6:

[0169] The device displays a customized AI agent interface to the user. As the user interacts with the AI ​​agent, the device sends this information to the server in real time.

[0170] Step 7:

[0171] The emotion engine analyzes the user's new emotional state, and the server generates the next response based on that analysis. The device then communicates this to the user, enabling continuous and mutually influential dialogue.

[0172] (Example 2)

[0173] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0174] Current digital agents struggle to respond flexibly to the individual emotional states of users, resulting in uniform and impersonal interactions. Traditional agents have insufficient capabilities to provide dynamic responses based on the user's specific feelings and settings, making it difficult to deliver personalized experiences.

[0175] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0176] In this invention, the server includes selection means for selecting user-modifiable visual and auditory elements, generation means for generating a digital agent based on the selected visual and auditory elements, analysis means for analyzing the user's emotions, response generation means for dynamically generating a response from the digital agent based on the analyzed emotion data, and transmission means for transmitting the generated digital agent to the user's device. This provides personalized dynamic responses that respond to the user's emotions, enabling interactions that are more tailored to individual users.

[0177] A "user" is the entity that inputs information into or customizes a system or digital agent.

[0178] "Selection method" refers to a function or interface that a user uses to change or configure visual or audio elements.

[0179] "Visual elements" are the components that make up the appearance and avatar of a digital agent, and are the visual elements that users can select.

[0180] "Speech elements" are the components that make up the types and patterns of sounds emitted by a digital agent, and are the sound components that the user can select.

[0181] "Generation means" refers to a process or technology for constructing a digital agent based on selected visual and auditory elements.

[0182] A "digital agent" is a software entity that handles user interaction and communication, and is equipped with visual and auditory elements.

[0183] "Analysis methods" refer to processing and technology used to determine emotions based on data received from users.

[0184] "Response generation means" refers to a process or technology that dynamically creates responses to be emitted by a digital agent based on emotional data obtained by analysis means.

[0185] "Transmission means" refers to the process or system structure for delivering the generated digital agent and its response to the user's device or terminal.

[0186] This invention is a system in which a user customizes a digital agent, and that agent dynamically generates responses according to the user's emotions. Its specific form is shown below.

[0187] The server generates visual and auditory elements of the digital agent based on user selections. The user configures their visual avatar and voice settings using a terminal, and these settings are sent from the terminal to the server. The server utilizes a generative AI model to build a digital agent incorporating the user's specified settings.

[0188] The user accesses the system using a terminal to initiate an interaction with the agent. The terminal captures the user's voice and facial expressions in real time and sends this data to the server. The server uses emotion analysis software to analyze this data and identify the user's emotional state. Based on the analyzed emotion data, the server uses a generative AI model to generate responses for the digital agent. This process allows the agent to engage in personalized conversations tailored to the user.

[0189] For example, if a user requests, "I'm a little tired today, so I'd like to hear something uplifting," the emotion engine analyzes the user's tone of voice and the content of their request, and the server has the agent generate an appropriate response. This allows the user to enjoy content that will help them feel refreshed.

[0190] A concrete example of a prompt in this system would be something like, "How does the AI ​​agent support the user when they feel like relaxing?" In this way, users can receive rich interactions that are tailored to their own emotional state.

[0191] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0192] Step 1:

[0193] The user accesses the platform using a device and selects the visual and audio elements of a digital agent. The device records the selected avatar design and voice type as user input. The device sends this selection data to a server, which analyzes the received data to prepare the information necessary to generate a custom-configured digital agent.

[0194] Step 2:

[0195] Based on the configuration data received, the server uses a generative AI model to embody the selected visual and audio elements. This generation process combines visual designs and audio files to create a user-specific digital agent on a computer. The generated agent data is stored in a database, linked to the user's unique ID.

[0196] Step 3:

[0197] To initiate interaction with the agent, the user inputs instructions via voice or text from the terminal. The terminal captures the user's speech and facial expressions and sends this data to the server. The server passes this input data to an emotion analysis engine for analysis, which identifies the user's current emotions. The analyzed emotion information is output as a specific emotion state.

[0198] Step 4:

[0199] The server takes the analyzed emotional state as input and uses a generative AI model to generate responses for the digital agent. Specifically, the generative AI model uses prompt sentences to create appropriate responses and arranges them into a dialogue scenario. During this process, visual representations and voice tone are also adjusted to match the emotion. The response data is temporarily stored and prepared for transmission to the terminal.

[0200] Step 5:

[0201] The device receives response data sent from the server and displays and plays it back in a way that is more user-friendly. The user receives the agent's responses visually and audibly and continues the interaction. The device tracks this interaction in real time and sends data back to the server for the next interaction as needed. This allows the user to continuously receive an emotionally resonant and personalized experience.

[0202] (Application Example 2)

[0203] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0204] Providing dynamic responses that respond to user emotions and situations is challenging in modern home appliances and artificial intelligence assistants. In particular, creating a customized experience for each user in a home environment requires systems with advanced emotion analysis and adaptive capabilities. Furthermore, there is a lack of integrated platforms that enable enjoyable experiences and stress reduction. Therefore, the challenge lies in integrating the provision of services that respond to user emotions with the management of home appliances.

[0205] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0206] In this invention, the server includes selection means for selecting the visual and auditory aspects of an intelligent agent that can be individually configured by the user; generation means for generating an intelligent agent based on the selected visual and auditory aspects; distribution means for delivering the generated intelligent agent to the user's device; emotion analysis means for analyzing the user's emotions and dynamically adjusting the intelligent agent's response; and operation means for using the intelligent agent to operate and manage household appliances in a home environment. This enables personalized service delivery adapted to each user's emotions and effective management of household appliances.

[0207] An "intelligent agent" is a program designed to provide information and various forms of support through interaction with the user.

[0208] "Visual" refers to the design and expressive elements related to the appearance and interface of an intelligent agent, and is the information that users perceive visually.

[0209] "Voice" refers to the characteristics and features of the voice output used when an intelligent agent interacts with a user, and is information transmitted through the user's hearing.

[0210] A "selection method" is a function or interface that allows the user to individually configure the visual and auditory aspects of an intelligent agent.

[0211] "Generative means" refers to a mechanism or process for constructing or realizing an intelligent agent based on selected visual and auditory information.

[0212] "Distribution means" refers to methods and technologies for transmitting the generated intelligent agent to the user's device and making it available for use.

[0213] "Emotional analysis methods" refer to technologies and systems for extracting and analyzing emotions from a user's speech, facial expressions, and other data.

[0214] "Operating means" refers to a method or technique by which an intelligent agent causes various devices and equipment in the home environment to operate according to instructions.

[0215] The system for implementing this invention analyzes the user's emotions and enables an intelligent agent to respond dynamically in the home environment based on the results. The server receives the user's specified visual and auditory settings and customizes the intelligent agent. The generated intelligent agent is delivered to the user's terminal and provides services tailored to the user's needs through actual interaction.

[0216] The server uses emotion analysis tools to receive user voice and facial expression data and processes it to identify emotions. This process includes analyzing voice tone and language patterns, and dynamically adjusting the agent's response according to the identified emotion. The software used incorporates speech recognition technology and facial recognition algorithms.

[0217] The device provides an environment where users can operate the interface visually and audibly, enabling them to control home appliances. The intelligent agent plays a role in enriching daily life by providing emotion-responsive voice guidance and controlling devices.

[0218] For example, if a user says, "I'm so tired today," the intelligent agent can sense the stress from the voice and respond with something like, "I'll play some relaxing music." In this case, an example of a prompt would be, "Good evening, how are you feeling today?" which would allow the agent to begin responding in a way that is appropriate to the user's situation.

[0219] These features allow users to enjoy a more personalized experience within their homes.

[0220] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0221] Step 1:

[0222] The user uses a terminal to select the visual and auditory characteristics of the intelligent agent. The terminal receives input from the user and sends the selected configuration data to the server. In this process, the visual theme and auditory characteristics specified by the user are processed as data. The output is the customized configuration data transmitted to the server.

[0223] Step 2:

[0224] The server generates an intelligent agent based on the received configuration data. The server uses the generated AI model to construct an agent that responds to the selected visuals and sounds. Configuration data is used as input, and a customized intelligent agent is generated as output. This step specifically involves supplying data to the AI ​​model and creating the agent as a result.

[0225] Step 3:

[0226] The generated intelligent agent is delivered from the server to the user's terminal. The server transmits data to the terminal via the network connection. The input is the data of the generated agent, and the output is the completion of delivery to the user's terminal. In this process, data transfer and verification are specifically performed.

[0227] Step 4:

[0228] The terminal prepares to begin interaction with the user. The user initiates a conversation with the agent and provides voice input to the terminal. The terminal collects the voice data and sends it back to the server. The input is the user's voice, and the output is sent to the server as voice data. Specifically, voice recording and data transfer occur at this time.

[0229] Step 5:

[0230] The server performs sentiment analysis based on voice data. It utilizes a generative AI model to analyze the user's emotions from their voice tone and content. The input for this step is voice data, and the output is identified sentiment data. Specifically, the analysis involves data analysis using speech recognition technology.

[0231] Step 6:

[0232] The server utilizes the analysis results to generate responses from the intelligent agent. It supplies prompt sentences corresponding to the generated sentiment data to the AI ​​model, creating an adapted response. The input is sentiment data, and the output is the adjusted agent response. In this process, the response is specifically customized by supplying prompt sentences.

[0233] Step 7:

[0234] The terminal provides the user with a pre-configured response. The terminal provides feedback to the user through display and audio output. The input is response data from the server, and the output is the user's perceived feedback. Specifically, the terminal plays audio and displays information to allow the user to confirm the response.

[0235] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0236] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0237] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0238] [Second Embodiment]

[0239] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0240] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0241] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0242] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0243] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0244] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0245] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0246] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0247] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0248] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0249] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0250] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0251] The system of this invention allows users to select the visual and voice characteristics of an artificial intelligence agent according to their preferences, thereby generating and utilizing a personalized agent. The system also provides a function for creators to sell their independently developed agents on a marketplace.

[0252] The entire system relies on interaction between the server, terminal, and user to reflect individual user settings. The following describes the processing flow in natural language.

[0253] 1. User customization

[0254] The user uses a device to select customization options for the artificial intelligence agent, including its visuals, voice, and skills. The device retrieves this selection information and sends it to the server.

[0255] 2. Agent generation

[0256] The server generates an artificial intelligence agent based on the selection information received from the user. Here, pre-prepared visual materials and audio data are combined, and natural language processing capabilities are integrated into the agent.

[0257] 3. Agent distribution and use

[0258] The server delivers the generated artificial intelligence agent to the user's device and configures it for immediate use. The user can then activate the agent on their device and have it perform various tasks.

[0259] 4. Using the Marketplace

[0260] The system provides a means for creators to post artificial intelligence agents on the marketplace, making them available for purchase by other users. The server manages the agent information posted by creators and displays it in a list on the platform.

[0261] Specific example

[0262] Personal assistant customization

[0263] User X wanted to customize a personal assistant to improve his work efficiency. He selected a friendly male voice and an anime-style character with business-specific skills on the platform. Based on these selections, the server generated an agent tailored to User X's needs and delivered it to his device.

[0264] Agent sales by creators

[0265] Creator Y developed an agent specifically for language learning. This agent supports multiple languages ​​and includes a pronunciation practice function. He posted this product on a marketplace, and it was managed on a server so that other users could purchase it.

[0266] This invention provides a new platform that explores personalization and creativity for both users and creators, enabling the generation and use of advanced AI agents.

[0267] The following describes the processing flow.

[0268] Step 1:

[0269] The user logs into the platform using their device. The device retrieves the user's login information and sends it to the server for authentication.

[0270] Step 2:

[0271] The server checks the received login information against its database and returns the authentication result to the terminal. If authentication is successful, the user is shown a customized interface.

[0272] Step 3:

[0273] Users access a customizable interface to determine the AI ​​agent's visuals, voice, and skills. The device collects the user's selected options and sends them to the server.

[0274] Step 4:

[0275] The server starts the generation process of the AI agent based on the user's selection data. It acquires the selected visual and audio data, integrates the natural language processing model, and generates a custom agent.

[0276] Step 5:

[0277] The generated AI agent is saved on the server and delivered to the user's terminal. The terminal provides the user with an interface that can launch the agent.

[0278] Step 6:

[0279] The user starts interacting with the AI agent through the terminal. The agent executes various tasks according to the user's requests and updates information by communicating with the server if necessary.

[0280] Step 7:

[0281] The creator accesses the marketplace and registers their own AI agent. The terminal sends the information posted by the creator to the server, and the server adds this to the market list.

[0282] Step 8:

[0283] The server manages the AI agents listed on the marketplace and makes them available for other users to purchase. When a user attempts to purchase, the server manages the purchase process and delivers the provided agent to the purchaser's terminal.

[0284] (Example 1)

[0285] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0286] In modern information processing, it is required that users can utilize customized information processing agents according to their individual needs and maximize their functions. However, in existing systems, the process for users to easily select visual representations and voices according to their preferences and generate individualized agents is complicated, and there is no established means for sharing and selling such agents to other users. It is necessary to solve this problem and provide an effective sales and sharing platform for agents among users.

[0287] The specific processing by the specific processing unit 290 of the data processing device 12 in Embodiment 1 is realized by the following means.

[0288] In this invention, the server includes a selection means for the user to select the visual representation and voice of an information processing agent that can be individually set, a generation means for generating an information processing agent based on the selected visual representation and voice, and a transmission means for delivering the generated information processing agent to the user's information processing device. Thereby, users can easily generate and utilize personalized agents, and a platform is provided that can smoothly conduct the sales and sharing of agents.

[0289] The "selection means" is an interface or mechanism for the user to individually set the visual and voice of an information processing agent.

[0290] The "generation means" is a process or system for creating an information processing agent based on the selected visual and voice information.

[0291] The "transmission means" is a method or device for transferring the generated information processing agent to the user's information processing device.

[0292] The "operation means" is a means for the user to experience the functions of the generated information processing agent through interaction.

[0293] "Registration method" refers to the procedure or mechanism for making the created information processing agent accessible to other users at the sales location.

[0294] A "data integration method" is a way to incorporate historical user data into an information processing agent to provide a consistent user experience.

[0295] "Commercial transaction management measures" refer to measures or platforms for managing the publication and commercial transactions of information processing agents.

[0296] The system of this invention allows users to individually configure, generate, and utilize information processing agents. The system is primarily realized through the interaction between a server, a terminal, and a user.

[0297] Users customize the agent's visual representation and voice via a terminal. The terminal is equipped with a dedicated application or web interface, which is used to collect the selected customization information. This selected information is securely transmitted to a server. Secure communication protocols such as SSL can be used for transmission.

[0298] The server generates an information processing agent based on the customized information received from the user. Specifically, the server has a database where visual materials, audio data, and skill sets are stored. Necessary components are retrieved from this database and integrated with the artificial intelligence engine using a generation method. Programming languages ​​such as Python and natural language processing libraries are often used for this process.

[0299] The generated information processing agent is packaged and distributed to the user's terminal. When the agent becomes operational on the terminal, the user can start the agent and utilize various functions. As a specific example, considering a personal assistant agent that a user uses to improve work efficiency, an agent with a friendly voice and business-specific functions is selected, and tasks can be executed immediately.

[0300] Furthermore, a creator can create their own information processing agent and sell it on a dedicated marketplace. Along with the registration from the creator, the server provides means for business transaction management to facilitate the sharing and trading of agents among users.

[0301] Examples of prompt sentences include the following.

[0302] "How to generate a personal assistant agent with a friendly male voice and skills specialized in business?"

[0303] With this system, the user can create and use their own agent, and the creator can开拓 a new market.

[0304] The flow of the specific process in Example 1 will be described using FIG. 11.

[0305] Step 1:

[0306] The user selects the visual and audio settings of the information processing agent using the terminal application or web interface. As input, the user checks the option list of visual designs and audio samples and makes a selection. The output is a set of selected visual representation IDs and audio IDs, which are constructed as the base data for the next process.

[0307] Step 2:

[0308] The terminal sends the user-selected customization information to the server as a data packet. Specifically, the selected information is packaged as JSON data and securely transmitted to the server over the internet. The input is the user's selected information, and the output is the received data record on the server.

[0309] Step 3:

[0310] The server analyzes the received customization information and extracts relevant visual materials, audio data, and skill sets from the database. The input is user selection information, and the output is agent data integrating these materials. Specifically, an SQL query is generated based on the selected ID, and the corresponding materials are retrieved from the database.

[0311] Step 4:

[0312] The server generates an information processing agent using the acquired material. Python scripts and natural language processing libraries are used for this generation. The input is a set of material, and the output is a working agent package. At this stage, natural language processing capabilities are incorporated, and the agent's conversational skills are formed.

[0313] Step 5:

[0314] The server packages the generated information processing agent in a compressed format and sends a download link to the user's device. The input is the generated agent and user information, and the output is the downloadable link information. Specifically, the link is notified to the device via email or app notification.

[0315] Step 6:

[0316] The user downloads and installs the agent on their terminal. The terminal then performs the necessary installation process to make the agent ready to run. The input is the downloaded agent package, and the output is the agent as a usable application.

[0317] Step 7:

[0318] The user activates an information processing agent and utilizes its customized functions. Specifically, the agent receives voice input and text commands and returns responses in natural language. The input is the user's operation instructions, and the output is the agent's corresponding action.

[0319] Step 8:

[0320] Creators use an agent development tool to create new agents and post them to the marketplace. The input is agent data provided by the creator, and the output is agent registration to the marketplace. The server manages this using commercial transaction management tools.

[0321] (Application Example 1)

[0322] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0323] When consumers shop online or in virtual environments, they need an environment where they can quickly and efficiently find products that match their individual preferences. However, conventional systems do not adequately provide personalized product suggestions that suit consumers' preferences, resulting in a time-consuming process for consumers to find what they want.

[0324] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0325] In this invention, the server includes a selection means for selecting the visual and auditory aspects of an artificial intelligence agent that can be individually configured by the user; a generation means for generating an artificial intelligence agent based on the selected visual and auditory aspects; a transmission means for sending the generated artificial intelligence agent to the user's information terminal; and a support means for suggesting products tailored to the user's preferences in a virtual store. This enables consumers to receive a personalized shopping experience and efficiently find products.

[0326] A "user" is the consumer themselves who uses the system to select and customize products.

[0327] "Individually configurable" refers to a state where users can freely select and change specific elements of the system based on their own preferences.

[0328] An "artificial intelligence agent" is a type of software that automatically performs various tasks based on user instructions.

[0329] "Visual" refers to the appearance or visual representation of the agent.

[0330] "Voice" refers to the expression of the agent's voice and sounds, and is a means of communication with the user through voice.

[0331] "Selection method" refers to an interface or technique that allows users to choose specific attributes of an agent according to their preferences.

[0332] "Generation means" refers to the function that manages the process of creating artificial intelligence agents based on selected visual and audio data.

[0333] "Transmission means" refers to the methods and functions for distributing the generated agent to the user's information terminal.

[0334] A "virtual store" is a virtual sales environment operated on the internet, where users can conduct online shopping.

[0335] "Support methods" refer to the technologies and processes used to suggest products tailored to the individual preferences of users.

[0336] To realize this invention, the following system configuration is used.

[0337] First, the server generates an artificial intelligence agent based on user input. The user sets their visual and auditory preferences through the terminal, and this information is sent to the server. The server receives the selection information and, using pre-prepared materials, generates a customized artificial intelligence agent based on the selection. This generation process uses advanced software such as Unity or Unreal Engine to construct realistic visual representations.

[0338] Next, the generated agent is sent to the user's device. The user can activate the agent within the virtual store and receive assistance in selecting products. On the device, a speech recognition API such as Google Cloud Speech-to-Text processes the user's voice commands and queries the product database.

[0339] Cloud services such as Amazon Web Services and Microsoft Azure are used for data processing and computation. This allows product recommendations to be generated by leveraging consumers' past purchase history and current trend information through machine learning libraries such as TensorFlow and PyTorch.

[0340] As a concrete example illustrating the mechanism, consider a scenario where a user visits a virtual store through smart glasses. When the user gives a voice command such as "Show me recommended electronic products," an agent provides a list of recommended products based on past purchase history and trends.

[0341] Furthermore, the following are specific examples of prompt statements that can be used in generative AI models:

[0342] Based on the user's past purchase history and current trends, please suggest the following products: categories of the user's interests. Filter by appearance and price range according to the following specifications.

[0343] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0344] Step 1:

[0345] The user selects the visual, auditory, and skill-based features of the artificial intelligence agent through their device. The input includes the user's preferences and required functions, and this information is sent to the server via the selection mechanism. The output generates customized information for the user.

[0346] Step 2:

[0347] The server generates an artificial intelligence agent using the user's customization information received. The input consists of user selections and pre-prepared material data. Unity or Unreal Engine is used for data processing to assemble personalized visuals. The AI ​​model generated during this process is configured to function based on specific prompt statements.

[0348] Step 3:

[0349] The generated artificial intelligence agent is sent from the server to the user's terminal. The server sends customized agent information to the terminal via a transmission method, preparing it for immediate use by the user. The output is the agent installed on the user's terminal.

[0350] Step 4:

[0351] On the device, the user accesses a virtual store and activates an agent. Using speech recognition, the user can give voice commands to the agent. The entered voice commands are converted into text data via the Google Cloud Speech-to-Text API.

[0352] Step 5:

[0353] The server queries a product database based on voice commands and provides the user with recommended product information. Inputs include converted text commands and the user's purchase history and trend data. TensorFlow or PyTorch is used to run the recommendation engine, and the output is a list of recommended products displayed on the user's device.

[0354] Step 6:

[0355] The user views the details of suggested products on the device and selects or purchases products using voice or touch commands. The input commands trigger the next step, displaying detailed information about the selected product.

[0356] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0357] This invention provides a system that can recognize a user's emotions and dynamically adjust the response of an artificial intelligence agent based on those emotions. In addition to customizing visuals and voices based on the user's individual settings, the system can analyze the user's emotional state using an emotion engine. This enables a more personalized user experience.

[0358] The system's implementation is based on interaction between servers, terminals, and users. The following describes the processing flow in natural language.

[0359] 1. User customization of agents

[0360] The user uses their device to select the visual and voice characteristics of the AI ​​agent on the platform. After customization, this information is sent from the device to the server, and a user-specific agent is prepared.

[0361] 2. Integration of the Emotional Engine

[0362] The server integrates an emotion engine with the AI ​​agent and analyzes user speech and facial expression data. This engine identifies emotions from the user's voice tone, language patterns, and facial expressions.

[0363] 3. Generating dynamic responses

[0364] After the emotion engine identifies the emotion, the server generates a response from the AI ​​agent based on the data. This enables conversations that are tailored to the user's emotional state. The agent further adjusts visual elements and voice tone to provide an appropriate response.

[0365] 4. User Interaction

[0366] The device displays the interface through which the optimized AI agent interacts with the user. Each time the user interacts with the agent, the emotion engine repeats the process in real time.

[0367] Specific example

[0368] A conversational agent for stress relief

[0369] User A wanted to relax when he felt stressed at work. He began interacting with an AI agent he had previously set up. The emotion engine sensed User A's stress level from his tone of voice and brief greetings, and the agent, with a calm voice and soothing visuals, presented topics and advice to help him relax.

[0370] Emotional response agents for entertainment

[0371] User B wanted to use the agent during a family meal. The agent analyzes the laughter and cheerful tone of User B's family and provides jokes and interesting trivia to liven up the conversation.

[0372] This system allows users to enjoy emotionally appropriate interactions tailored to their individual situations, creating richer experiences.

[0373] The following describes the processing flow.

[0374] Step 1:

[0375] The user accesses the platform using their device and logs into their account. The device sends the login information to the server for authentication.

[0376] Step 2:

[0377] The server confirms that authentication was successful and displays the user interface on the terminal. The user selects the visual and voice of the AI ​​agent on the interface.

[0378] Step 3:

[0379] Based on the user's selection, the device sends visual and audio customization information to the server. The server then uses this information to begin generating the specified AI agent.

[0380] Step 4:

[0381] The server integrates an emotion engine with the AI ​​agent and performs data processing to analyze the user's emotions. The emotion engine analyzes voice tone and facial expression data to identify the user's emotional state.

[0382] Step 5:

[0383] The server dynamically adjusts the AI ​​agent's response based on the analysis results of the emotion engine. This includes the agent's utterances, voice tone, and visuals.

[0384] Step 6:

[0385] The device displays a customized AI agent interface to the user. As the user interacts with the AI ​​agent, the device sends this information to the server in real time.

[0386] Step 7:

[0387] The emotion engine analyzes the user's new emotional state, and the server generates the next response based on that analysis. The device then communicates this to the user, enabling continuous and mutually influential dialogue.

[0388] (Example 2)

[0389] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0390] Current digital agents struggle to respond flexibly to the individual emotional states of users, resulting in uniform and impersonal interactions. Traditional agents have insufficient capabilities to provide dynamic responses based on the user's specific feelings and settings, making it difficult to deliver personalized experiences.

[0391] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0392] In this invention, the server includes selection means for selecting user-modifiable visual and auditory elements, generation means for generating a digital agent based on the selected visual and auditory elements, analysis means for analyzing the user's emotions, response generation means for dynamically generating a response from the digital agent based on the analyzed emotion data, and transmission means for transmitting the generated digital agent to the user's device. This provides personalized dynamic responses that respond to the user's emotions, enabling interactions that are more tailored to individual users.

[0393] A "user" is the entity that inputs information into or customizes a system or digital agent.

[0394] "Selection method" refers to a function or interface that a user uses to change or configure visual or audio elements.

[0395] "Visual elements" are the components that make up the appearance and avatar of a digital agent, and are the visual elements that users can select.

[0396] "Speech elements" are the components that make up the types and patterns of sounds emitted by a digital agent, and are the sound components that the user can select.

[0397] "Generation means" refers to a process or technology for constructing a digital agent based on selected visual and auditory elements.

[0398] A "digital agent" is a software entity that handles user interaction and communication, and is equipped with visual and auditory elements.

[0399] "Analysis methods" refer to processing and technology used to determine emotions based on data received from users.

[0400] "Response generation means" refers to a process or technology that dynamically creates responses to be emitted by a digital agent based on emotional data obtained by analysis means.

[0401] "Transmission means" refers to the process or system structure for delivering the generated digital agent and its response to the user's device or terminal.

[0402] This invention is a system in which a user customizes a digital agent, and that agent dynamically generates responses according to the user's emotions. Its specific form is shown below.

[0403] The server generates visual and auditory elements of the digital agent based on user selections. The user configures their visual avatar and voice settings using a terminal, and these settings are sent from the terminal to the server. The server utilizes a generative AI model to build a digital agent incorporating the user's specified settings.

[0404] The user accesses the system using a terminal to initiate an interaction with the agent. The terminal captures the user's voice and facial expressions in real time and sends this data to the server. The server uses emotion analysis software to analyze this data and identify the user's emotional state. Based on the analyzed emotion data, the server uses a generative AI model to generate responses for the digital agent. This process allows the agent to engage in personalized conversations tailored to the user.

[0405] For example, if a user requests, "I'm a little tired today, so I'd like to hear something uplifting," the emotion engine analyzes the user's tone of voice and the content of their request, and the server has the agent generate an appropriate response. This allows the user to enjoy content that will help them feel refreshed.

[0406] A concrete example of a prompt in this system would be something like, "How does the AI ​​agent support the user when they feel like relaxing?" In this way, users can receive rich interactions that are tailored to their own emotional state.

[0407] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0408] Step 1:

[0409] The user accesses the platform using a device and selects the visual and audio elements of a digital agent. The device records the selected avatar design and voice type as user input. The device sends this selection data to a server, which analyzes the received data to prepare the information necessary to generate a custom-configured digital agent.

[0410] Step 2:

[0411] Based on the configuration data received, the server uses a generative AI model to embody the selected visual and audio elements. This generation process combines visual designs and audio files to create a user-specific digital agent on a computer. The generated agent data is stored in a database, linked to the user's unique ID.

[0412] Step 3:

[0413] To initiate interaction with the agent, the user inputs instructions via voice or text from the terminal. The terminal captures the user's speech and facial expressions and sends this data to the server. The server passes this input data to an emotion analysis engine for analysis, which identifies the user's current emotions. The analyzed emotion information is output as a specific emotion state.

[0414] Step 4:

[0415] The server takes the analyzed emotional state as input and uses a generative AI model to generate responses for the digital agent. Specifically, the generative AI model uses prompt sentences to create appropriate responses and arranges them into a dialogue scenario. During this process, visual representations and voice tone are also adjusted to match the emotion. The response data is temporarily stored and prepared for transmission to the terminal.

[0416] Step 5:

[0417] The device receives response data sent from the server and displays and plays it back in a way that is more user-friendly. The user receives the agent's responses visually and audibly and continues the interaction. The device tracks this interaction in real time and sends data back to the server for the next interaction as needed. This allows the user to continuously receive an emotionally resonant and personalized experience.

[0418] (Application Example 2)

[0419] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0420] Providing dynamic responses that respond to user emotions and situations is challenging in modern home appliances and artificial intelligence assistants. In particular, creating a customized experience for each user in a home environment requires systems with advanced emotion analysis and adaptive capabilities. Furthermore, there is a lack of integrated platforms that enable enjoyable experiences and stress reduction. Therefore, the challenge lies in integrating the provision of services that respond to user emotions with the management of home appliances.

[0421] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0422] In this invention, the server includes selection means for selecting the visual and auditory aspects of an intelligent agent that can be individually configured by the user; generation means for generating an intelligent agent based on the selected visual and auditory aspects; distribution means for delivering the generated intelligent agent to the user's device; emotion analysis means for analyzing the user's emotions and dynamically adjusting the intelligent agent's response; and operation means for using the intelligent agent to operate and manage household appliances in a home environment. This enables personalized service delivery adapted to each user's emotions and effective management of household appliances.

[0423] An "intelligent agent" is a program designed to provide information and various forms of support through interaction with the user.

[0424] "Visual" refers to the design and expressive elements related to the appearance and interface of an intelligent agent, and is the information that users perceive visually.

[0425] "Voice" refers to the characteristics and features of the voice output used when an intelligent agent interacts with a user, and is information transmitted through the user's hearing.

[0426] A "selection method" is a function or interface that allows the user to individually configure the visual and auditory aspects of an intelligent agent.

[0427] "Generative means" refers to a mechanism or process for constructing or realizing an intelligent agent based on selected visual and auditory information.

[0428] "Distribution means" refers to methods and technologies for transmitting the generated intelligent agent to the user's device and making it available for use.

[0429] "Emotional analysis methods" refer to technologies and systems for extracting and analyzing emotions from a user's speech, facial expressions, and other data.

[0430] "Operating means" refers to a method or technique by which an intelligent agent causes various devices and equipment in the home environment to operate according to instructions.

[0431] The system for implementing this invention analyzes the user's emotions and enables an intelligent agent to respond dynamically in the home environment based on the results. The server receives the user's specified visual and auditory settings and customizes the intelligent agent. The generated intelligent agent is delivered to the user's terminal and provides services tailored to the user's needs through actual interaction.

[0432] The server uses emotion analysis tools to receive user voice and facial expression data and processes it to identify emotions. This process includes analyzing voice tone and language patterns, and dynamically adjusting the agent's response according to the identified emotion. The software used incorporates speech recognition technology and facial recognition algorithms.

[0433] The device provides an environment where users can operate the interface visually and audibly, enabling them to control home appliances. The intelligent agent plays a role in enriching daily life by providing emotion-responsive voice guidance and controlling devices.

[0434] For example, if a user says, "I'm so tired today," the intelligent agent can sense the stress from the voice and respond with something like, "I'll play some relaxing music." In this case, an example of a prompt would be, "Good evening, how are you feeling today?" which would allow the agent to begin responding in a way that is appropriate to the user's situation.

[0435] These features allow users to enjoy a more personalized experience within their homes.

[0436] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0437] Step 1:

[0438] The user uses a terminal to select the visual and auditory characteristics of the intelligent agent. The terminal receives input from the user and sends the selected configuration data to the server. In this process, the visual theme and auditory characteristics specified by the user are processed as data. The output is the customized configuration data transmitted to the server.

[0439] Step 2:

[0440] The server generates an intelligent agent based on the received configuration data. The server uses the generated AI model to construct an agent that responds to the selected visuals and sounds. Configuration data is used as input, and a customized intelligent agent is generated as output. This step specifically involves supplying data to the AI ​​model and creating the agent as a result.

[0441] Step 3:

[0442] The generated intelligent agent is delivered from the server to the user's terminal. The server transmits data to the terminal via the network connection. The input is the data of the generated agent, and the output is the completion of delivery to the user's terminal. In this process, data transfer and verification are specifically performed.

[0443] Step 4:

[0444] The terminal prepares to begin interaction with the user. The user initiates a conversation with the agent and provides voice input to the terminal. The terminal collects the voice data and sends it back to the server. The input is the user's voice, and the output is sent to the server as voice data. Specifically, voice recording and data transfer occur at this time.

[0445] Step 5:

[0446] The server performs sentiment analysis based on voice data. It utilizes a generative AI model to analyze the user's emotions from their voice tone and content. The input for this step is voice data, and the output is identified sentiment data. Specifically, the analysis involves data analysis using speech recognition technology.

[0447] Step 6:

[0448] The server utilizes the analysis results to generate responses from the intelligent agent. It supplies prompt sentences corresponding to the generated sentiment data to the AI ​​model, creating an adapted response. The input is sentiment data, and the output is the adjusted agent response. In this process, the response is specifically customized by supplying prompt sentences.

[0449] Step 7:

[0450] The terminal provides the user with a pre-configured response. The terminal provides feedback to the user through display and audio output. The input is response data from the server, and the output is the user's perceived feedback. Specifically, the terminal plays audio and displays information to allow the user to confirm the response.

[0451] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0452] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0453] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0454] [Third Embodiment]

[0455] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0456] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0457] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0458] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0459] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0460] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0461] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0462] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0463] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0464] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0465] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0466] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0467] The system of this invention allows users to select the visual and voice characteristics of an artificial intelligence agent according to their preferences, thereby generating and utilizing a personalized agent. The system also provides a function for creators to sell their independently developed agents on a marketplace.

[0468] The entire system relies on interaction between the server, terminal, and user to reflect individual user settings. The following describes the processing flow in natural language.

[0469] 1. User customization

[0470] The user uses a device to select customization options for the artificial intelligence agent, including its visuals, voice, and skills. The device retrieves this selection information and sends it to the server.

[0471] 2. Agent generation

[0472] The server generates an artificial intelligence agent based on the selection information received from the user. Here, pre-prepared visual materials and audio data are combined, and natural language processing capabilities are integrated into the agent.

[0473] 3. Agent distribution and use

[0474] The server delivers the generated artificial intelligence agent to the user's device and configures it for immediate use. The user can then activate the agent on their device and have it perform various tasks.

[0475] 4. Using the Marketplace

[0476] The system provides a means for creators to post artificial intelligence agents on the marketplace, making them available for purchase by other users. The server manages the agent information posted by creators and displays it in a list on the platform.

[0477] Specific example

[0478] Personal assistant customization

[0479] User X wanted to customize a personal assistant to improve his work efficiency. He selected a friendly male voice and an anime-style character with business-specific skills on the platform. Based on these selections, the server generated an agent tailored to User X's needs and delivered it to his device.

[0480] Agent sales by creators

[0481] Creator Y developed an agent specifically for language learning. This agent supports multiple languages ​​and includes a pronunciation practice function. He posted this product on a marketplace, and it was managed on a server so that other users could purchase it.

[0482] This invention provides a new platform that explores personalization and creativity for both users and creators, enabling the generation and use of advanced AI agents.

[0483] The following describes the processing flow.

[0484] Step 1:

[0485] The user logs into the platform using their device. The device retrieves the user's login information and sends it to the server for authentication.

[0486] Step 2:

[0487] The server checks the received login information against its database and returns the authentication result to the terminal. If authentication is successful, the user is shown a customized interface.

[0488] Step 3:

[0489] Users access a customizable interface to determine the AI ​​agent's visuals, voice, and skills. The device collects the user's selected options and sends them to the server.

[0490] Step 4:

[0491] The server initiates the AI ​​agent generation process based on the user's selection data. It retrieves the selected visual and audio data and integrates it with a natural language processing model to generate a custom agent.

[0492] Step 5:

[0493] The generated AI agent is stored on the server and delivered to the user's device. The device then provides the user with an interface that allows them to launch the agent.

[0494] Step 6:

[0495] The user initiates interaction with the AI ​​agent through their device. The agent performs various tasks in response to the user's requests and updates information by communicating with the server as needed.

[0496] Step 7:

[0497] Creators access the marketplace and register their own AI agent. The information posted by the creator is sent from the device to the server, which adds it to the market list.

[0498] Step 8:

[0499] The server manages AI agents listed on the marketplace, making them available for other users to purchase. When a user attempts to purchase an agent, the server manages the purchase process and delivers the provided agent to the buyer's device.

[0500] (Example 1)

[0501] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0502] In modern information processing, there is a need for users to utilize customized information processing agents tailored to their individual needs and to maximize their functionality. However, existing systems make the process of generating personalized agents by allowing users to easily select visual representations and sounds according to their preferences cumbersome, and there are no established means for sharing or selling these agents to other users. It is necessary to solve this problem and provide an effective platform for the sale and sharing of agents among users.

[0503] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0504] In this invention, the server includes selection means for selecting a visual representation and sound of an information processing agent that can be individually configured by the user, generation means for generating an information processing agent based on the selected visual representation and sound, and transmission means for distributing the generated information processing agent to the user's information processing device. This provides a platform that allows users to easily generate and use personalized agents, and also facilitates the sale and sharing of agents.

[0505] A "selection mechanism" is an interface or mechanism that allows users to individually configure the visual and auditory settings of an information processing agent.

[0506] "Generation means" refers to a process or system for creating an information processing agent based on selected visual and auditory information.

[0507] "Transmission means" refers to a method or apparatus for transferring the generated information processing agent to the user's information processing device.

[0508] "Operation means" refers to means of interaction that allow the user to experience the functions of the generated information processing agent.

[0509] "Registration method" refers to the procedure or mechanism for making the created information processing agent accessible to other users at the sales location.

[0510] A "data integration method" is a way to incorporate historical user data into an information processing agent to provide a consistent user experience.

[0511] "Commercial transaction management measures" refer to measures or platforms for managing the publication and commercial transactions of information processing agents.

[0512] The system of this invention allows users to individually configure, generate, and utilize information processing agents. The system is primarily realized through the interaction between a server, a terminal, and a user.

[0513] Users customize the agent's visual representation and voice via a terminal. The terminal is equipped with a dedicated application or web interface, which is used to collect the selected customization information. This selected information is securely transmitted to a server. Secure communication protocols such as SSL can be used for transmission.

[0514] The server generates an information processing agent based on the customized information received from the user. Specifically, the server has a database where visual materials, audio data, and skill sets are stored. Necessary components are retrieved from this database and integrated with the artificial intelligence engine using a generation method. Programming languages ​​such as Python and natural language processing libraries are often used for this process.

[0515] The generated information processing agent is packaged and delivered to the user's terminal. Once the agent is operational on the terminal, the user can activate it and utilize its various functions. For example, considering a personal assistant agent used by a user to improve work efficiency, an agent with a friendly voice and business-specific functions can be selected, allowing for immediate task execution.

[0516] Furthermore, creators can create their own information processing agents and sell them on a dedicated marketplace. The server provides a means of managing commercial transactions along with the registration of creators, facilitating the sharing and trading of agents among users.

[0517] Examples of prompt messages include the following:

[0518] "How can we create a personal assistant agent with a friendly male voice and business-specific skills?"

[0519] This system allows users to create and use their own agents, and enables creators to open up new markets.

[0520] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0521] Step 1:

[0522] The user selects the visual and audio settings for the information processing agent using a terminal application or web interface. As input, the user reviews and selects from a list of visual design options and audio samples. The output is a set of selected visual representation IDs and audio IDs, which are then constructed as the foundational data for subsequent processing.

[0523] Step 2:

[0524] The terminal sends the user-selected customization information to the server as a data packet. Specifically, the selected information is packaged as JSON data and securely transmitted to the server over the internet. The input is the user's selected information, and the output is the received data record on the server.

[0525] Step 3:

[0526] The server analyzes the received customization information and extracts relevant visual materials, audio data, and skill sets from the database. The input is user selection information, and the output is agent data integrating these materials. Specifically, an SQL query is generated based on the selected ID, and the corresponding materials are retrieved from the database.

[0527] Step 4:

[0528] The server generates an information processing agent using the acquired material. Python scripts and natural language processing libraries are used for this generation. The input is a set of material, and the output is a working agent package. At this stage, natural language processing capabilities are incorporated, and the agent's conversational skills are formed.

[0529] Step 5:

[0530] The server packages the generated information processing agent in a compressed format and sends a download link to the user's device. The input is the generated agent and user information, and the output is the downloadable link information. Specifically, the link is notified to the device via email or app notification.

[0531] Step 6:

[0532] The user downloads and installs the agent on their terminal. The terminal then performs the necessary installation process to make the agent ready to run. The input is the downloaded agent package, and the output is the agent as a usable application.

[0533] Step 7:

[0534] The user activates an information processing agent and utilizes its customized functions. Specifically, the agent receives voice input and text commands and returns responses in natural language. The input is the user's operation instructions, and the output is the agent's corresponding action.

[0535] Step 8:

[0536] Creators use an agent development tool to create new agents and post them to the marketplace. The input is agent data provided by the creator, and the output is agent registration to the marketplace. The server manages this using commercial transaction management tools.

[0537] (Application Example 1)

[0538] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0539] When consumers shop online or in virtual environments, they need an environment where they can quickly and efficiently find products that match their individual preferences. However, conventional systems do not adequately provide personalized product suggestions that suit consumers' preferences, resulting in a time-consuming process for consumers to find what they want.

[0540] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0541] In this invention, the server includes a selection means for selecting the visual and auditory aspects of an artificial intelligence agent that can be individually configured by the user; a generation means for generating an artificial intelligence agent based on the selected visual and auditory aspects; a transmission means for sending the generated artificial intelligence agent to the user's information terminal; and a support means for suggesting products tailored to the user's preferences in a virtual store. This enables consumers to receive a personalized shopping experience and efficiently find products.

[0542] A "user" is the consumer themselves who uses the system to select and customize products.

[0543] "Individually configurable" refers to a state where users can freely select and change specific elements of the system based on their own preferences.

[0544] An "artificial intelligence agent" is a type of software that automatically performs various tasks based on user instructions.

[0545] "Visual" refers to the appearance or visual representation of the agent.

[0546] "Voice" refers to the expression of the agent's voice and sounds, and is a means of communication with the user through voice.

[0547] "Selection method" refers to an interface or technique that allows users to choose specific attributes of an agent according to their preferences.

[0548] "Generation means" refers to the function that manages the process of creating artificial intelligence agents based on selected visual and audio data.

[0549] "Transmission means" refers to the methods and functions for distributing the generated agent to the user's information terminal.

[0550] A "virtual store" is a virtual sales environment operated on the internet, where users can conduct online shopping.

[0551] "Support methods" refer to the technologies and processes used to suggest products tailored to the individual preferences of users.

[0552] To realize this invention, the following system configuration is used.

[0553] First, the server generates an artificial intelligence agent based on user input. The user sets their visual and auditory preferences through the terminal, and this information is sent to the server. The server receives the selection information and, using pre-prepared materials, generates a customized artificial intelligence agent based on the selection. This generation process uses advanced software such as Unity or Unreal Engine to construct realistic visual representations.

[0554] Next, the generated agent is sent to the user's device. The user can activate the agent within the virtual store and receive assistance in selecting products. On the device, a speech recognition API such as Google Cloud Speech-to-Text processes the user's voice commands and queries the product database.

[0555] Cloud services such as Amazon Web Services and Microsoft Azure are used for data processing and computation. This allows product recommendations to be generated by leveraging consumers' past purchase history and current trend information through machine learning libraries such as TensorFlow and PyTorch.

[0556] As a concrete example illustrating the mechanism, consider a scenario where a user visits a virtual store through smart glasses. When the user gives a voice command such as "Show me recommended electronic products," an agent provides a list of recommended products based on past purchase history and trends.

[0557] Furthermore, the following are specific examples of prompt statements that can be used in generative AI models:

[0558] Based on the user's past purchase history and current trends, please suggest the following products: categories of the user's interests. Filter by appearance and price range according to the following specifications.

[0559] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0560] Step 1:

[0561] The user selects the visual, auditory, and skill-based features of the artificial intelligence agent through their device. The input includes the user's preferences and required functions, and this information is sent to the server via the selection mechanism. The output generates customized information for the user.

[0562] Step 2:

[0563] The server generates an artificial intelligence agent using the user's customization information received. The input consists of user selections and pre-prepared material data. Unity or Unreal Engine is used for data processing to assemble personalized visuals. The AI ​​model generated during this process is configured to function based on specific prompt statements.

[0564] Step 3:

[0565] The generated artificial intelligence agent is sent from the server to the user's terminal. The server sends customized agent information to the terminal via a transmission method, preparing it for immediate use by the user. The output is the agent installed on the user's terminal.

[0566] Step 4:

[0567] On the device, the user accesses a virtual store and activates an agent. Using speech recognition, the user can give voice commands to the agent. The entered voice commands are converted into text data via the Google Cloud Speech-to-Text API.

[0568] Step 5:

[0569] The server queries a product database based on voice commands and provides the user with recommended product information. Inputs include converted text commands and the user's purchase history and trend data. TensorFlow or PyTorch is used to run the recommendation engine, and the output is a list of recommended products displayed on the user's device.

[0570] Step 6:

[0571] The user views the details of suggested products on the device and selects or purchases products using voice or touch commands. The input commands trigger the next step, displaying detailed information about the selected product.

[0572] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0573] This invention provides a system that can recognize a user's emotions and dynamically adjust the response of an artificial intelligence agent based on those emotions. In addition to customizing visuals and voices based on the user's individual settings, the system can analyze the user's emotional state using an emotion engine. This enables a more personalized user experience.

[0574] The system's implementation is based on interaction between servers, terminals, and users. The following describes the processing flow in natural language.

[0575] 1. User customization of agents

[0576] The user uses their device to select the visual and voice characteristics of the AI ​​agent on the platform. After customization, this information is sent from the device to the server, and a user-specific agent is prepared.

[0577] 2. Integration of the Emotional Engine

[0578] The server integrates an emotion engine with the AI ​​agent and analyzes user speech and facial expression data. This engine identifies emotions from the user's voice tone, language patterns, and facial expressions.

[0579] 3. Generating dynamic responses

[0580] After the emotion engine identifies the emotion, the server generates a response from the AI ​​agent based on the data. This enables conversations that are tailored to the user's emotional state. The agent further adjusts visual elements and voice tone to provide an appropriate response.

[0581] 4. User Interaction

[0582] The device displays the interface through which the optimized AI agent interacts with the user. Each time the user interacts with the agent, the emotion engine repeats the process in real time.

[0583] Specific example

[0584] A conversational agent for stress relief

[0585] User A wanted to relax when he felt stressed at work. He began interacting with an AI agent he had previously set up. The emotion engine sensed User A's stress level from his tone of voice and brief greetings, and the agent, with a calm voice and soothing visuals, presented topics and advice to help him relax.

[0586] Emotional response agents for entertainment

[0587] User B wanted to use the agent during a family meal. The agent analyzes the laughter and cheerful tone of User B's family and provides jokes and interesting trivia to liven up the conversation.

[0588] This system allows users to enjoy emotionally appropriate interactions tailored to their individual situations, creating richer experiences.

[0589] The following describes the processing flow.

[0590] Step 1:

[0591] The user accesses the platform using their device and logs into their account. The device sends the login information to the server for authentication.

[0592] Step 2:

[0593] The server confirms that authentication was successful and displays the user interface on the terminal. The user selects the visual and voice of the AI ​​agent on the interface.

[0594] Step 3:

[0595] Based on the user's selection, the device sends visual and audio customization information to the server. The server then uses this information to begin generating the specified AI agent.

[0596] Step 4:

[0597] The server integrates an emotion engine with the AI ​​agent and performs data processing to analyze the user's emotions. The emotion engine analyzes voice tone and facial expression data to identify the user's emotional state.

[0598] Step 5:

[0599] The server dynamically adjusts the AI ​​agent's response based on the analysis results of the emotion engine. This includes the agent's utterances, voice tone, and visuals.

[0600] Step 6:

[0601] The device displays a customized AI agent interface to the user. As the user interacts with the AI ​​agent, the device sends this information to the server in real time.

[0602] Step 7:

[0603] The emotion engine analyzes the user's new emotional state, and the server generates the next response based on that analysis. The device then communicates this to the user, enabling continuous and mutually influential dialogue.

[0604] (Example 2)

[0605] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0606] Current digital agents struggle to respond flexibly to the individual emotional states of users, resulting in uniform and impersonal interactions. Traditional agents have insufficient capabilities to provide dynamic responses based on the user's specific feelings and settings, making it difficult to deliver personalized experiences.

[0607] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0608] In this invention, the server includes selection means for selecting user-modifiable visual and auditory elements, generation means for generating a digital agent based on the selected visual and auditory elements, analysis means for analyzing the user's emotions, response generation means for dynamically generating a response from the digital agent based on the analyzed emotion data, and transmission means for transmitting the generated digital agent to the user's device. This provides personalized dynamic responses that respond to the user's emotions, enabling interactions that are more tailored to individual users.

[0609] A "user" is the entity that inputs information into or customizes a system or digital agent.

[0610] "Selection method" refers to a function or interface that a user uses to change or configure visual or audio elements.

[0611] "Visual elements" are the components that make up the appearance and avatar of a digital agent, and are the visual elements that users can select.

[0612] "Speech elements" are the components that make up the types and patterns of sounds emitted by a digital agent, and are the sound components that the user can select.

[0613] "Generation means" refers to a process or technology for constructing a digital agent based on selected visual and auditory elements.

[0614] A "digital agent" is a software entity that handles user interaction and communication, and is equipped with visual and auditory elements.

[0615] "Analysis methods" refer to processing and technology used to determine emotions based on data received from users.

[0616] "Response generation means" refers to a process or technology that dynamically creates responses to be emitted by a digital agent based on emotional data obtained by analysis means.

[0617] "Transmission means" refers to the process or system structure for delivering the generated digital agent and its response to the user's device or terminal.

[0618] This invention is a system in which a user customizes a digital agent, and that agent dynamically generates responses according to the user's emotions. Its specific form is shown below.

[0619] The server generates visual and auditory elements of the digital agent based on user selections. The user configures their visual avatar and voice settings using a terminal, and these settings are sent from the terminal to the server. The server utilizes a generative AI model to build a digital agent incorporating the user's specified settings.

[0620] The user accesses the system using a terminal to initiate an interaction with the agent. The terminal captures the user's voice and facial expressions in real time and sends this data to the server. The server uses emotion analysis software to analyze this data and identify the user's emotional state. Based on the analyzed emotion data, the server uses a generative AI model to generate responses for the digital agent. This process allows the agent to engage in personalized conversations tailored to the user.

[0621] For example, if a user requests, "I'm a little tired today, so I'd like to hear something uplifting," the emotion engine analyzes the user's tone of voice and the content of their request, and the server has the agent generate an appropriate response. This allows the user to enjoy content that will help them feel refreshed.

[0622] A concrete example of a prompt in this system would be something like, "How does the AI ​​agent support the user when they feel like relaxing?" In this way, users can receive rich interactions that are tailored to their own emotional state.

[0623] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0624] Step 1:

[0625] The user accesses the platform using a device and selects the visual and audio elements of a digital agent. The device records the selected avatar design and voice type as user input. The device sends this selection data to a server, which analyzes the received data to prepare the information necessary to generate a custom-configured digital agent.

[0626] Step 2:

[0627] Based on the configuration data received, the server uses a generative AI model to embody the selected visual and audio elements. This generation process combines visual designs and audio files to create a user-specific digital agent on a computer. The generated agent data is stored in a database, linked to the user's unique ID.

[0628] Step 3:

[0629] To initiate interaction with the agent, the user inputs instructions via voice or text from the terminal. The terminal captures the user's speech and facial expressions and sends this data to the server. The server passes this input data to an emotion analysis engine for analysis, which identifies the user's current emotions. The analyzed emotion information is output as a specific emotion state.

[0630] Step 4:

[0631] The server takes the analyzed emotional state as input and uses a generative AI model to generate responses for the digital agent. Specifically, the generative AI model uses prompt sentences to create appropriate responses and arranges them into a dialogue scenario. During this process, visual representations and voice tone are also adjusted to match the emotion. The response data is temporarily stored and prepared for transmission to the terminal.

[0632] Step 5:

[0633] The device receives response data sent from the server and displays and plays it back in a way that is more user-friendly. The user receives the agent's responses visually and audibly and continues the interaction. The device tracks this interaction in real time and sends data back to the server for the next interaction as needed. This allows the user to continuously receive an emotionally resonant and personalized experience.

[0634] (Application Example 2)

[0635] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0636] Providing dynamic responses that respond to user emotions and situations is challenging in modern home appliances and artificial intelligence assistants. In particular, creating a customized experience for each user in a home environment requires systems with advanced emotion analysis and adaptive capabilities. Furthermore, there is a lack of integrated platforms that enable enjoyable experiences and stress reduction. Therefore, the challenge lies in integrating the provision of services that respond to user emotions with the management of home appliances.

[0637] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0638] In this invention, the server includes selection means for selecting the visual and auditory aspects of an intelligent agent that can be individually configured by the user; generation means for generating an intelligent agent based on the selected visual and auditory aspects; distribution means for delivering the generated intelligent agent to the user's device; emotion analysis means for analyzing the user's emotions and dynamically adjusting the intelligent agent's response; and operation means for using the intelligent agent to operate and manage household appliances in a home environment. This enables personalized service delivery adapted to each user's emotions and effective management of household appliances.

[0639] An "intelligent agent" is a program designed to provide information and various forms of support through interaction with the user.

[0640] "Visual" refers to the design and expressive elements related to the appearance and interface of an intelligent agent, and is the information that users perceive visually.

[0641] "Voice" refers to the characteristics and features of the voice output used when an intelligent agent interacts with a user, and is information transmitted through the user's hearing.

[0642] A "selection method" is a function or interface that allows the user to individually configure the visual and auditory aspects of an intelligent agent.

[0643] "Generative means" refers to a mechanism or process for constructing or realizing an intelligent agent based on selected visual and auditory information.

[0644] "Distribution means" refers to methods and technologies for transmitting the generated intelligent agent to the user's device and making it available for use.

[0645] "Emotional analysis methods" refer to technologies and systems for extracting and analyzing emotions from a user's speech, facial expressions, and other data.

[0646] "Operating means" refers to a method or technique by which an intelligent agent causes various devices and equipment in the home environment to operate according to instructions.

[0647] The system for implementing this invention analyzes the user's emotions and enables an intelligent agent to respond dynamically in the home environment based on the results. The server receives the user's specified visual and auditory settings and customizes the intelligent agent. The generated intelligent agent is delivered to the user's terminal and provides services tailored to the user's needs through actual interaction.

[0648] The server uses emotion analysis tools to receive user voice and facial expression data and processes it to identify emotions. This process includes analyzing voice tone and language patterns, and dynamically adjusting the agent's response according to the identified emotion. The software used incorporates speech recognition technology and facial recognition algorithms.

[0649] The device provides an environment where users can operate the interface visually and audibly, enabling them to control home appliances. The intelligent agent plays a role in enriching daily life by providing emotion-responsive voice guidance and controlling devices.

[0650] For example, if a user says, "I'm so tired today," the intelligent agent can sense the stress from the voice and respond with something like, "I'll play some relaxing music." In this case, an example of a prompt would be, "Good evening, how are you feeling today?" which would allow the agent to begin responding in a way that is appropriate to the user's situation.

[0651] These features allow users to enjoy a more personalized experience within their homes.

[0652] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0653] Step 1:

[0654] The user uses a terminal to select the visual and auditory characteristics of the intelligent agent. The terminal receives input from the user and sends the selected configuration data to the server. In this process, the visual theme and auditory characteristics specified by the user are processed as data. The output is the customized configuration data transmitted to the server.

[0655] Step 2:

[0656] The server generates an intelligent agent based on the received configuration data. The server uses the generated AI model to construct an agent that responds to the selected visuals and sounds. Configuration data is used as input, and a customized intelligent agent is generated as output. This step specifically involves supplying data to the AI ​​model and creating the agent as a result.

[0657] Step 3:

[0658] The generated intelligent agent is delivered from the server to the user's terminal. The server transmits data to the terminal via the network connection. The input is the data of the generated agent, and the output is the completion of delivery to the user's terminal. In this process, data transfer and verification are specifically performed.

[0659] Step 4:

[0660] The terminal prepares to begin interaction with the user. The user initiates a conversation with the agent and provides voice input to the terminal. The terminal collects the voice data and sends it back to the server. The input is the user's voice, and the output is sent to the server as voice data. Specifically, voice recording and data transfer occur at this time.

[0661] Step 5:

[0662] The server performs sentiment analysis based on voice data. It utilizes a generative AI model to analyze the user's emotions from their voice tone and content. The input for this step is voice data, and the output is identified sentiment data. Specifically, the analysis involves data analysis using speech recognition technology.

[0663] Step 6:

[0664] The server utilizes the analysis results to generate responses from the intelligent agent. It supplies prompt sentences corresponding to the generated sentiment data to the AI ​​model, creating an adapted response. The input is sentiment data, and the output is the adjusted agent response. In this process, the response is specifically customized by supplying prompt sentences.

[0665] Step 7:

[0666] The terminal provides the user with a pre-configured response. The terminal provides feedback to the user through display and audio output. The input is response data from the server, and the output is the user's perceived feedback. Specifically, the terminal plays audio and displays information to allow the user to confirm the response.

[0667] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0668] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0669] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0670] [Fourth Embodiment]

[0671] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0672] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0673] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0674] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0675] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0676] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0677] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0678] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0679] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0680] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0681] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0682] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0683] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0684] The system of this invention allows users to select the visual and voice characteristics of an artificial intelligence agent according to their preferences, thereby generating and utilizing a personalized agent. The system also provides a function for creators to sell their independently developed agents on a marketplace.

[0685] The entire system relies on interaction between the server, terminal, and user to reflect individual user settings. The following describes the processing flow in natural language.

[0686] 1. User customization

[0687] The user uses a device to select customization options for the artificial intelligence agent, including its visuals, voice, and skills. The device retrieves this selection information and sends it to the server.

[0688] 2. Agent generation

[0689] The server generates an artificial intelligence agent based on the selection information received from the user. Here, pre-prepared visual materials and audio data are combined, and natural language processing capabilities are integrated into the agent.

[0690] 3. Agent distribution and use

[0691] The server delivers the generated artificial intelligence agent to the user's device and configures it for immediate use. The user can then activate the agent on their device and have it perform various tasks.

[0692] 4. Using the Marketplace

[0693] The system provides a means for creators to post artificial intelligence agents on the marketplace, making them available for purchase by other users. The server manages the agent information posted by creators and displays it in a list on the platform.

[0694] Specific example

[0695] Personal assistant customization

[0696] User X wanted to customize a personal assistant to improve his work efficiency. He selected a friendly male voice and an anime-style character with business-specific skills on the platform. Based on these selections, the server generated an agent tailored to User X's needs and delivered it to his device.

[0697] Agent sales by creators

[0698] Creator Y developed an agent specifically for language learning. This agent supports multiple languages ​​and includes a pronunciation practice function. He posted this product on a marketplace, and it was managed on a server so that other users could purchase it.

[0699] This invention provides a new platform that explores personalization and creativity for both users and creators, enabling the generation and use of advanced AI agents.

[0700] The following describes the processing flow.

[0701] Step 1:

[0702] The user logs into the platform using their device. The device retrieves the user's login information and sends it to the server for authentication.

[0703] Step 2:

[0704] The server checks the received login information against its database and returns the authentication result to the terminal. If authentication is successful, the user is shown a customized interface.

[0705] Step 3:

[0706] Users access a customizable interface to determine the AI ​​agent's visuals, voice, and skills. The device collects the user's selected options and sends them to the server.

[0707] Step 4:

[0708] The server initiates the AI ​​agent generation process based on the user's selection data. It retrieves the selected visual and audio data and integrates it with a natural language processing model to generate a custom agent.

[0709] Step 5:

[0710] The generated AI agent is stored on the server and delivered to the user's device. The device then provides the user with an interface that allows them to launch the agent.

[0711] Step 6:

[0712] The user initiates interaction with the AI ​​agent through their device. The agent performs various tasks in response to the user's requests and updates information by communicating with the server as needed.

[0713] Step 7:

[0714] Creators access the marketplace and register their own AI agent. The information posted by the creator is sent from the device to the server, which adds it to the market list.

[0715] Step 8:

[0716] The server manages AI agents listed on the marketplace, making them available for other users to purchase. When a user attempts to purchase an agent, the server manages the purchase process and delivers the provided agent to the buyer's device.

[0717] (Example 1)

[0718] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0719] In modern information processing, there is a need for users to utilize customized information processing agents tailored to their individual needs and to maximize their functionality. However, existing systems make the process of generating personalized agents by allowing users to easily select visual representations and sounds according to their preferences cumbersome, and there are no established means for sharing or selling these agents to other users. It is necessary to solve this problem and provide an effective platform for the sale and sharing of agents among users.

[0720] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0721] In this invention, the server includes selection means for selecting a visual representation and sound of an information processing agent that can be individually configured by the user, generation means for generating an information processing agent based on the selected visual representation and sound, and transmission means for distributing the generated information processing agent to the user's information processing device. This provides a platform that allows users to easily generate and use personalized agents, and also facilitates the sale and sharing of agents.

[0722] A "selection mechanism" is an interface or mechanism that allows users to individually configure the visual and auditory settings of an information processing agent.

[0723] "Generation means" refers to a process or system for creating an information processing agent based on selected visual and auditory information.

[0724] "Transmission means" refers to a method or apparatus for transferring the generated information processing agent to the user's information processing device.

[0725] "Operation means" refers to means of interaction that allow the user to experience the functions of the generated information processing agent.

[0726] "Registration method" refers to the procedure or mechanism for making the created information processing agent accessible to other users at the sales location.

[0727] A "data integration method" is a way to incorporate historical user data into an information processing agent to provide a consistent user experience.

[0728] "Commercial transaction management measures" refer to measures or platforms for managing the publication and commercial transactions of information processing agents.

[0729] The system of this invention allows users to individually configure, generate, and utilize information processing agents. The system is primarily realized through the interaction between a server, a terminal, and a user.

[0730] Users customize the agent's visual representation and voice via a terminal. The terminal is equipped with a dedicated application or web interface, which is used to collect the selected customization information. This selected information is securely transmitted to a server. Secure communication protocols such as SSL can be used for transmission.

[0731] The server generates an information processing agent based on the customized information received from the user. Specifically, the server has a database where visual materials, audio data, and skill sets are stored. Necessary components are retrieved from this database and integrated with the artificial intelligence engine using a generation method. Programming languages ​​such as Python and natural language processing libraries are often used for this process.

[0732] The generated information processing agent is packaged and delivered to the user's terminal. Once the agent is operational on the terminal, the user can activate it and utilize its various functions. For example, considering a personal assistant agent used by a user to improve work efficiency, an agent with a friendly voice and business-specific functions can be selected, allowing for immediate task execution.

[0733] Furthermore, creators can create their own information processing agents and sell them on a dedicated marketplace. The server provides a means of managing commercial transactions along with the registration of creators, facilitating the sharing and trading of agents among users.

[0734] Examples of prompt messages include the following:

[0735] "How can we create a personal assistant agent with a friendly male voice and business-specific skills?"

[0736] This system allows users to create and use their own agents, and enables creators to open up new markets.

[0737] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0738] Step 1:

[0739] The user selects the visual and audio settings for the information processing agent using a terminal application or web interface. As input, the user reviews and selects from a list of visual design options and audio samples. The output is a set of selected visual representation IDs and audio IDs, which are then constructed as the foundational data for subsequent processing.

[0740] Step 2:

[0741] The terminal sends the user-selected customization information to the server as a data packet. Specifically, the selected information is packaged as JSON data and securely transmitted to the server over the internet. The input is the user's selected information, and the output is the received data record on the server.

[0742] Step 3:

[0743] The server analyzes the received customization information and extracts relevant visual materials, audio data, and skill sets from the database. The input is user selection information, and the output is agent data integrating these materials. Specifically, an SQL query is generated based on the selected ID, and the corresponding materials are retrieved from the database.

[0744] Step 4:

[0745] The server generates an information processing agent using the acquired material. Python scripts and natural language processing libraries are used for this generation. The input is a set of material, and the output is a working agent package. At this stage, natural language processing capabilities are incorporated, and the agent's conversational skills are formed.

[0746] Step 5:

[0747] The server packages the generated information processing agent in a compressed format and sends a download link to the user's device. The input is the generated agent and user information, and the output is the downloadable link information. Specifically, the link is notified to the device via email or app notification.

[0748] Step 6:

[0749] The user downloads and installs the agent on their terminal. The terminal then performs the necessary installation process to make the agent ready to run. The input is the downloaded agent package, and the output is the agent as a usable application.

[0750] Step 7:

[0751] The user activates an information processing agent and utilizes its customized functions. Specifically, the agent receives voice input and text commands and returns responses in natural language. The input is the user's operation instructions, and the output is the agent's corresponding action.

[0752] Step 8:

[0753] Creators use an agent development tool to create new agents and post them to the marketplace. The input is agent data provided by the creator, and the output is agent registration to the marketplace. The server manages this using commercial transaction management tools.

[0754] (Application Example 1)

[0755] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0756] When consumers shop online or in virtual environments, they need an environment where they can quickly and efficiently find products that match their individual preferences. However, conventional systems do not adequately provide personalized product suggestions that suit consumers' preferences, resulting in a time-consuming process for consumers to find what they want.

[0757] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0758] In this invention, the server includes a selection means for selecting the visual and auditory aspects of an artificial intelligence agent that can be individually configured by the user; a generation means for generating an artificial intelligence agent based on the selected visual and auditory aspects; a transmission means for sending the generated artificial intelligence agent to the user's information terminal; and a support means for suggesting products tailored to the user's preferences in a virtual store. This enables consumers to receive a personalized shopping experience and efficiently find products.

[0759] A "user" is the consumer themselves who uses the system to select and customize products.

[0760] "Individually configurable" refers to a state where users can freely select and change specific elements of the system based on their own preferences.

[0761] An "artificial intelligence agent" is a type of software that automatically performs various tasks based on user instructions.

[0762] "Visual" refers to the appearance or visual representation of the agent.

[0763] "Voice" refers to the expression of the agent's voice and sounds, and is a means of communication with the user through voice.

[0764] "Selection method" refers to an interface or technique that allows users to choose specific attributes of an agent according to their preferences.

[0765] "Generation means" refers to the function that manages the process of creating artificial intelligence agents based on selected visual and audio data.

[0766] "Transmission means" refers to the methods and functions for distributing the generated agent to the user's information terminal.

[0767] A "virtual store" is a virtual sales environment operated on the internet, where users can conduct online shopping.

[0768] "Support methods" refer to the technologies and processes used to suggest products tailored to the individual preferences of users.

[0769] To realize this invention, the following system configuration is used.

[0770] First, the server generates an artificial intelligence agent based on user input. The user sets their visual and auditory preferences through the terminal, and this information is sent to the server. The server receives the selection information and, using pre-prepared materials, generates a customized artificial intelligence agent based on the selection. This generation process uses advanced software such as Unity or Unreal Engine to construct realistic visual representations.

[0771] Next, the generated agent is sent to the user's device. The user can activate the agent within the virtual store and receive assistance in selecting products. On the device, a speech recognition API such as Google Cloud Speech-to-Text processes the user's voice commands and queries the product database.

[0772] Cloud services such as Amazon Web Services and Microsoft Azure are used for data processing and computation. This allows product recommendations to be generated by leveraging consumers' past purchase history and current trend information through machine learning libraries such as TensorFlow and PyTorch.

[0773] As a concrete example illustrating the mechanism, consider a scenario where a user visits a virtual store through smart glasses. When the user gives a voice command such as "Show me recommended electronic products," an agent provides a list of recommended products based on past purchase history and trends.

[0774] Furthermore, the following are specific examples of prompt statements that can be used in generative AI models:

[0775] Based on the user's past purchase history and current trends, please suggest the following products: categories of the user's interests. Filter by appearance and price range according to the following specifications.

[0776] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0777] Step 1:

[0778] The user selects the visual, auditory, and skill-based features of the artificial intelligence agent through their device. The input includes the user's preferences and required functions, and this information is sent to the server via the selection mechanism. The output generates customized information for the user.

[0779] Step 2:

[0780] The server generates an artificial intelligence agent using the user's customization information received. The input consists of user selections and pre-prepared material data. Unity or Unreal Engine is used for data processing to assemble personalized visuals. The AI ​​model generated during this process is configured to function based on specific prompt statements.

[0781] Step 3:

[0782] The generated artificial intelligence agent is sent from the server to the user's terminal. The server sends customized agent information to the terminal via a transmission method, preparing it for immediate use by the user. The output is the agent installed on the user's terminal.

[0783] Step 4:

[0784] On the device, the user accesses a virtual store and activates an agent. Using speech recognition, the user can give voice commands to the agent. The entered voice commands are converted into text data via the Google Cloud Speech-to-Text API.

[0785] Step 5:

[0786] The server queries a product database based on voice commands and provides the user with recommended product information. Inputs include converted text commands and the user's purchase history and trend data. TensorFlow or PyTorch is used to run the recommendation engine, and the output is a list of recommended products displayed on the user's device.

[0787] Step 6:

[0788] The user views the details of suggested products on the device and selects or purchases products using voice or touch commands. The input commands trigger the next step, displaying detailed information about the selected product.

[0789] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0790] This invention provides a system that can recognize a user's emotions and dynamically adjust the response of an artificial intelligence agent based on those emotions. In addition to customizing visuals and voices based on the user's individual settings, the system can analyze the user's emotional state using an emotion engine. This enables a more personalized user experience.

[0791] The system's implementation is based on interaction between servers, terminals, and users. The following describes the processing flow in natural language.

[0792] 1. User customization of agents

[0793] The user uses their device to select the visual and voice characteristics of the AI ​​agent on the platform. After customization, this information is sent from the device to the server, and a user-specific agent is prepared.

[0794] 2. Integration of the Emotional Engine

[0795] The server integrates an emotion engine with the AI ​​agent and analyzes user speech and facial expression data. This engine identifies emotions from the user's voice tone, language patterns, and facial expressions.

[0796] 3. Generating dynamic responses

[0797] After the emotion engine identifies the emotion, the server generates a response from the AI ​​agent based on the data. This enables conversations that are tailored to the user's emotional state. The agent further adjusts visual elements and voice tone to provide an appropriate response.

[0798] 4. User Interaction

[0799] The device displays the interface through which the optimized AI agent interacts with the user. Each time the user interacts with the agent, the emotion engine repeats the process in real time.

[0800] Specific example

[0801] A conversational agent for stress relief

[0802] User A wanted to relax when he felt stressed at work. He began interacting with an AI agent he had previously set up. The emotion engine sensed User A's stress level from his tone of voice and brief greetings, and the agent, with a calm voice and soothing visuals, presented topics and advice to help him relax.

[0803] Emotional response agents for entertainment

[0804] User B wanted to use the agent during a family meal. The agent analyzes the laughter and cheerful tone of User B's family and provides jokes and interesting trivia to liven up the conversation.

[0805] This system allows users to enjoy emotionally appropriate interactions tailored to their individual situations, creating richer experiences.

[0806] The following describes the processing flow.

[0807] Step 1:

[0808] The user accesses the platform using their device and logs into their account. The device sends the login information to the server for authentication.

[0809] Step 2:

[0810] The server confirms that authentication was successful and displays the user interface on the terminal. The user selects the visual and voice of the AI ​​agent on the interface.

[0811] Step 3:

[0812] Based on the user's selection, the device sends visual and audio customization information to the server. The server then uses this information to begin generating the specified AI agent.

[0813] Step 4:

[0814] The server integrates an emotion engine with the AI ​​agent and performs data processing to analyze the user's emotions. The emotion engine analyzes voice tone and facial expression data to identify the user's emotional state.

[0815] Step 5:

[0816] The server dynamically adjusts the AI ​​agent's response based on the analysis results of the emotion engine. This includes the agent's utterances, voice tone, and visuals.

[0817] Step 6:

[0818] The device displays a customized AI agent interface to the user. As the user interacts with the AI ​​agent, the device sends this information to the server in real time.

[0819] Step 7:

[0820] The emotion engine analyzes the user's new emotional state, and the server generates the next response based on that analysis. The device then communicates this to the user, enabling continuous and mutually influential dialogue.

[0821] (Example 2)

[0822] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0823] Current digital agents struggle to respond flexibly to the individual emotional states of users, resulting in uniform and impersonal interactions. Traditional agents have insufficient capabilities to provide dynamic responses based on the user's specific feelings and settings, making it difficult to deliver personalized experiences.

[0824] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0825] In this invention, the server includes selection means for selecting user-modifiable visual and auditory elements, generation means for generating a digital agent based on the selected visual and auditory elements, analysis means for analyzing the user's emotions, response generation means for dynamically generating a response from the digital agent based on the analyzed emotion data, and transmission means for transmitting the generated digital agent to the user's device. This provides personalized dynamic responses that respond to the user's emotions, enabling interactions that are more tailored to individual users.

[0826] A "user" is the entity that inputs information into or customizes a system or digital agent.

[0827] "Selection method" refers to a function or interface that a user uses to change or configure visual or audio elements.

[0828] "Visual elements" are the components that make up the appearance and avatar of a digital agent, and are the visual elements that users can select.

[0829] "Speech elements" are the components that make up the types and patterns of sounds emitted by a digital agent, and are the sound components that the user can select.

[0830] "Generation means" refers to a process or technology for constructing a digital agent based on selected visual and auditory elements.

[0831] A "digital agent" is a software entity that handles user interaction and communication, and is equipped with visual and auditory elements.

[0832] "Analysis methods" refer to processing and technology used to determine emotions based on data received from users.

[0833] "Response generation means" refers to a process or technology that dynamically creates responses to be emitted by a digital agent based on emotional data obtained by analysis means.

[0834] "Transmission means" refers to the process or system structure for delivering the generated digital agent and its response to the user's device or terminal.

[0835] This invention is a system in which a user customizes a digital agent, and that agent dynamically generates responses according to the user's emotions. Its specific form is shown below.

[0836] The server generates visual and auditory elements of the digital agent based on user selections. The user configures their visual avatar and voice settings using a terminal, and these settings are sent from the terminal to the server. The server utilizes a generative AI model to build a digital agent incorporating the user's specified settings.

[0837] The user accesses the system using a terminal to initiate an interaction with the agent. The terminal captures the user's voice and facial expressions in real time and sends this data to the server. The server uses emotion analysis software to analyze this data and identify the user's emotional state. Based on the analyzed emotion data, the server uses a generative AI model to generate responses for the digital agent. This process allows the agent to engage in personalized conversations tailored to the user.

[0838] For example, if a user requests, "I'm a little tired today, so I'd like to hear something uplifting," the emotion engine analyzes the user's tone of voice and the content of their request, and the server has the agent generate an appropriate response. This allows the user to enjoy content that will help them feel refreshed.

[0839] A concrete example of a prompt in this system would be something like, "How does the AI ​​agent support the user when they feel like relaxing?" In this way, users can receive rich interactions that are tailored to their own emotional state.

[0840] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0841] Step 1:

[0842] The user accesses the platform using a device and selects the visual and audio elements of a digital agent. The device records the selected avatar design and voice type as user input. The device sends this selection data to a server, which analyzes the received data to prepare the information necessary to generate a custom-configured digital agent.

[0843] Step 2:

[0844] Based on the configuration data received, the server uses a generative AI model to embody the selected visual and audio elements. This generation process combines visual designs and audio files to create a user-specific digital agent on a computer. The generated agent data is stored in a database, linked to the user's unique ID.

[0845] Step 3:

[0846] To initiate interaction with the agent, the user inputs instructions via voice or text from the terminal. The terminal captures the user's speech and facial expressions and sends this data to the server. The server passes this input data to an emotion analysis engine for analysis, which identifies the user's current emotions. The analyzed emotion information is output as a specific emotion state.

[0847] Step 4:

[0848] The server takes the analyzed emotional state as input and uses a generative AI model to generate responses for the digital agent. Specifically, the generative AI model uses prompt sentences to create appropriate responses and arranges them into a dialogue scenario. During this process, visual representations and voice tone are also adjusted to match the emotion. The response data is temporarily stored and prepared for transmission to the terminal.

[0849] Step 5:

[0850] The device receives response data sent from the server and displays and plays it back in a way that is more user-friendly. The user receives the agent's responses visually and audibly and continues the interaction. The device tracks this interaction in real time and sends data back to the server for the next interaction as needed. This allows the user to continuously receive an emotionally resonant and personalized experience.

[0851] (Application Example 2)

[0852] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0853] Providing dynamic responses that respond to user emotions and situations is challenging in modern home appliances and artificial intelligence assistants. In particular, creating a customized experience for each user in a home environment requires systems with advanced emotion analysis and adaptive capabilities. Furthermore, there is a lack of integrated platforms that enable enjoyable experiences and stress reduction. Therefore, the challenge lies in integrating the provision of services that respond to user emotions with the management of home appliances.

[0854] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0855] In this invention, the server includes selection means for selecting the visual and auditory aspects of an intelligent agent that can be individually configured by the user; generation means for generating an intelligent agent based on the selected visual and auditory aspects; distribution means for delivering the generated intelligent agent to the user's device; emotion analysis means for analyzing the user's emotions and dynamically adjusting the intelligent agent's response; and operation means for using the intelligent agent to operate and manage household appliances in a home environment. This enables personalized service delivery adapted to each user's emotions and effective management of household appliances.

[0856] An "intelligent agent" is a program designed to provide information and various forms of support through interaction with the user.

[0857] "Visual" refers to the design and expressive elements related to the appearance and interface of an intelligent agent, and is the information that users perceive visually.

[0858] "Voice" refers to the characteristics and features of the voice output used when an intelligent agent interacts with a user, and is information transmitted through the user's hearing.

[0859] A "selection method" is a function or interface that allows the user to individually configure the visual and auditory aspects of an intelligent agent.

[0860] "Generative means" refers to a mechanism or process for constructing or realizing an intelligent agent based on selected visual and auditory information.

[0861] "Distribution means" refers to methods and technologies for transmitting the generated intelligent agent to the user's device and making it available for use.

[0862] "Emotional analysis methods" refer to technologies and systems for extracting and analyzing emotions from a user's speech, facial expressions, and other data.

[0863] "Operating means" refers to a method or technique by which an intelligent agent causes various devices and equipment in the home environment to operate according to instructions.

[0864] The system for implementing this invention analyzes the user's emotions and enables an intelligent agent to respond dynamically in the home environment based on the results. The server receives the user's specified visual and auditory settings and customizes the intelligent agent. The generated intelligent agent is delivered to the user's terminal and provides services tailored to the user's needs through actual interaction.

[0865] The server uses emotion analysis tools to receive user voice and facial expression data and processes it to identify emotions. This process includes analyzing voice tone and language patterns, and dynamically adjusting the agent's response according to the identified emotion. The software used incorporates speech recognition technology and facial recognition algorithms.

[0866] The device provides an environment where users can operate the interface visually and audibly, enabling them to control home appliances. The intelligent agent plays a role in enriching daily life by providing emotion-responsive voice guidance and controlling devices.

[0867] For example, if a user says, "I'm so tired today," the intelligent agent can sense the stress from the voice and respond with something like, "I'll play some relaxing music." In this case, an example of a prompt would be, "Good evening, how are you feeling today?" which would allow the agent to begin responding in a way that is appropriate to the user's situation.

[0868] These features allow users to enjoy a more personalized experience within their homes.

[0869] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0870] Step 1:

[0871] The user uses a terminal to select the visual and auditory characteristics of the intelligent agent. The terminal receives input from the user and sends the selected configuration data to the server. In this process, the visual theme and auditory characteristics specified by the user are processed as data. The output is the customized configuration data transmitted to the server.

[0872] Step 2:

[0873] The server generates an intelligent agent based on the received configuration data. The server uses the generated AI model to construct an agent that responds to the selected visuals and sounds. Configuration data is used as input, and a customized intelligent agent is generated as output. This step specifically involves supplying data to the AI ​​model and creating the agent as a result.

[0874] Step 3:

[0875] The generated intelligent agent is delivered from the server to the user's terminal. The server transmits data to the terminal via the network connection. The input is the data of the generated agent, and the output is the completion of delivery to the user's terminal. In this process, data transfer and verification are specifically performed.

[0876] Step 4:

[0877] The terminal prepares to begin interaction with the user. The user initiates a conversation with the agent and provides voice input to the terminal. The terminal collects the voice data and sends it back to the server. The input is the user's voice, and the output is sent to the server as voice data. Specifically, voice recording and data transfer occur at this time.

[0878] Step 5:

[0879] The server performs sentiment analysis based on voice data. It utilizes a generative AI model to analyze the user's emotions from their voice tone and content. The input for this step is voice data, and the output is identified sentiment data. Specifically, the analysis involves data analysis using speech recognition technology.

[0880] Step 6:

[0881] The server utilizes the analysis results to generate responses from the intelligent agent. It supplies prompt sentences corresponding to the generated sentiment data to the AI ​​model, creating an adapted response. The input is sentiment data, and the output is the adjusted agent response. In this process, the response is specifically customized by supplying prompt sentences.

[0882] Step 7:

[0883] The terminal provides the user with a pre-configured response. The terminal provides feedback to the user through display and audio output. The input is response data from the server, and the output is the user's perceived feedback. Specifically, the terminal plays audio and displays information to allow the user to confirm the response.

[0884] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0885] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0886] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0887] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0888] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0889] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0890] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0891] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0892] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0893] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0894] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0895] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0896] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0897] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0898] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0899] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0900] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0901] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0902] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0903] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0904] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0905] The following is further disclosed regarding the embodiments described above.

[0906] (Claim 1)

[0907] A selection method for users to individually configure the visual and voice characteristics of an artificial intelligence agent,

[0908] A generation means for generating an artificial intelligence agent based on selected visuals and audio,

[0909] A distribution method for delivering the generated artificial intelligence agent to the user's device,

[0910] A system that includes this.

[0911] (Claim 2)

[0912] The system according to claim 1, comprising means for transferring the user's past data to the generated artificial intelligence agent.

[0913] (Claim 3)

[0914] The system according to claim 1, comprising posting means for posting and selling artificial intelligence agents on a marketplace.

[0915] "Example 1"

[0916] (Claim 1)

[0917] A selection means for selecting the visual representation and sound of an information processing agent that can be individually configured by the user,

[0918] A generation means for generating an information processing agent based on selected visual representations and sounds,

[0919] A transmission means for distributing the generated information processing agent to the user's information processing device,

[0920] An operating method that allows the user to experience the functions of the generated information processing agent,

[0921] A means of registering information processing agents to be posted to a sales site so that other users can download them,

[0922] A system that includes this.

[0923] (Claim 2)

[0924] The system according to claim 1, further comprising data integration means for integrating the user's past usage data into the generated information processing agent.

[0925] (Claim 3)

[0926] The system according to claim 1, comprising a commercial transaction management means for making information processing agents publicly available and trading them at a sales office.

[0927] "Application Example 1"

[0928] (Claim 1)

[0929] A selection means for users to individually configure the visual and auditory aspects of an artificial intelligence agent,

[0930] A generation means for creating an artificial intelligence agent based on selected visual and auditory information,

[0931] A transmission means for sending the generated artificial intelligence agent to the user's information terminal,

[0932] A support method for suggesting products tailored to the user's preferences in a virtual store,

[0933] A system that includes this.

[0934] (Claim 2)

[0935] The system according to claim 1, comprising means for transferring consumer's past usage data to the generated artificial intelligence agent.

[0936] (Claim 3)

[0937] The system according to claim 1, comprising posting means for posting and selling artificial intelligence agents to an electronic marketplace.

[0938] "Example 2 of combining an emotion engine"

[0939] (Claim 1)

[0940] A selection means for users to select modifiable visual and audio elements,

[0941] A generation means for generating a digital agent based on selected visual and auditory elements,

[0942] Analytical methods for analyzing user emotions,

[0943] A response generation means that dynamically generates a response of a digital agent based on analyzed emotional data,

[0944] A transmission means for sending the generated digital agent to the user's device,

[0945] ...

[0946] A system that includes this.

[0947] (Claim 2)

[0948] The system according to claim 1, comprising means for adjusting the analysis means based on the user's initial settings.

[0949] (Claim 3)

[0950] The system according to claim 1, comprising posting means for posting and providing digital agents to a list.

[0951] "Application example 2 when combining with an emotional engine"

[0952] (Claim 1)

[0953] A selection means for users to individually configure the visual and auditory aspects of an intelligent agent,

[0954] A generation means for generating an intelligent agent based on selected visual and auditory information,

[0955] A distribution means for delivering the generated intelligent agent to the user's device,

[0956] An emotion analysis means that analyzes the user's emotions and dynamically adjusts the response of an intelligent agent,

[0957] An operating means for operating and managing household appliances in a home environment using an intelligent agent,

[0958] A system that includes this.

[0959] (Claim 2)

[0960] The system according to claim 1, comprising means for transferring the user's historical information to the generated intelligent agent.

[0961] (Claim 3)

[0962] The system according to claim 1, comprising posting means for posting and buying / selling intelligent agents on a trading platform. [Explanation of Symbols]

[0963] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A selection method for users to individually configure the visual and voice characteristics of an artificial intelligence agent, A generation means for generating an artificial intelligence agent based on selected visuals and audio, A distribution method for delivering the generated artificial intelligence agent to the user's device, A system that includes this.

2. The system according to claim 1, comprising means for transferring the user's past data to the generated artificial intelligence agent.

3. The system according to claim 1, comprising posting means for posting and selling artificial intelligence agents on a marketplace.

Images