system

Abstract

We provide the system. [Solution] Means of acquiring personal information, A means for training a machine learning model based on the aforementioned personal information, A means for generating individual responses using the machine learning model based on user inquiries, Means for providing the aforementioned individual responses to the user, Methods for selecting partner services, A means for notifying the user of the aforementioned partner service, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The current artificial intelligence can only provide general answers, and there is a problem that it is difficult to provide individual advice based on the individual situation and past history of the user. For this reason, it has not been able to function as an intimate and reliable counselor for the user, and it has been difficult to improve the user satisfaction. In addition, there is a lack of a mechanism that can efficiently propose partnership services optimized for the individual needs of the user. [[ID=三十六]]

Means for Solving the Problems

[0005] This invention provides a means for acquiring personal information and for training a machine learning model based on said personal information, and for generating individualized responses suitable for the user's situation using the machine learning model based on user inquiries. Furthermore, it constructs a system that enables service proposals tailored to user needs by selecting partner services and notifying the user of the selection. This aims to improve user satisfaction by providing attentive service that aligns with user requests and by uncovering potential needs.

[0006] "Personal information" refers to privacy-related information such as the user's name, address, contact information, activity history, and health data.

[0007] A "machine learning model" refers to an algorithm or computational model that analyzes data, automatically learns patterns and rules, and makes predictions and decisions.

[0008] "Individualized responses" refer to specific answers and suggestions tailored to a particular user, generated based on the user's individual circumstances and past interactions.

[0009] "Partner services" refer to products and services provided by partner companies or related businesses that are recommended by the system according to the user's needs.

[0010] "Notification" refers to the act or function of a system conveying information or messages to a user. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4]This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the numbered communication I / F (Interface) is an interface that includes a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] This invention provides a personal AI system that enables the provision of information and service suggestions tailored to the individual needs of users. The system consists of three main components: a server, a terminal, and a user.

[0033] Data collection

[0034] Users connect to this system via their devices after consenting to the provision of their personal information. The devices collect information and behavioral data from the user (e.g., social media posts, schedule information, etc.) and send it to the server. The server receives this data and stores it in a database in an encrypted format.

[0035] Data analysis and response generation

[0036] The server trains a machine learning model based on stored user data. This model analyzes the user's past behavior and current situation. When a user asks a question to the system using a terminal, the terminal sends the input to the server. The server uses the machine learning model to generate a personalized response optimized for the user and presents it to the user through the terminal.

[0037] Specific example

[0038] For example, if a user inputs a concern such as "I've been having trouble sleeping lately" into the system, the server will refer to past behavioral data and health information to generate specific advice to improve sleep quality (e.g., reviewing bedtime routines, relaxation techniques, etc.). This information is then provided to the user via their device.

[0039] Selection and notification of partner services

[0040] Furthermore, the server selects services from partner services that it deems appropriate for the user. This is done using an advertising optimization algorithm to select partner services that match the user's individual needs. The selection results are delivered to the device and the user is notified, allowing the user to learn about services that interest them.

[0041] In this way, the present invention aims to enhance user satisfaction by enabling responses tailored to individual user needs and service proposals that meet their interests.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] Users log in to the system via their device and give their consent regarding the collection of personal information. In this process, the device collects the user's basic profile information.

[0045] Step 2:

[0046] The device monitors user activity data (e.g., social media posts, schedules, health information) and sends this data to the server. Real-time updates ensure that the server reflects the user's latest information.

[0047] Step 3:

[0048] The server stores data received from the terminal in a database. The stored data is kept in an encrypted format to protect privacy.

[0049] Step 4:

[0050] The server periodically preprocesses the data in the database, preparing it as training data for machine learning models. This includes data cleaning and filtering.

[0051] Step 5:

[0052] The server trains a machine learning model using pre-processed data. The model learns user behavior patterns and preferences, enabling predictions tailored to individual needs.

[0053] Step 6:

[0054] When a user enters a specific question or request into the device, the device forwards that request to the server. The format and content of the input are then validated.

[0055] Step 7:

[0056] The server applies a machine learning model based on the user's request and generates the optimal individual response by referring to past data.

[0057] Step 8:

[0058] The generated response is sent from the server to the terminal and displayed to the user. This allows the user to receive advice and information appropriate to their situation.

[0059] Step 9:

[0060] The server selects partner services based on user data and applies an advertising optimization algorithm to choose the appropriate service.

[0061] Step 10:

[0062] Information about selected partner services is notified to the user via their device, allowing them to receive service suggestions tailored to their interests.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] Extracting meaningful information from vast amounts of personal and behavioral data, and providing appropriate responses and service suggestions tailored to individual user needs, is a major challenge in today's information society. A system that efficiently solves this problem and improves user satisfaction is needed.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes a device for acquiring personal information, a device for training an information processing model, and a device for clustering user behavior data. This enables the provision of information and service proposals optimized for the individual needs of users.

[0068] "Personal information" refers to unique data about a user, including but not limited to name, address, contact information, and activity history.

[0069] An "information processing model" refers to an algorithm or mathematical model that is trained on personal information and behavioral data to generate user-optimized responses.

[0070] "Third-party services" refer to services provided by external organizations or companies other than the server provider, and are selected based on the user's needs.

[0071] "Encryption" refers to the process of transforming the content of data using specific keys and algorithms in order to prevent unauthorized access to the data.

[0072] Clustering is a data analysis technique that divides a dataset into groups based on similarity, and is used to discover meaningful patterns or groups.

[0073] This personal AI system embodies the technology for providing information and service suggestions tailored to user needs. The system primarily consists of three elements: a server, a terminal, and a user.

[0074] The server first receives personal information and behavioral data transmitted from the terminal and stores it in an encrypted format for secure management. Industry-standard methods such as AES are used for encryption. This makes it possible to store data efficiently while ensuring user privacy.

[0075] The server trains an information processing model using stored data to optimize responses and suggestions to the user. The information processing model used includes common machine learning algorithms, and libraries such as Scikit-learn are utilized for analysis. The model analyzes the user's past behavioral data and reveals specific patterns through clustering.

[0076] On the other hand, the terminal functions as an interface connecting the user and the server. When a user makes an inquiry, the terminal forwards the request to the server. The server generates a response using a generation AI model, such as a natural language processing model, and presents it to the user via the terminal. Models such as OpenAI's GPT-3® are used for this response generation.

[0077] For example, if a user enters "I want to improve my sleep quality" into their device, the server analyzes collected health data and behavioral patterns to suggest improvements to their pre-sleep routine and relaxation techniques. This advice is displayed on the device in text format, allowing the user to improve their behavior based on it.

[0078] An example of a prompt message could be an input in the form of, "Analyze the user's sleep patterns and provide advice for improving their sleep quality."

[0079] In this way, the system aims to improve the user experience by providing information and service suggestions tailored to the individual needs of each user.

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] The user connects to the system through their device and consents to providing personal information. The device configures what information to collect and begins gathering behavioral data such as social media posts and schedule information. This input information includes the user's personal information and behavioral data. The device organizes this data into an appropriate format and sends it to the server.

[0083] Step 2:

[0084] The server receives data sent from the terminal, encrypts it, and stores it in a database. Inputs are user personal information and behavioral data, and outputs are encrypted database entries. The server uses encryption algorithms such as AES to protect the information.

[0085] Step 3:

[0086] The server trains an information processing model using stored data. The input is encrypted user data, and the output is an information processing model optimized for the user. The server uses machine learning libraries such as Scikit-learn and applies clustering algorithms to extract patterns.

[0087] Step 4:

[0088] When a user enters a specific question or request into the terminal, the terminal forwards that query to the server. The input is the text containing the user's question, and the output is the query sent to the server. The terminal sends this to the server in JSON format.

[0089] Step 5:

[0090] The server uses a generative AI model to generate appropriate responses from user requests. The input consists of user questions and prompts reflecting various data, while the output is the response to the user. The server utilizes various natural language processing technologies as its generative AI model, sometimes employing OpenAI's GPT-3.

[0091] Step 6:

[0092] Once a response is generated, the server sends it back to the terminal. The terminal displays the received response in its user interface. The input is the response message received from the server, and the output is the display on the terminal screen. The terminal adjusts the format to display advice and suggestions in a way that is easy for the user to understand.

[0093] Step 7:

[0094] The server selects third-party services based on user needs. Inputs are user behavior data and related information, while outputs are information about the selected third-party services. The server applies an advertising optimization algorithm to select the most suitable service based on the collected data.

[0095] Step 8:

[0096] The selected third-party service information is notified to the user via the terminal. The input is the selected service information, and the output is a notification message to the user. The terminal notifies the user in a timely manner and encourages feedback to pique their interest.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] In modern society, providing personalized services tailored to individual lifestyles and preferences is crucial. However, existing systems struggle to provide timely, individualized services and information that meet the specific needs of each user. Furthermore, consumer robots designed to support daily life within the home do not adequately meet user needs.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes means for acquiring personal information, means for training a data analysis model, and means for generating individual responses based on user inquiries. This enables the provision of information and service proposals tailored to the individual needs of users.

[0102] "Personal information" refers to data about the user, including information such as name, place of residence, contact information, and activity history.

[0103] A "data analysis model" is a computational model that uses machine learning and artificial intelligence technologies to analyze large amounts of data and identify specific patterns or make predictions.

[0104] An "information terminal" is a device that enables interaction between a user and a system, and includes smartphones, tablets, computers, and other similar devices.

[0105] "External services" refer to third-party services and solutions selected to partner with this system and provide additional value to users.

[0106] "Mechanical devices" refer to machines intended to support users within the home, and include robots and the like.

[0107] "Behavioral data" refers to records of a user's activities and choices in their daily life, including travel routes, purchase history, and device usage history.

[0108] The system for realizing this invention collects personal information and provides user-optimized responses and services. The server has high-performance computing capabilities and analyzes user behavior data using data analysis models. Specifically, the server trains machine learning models using Tensorflow® or PyTorch and generates analysis results. This enables responses that can appropriately address the individual needs of users.

[0109] Users interact with the system through information terminals such as smartphones and tablets. These terminals transmit collected personal information to the server and then present the user with responses and service suggestions received from the server.

[0110] Furthermore, robots, as mechanical devices, are operated to support users' lives. These robots connect to the cloud and provide in-home technical services based on instructions from servers. This includes health management, schedule management, and assistance with operating other household devices.

[0111] For example, if a user prompts the robot with "Choose clothes to match today's schedule," the server analyzes behavioral data and weather information and communicates appropriate clothing choices to the robot. Based on this information, the robot then makes clothing suggestions to the user.

[0112] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0113] Step 1:

[0114] Users input personal information and specific needs through information devices such as smartphones and tablets. The input information is transmitted to the server by the device. The input includes user behavior data and prompt messages.

[0115] Step 2:

[0116] The server stores the received personal information in a database using an encrypted protocol. The stored information is used for later data analysis.

[0117] Step 3:

[0118] The server uses machine learning models to analyze data based on stored user behavior data. Software such as TensorFlow and PyTorch are used to predict user patterns and needs based on the input data.

[0119] Step 4:

[0120] Based on user prompts, the server generates the optimal response based on the analysis results. The generation AI model creates individual responses tailored to the user's requests.

[0121] Step 5:

[0122] The generated response is sent from the server to the information terminal. The information terminal then presents the received response to the user. This allows the user to receive information tailored to their specific needs.

[0123] Step 6:

[0124] The server transmits control information to partnered external services and in-home robots as needed. This includes health management, scheduling, and assistance with operating home devices.

[0125] Step 7:

[0126] The robot performs specific actions for the user based on information provided by the server. For example, it might suggest appropriate clothing or adjust the schedule in response to the user's request.

[0127] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0128] This invention is a personal AI system incorporating an emotion engine that recognizes user emotions, enabling the provision of information and service suggestions tailored to the user's individual needs. The system consists of a server, a terminal, and the user as its main components, and achieves advanced personalization using the emotion engine.

[0129] Data collection and sentiment recognition

[0130] Users log in to the system via their device and consent to the collection of personal information. Based on this, the device collects the user's text messages, voice data, and other behavioral data. The device has a built-in emotion engine that can analyze the text and voice data collected from the user in real time and recognize the user's emotional state.

[0131] Data analysis and response generation

[0132] The server stores data received from the terminal and sentiment recognition results, and uses this to train a machine learning model. When a user enters a question or request for advice, the terminal sends it to the server. Based on the machine learning model and the sentiment engine's results, the server generates an optimal, personalized response that also takes into account the user's emotional state. This response may include encouraging messages or emotionally sensitive advice.

[0133] Specific example

[0134] For example, if a user inputs "I've been feeling very anxious lately" into the system, the emotion engine detects the user's anxiety, and the server, referencing past data and emotion recognition results, generates advice to reduce stress. Specifically, this advice may include suggestions such as deep breathing exercises or recommendations for calming music.

[0135] Selection and notification of partner services

[0136] Furthermore, the server utilizes the results of the emotion engine to select suitable partner services for the user and applies an advertising optimization algorithm. The selected service information is notified to the device, and suggestions are made based on the user's emotional state. This approach allows for the appropriate provision of services that are likely to interest the user.

[0137] Thus, the present invention aims to improve user satisfaction by combining an emotion engine and machine learning to enable more intimate and personalized services for users.

[0138] The following describes the processing flow.

[0139] Step 1:

[0140] Users log in to the system via their device and go through a process to grant permission for the collection of personal information and sentiment recognition. This allows them to begin using the system.

[0141] Step 2:

[0142] The device monitors text and voice input in real time according to the user's usage and prepares to collect this data. At the same time, the emotion engine starts up and prepares to analyze emotions from the input data.

[0143] Step 3:

[0144] The emotion engine analyzes user-inputted text and audio in real time and recognizes emotions from the content. For example, emotions are determined based on the tone of the text, the pitch of the voice, and other factors. The recognition results are immediately sent to the server.

[0145] Step 4:

[0146] The server securely stores the sentiment analysis results and behavioral data received from the terminals in an encrypted database. This allows for the accumulation of historical data for each user.

[0147] Step 5:

[0148] When a user enters a specific question or request into their device, the device sends it to the server. This request also includes the sentiment recognition results.

[0149] Step 6:

[0150] The server processes the received request and applies a machine learning model to create a personalized response best suited to the user's situation. This process also takes emotion recognition into account; for example, if the user is feeling anxious, it will generate a reassuring response.

[0151] Step 7:

[0152] The generated response is sent from the server to the terminal and then presented to the user. The user can then receive detailed advice and emotionally responsive support.

[0153] Step 8:

[0154] Based on emotion recognition results and user data, the server selects the partner services that are predicted to be of the user's greatest interest. An advertising optimization algorithm is used in this selection process.

[0155] Step 9:

[0156] Information about proposed partner services will be notified to the user via their device. This notification uses language that is considerate of the user's emotional state and is designed to be easily engaging.

[0157] (Example 2)

[0158] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] In modern society, users demand information and service suggestions tailored to their specific situations and emotional states. However, conventional systems have struggled to generate responses that take into account the user's emotional state, making personalized service delivery difficult. Furthermore, there is a growing need to effectively utilize text and audio data to provide user-friendly suggestions.

[0160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0161] In this invention, the server includes means for recognizing the user's emotional state, means for training a machine learning model based on personal information and the user's emotional state, and means for notifying the user of selected partner services. This enables personalized responses and service provision in accordance with the user's emotions.

[0162] "Emotional state" refers to the psychological or emotional state of a user, as analyzed from their text and voice data.

[0163] "Personal information" refers to a collection of information that includes data such as text messages, voice data, and behavioral information about a user.

[0164] A "machine learning model" refers to a computational model that includes algorithms to learn from data and generate responses and suggestions that are appropriate for the user.

[0165] A "terminal" refers to a device such as a computer, smartphone, or tablet that a user uses to access a system.

[0166] "Partner services" refer to external related services or products offered to users that are selected based on the user's current emotional state and interests.

[0167] A "response" refers to a reply or action that is generated and provided in response to user input.

[0168] An "emotion engine" refers to a technology that analyzes a user's text or voice data to recognize their emotional state.

[0169] This invention is a personal AI system that combines an emotion engine for analyzing user emotions with a machine learning model. This system provides information and service suggestions tailored to the user's individual needs, achieving a high level of personalization. The main components of the system include a server, a terminal, and the user.

[0170] Users log in to the system using their devices and consent to the collection of personal information. The devices have an emotion engine built in that analyzes text messages and voice data collected from users in real time. By analyzing this data, the emotion engine can identify the user's emotional state.

[0171] The server stores data sent from the terminal and emotion recognition results, and uses this to train a machine learning model. When a user inputs a question or request for advice into the terminal, the terminal sends it to the server, which uses the machine learning model and the emotion engine's analysis results to generate the optimal response. This response may include encouragement or advice tailored to the user's emotional state.

[0172] As a concrete example, consider a scenario where a user enters "I've been feeling very anxious lately" as a prompt. In this case, the emotion engine detects the user's anxiety, and the server generates advice to alleviate the anxiety based on accumulated historical data and the emotion recognition results. This advice may include recommendations for deep breathing exercises or relaxation music.

[0173] Furthermore, the server selects partner services based on the results of the emotion engine and applies an advertising optimization algorithm. This allows it to select services that are appropriate to the user's emotional state and notify the user through their device. This approach makes it possible to provide services that appropriately attract the user's interest.

[0174] This system utilizes an emotion engine and machine learning to provide users with more personalized and intimate services, aiming to improve user satisfaction.

[0175] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0176] Step 1:

[0177] The user logs into the system via a terminal. During the login process, the user enters information to consent to the collection of personal information for sentiment analysis. Based on this input, the terminal begins collecting the user's text messages and voice data and storing them in a database.

[0178] Step 2:

[0179] The emotion engine built into the device analyzes collected text messages and voice data in real time. This data is used as input for emotion analysis, and natural language processing techniques are employed to generate emotion indicators. The output is an emotion tag indicating the user's emotional state, which is used in subsequent processing steps.

[0180] Step 3:

[0181] The device sends text and audio data, along with generated emotion tags, to the server. This is passed to the server as input, and the server generates output that stores the received data in the user profile.

[0182] Step 4:

[0183] The server uses accumulated data as input to train a machine learning model. Specifically, it updates the model's weights using past user data and current sentiment data. The output is an improved model that enables sophisticated response generation.

[0184] Step 5:

[0185] The user enters a question or request for advice into the terminal. This input is sent to the server, which generates a response based on sentiment tags and a machine learning model. The output is a personalized response that includes encouragement and advice tailored to the user's emotional state.

[0186] Step 6:

[0187] The server sends the generated individual response to the terminal, which then notifies the user. The user receives this response and can try out the available actions or relaxation techniques.

[0188] Step 7:

[0189] The server selects partner services based on the results of the emotion engine and applies an advertising optimization algorithm. This process takes input to select services that take the user's emotional state into consideration. The output is the service information best suited to the user and is provided to the user through their terminal.

[0190] (Application Example 2)

[0191] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0192] In recent years, smart systems aimed at improving convenience within the home have become increasingly popular. However, conventional systems have been unable to adjust their operation to take into account the user's emotional state, making them insufficient in providing the optimal support the user needs. In particular, there is a need for technology that can recognize the user's emotions in real time and dynamically control the operation of home appliances based on this.

[0193] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0194] In this invention, the server includes means for acquiring individual data, means for training a learning model, and means for acquiring user voice data and text messages and recognizing emotions. This enables optimal control of home appliances in accordance with the user's emotions.

[0195] "Individual data" refers to unique information related to a user, such as behavioral information and preference information, or other information about a specific user.

[0196] A "learning model" is an algorithm that enables prediction and classification based on data, and is particularly used in machine learning.

[0197] A "response" refers to the information or instructions generated in response to a user inquiry, providing results that meet the user's requirements.

[0198] "Partner services" refer to services provided by external businesses or systems to users by the system.

[0199] "Voice data" refers to data that records the voice spoken by a user as digital information, and is used for speech recognition and sentiment analysis.

[0200] A "text message" refers to a message sent by a user as a string of characters, and is data that is subject to sentiment recognition and information analysis.

[0201] "Emotions" refer to information that indicates the user's mental state, and the psychological state inferred from voice and text.

[0202] "Household appliances" refer to various devices and equipment installed in homes that operate on electricity, including air conditioners, lighting, and entertainment equipment.

[0203] This invention is a system for controlling home appliances based on user emotion recognition. This system acquires user voice data and text messages and uses that data to analyze emotions in real time. The terminal is connected to various home appliances installed in the home and instructs those appliances to perform actions according to the user's emotional state.

[0204] The server first acquires individual data, including user voice data and text messages. Next, it trains a learning model based on the acquired data to recognize the user's emotions. After the emotions are recognized, the results are used to control the operation of household appliances. These appliances include lighting, air conditioners, and music players, and the optimal operating state is instructed based on the user's specific emotional state.

[0205] For example, if a user says "I'm tired today," the device will pick up the audio, and the emotion recognition module will determine that the user is "tired." Based on this determination, the lighting will be adjusted to a softer brightness, and relaxing music will be played to improve the user's comfort. By applying a generative AI model, a prompt such as "When a user is feeling stressed, what advice should be displayed to help them relax?" can be used.

[0206] In this way, we can realize a system that provides meticulous service tailored to the user's emotions.

[0207] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0208] Step 1:

[0209] The server retrieves user voice data and text messages from the terminal. The input is raw voice data and text messages, which are converted into a digital format for processing by the emotion recognition module. Analyzing the converted data prepares it for the next step.

[0210] Step 2:

[0211] The server uses a learning model to perform emotion recognition based on the acquired data. The input digital data is passed through the emotion recognition algorithm to determine the user's emotional state. The output is an emotional state label, such as "fatigue" or "anxiety."

[0212] Step 3:

[0213] The server generates control instructions for household appliances based on the results of emotion recognition. The input is an emotion label determined by emotion recognition, and this, along with a pre-configured set of rules, is used to determine which appliances to operate and how. For example, if "fatigue" is detected, the server will generate instructions to dim the lights and play relaxing music.

[0214] Step 4:

[0215] The terminal receives control instructions from the server and executes specific actions on household appliances. The input is the control instructions from the server, and the output is the actual physical operation of the appliance. Actions such as adjusting lighting or playing music are taken to enhance user comfort.

[0216] Step 5:

[0217] Users experience the environmental changes adjusted by the system and send feedback to their terminals as needed. The input is the user's subjective impression of the system changes, and the output is used for subsequent processing, contributing to further improvements in the system's accuracy.

[0218] This series of processes enables nuanced control of home appliances based on user emotions.

[0219] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0220] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0221] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0222] [Second Embodiment]

[0223] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0224] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0225] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0226] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0227] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0228] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0229] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0230] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0231] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0232] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0233] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0234] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0235] This invention provides a personal AI system that enables the provision of information and service suggestions tailored to the individual needs of users. The system consists of three main components: a server, a terminal, and a user.

[0236] Data collection

[0237] Users connect to this system via their devices after consenting to the provision of their personal information. The devices collect information and behavioral data from the user (e.g., social media posts, schedule information, etc.) and send it to the server. The server receives this data and stores it in a database in an encrypted format.

[0238] Data analysis and response generation

[0239] The server trains a machine learning model based on stored user data. This model analyzes the user's past behavior and current situation. When a user asks a question to the system using a terminal, the terminal sends the input to the server. The server uses the machine learning model to generate a personalized response optimized for the user and presents it to the user through the terminal.

[0240] Specific example

[0241] For example, if a user inputs a concern such as "I've been having trouble sleeping lately" into the system, the server will refer to past behavioral data and health information to generate specific advice to improve sleep quality (e.g., reviewing bedtime routines, relaxation techniques, etc.). This information is then provided to the user via their device.

[0242] Selection and notification of partner services

[0243] Furthermore, the server selects services from partner services that it deems appropriate for the user. This is done using an advertising optimization algorithm to select partner services that match the user's individual needs. The selection results are delivered to the device and the user is notified, allowing the user to learn about services that interest them.

[0244] In this way, the present invention aims to enhance user satisfaction by enabling responses tailored to individual user needs and service proposals that meet their interests.

[0245] The following describes the processing flow.

[0246] Step 1:

[0247] Users log in to the system via their device and give their consent regarding the collection of personal information. In this process, the device collects the user's basic profile information.

[0248] Step 2:

[0249] The device monitors user activity data (e.g., social media posts, schedules, health information) and sends this data to the server. Real-time updates ensure that the server reflects the user's latest information.

[0250] Step 3:

[0251] The server stores data received from the terminal in a database. The stored data is kept in an encrypted format to protect privacy.

[0252] Step 4:

[0253] The server periodically preprocesses the data in the database, preparing it as training data for machine learning models. This includes data cleaning and filtering.

[0254] Step 5:

[0255] The server trains a machine learning model using pre-processed data. The model learns user behavior patterns and preferences, enabling predictions tailored to individual needs.

[0256] Step 6:

[0257] When a user enters a specific question or request into the device, the device forwards that request to the server. The format and content of the input are then validated.

[0258] Step 7:

[0259] The server applies a machine learning model based on the user's request and generates the optimal individual response by referring to past data.

[0260] Step 8:

[0261] The generated response is sent from the server to the terminal and displayed to the user. This allows the user to receive advice and information appropriate to their situation.

[0262] Step 9:

[0263] The server selects partner services based on user data and applies an advertising optimization algorithm to choose the appropriate service.

[0264] Step 10:

[0265] Information about selected partner services is notified to the user via their device, allowing them to receive service suggestions tailored to their interests.

[0266] (Example 1)

[0267] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0268] Extracting meaningful information from vast amounts of personal and behavioral data, and providing appropriate responses and service suggestions tailored to individual user needs, is a major challenge in today's information society. A system that efficiently solves this problem and improves user satisfaction is needed.

[0269] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0270] In this invention, the server includes a device for acquiring personal information, a device for training an information processing model, and a device for clustering user behavior data. This enables the provision of information and service proposals optimized for the individual needs of users.

[0271] "Personal information" refers to unique data about a user, including but not limited to name, address, contact information, and activity history.

[0272] An "information processing model" refers to an algorithm or mathematical model that is trained on personal information and behavioral data to generate user-optimized responses.

[0273] "Third-party services" refer to services provided by external organizations or companies other than the server provider, and are selected based on the user's needs.

[0274] "Encryption" refers to the process of transforming the content of data using specific keys and algorithms in order to prevent unauthorized access to the data.

[0275] Clustering is a data analysis technique that divides a dataset into groups based on similarity, and is used to discover meaningful patterns or groups.

[0276] This personal AI system embodies the technology for providing information and service suggestions tailored to user needs. The system primarily consists of three elements: a server, a terminal, and a user.

[0277] The server first receives personal information and behavioral data transmitted from the terminal and stores it in an encrypted format for secure management. Industry-standard methods such as AES are used for encryption. This makes it possible to store data efficiently while ensuring user privacy.

[0278] The server trains an information processing model using the stored data to optimize responses and suggestions for users. The information processing model used here includes general machine learning algorithms, and libraries such as Scikit-learn are utilized for analysis. The model analyzes the user's past behavior data and reveals specific patterns through clustering.

[0279] On the other hand, the terminal functions as an interface connecting the user and the server. When there is an inquiry from the user, the terminal transfers the request to the server. The server generates a response using an AI model, such as a natural language processing model, and presents it to the user via the terminal. Models like OpenAI's GPT-3 are used for this response generation.

[0280] As an example, when the user inputs "want to improve sleep quality" into the terminal, the server analyzes the collected health data and behavioral patterns and proposes improvements to the bedtime routine and relaxation techniques. This advice is displayed on the terminal in text format, and the user can improve their actions based on it.

[0281] As an example of a prompt sentence, an input in the form of "Analyze the user's sleep pattern and provide advice for improvement" can be considered.

[0282] In this way, the system aims to provide information and service suggestions tailored to the individual needs of the user and improve the user experience.

[0283] The flow of the specific process in Example 1 will be described using FIG. 11.

[0284] Step 1:

[0285] The user connects to the system through the terminal and agrees to provide personal information. The terminal sets which information to collect and begins to gather behavioral data such as SNS posts and schedule information. This input information includes the user's personal information and behavioral data. The terminal organizes these data into an appropriate format and sends them to the server.

[0286] Step 2:

[0287] The server receives the data sent from the terminal, encrypts it, and stores it in the database. The input is the user's personal information and behavioral data, and the output is an encrypted database entry. The server uses an encryption algorithm such as AES to protect the information.

[0288] Step 3:

[0289] The server trains an information processing model using the stored data. The input is the encrypted user data, and the output is an information processing model optimized for the user. The server uses a machine learning library such as Scikit-learn and applies a clustering algorithm to extract patterns.

[0290] Step 4:

[0291] When the user inputs a specific question or request into the terminal, the terminal forwards the query to the server. The input is the text containing the user's question, and the output is the transmission of the query to the server. The terminal sends this to the server in JSON format.

[0292] Step 5:

[0293] The server uses a generative AI model to generate an appropriate response from the user's request. The input is the user's question and a prompt reflecting various data, and the output is a response to the user. The server utilizes various natural language processing technologies as the generative AI model and may use OpenAI's GPT-3, etc.

[0294] Step 6:

[0295] Once a response is generated, the server sends it back to the terminal. The terminal displays the received response in its user interface. The input is the response message received from the server, and the output is the display on the terminal screen. The terminal adjusts the format to display advice and suggestions in a way that is easy for the user to understand.

[0296] Step 7:

[0297] The server selects third-party services based on user needs. Inputs are user behavior data and related information, while outputs are information about the selected third-party services. The server applies an advertising optimization algorithm to select the most suitable service based on the collected data.

[0298] Step 8:

[0299] The selected third-party service information is notified to the user via the terminal. The input is the selected service information, and the output is a notification message to the user. The terminal notifies the user in a timely manner and encourages feedback to pique their interest.

[0300] (Application Example 1)

[0301] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0302] In modern society, providing personalized services tailored to individual lifestyles and preferences is crucial. However, existing systems struggle to provide timely, individualized services and information that meet the specific needs of each user. Furthermore, consumer robots designed to support daily life within the home do not adequately meet user needs.

[0303] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following respective means.

[0304] In this invention, the server includes means for acquiring personal information, means for training a data analysis model, and means for generating an individual response based on an inquiry from a user. Thereby, it becomes possible to provide information and propose services according to the individual needs of the user.

[0305] "Personal information" is data regarding the user, and includes information such as name, place of residence, contact information, and action history.

[0306] "Data analysis model" is a computational model for analyzing a large amount of data using machine learning or artificial intelligence technology and making specific patterns or predictions.

[0307] "Information terminal" is a device that enables interaction between the user and the system, and includes smartphones, tablets, computers, and the like.

[0308] "External service" is a service or solution of a third party that partners with this system and is selected to provide additional value to the user.

[0309] "Mechanical device" is a machine intended to support the user within the home, and includes robots and the like.

[0310] "Action data" is a record of the activities and choices in the daily life of the user, and includes movement routes, purchase histories, device operation histories, and the like.

[0311] The system for realizing this invention collects personal information and provides user-optimized responses and services. The server has high-performance computing capabilities and analyzes user behavior data using data analysis models. Specifically, the server trains machine learning models using TensorFlow, PyTorch, etc., and generates analysis results. This enables responses that can appropriately address the individual needs of users.

[0312] Users interact with the system through information terminals such as smartphones and tablets. These terminals transmit collected personal information to the server and then present the user with responses and service suggestions received from the server.

[0313] Furthermore, robots, as mechanical devices, are operated to support users' lives. These robots connect to the cloud and provide in-home technical services based on instructions from servers. This includes health management, schedule management, and assistance with operating other household devices.

[0314] For example, if a user prompts the robot with "Choose clothes to match today's schedule," the server analyzes behavioral data and weather information and communicates appropriate clothing choices to the robot. Based on this information, the robot then makes clothing suggestions to the user.

[0315] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0316] Step 1:

[0317] Users input personal information and specific needs through information devices such as smartphones and tablets. The input information is transmitted to the server by the device. The input includes user behavior data and prompt messages.

[0318] Step 2:

[0319] The server stores the received personal information in a database using an encrypted protocol. The stored information is used for later data analysis.

[0320] Step 3:

[0321] The server uses machine learning models to analyze data based on stored user behavior data. Software such as TensorFlow and PyTorch are used to predict user patterns and needs based on the input data.

[0322] Step 4:

[0323] Based on user prompts, the server generates the optimal response based on the analysis results. The generation AI model creates individual responses tailored to the user's requests.

[0324] Step 5:

[0325] The generated response is sent from the server to the information terminal. The information terminal then presents the received response to the user. This allows the user to receive information tailored to their specific needs.

[0326] Step 6:

[0327] The server transmits control information to partnered external services and in-home robots as needed. This includes health management, scheduling, and assistance with operating home devices.

[0328] Step 7:

[0329] The robot performs specific actions for the user based on information provided by the server. For example, it might suggest appropriate clothing or adjust the schedule in response to the user's request.

[0330] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0331] This invention is a personal AI system incorporating an emotion engine that recognizes user emotions, enabling the provision of information and service suggestions tailored to the user's individual needs. The system consists of a server, a terminal, and the user as its main components, and achieves advanced personalization using the emotion engine.

[0332] Data collection and sentiment recognition

[0333] Users log in to the system via their device and consent to the collection of personal information. Based on this, the device collects the user's text messages, voice data, and other behavioral data. The device has a built-in emotion engine that can analyze the text and voice data collected from the user in real time and recognize the user's emotional state.

[0334] Data analysis and response generation

[0335] The server stores data received from the terminal and sentiment recognition results, and uses this to train a machine learning model. When a user enters a question or request for advice, the terminal sends it to the server. Based on the machine learning model and the sentiment engine's results, the server generates an optimal, personalized response that also takes into account the user's emotional state. This response may include encouraging messages or emotionally sensitive advice.

[0336] Specific example

[0337] For example, if a user inputs "I've been feeling very anxious lately" into the system, the emotion engine detects the user's anxiety, and the server, referencing past data and emotion recognition results, generates advice to reduce stress. Specifically, this advice may include suggestions such as deep breathing exercises or recommendations for calming music.

[0338] Selection and notification of partner services

[0339] Furthermore, the server utilizes the results of the emotion engine to select suitable partner services for the user and applies an advertising optimization algorithm. The selected service information is notified to the device, and suggestions are made based on the user's emotional state. This approach allows for the appropriate provision of services that are likely to interest the user.

[0340] Thus, the present invention aims to improve user satisfaction by combining an emotion engine and machine learning to enable more intimate and personalized services for users.

[0341] The following describes the processing flow.

[0342] Step 1:

[0343] Users log in to the system via their device and go through a process to grant permission for the collection of personal information and sentiment recognition. This allows them to begin using the system.

[0344] Step 2:

[0345] The device monitors text and voice input in real time according to the user's usage and prepares to collect this data. At the same time, the emotion engine starts up and prepares to analyze emotions from the input data.

[0346] Step 3:

[0347] The emotion engine analyzes user-inputted text and audio in real time and recognizes emotions from the content. For example, emotions are determined based on the tone of the text, the pitch of the voice, and other factors. The recognition results are immediately sent to the server.

[0348] Step 4:

[0349] The server securely stores the sentiment analysis results and behavioral data received from the terminals in an encrypted database. This allows for the accumulation of historical data for each user.

[0350] Step 5:

[0351] When a user enters a specific question or request into their device, the device sends it to the server. This request also includes the sentiment recognition results.

[0352] Step 6:

[0353] The server processes the received request and applies a machine learning model to create a personalized response best suited to the user's situation. This process also takes emotion recognition into account; for example, if the user is feeling anxious, it will generate a reassuring response.

[0354] Step 7:

[0355] The generated response is sent from the server to the terminal and then presented to the user. The user can then receive detailed advice and emotionally responsive support.

[0356] Step 8:

[0357] Based on emotion recognition results and user data, the server selects the partner services that are predicted to be of the user's greatest interest. An advertising optimization algorithm is used in this selection process.

[0358] Step 9:

[0359] Information about proposed partner services will be notified to the user via their device. This notification uses language that is considerate of the user's emotional state and is designed to be easily engaging.

[0360] (Example 2)

[0361] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0362] In modern society, users demand information and service suggestions tailored to their specific situations and emotional states. However, conventional systems have struggled to generate responses that take into account the user's emotional state, making personalized service delivery difficult. Furthermore, there is a growing need to effectively utilize text and audio data to provide user-friendly suggestions.

[0363] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0364] In this invention, the server includes means for recognizing the user's emotional state, means for training a machine learning model based on personal information and the user's emotional state, and means for notifying the user of selected partner services. This enables personalized responses and service provision in accordance with the user's emotions.

[0365] "Emotional state" refers to the psychological or emotional state of a user, as analyzed from their text and voice data.

[0366] "Personal information" refers to a collection of information that includes data such as text messages, voice data, and behavioral information about a user.

[0367] A "machine learning model" refers to a computational model that includes algorithms to learn from data and generate responses and suggestions that are appropriate for the user.

[0368] A "terminal" refers to a device such as a computer, smartphone, or tablet that a user uses to access a system.

[0369] "Partner services" refer to external related services or products offered to users that are selected based on the user's current emotional state and interests.

[0370] A "response" refers to a reply or action that is generated and provided in response to user input.

[0371] An "emotion engine" refers to a technology that analyzes a user's text or voice data to recognize their emotional state.

[0372] This invention is a personal AI system that combines an emotion engine for analyzing user emotions with a machine learning model. This system provides information and service suggestions tailored to the user's individual needs, achieving a high level of personalization. The main components of the system include a server, a terminal, and the user.

[0373] Users log in to the system using their devices and consent to the collection of personal information. The devices have an emotion engine built in that analyzes text messages and voice data collected from users in real time. By analyzing this data, the emotion engine can identify the user's emotional state.

[0374] The server stores data sent from the terminal and emotion recognition results, and uses this to train a machine learning model. When a user inputs a question or request for advice into the terminal, the terminal sends it to the server, which uses the machine learning model and the emotion engine's analysis results to generate the optimal response. This response may include encouragement or advice tailored to the user's emotional state.

[0375] As a concrete example, consider a scenario where a user enters "I've been feeling very anxious lately" as a prompt. In this case, the emotion engine detects the user's anxiety, and the server generates advice to alleviate the anxiety based on accumulated historical data and the emotion recognition results. This advice may include recommendations for deep breathing exercises or relaxation music.

[0376] Furthermore, the server selects partner services based on the results of the emotion engine and applies an advertising optimization algorithm. This allows it to select services that are appropriate to the user's emotional state and notify the user through their device. This approach makes it possible to provide services that appropriately attract the user's interest.

[0377] This system utilizes an emotion engine and machine learning to provide users with more personalized and intimate services, aiming to improve user satisfaction.

[0378] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0379] Step 1:

[0380] The user logs into the system via a terminal. During the login process, the user enters information to consent to the collection of personal information for sentiment analysis. Based on this input, the terminal begins collecting the user's text messages and voice data and storing them in a database.

[0381] Step 2:

[0382] The emotion engine built into the device analyzes collected text messages and voice data in real time. This data is used as input for emotion analysis, and natural language processing techniques are employed to generate emotion indicators. The output is an emotion tag indicating the user's emotional state, which is used in subsequent processing steps.

[0383] Step 3:

[0384] The device sends text and audio data, along with generated emotion tags, to the server. This is passed to the server as input, and the server generates output that stores the received data in the user profile.

[0385] Step 4:

[0386] The server uses accumulated data as input to train a machine learning model. Specifically, it updates the model's weights using past user data and current sentiment data. The output is an improved model that enables sophisticated response generation.

[0387] Step 5:

[0388] The user enters a question or request for advice into the terminal. This input is sent to the server, which generates a response based on sentiment tags and a machine learning model. The output is a personalized response that includes encouragement and advice tailored to the user's emotional state.

[0389] Step 6:

[0390] The server sends the generated individual response to the terminal, which then notifies the user. The user receives this response and can try out the available actions or relaxation techniques.

[0391] Step 7:

[0392] The server selects partner services based on the results of the emotion engine and applies an advertising optimization algorithm. This process takes input to select services that take the user's emotional state into consideration. The output is the service information best suited to the user and is provided to the user through their terminal.

[0393] (Application Example 2)

[0394] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0395] In recent years, smart systems aimed at improving convenience within the home have become increasingly popular. However, conventional systems have been unable to adjust their operation to take into account the user's emotional state, making them insufficient in providing the optimal support the user needs. In particular, there is a need for technology that can recognize the user's emotions in real time and dynamically control the operation of home appliances based on this.

[0396] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0397] In this invention, the server includes means for acquiring individual data, means for training a learning model, and means for acquiring user voice data and text messages and recognizing emotions. This enables optimal control of home appliances in accordance with the user's emotions.

[0398] "Individual data" refers to unique information related to a user, such as behavioral information and preference information, or other information about a specific user.

[0399] A "learning model" is an algorithm that enables prediction and classification based on data, and is particularly used in machine learning.

[0400] A "response" refers to the information or instructions generated in response to a user inquiry, providing results that meet the user's requirements.

[0401] "Partner services" refer to services provided by external businesses or systems to users by the system.

[0402] "Voice data" refers to data that records the voice spoken by a user as digital information, and is used for speech recognition and sentiment analysis.

[0403] A "text message" refers to a message sent by a user as a string of characters, and is data that is subject to sentiment recognition and information analysis.

[0404] "Emotions" refer to information that indicates the user's mental state, and the psychological state inferred from voice and text.

[0405] "Household appliances" refer to various devices and equipment installed in homes that operate on electricity, including air conditioners, lighting, and entertainment equipment.

[0406] This invention is a system for controlling home appliances based on user emotion recognition. This system acquires user voice data and text messages and uses that data to analyze emotions in real time. The terminal is connected to various home appliances installed in the home and instructs those appliances to perform actions according to the user's emotional state.

[0407] The server first acquires individual data, including user voice data and text messages. Next, it trains a learning model based on the acquired data to recognize the user's emotions. After the emotions are recognized, the results are used to control the operation of household appliances. These appliances include lighting, air conditioners, and music players, and the optimal operating state is instructed based on the user's specific emotional state.

[0408] For example, if a user says "I'm tired today," the device will pick up the audio, and the emotion recognition module will determine that the user is "tired." Based on this determination, the lighting will be adjusted to a softer brightness, and relaxing music will be played to improve the user's comfort. By applying a generative AI model, a prompt such as "When a user is feeling stressed, what advice should be displayed to help them relax?" can be used.

[0409] In this way, we can realize a system that provides meticulous service tailored to the user's emotions.

[0410] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0411] Step 1:

[0412] The server retrieves user voice data and text messages from the terminal. The input is raw voice data and text messages, which are converted into a digital format for processing by the emotion recognition module. Analyzing the converted data prepares it for the next step.

[0413] Step 2:

[0414] The server uses a learning model to perform emotion recognition based on the acquired data. The input digital data is passed through the emotion recognition algorithm to determine the user's emotional state. The output is an emotional state label, such as "fatigue" or "anxiety."

[0415] Step 3:

[0416] The server generates control instructions for household appliances based on the results of emotion recognition. The input is an emotion label determined by emotion recognition, and this, along with a pre-configured set of rules, is used to determine which appliances to operate and how. For example, if "fatigue" is detected, the server will generate instructions to dim the lights and play relaxing music.

[0417] Step 4:

[0418] The terminal receives control instructions from the server and executes specific actions on household appliances. The input is the control instructions from the server, and the output is the actual physical operation of the appliance. Actions such as adjusting lighting or playing music are taken to enhance user comfort.

[0419] Step 5:

[0420] Users experience the environmental changes adjusted by the system and send feedback to their terminals as needed. The input is the user's subjective impression of the system changes, and the output is used for subsequent processing, contributing to further improvements in the system's accuracy.

[0421] This series of processes enables nuanced control of home appliances based on user emotions.

[0422] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0423] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0424] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0425] [Third Embodiment]

[0426] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0427] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0428] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0429] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0430] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0431] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0432] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0433] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0434] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0435] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0436] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0437] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0438] This invention provides a personal AI system that enables the provision of information and service suggestions tailored to the individual needs of users. The system consists of three main components: a server, a terminal, and a user.

[0439] Data collection

[0440] Users connect to this system via their devices after consenting to the provision of their personal information. The devices collect information and behavioral data from the user (e.g., social media posts, schedule information, etc.) and send it to the server. The server receives this data and stores it in a database in an encrypted format.

[0441] Data analysis and response generation

[0442] The server trains a machine learning model based on stored user data. This model analyzes the user's past behavior and current situation. When a user asks a question to the system using a terminal, the terminal sends the input to the server. The server uses the machine learning model to generate a personalized response optimized for the user and presents it to the user through the terminal.

[0443] Specific example

[0444] For example, if a user inputs a concern such as "I've been having trouble sleeping lately" into the system, the server will refer to past behavioral data and health information to generate specific advice to improve sleep quality (e.g., reviewing bedtime routines, relaxation techniques, etc.). This information is then provided to the user via their device.

[0445] Selection and notification of partner services

[0446] Furthermore, the server selects services from partner services that it deems appropriate for the user. This is done using an advertising optimization algorithm to select partner services that match the user's individual needs. The selection results are delivered to the device and the user is notified, allowing the user to learn about services that interest them.

[0447] In this way, the present invention aims to enhance user satisfaction by enabling responses tailored to individual user needs and service proposals that meet their interests.

[0448] The following describes the processing flow.

[0449] Step 1:

[0450] Users log in to the system via their device and give their consent regarding the collection of personal information. In this process, the device collects the user's basic profile information.

[0451] Step 2:

[0452] The device monitors user activity data (e.g., social media posts, schedules, health information) and sends this data to the server. Real-time updates ensure that the server reflects the user's latest information.

[0453] Step 3:

[0454] The server stores data received from the terminal in a database. The stored data is kept in an encrypted format to protect privacy.

[0455] Step 4:

[0456] The server periodically preprocesses the data in the database, preparing it as training data for machine learning models. This includes data cleaning and filtering.

[0457] Step 5:

[0458] The server trains a machine learning model using pre-processed data. The model learns user behavior patterns and preferences, enabling predictions tailored to individual needs.

[0459] Step 6:

[0460] When a user enters a specific question or request into the device, the device forwards that request to the server. The format and content of the input are then validated.

[0461] Step 7:

[0462] The server applies a machine learning model based on the user's request and generates the optimal individual response by referring to past data.

[0463] Step 8:

[0464] The generated response is sent from the server to the terminal and displayed to the user. This allows the user to receive advice and information appropriate to their situation.

[0465] Step 9:

[0466] The server selects partner services based on user data and applies an advertising optimization algorithm to choose the appropriate service.

[0467] Step 10:

[0468] Information about selected partner services is notified to the user via their device, allowing them to receive service suggestions tailored to their interests.

[0469] (Example 1)

[0470] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0471] Extracting meaningful information from vast amounts of personal and behavioral data, and providing appropriate responses and service suggestions tailored to individual user needs, is a major challenge in today's information society. A system that efficiently solves this problem and improves user satisfaction is needed.

[0472] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0473] In this invention, the server includes a device for acquiring personal information, a device for training an information processing model, and a device for clustering user behavior data. This enables the provision of information and service proposals optimized for the individual needs of users.

[0474] "Personal information" refers to unique data about a user, including but not limited to name, address, contact information, and activity history.

[0475] An "information processing model" refers to an algorithm or mathematical model that is trained on personal information and behavioral data to generate user-optimized responses.

[0476] "Third-party services" refer to services provided by external organizations or companies other than the server provider, and are selected based on the user's needs.

[0477] "Encryption" refers to the process of transforming the content of data using specific keys and algorithms in order to prevent unauthorized access to the data.

[0478] Clustering is a data analysis technique that divides a dataset into groups based on similarity, and is used to discover meaningful patterns or groups.

[0479] This personal AI system embodies the technology for providing information and service suggestions tailored to user needs. The system primarily consists of three elements: a server, a terminal, and a user.

[0480] The server first receives personal information and behavioral data transmitted from the terminal and stores it in an encrypted format for secure management. Industry-standard methods such as AES are used for encryption. This makes it possible to store data efficiently while ensuring user privacy.

[0481] The server trains an information processing model using stored data to optimize responses and suggestions to the user. The information processing model used includes common machine learning algorithms, and libraries such as Scikit-learn are utilized for analysis. The model analyzes the user's past behavioral data and reveals specific patterns through clustering.

[0482] On the other hand, the terminal functions as an interface connecting the user and the server. When a user makes an inquiry, the terminal forwards the request to the server. The server generates a response using a response generation AI model, such as a natural language processing model, and presents it to the user via the terminal. A model like OpenAI's GPT-3 is used for this response generation.

[0483] For example, if a user enters "I want to improve my sleep quality" into their device, the server analyzes collected health data and behavioral patterns to suggest improvements to their pre-sleep routine and relaxation techniques. This advice is displayed on the device in text format, allowing the user to improve their behavior based on it.

[0484] An example of a prompt message could be an input in the form of, "Analyze the user's sleep patterns and provide advice for improving their sleep quality."

[0485] In this way, the system aims to improve the user experience by providing information and service suggestions tailored to the individual needs of each user.

[0486] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0487] Step 1:

[0488] The user connects to the system through their device and consents to providing personal information. The device configures what information to collect and begins gathering behavioral data such as social media posts and schedule information. This input information includes the user's personal information and behavioral data. The device organizes this data into an appropriate format and sends it to the server.

[0489] Step 2:

[0490] The server receives data sent from the terminal, encrypts it, and stores it in a database. Inputs are user personal information and behavioral data, and outputs are encrypted database entries. The server uses encryption algorithms such as AES to protect the information.

[0491] Step 3:

[0492] The server trains an information processing model using stored data. The input is encrypted user data, and the output is an information processing model optimized for the user. The server uses machine learning libraries such as Scikit-learn and applies clustering algorithms to extract patterns.

[0493] Step 4:

[0494] When a user enters a specific question or request into the terminal, the terminal forwards that query to the server. The input is the text containing the user's question, and the output is the query sent to the server. The terminal sends this to the server in JSON format.

[0495] Step 5:

[0496] The server uses a generative AI model to generate appropriate responses from user requests. The input consists of user questions and prompts reflecting various data, while the output is the response to the user. The server utilizes various natural language processing technologies as its generative AI model, sometimes employing OpenAI's GPT-3.

[0497] Step 6:

[0498] Once a response is generated, the server sends it back to the terminal. The terminal displays the received response in its user interface. The input is the response message received from the server, and the output is the display on the terminal screen. The terminal adjusts the format to display advice and suggestions in a way that is easy for the user to understand.

[0499] Step 7:

[0500] The server selects third-party services based on user needs. Inputs are user behavior data and related information, while outputs are information about the selected third-party services. The server applies an advertising optimization algorithm to select the most suitable service based on the collected data.

[0501] Step 8:

[0502] The selected third-party service information is notified to the user via the terminal. The input is the selected service information, and the output is a notification message to the user. The terminal notifies the user in a timely manner and encourages feedback to pique their interest.

[0503] (Application Example 1)

[0504] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0505] In modern society, providing personalized services tailored to individual lifestyles and preferences is crucial. However, existing systems struggle to provide timely, individualized services and information that meet the specific needs of each user. Furthermore, consumer robots designed to support daily life within the home do not adequately meet user needs.

[0506] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0507] In this invention, the server includes means for acquiring personal information, means for training a data analysis model, and means for generating individual responses based on user inquiries. This enables the provision of information and service proposals tailored to the individual needs of users.

[0508] "Personal information" refers to data about the user, including information such as name, place of residence, contact information, and activity history.

[0509] A "data analysis model" is a computational model that uses machine learning and artificial intelligence technologies to analyze large amounts of data and identify specific patterns or make predictions.

[0510] An "information terminal" is a device that enables interaction between a user and a system, and includes smartphones, tablets, computers, and other similar devices.

[0511] "External services" refer to third-party services and solutions selected to partner with this system and provide additional value to users.

[0512] "Mechanical devices" refer to machines intended to support users within the home, and include robots and the like.

[0513] "Behavioral data" refers to records of a user's activities and choices in their daily life, including travel routes, purchase history, and device usage history.

[0514] The system for realizing this invention collects personal information and provides user-optimized responses and services. The server has high-performance computing capabilities and analyzes user behavior data using data analysis models. Specifically, the server trains machine learning models using TensorFlow, PyTorch, etc., and generates analysis results. This enables responses that can appropriately address the individual needs of users.

[0515] Users interact with the system through information terminals such as smartphones and tablets. These terminals transmit collected personal information to the server and then present the user with responses and service suggestions received from the server.

[0516] Furthermore, robots, as mechanical devices, are operated to support users' lives. These robots connect to the cloud and provide in-home technical services based on instructions from servers. This includes health management, schedule management, and assistance with operating other household devices.

[0517] For example, if a user prompts the robot with "Choose clothes to match today's schedule," the server analyzes behavioral data and weather information and communicates appropriate clothing choices to the robot. Based on this information, the robot then makes clothing suggestions to the user.

[0518] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0519] Step 1:

[0520] Users input personal information and specific needs through information devices such as smartphones and tablets. The input information is transmitted to the server by the device. The input includes user behavior data and prompt messages.

[0521] Step 2:

[0522] The server stores the received personal information in a database using an encrypted protocol. The stored information is used for later data analysis.

[0523] Step 3:

[0524] The server uses machine learning models to analyze data based on stored user behavior data. Software such as TensorFlow and PyTorch are used to predict user patterns and needs based on the input data.

[0525] Step 4:

[0526] Based on user prompts, the server generates the optimal response based on the analysis results. The generation AI model creates individual responses tailored to the user's requests.

[0527] Step 5:

[0528] The generated response is sent from the server to the information terminal. The information terminal then presents the received response to the user. This allows the user to receive information tailored to their specific needs.

[0529] Step 6:

[0530] The server transmits control information to partnered external services and in-home robots as needed. This includes health management, scheduling, and assistance with operating home devices.

[0531] Step 7:

[0532] The robot performs specific actions for the user based on information provided by the server. For example, it might suggest appropriate clothing or adjust the schedule in response to the user's request.

[0533] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0534] This invention is a personal AI system incorporating an emotion engine that recognizes user emotions, enabling the provision of information and service suggestions tailored to the user's individual needs. The system consists of a server, a terminal, and the user as its main components, and achieves advanced personalization using the emotion engine.

[0535] Data collection and sentiment recognition

[0536] Users log in to the system via their device and consent to the collection of personal information. Based on this, the device collects the user's text messages, voice data, and other behavioral data. The device has a built-in emotion engine that can analyze the text and voice data collected from the user in real time and recognize the user's emotional state.

[0537] Data analysis and response generation

[0538] The server stores data received from the terminal and sentiment recognition results, and uses this to train a machine learning model. When a user enters a question or request for advice, the terminal sends it to the server. Based on the machine learning model and the sentiment engine's results, the server generates an optimal, personalized response that also takes into account the user's emotional state. This response may include encouraging messages or emotionally sensitive advice.

[0539] Specific example

[0540] For example, if a user inputs "I've been feeling very anxious lately" into the system, the emotion engine detects the user's anxiety, and the server, referencing past data and emotion recognition results, generates advice to reduce stress. Specifically, this advice may include suggestions such as deep breathing exercises or recommendations for calming music.

[0541] Selection and notification of partner services

[0542] Furthermore, the server utilizes the results of the emotion engine to select suitable partner services for the user and applies an advertising optimization algorithm. The selected service information is notified to the device, and suggestions are made based on the user's emotional state. This approach allows for the appropriate provision of services that are likely to interest the user.

[0543] Thus, the present invention aims to improve user satisfaction by combining an emotion engine and machine learning to enable more intimate and personalized services for users.

[0544] The following describes the processing flow.

[0545] Step 1:

[0546] Users log in to the system via their device and go through a process to grant permission for the collection of personal information and sentiment recognition. This allows them to begin using the system.

[0547] Step 2:

[0548] The device monitors text and voice input in real time according to the user's usage and prepares to collect this data. At the same time, the emotion engine starts up and prepares to analyze emotions from the input data.

[0549] Step 3:

[0550] The emotion engine analyzes user-inputted text and audio in real time and recognizes emotions from the content. For example, emotions are determined based on the tone of the text, the pitch of the voice, and other factors. The recognition results are immediately sent to the server.

[0551] Step 4:

[0552] The server securely stores the sentiment analysis results and behavioral data received from the terminals in an encrypted database. This allows for the accumulation of historical data for each user.

[0553] Step 5:

[0554] When a user enters a specific question or request into their device, the device sends it to the server. This request also includes the sentiment recognition results.

[0555] Step 6:

[0556] The server processes the received request and applies a machine learning model to create a personalized response best suited to the user's situation. This process also takes emotion recognition into account; for example, if the user is feeling anxious, it will generate a reassuring response.

[0557] Step 7:

[0558] The generated response is sent from the server to the terminal and then presented to the user. The user can then receive detailed advice and emotionally responsive support.

[0559] Step 8:

[0560] Based on emotion recognition results and user data, the server selects the partner services that are predicted to be of the user's greatest interest. An advertising optimization algorithm is used in this selection process.

[0561] Step 9:

[0562] Information about proposed partner services will be notified to the user via their device. This notification uses language that is considerate of the user's emotional state and is designed to be easily engaging.

[0563] (Example 2)

[0564] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0565] In modern society, users demand information and service suggestions tailored to their specific situations and emotional states. However, conventional systems have struggled to generate responses that take into account the user's emotional state, making personalized service delivery difficult. Furthermore, there is a growing need to effectively utilize text and audio data to provide user-friendly suggestions.

[0566] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0567] In this invention, the server includes means for recognizing the user's emotional state, means for training a machine learning model based on personal information and the user's emotional state, and means for notifying the user of selected partner services. This enables personalized responses and service provision in accordance with the user's emotions.

[0568] "Emotional state" refers to the psychological or emotional state of a user, as analyzed from their text and voice data.

[0569] "Personal information" refers to a collection of information that includes data such as text messages, voice data, and behavioral information about a user.

[0570] A "machine learning model" refers to a computational model that includes algorithms to learn from data and generate responses and suggestions that are appropriate for the user.

[0571] A "terminal" refers to a device such as a computer, smartphone, or tablet that a user uses to access a system.

[0572] "Partner services" refer to external related services or products offered to users that are selected based on the user's current emotional state and interests.

[0573] A "response" refers to a reply or action that is generated and provided in response to user input.

[0574] An "emotion engine" refers to a technology that analyzes a user's text or voice data to recognize their emotional state.

[0575] This invention is a personal AI system that combines an emotion engine for analyzing user emotions with a machine learning model. This system provides information and service suggestions tailored to the user's individual needs, achieving a high level of personalization. The main components of the system include a server, a terminal, and the user.

[0576] Users log in to the system using their devices and consent to the collection of personal information. The devices have an emotion engine built in that analyzes text messages and voice data collected from users in real time. By analyzing this data, the emotion engine can identify the user's emotional state.

[0577] The server stores data sent from the terminal and emotion recognition results, and uses this to train a machine learning model. When a user inputs a question or request for advice into the terminal, the terminal sends it to the server, which uses the machine learning model and the emotion engine's analysis results to generate the optimal response. This response may include encouragement or advice tailored to the user's emotional state.

[0578] As a concrete example, consider a scenario where a user enters "I've been feeling very anxious lately" as a prompt. In this case, the emotion engine detects the user's anxiety, and the server generates advice to alleviate the anxiety based on accumulated historical data and the emotion recognition results. This advice may include recommendations for deep breathing exercises or relaxation music.

[0579] Furthermore, the server selects partner services based on the results of the emotion engine and applies an advertising optimization algorithm. This allows it to select services that are appropriate to the user's emotional state and notify the user through their device. This approach makes it possible to provide services that appropriately attract the user's interest.

[0580] This system utilizes an emotion engine and machine learning to provide users with more personalized and intimate services, aiming to improve user satisfaction.

[0581] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0582] Step 1:

[0583] The user logs into the system via a terminal. During the login process, the user enters information to consent to the collection of personal information for sentiment analysis. Based on this input, the terminal begins collecting the user's text messages and voice data and storing them in a database.

[0584] Step 2:

[0585] The emotion engine built into the device analyzes collected text messages and voice data in real time. This data is used as input for emotion analysis, and natural language processing techniques are employed to generate emotion indicators. The output is an emotion tag indicating the user's emotional state, which is used in subsequent processing steps.

[0586] Step 3:

[0587] The device sends text and audio data, along with generated emotion tags, to the server. This is passed to the server as input, and the server generates output that stores the received data in the user profile.

[0588] Step 4:

[0589] The server uses accumulated data as input to train a machine learning model. Specifically, it updates the model's weights using past user data and current sentiment data. The output is an improved model that enables sophisticated response generation.

[0590] Step 5:

[0591] The user enters a question or request for advice into the terminal. This input is sent to the server, which generates a response based on sentiment tags and a machine learning model. The output is a personalized response that includes encouragement and advice tailored to the user's emotional state.

[0592] Step 6:

[0593] The server sends the generated individual response to the terminal, which then notifies the user. The user receives this response and can try out the available actions or relaxation techniques.

[0594] Step 7:

[0595] The server selects partner services based on the results of the emotion engine and applies an advertising optimization algorithm. This process takes input to select services that take the user's emotional state into consideration. The output is the service information best suited to the user and is provided to the user through their terminal.

[0596] (Application Example 2)

[0597] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0598] In recent years, smart systems aimed at improving convenience within the home have become increasingly popular. However, conventional systems have been unable to adjust their operation to take into account the user's emotional state, making them insufficient in providing the optimal support the user needs. In particular, there is a need for technology that can recognize the user's emotions in real time and dynamically control the operation of home appliances based on this.

[0599] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0600] In this invention, the server includes means for acquiring individual data, means for training a learning model, and means for acquiring user voice data and text messages and recognizing emotions. This enables optimal control of home appliances in accordance with the user's emotions.

[0601] "Individual data" refers to unique information related to a user, such as behavioral information and preference information, or other information about a specific user.

[0602] A "learning model" is an algorithm that enables prediction and classification based on data, and is particularly used in machine learning.

[0603] A "response" refers to the information or instructions generated in response to a user inquiry, providing results that meet the user's requirements.

[0604] "Partner services" refer to services provided by external businesses or systems to users by the system.

[0605] "Voice data" refers to data that records the voice spoken by a user as digital information, and is used for speech recognition and sentiment analysis.

[0606] A "text message" refers to a message sent by a user as a string of characters, and is data that is subject to sentiment recognition and information analysis.

[0607] "Emotions" refer to information that indicates the user's mental state, and the psychological state inferred from voice and text.

[0608] "Household appliances" refer to various devices and equipment installed in homes that operate on electricity, including air conditioners, lighting, and entertainment equipment.

[0609] This invention is a system for controlling home appliances based on user emotion recognition. This system acquires user voice data and text messages and uses that data to analyze emotions in real time. The terminal is connected to various home appliances installed in the home and instructs those appliances to perform actions according to the user's emotional state.

[0610] The server first acquires individual data, including user voice data and text messages. Next, it trains a learning model based on the acquired data to recognize the user's emotions. After the emotions are recognized, the results are used to control the operation of household appliances. These appliances include lighting, air conditioners, and music players, and the optimal operating state is instructed based on the user's specific emotional state.

[0611] For example, if a user says "I'm tired today," the device will pick up the audio, and the emotion recognition module will determine that the user is "tired." Based on this determination, the lighting will be adjusted to a softer brightness, and relaxing music will be played to improve the user's comfort. By applying a generative AI model, a prompt such as "When a user is feeling stressed, what advice should be displayed to help them relax?" can be used.

[0612] In this way, we can realize a system that provides meticulous service tailored to the user's emotions.

[0613] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0614] Step 1:

[0615] The server retrieves user voice data and text messages from the terminal. The input is raw voice data and text messages, which are converted into a digital format for processing by the emotion recognition module. Analyzing the converted data prepares it for the next step.

[0616] Step 2:

[0617] The server uses a learning model to perform emotion recognition based on the acquired data. The input digital data is passed through the emotion recognition algorithm to determine the user's emotional state. The output is an emotional state label, such as "fatigue" or "anxiety."

[0618] Step 3:

[0619] The server generates control instructions for household appliances based on the results of emotion recognition. The input is an emotion label determined by emotion recognition, and this, along with a pre-configured set of rules, is used to determine which appliances to operate and how. For example, if "fatigue" is detected, the server will generate instructions to dim the lights and play relaxing music.

[0620] Step 4:

[0621] The terminal receives control instructions from the server and executes specific actions on household appliances. The input is the control instructions from the server, and the output is the actual physical operation of the appliance. Actions such as adjusting lighting or playing music are taken to enhance user comfort.

[0622] Step 5:

[0623] Users experience the environmental changes adjusted by the system and send feedback to their terminals as needed. The input is the user's subjective impression of the system changes, and the output is used for subsequent processing, contributing to further improvements in the system's accuracy.

[0624] This series of processes enables nuanced control of home appliances based on user emotions.

[0625] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0626] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0627] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0628] [Fourth Embodiment]

[0629] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0630] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0631] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0632] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0633] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0634] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0635] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0636] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0637] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0638] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0639] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0640] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0641] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0642] This invention provides a personal AI system that enables the provision of information and service suggestions tailored to the individual needs of users. The system consists of three main components: a server, a terminal, and a user.

[0643] Data collection

[0644] Users connect to this system via their devices after consenting to the provision of their personal information. The devices collect information and behavioral data from the user (e.g., social media posts, schedule information, etc.) and send it to the server. The server receives this data and stores it in a database in an encrypted format.

[0645] Data analysis and response generation

[0646] The server trains a machine learning model based on stored user data. This model analyzes the user's past behavior and current situation. When a user asks a question to the system using a terminal, the terminal sends the input to the server. The server uses the machine learning model to generate a personalized response optimized for the user and presents it to the user through the terminal.

[0647] Specific example

[0648] For example, if a user inputs a concern such as "I've been having trouble sleeping lately" into the system, the server will refer to past behavioral data and health information to generate specific advice to improve sleep quality (e.g., reviewing bedtime routines, relaxation techniques, etc.). This information is then provided to the user via their device.

[0649] Selection and notification of partner services

[0650] Furthermore, the server selects services from partner services that it deems appropriate for the user. This is done using an advertising optimization algorithm to select partner services that match the user's individual needs. The selection results are delivered to the device and the user is notified, allowing the user to learn about services that interest them.

[0651] In this way, the present invention aims to enhance user satisfaction by enabling responses tailored to individual user needs and service proposals that meet their interests.

[0652] The following describes the processing flow.

[0653] Step 1:

[0654] Users log in to the system via their device and give their consent regarding the collection of personal information. In this process, the device collects the user's basic profile information.

[0655] Step 2:

[0656] The device monitors user activity data (e.g., social media posts, schedules, health information) and sends this data to the server. Real-time updates ensure that the server reflects the user's latest information.

[0657] Step 3:

[0658] The server stores data received from the terminal in a database. The stored data is kept in an encrypted format to protect privacy.

[0659] Step 4:

[0660] The server periodically preprocesses the data in the database, preparing it as training data for machine learning models. This includes data cleaning and filtering.

[0661] Step 5:

[0662] The server trains a machine learning model using pre-processed data. The model learns user behavior patterns and preferences, enabling predictions tailored to individual needs.

[0663] Step 6:

[0664] When a user enters a specific question or request into the device, the device forwards that request to the server. The format and content of the input are then validated.

[0665] Step 7:

[0666] The server applies a machine learning model based on the user's request and generates the optimal individual response by referring to past data.

[0667] Step 8:

[0668] The generated response is sent from the server to the terminal and displayed to the user. This allows the user to receive advice and information appropriate to their situation.

[0669] Step 9:

[0670] The server selects partner services based on user data and applies an advertising optimization algorithm to choose the appropriate service.

[0671] Step 10:

[0672] Information about selected partner services is notified to the user via their device, allowing them to receive service suggestions tailored to their interests.

[0673] (Example 1)

[0674] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0675] Extracting meaningful information from vast amounts of personal and behavioral data, and providing appropriate responses and service suggestions tailored to individual user needs, is a major challenge in today's information society. A system that efficiently solves this problem and improves user satisfaction is needed.

[0676] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0677] In this invention, the server includes a device for acquiring personal information, a device for training an information processing model, and a device for clustering user behavior data. This enables the provision of information and service proposals optimized for the individual needs of users.

[0678] "Personal information" refers to unique data about a user, including but not limited to name, address, contact information, and activity history.

[0679] An "information processing model" refers to an algorithm or mathematical model that is trained on personal information and behavioral data to generate user-optimized responses.

[0680] "Third-party services" refer to services provided by external organizations or companies other than the server provider, and are selected based on the user's needs.

[0681] "Encryption" refers to the process of transforming the content of data using specific keys and algorithms in order to prevent unauthorized access to the data.

[0682] Clustering is a data analysis technique that divides a dataset into groups based on similarity, and is used to discover meaningful patterns or groups.

[0683] This personal AI system embodies the technology for providing information and service suggestions tailored to user needs. The system primarily consists of three elements: a server, a terminal, and a user.

[0684] The server first receives personal information and behavioral data transmitted from the terminal and stores it in an encrypted format for secure management. Industry-standard methods such as AES are used for encryption. This makes it possible to store data efficiently while ensuring user privacy.

[0685] The server trains an information processing model using stored data to optimize responses and suggestions to the user. The information processing model used includes common machine learning algorithms, and libraries such as Scikit-learn are utilized for analysis. The model analyzes the user's past behavioral data and reveals specific patterns through clustering.

[0686] On the other hand, the terminal functions as an interface connecting the user and the server. When a user makes an inquiry, the terminal forwards the request to the server. The server generates a response using a response generation AI model, such as a natural language processing model, and presents it to the user via the terminal. A model like OpenAI's GPT-3 is used for this response generation.

[0687] For example, if a user enters "I want to improve my sleep quality" into their device, the server analyzes collected health data and behavioral patterns to suggest improvements to their pre-sleep routine and relaxation techniques. This advice is displayed on the device in text format, allowing the user to improve their behavior based on it.

[0688] An example of a prompt message could be an input in the form of, "Analyze the user's sleep patterns and provide advice for improving their sleep quality."

[0689] In this way, the system aims to improve the user experience by providing information and service suggestions tailored to the individual needs of each user.

[0690] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0691] Step 1:

[0692] The user connects to the system through their device and consents to providing personal information. The device configures what information to collect and begins gathering behavioral data such as social media posts and schedule information. This input information includes the user's personal information and behavioral data. The device organizes this data into an appropriate format and sends it to the server.

[0693] Step 2:

[0694] The server receives data sent from the terminal, encrypts it, and stores it in a database. Inputs are user personal information and behavioral data, and outputs are encrypted database entries. The server uses encryption algorithms such as AES to protect the information.

[0695] Step 3:

[0696] The server trains an information processing model using stored data. The input is encrypted user data, and the output is an information processing model optimized for the user. The server uses machine learning libraries such as Scikit-learn and applies clustering algorithms to extract patterns.

[0697] Step 4:

[0698] When a user enters a specific question or request into the terminal, the terminal forwards that query to the server. The input is the text containing the user's question, and the output is the query sent to the server. The terminal sends this to the server in JSON format.

[0699] Step 5:

[0700] The server uses a generative AI model to generate appropriate responses from user requests. The input consists of user questions and prompts reflecting various data, while the output is the response to the user. The server utilizes various natural language processing technologies as its generative AI model, sometimes employing OpenAI's GPT-3.

[0701] Step 6:

[0702] Once a response is generated, the server sends it back to the terminal. The terminal displays the received response in its user interface. The input is the response message received from the server, and the output is the display on the terminal screen. The terminal adjusts the format to display advice and suggestions in a way that is easy for the user to understand.

[0703] Step 7:

[0704] The server selects third-party services based on user needs. Inputs are user behavior data and related information, while outputs are information about the selected third-party services. The server applies an advertising optimization algorithm to select the most suitable service based on the collected data.

[0705] Step 8:

[0706] The selected third-party service information is notified to the user via the terminal. The input is the selected service information, and the output is a notification message to the user. The terminal notifies the user in a timely manner and encourages feedback to pique their interest.

[0707] (Application Example 1)

[0708] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0709] In modern society, providing personalized services tailored to individual lifestyles and preferences is crucial. However, existing systems struggle to provide timely, individualized services and information that meet the specific needs of each user. Furthermore, consumer robots designed to support daily life within the home do not adequately meet user needs.

[0710] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0711] In this invention, the server includes means for acquiring personal information, means for training a data analysis model, and means for generating individual responses based on user inquiries. This enables the provision of information and service proposals tailored to the individual needs of users.

[0712] "Personal information" refers to data about the user, including information such as name, place of residence, contact information, and activity history.

[0713] A "data analysis model" is a computational model that uses machine learning and artificial intelligence technologies to analyze large amounts of data and identify specific patterns or make predictions.

[0714] An "information terminal" is a device that enables interaction between a user and a system, and includes smartphones, tablets, computers, and other similar devices.

[0715] "External services" refer to third-party services and solutions selected to partner with this system and provide additional value to users.

[0716] "Mechanical devices" refer to machines intended to support users within the home, and include robots and the like.

[0717] "Behavioral data" refers to records of a user's activities and choices in their daily life, including travel routes, purchase history, and device usage history.

[0718] The system for realizing this invention collects personal information and provides user-optimized responses and services. The server has high-performance computing capabilities and analyzes user behavior data using data analysis models. Specifically, the server trains machine learning models using TensorFlow, PyTorch, etc., and generates analysis results. This enables responses that can appropriately address the individual needs of users.

[0719] Users interact with the system through information terminals such as smartphones and tablets. These terminals transmit collected personal information to the server and then present the user with responses and service suggestions received from the server.

[0720] Furthermore, robots, as mechanical devices, are operated to support users' lives. These robots connect to the cloud and provide in-home technical services based on instructions from servers. This includes health management, schedule management, and assistance with operating other household devices.

[0721] For example, if a user prompts the robot with "Choose clothes to match today's schedule," the server analyzes behavioral data and weather information and communicates appropriate clothing choices to the robot. Based on this information, the robot then makes clothing suggestions to the user.

[0722] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0723] Step 1:

[0724] Users input personal information and specific needs through information devices such as smartphones and tablets. The input information is transmitted to the server by the device. The input includes user behavior data and prompt messages.

[0725] Step 2:

[0726] The server stores the received personal information in a database using an encrypted protocol. The stored information is used for later data analysis.

[0727] Step 3:

[0728] The server uses machine learning models to analyze data based on stored user behavior data. Software such as TensorFlow and PyTorch are used to predict user patterns and needs based on the input data.

[0729] Step 4:

[0730] Based on user prompts, the server generates the optimal response based on the analysis results. The generation AI model creates individual responses tailored to the user's requests.

[0731] Step 5:

[0732] The generated response is sent from the server to the information terminal. The information terminal then presents the received response to the user. This allows the user to receive information tailored to their specific needs.

[0733] Step 6:

[0734] The server transmits control information to partnered external services and in-home robots as needed. This includes health management, scheduling, and assistance with operating home devices.

[0735] Step 7:

[0736] The robot performs specific actions for the user based on information provided by the server. For example, it might suggest appropriate clothing or adjust the schedule in response to the user's request.

[0737] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0738] This invention is a personal AI system incorporating an emotion engine that recognizes user emotions, enabling the provision of information and service suggestions tailored to the user's individual needs. The system consists of a server, a terminal, and the user as its main components, and achieves advanced personalization using the emotion engine.

[0739] Data collection and sentiment recognition

[0740] Users log in to the system via their device and consent to the collection of personal information. Based on this, the device collects the user's text messages, voice data, and other behavioral data. The device has a built-in emotion engine that can analyze the text and voice data collected from the user in real time and recognize the user's emotional state.

[0741] Data analysis and response generation

[0742] The server stores data received from the terminal and sentiment recognition results, and uses this to train a machine learning model. When a user enters a question or request for advice, the terminal sends it to the server. Based on the machine learning model and the sentiment engine's results, the server generates an optimal, personalized response that also takes into account the user's emotional state. This response may include encouraging messages or emotionally sensitive advice.

[0743] Specific example

[0744] For example, if a user inputs "I've been feeling very anxious lately" into the system, the emotion engine detects the user's anxiety, and the server, referencing past data and emotion recognition results, generates advice to reduce stress. Specifically, this advice may include suggestions such as deep breathing exercises or recommendations for calming music.

[0745] Selection and notification of partner services

[0746] Furthermore, the server utilizes the results of the emotion engine to select suitable partner services for the user and applies an advertising optimization algorithm. The selected service information is notified to the device, and suggestions are made based on the user's emotional state. This approach allows for the appropriate provision of services that are likely to interest the user.

[0747] Thus, the present invention aims to improve user satisfaction by combining an emotion engine and machine learning to enable more intimate and personalized services for users.

[0748] The following describes the processing flow.

[0749] Step 1:

[0750] Users log in to the system via their device and go through a process to grant permission for the collection of personal information and sentiment recognition. This allows them to begin using the system.

[0751] Step 2:

[0752] The device monitors text and voice input in real time according to the user's usage and prepares to collect this data. At the same time, the emotion engine starts up and prepares to analyze emotions from the input data.

[0753] Step 3:

[0754] The emotion engine analyzes user-inputted text and audio in real time and recognizes emotions from the content. For example, emotions are determined based on the tone of the text, the pitch of the voice, and other factors. The recognition results are immediately sent to the server.

[0755] Step 4:

[0756] The server securely stores the sentiment analysis results and behavioral data received from the terminals in an encrypted database. This allows for the accumulation of historical data for each user.

[0757] Step 5:

[0758] When a user enters a specific question or request into their device, the device sends it to the server. This request also includes the sentiment recognition results.

[0759] Step 6:

[0760] The server processes the received request and applies a machine learning model to create a personalized response best suited to the user's situation. This process also takes emotion recognition into account; for example, if the user is feeling anxious, it will generate a reassuring response.

[0761] Step 7:

[0762] The generated response is sent from the server to the terminal and then presented to the user. The user can then receive detailed advice and emotionally responsive support.

[0763] Step 8:

[0764] Based on emotion recognition results and user data, the server selects the partner services that are predicted to be of the user's greatest interest. An advertising optimization algorithm is used in this selection process.

[0765] Step 9:

[0766] Information about proposed partner services will be notified to the user via their device. This notification uses language that is considerate of the user's emotional state and is designed to be easily engaging.

[0767] (Example 2)

[0768] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0769] In modern society, users demand information and service suggestions tailored to their specific situations and emotional states. However, conventional systems have struggled to generate responses that take into account the user's emotional state, making personalized service delivery difficult. Furthermore, there is a growing need to effectively utilize text and audio data to provide user-friendly suggestions.

[0770] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0771] In this invention, the server includes means for recognizing the user's emotional state, means for training a machine learning model based on personal information and the user's emotional state, and means for notifying the user of selected partner services. This enables personalized responses and service provision in accordance with the user's emotions.

[0772] "Emotional state" refers to the psychological or emotional state of a user, as analyzed from their text and voice data.

[0773] "Personal information" refers to a collection of information that includes data such as text messages, voice data, and behavioral information about a user.

[0774] A "machine learning model" refers to a computational model that includes algorithms to learn from data and generate responses and suggestions that are appropriate for the user.

[0775] A "terminal" refers to a device such as a computer, smartphone, or tablet that a user uses to access a system.

[0776] "Partner services" refer to external related services or products offered to users that are selected based on the user's current emotional state and interests.

[0777] A "response" refers to a reply or action that is generated and provided in response to user input.

[0778] An "emotion engine" refers to a technology that analyzes a user's text or voice data to recognize their emotional state.

[0779] This invention is a personal AI system that combines an emotion engine for analyzing user emotions with a machine learning model. This system provides information and service suggestions tailored to the user's individual needs, achieving a high level of personalization. The main components of the system include a server, a terminal, and the user.

[0780] Users log in to the system using their devices and consent to the collection of personal information. The devices have an emotion engine built in that analyzes text messages and voice data collected from users in real time. By analyzing this data, the emotion engine can identify the user's emotional state.

[0781] The server stores data sent from the terminal and emotion recognition results, and uses this to train a machine learning model. When a user inputs a question or request for advice into the terminal, the terminal sends it to the server, which uses the machine learning model and the emotion engine's analysis results to generate the optimal response. This response may include encouragement or advice tailored to the user's emotional state.

[0782] As a concrete example, consider a scenario where a user enters "I've been feeling very anxious lately" as a prompt. In this case, the emotion engine detects the user's anxiety, and the server generates advice to alleviate the anxiety based on accumulated historical data and the emotion recognition results. This advice may include recommendations for deep breathing exercises or relaxation music.

[0783] Furthermore, the server selects partner services based on the results of the emotion engine and applies an advertising optimization algorithm. This allows it to select services that are appropriate to the user's emotional state and notify the user through their device. This approach makes it possible to provide services that appropriately attract the user's interest.

[0784] This system utilizes an emotion engine and machine learning to provide users with more personalized and intimate services, aiming to improve user satisfaction.

[0785] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0786] Step 1:

[0787] The user logs into the system via a terminal. During the login process, the user enters information to consent to the collection of personal information for sentiment analysis. Based on this input, the terminal begins collecting the user's text messages and voice data and storing them in a database.

[0788] Step 2:

[0789] The emotion engine built into the device analyzes collected text messages and voice data in real time. This data is used as input for emotion analysis, and natural language processing techniques are employed to generate emotion indicators. The output is an emotion tag indicating the user's emotional state, which is used in subsequent processing steps.

[0790] Step 3:

[0791] The device sends text and audio data, along with generated emotion tags, to the server. This is passed to the server as input, and the server generates output that stores the received data in the user profile.

[0792] Step 4:

[0793] The server uses accumulated data as input to train a machine learning model. Specifically, it updates the model's weights using past user data and current sentiment data. The output is an improved model that enables sophisticated response generation.

[0794] Step 5:

[0795] The user enters a question or request for advice into the terminal. This input is sent to the server, which generates a response based on sentiment tags and a machine learning model. The output is a personalized response that includes encouragement and advice tailored to the user's emotional state.

[0796] Step 6:

[0797] The server sends the generated individual response to the terminal, which then notifies the user. The user receives this response and can try out the available actions or relaxation techniques.

[0798] Step 7:

[0799] The server selects partner services based on the results of the emotion engine and applies an advertising optimization algorithm. This process takes input to select services that take the user's emotional state into consideration. The output is the service information best suited to the user and is provided to the user through their terminal.

[0800] (Application Example 2)

[0801] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0802] In recent years, smart systems aimed at improving convenience within the home have become increasingly popular. However, conventional systems have been unable to adjust their operation to take into account the user's emotional state, making them insufficient in providing the optimal support the user needs. In particular, there is a need for technology that can recognize the user's emotions in real time and dynamically control the operation of home appliances based on this.

[0803] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0804] In this invention, the server includes means for acquiring individual data, means for training a learning model, and means for acquiring user voice data and text messages and recognizing emotions. This enables optimal control of home appliances in accordance with the user's emotions.

[0805] "Individual data" refers to unique information related to a user, such as behavioral information and preference information, or other information about a specific user.

[0806] A "learning model" is an algorithm that enables prediction and classification based on data, and is particularly used in machine learning.

[0807] A "response" refers to the information or instructions generated in response to a user inquiry, providing results that meet the user's requirements.

[0808] "Partner services" refer to services provided by external businesses or systems to users by the system.

[0809] "Voice data" refers to data that records the voice spoken by a user as digital information, and is used for speech recognition and sentiment analysis.

[0810] A "text message" refers to a message sent by a user as a string of characters, and is data that is subject to sentiment recognition and information analysis.

[0811] "Emotions" refer to information that indicates the user's mental state, and the psychological state inferred from voice and text.

[0812] "Household appliances" refer to various devices and equipment installed in homes that operate on electricity, including air conditioners, lighting, and entertainment equipment.

[0813] This invention is a system for controlling home appliances based on user emotion recognition. This system acquires user voice data and text messages and uses that data to analyze emotions in real time. The terminal is connected to various home appliances installed in the home and instructs those appliances to perform actions according to the user's emotional state.

[0814] The server first acquires individual data, including user voice data and text messages. Next, it trains a learning model based on the acquired data to recognize the user's emotions. After the emotions are recognized, the results are used to control the operation of household appliances. These appliances include lighting, air conditioners, and music players, and the optimal operating state is instructed based on the user's specific emotional state.

[0815] For example, if a user says "I'm tired today," the device will pick up the audio, and the emotion recognition module will determine that the user is "tired." Based on this determination, the lighting will be adjusted to a softer brightness, and relaxing music will be played to improve the user's comfort. By applying a generative AI model, a prompt such as "When a user is feeling stressed, what advice should be displayed to help them relax?" can be used.

[0816] In this way, we can realize a system that provides meticulous service tailored to the user's emotions.

[0817] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0818] Step 1:

[0819] The server retrieves user voice data and text messages from the terminal. The input is raw voice data and text messages, which are converted into a digital format for processing by the emotion recognition module. Analyzing the converted data prepares it for the next step.

[0820] Step 2:

[0821] The server uses a learning model to perform emotion recognition based on the acquired data. The input digital data is passed through the emotion recognition algorithm to determine the user's emotional state. The output is an emotional state label, such as "fatigue" or "anxiety."

[0822] Step 3:

[0823] The server generates control instructions for household appliances based on the results of emotion recognition. The input is an emotion label determined by emotion recognition, and this, along with a pre-configured set of rules, is used to determine which appliances to operate and how. For example, if "fatigue" is detected, the server will generate instructions to dim the lights and play relaxing music.

[0824] Step 4:

[0825] The terminal receives control instructions from the server and executes specific actions on household appliances. The input is the control instructions from the server, and the output is the actual physical operation of the appliance. Actions such as adjusting lighting or playing music are taken to enhance user comfort.

[0826] Step 5:

[0827] Users experience the environmental changes adjusted by the system and send feedback to their terminals as needed. The input is the user's subjective impression of the system changes, and the output is used for subsequent processing, contributing to further improvements in the system's accuracy.

[0828] This series of processes enables nuanced control of home appliances based on user emotions.

[0829] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0830] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0831] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0832] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0833] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0834] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0835] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0836] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0837] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0838] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0839] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0840] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0841] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0842] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0843] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0844] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0845] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0846] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0847] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0848] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0849] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0850] The following is further disclosed regarding the embodiments described above.

[0851] (Claim 1)

[0852] Means of acquiring personal information,

[0853] A means for training a machine learning model based on the aforementioned personal information,

[0854] A means for generating individual responses using the machine learning model based on user inquiries,

[0855] Means for providing the aforementioned individual responses to the user,

[0856] Methods for selecting partner services,

[0857] A means for notifying the user of the aforementioned partner service,

[0858] A system that includes this.

[0859] (Claim 2)

[0860] The system according to claim 1, characterized in that the personal information includes user behavior data.

[0861] (Claim 3)

[0862] The system according to claim 1, characterized in that the machine learning model is retrained periodically.

[0863] "Example 1"

[0864] (Claim 1)

[0865] A device for acquiring personal information,

[0866] Based on the aforementioned personal information, an apparatus for training an information processing model,

[0867] A device that generates individual responses using the information processing model based on user inquiries,

[0868] A device that provides the user with the aforementioned individual responses,

[0869] A device for selecting third-party services,

[0870] A device that notifies the user of the aforementioned third-party service,

[0871] A device for encrypting the aforementioned data,

[0872] A device for clustering the user's behavioral data,

[0873] A system that includes this.

[0874] (Claim 2)

[0875] The system according to claim 1, characterized in that the personal information includes user activity data.

[0876] (Claim 3)

[0877] The system according to claim 1, characterized in that the information processing model is periodically retrained.

[0878] "Application Example 1"

[0879] (Claim 1)

[0880] Means of acquiring personal information,

[0881] A means for training a data analysis model based on the aforementioned personal information,

[0882] A means for generating individual responses using the data analysis model based on user inquiries,

[0883] Means for providing the aforementioned individual responses to the user via an information terminal,

[0884] Methods for selecting external services to partner with,

[0885] Means for notifying the information terminal of the aforementioned external service,

[0886] A means for controlling a mechanical device that provides individualized services to assist the user in their daily life,

[0887] A system that includes this.

[0888] (Claim 2)

[0889] The system according to claim 1, characterized in that the aforementioned personal information includes user behavior data.

[0890] (Claim 3)

[0891] The system according to claim 1, characterized in that the data analysis model is periodically retrained.

[0892] "Example 2 of combining an emotion engine"

[0893] (Claim 1)

[0894] A means of recognizing the user's emotional state,

[0895] Means of acquiring personal information,

[0896] A means for training a machine learning model based on the aforementioned personal information and the user's emotional state,

[0897] A means of receiving inquiries from users via a terminal,

[0898] Means for generating individual responses using the output of the machine learning model and emotion engine,

[0899] Means for providing the aforementioned individual responses to the user,

[0900] Methods for selecting partner services,

[0901] A means of notifying the user of the selected partner service,

[0902] A system that includes this.

[0903] (Claim 2)

[0904] The system according to claim 1, characterized in that the personal information includes user behavior information and voice information.

[0905] (Claim 3)

[0906] The system according to claim 1, characterized in that the machine learning model is periodically retrained and updated based on past data and sentiment recognition results.

[0907] "Application example 2 when combining with an emotional engine"

[0908] (Claim 1)

[0909] Means of obtaining individual data,

[0910] A means for training a learning model based on the aforementioned individual data,

[0911] A means for generating individual responses using the learning model based on user inquiries,

[0912] Means for providing the aforementioned individual responses to the user,

[0913] Methods for selecting partner services,

[0914] A means for notifying the user of the aforementioned partner service,

[0915] A means of acquiring user voice data and text messages and recognizing emotions,

[0916] Based on the aforementioned emotions, a means for controlling household appliances,

[0917] A system that includes this.

[0918] (Claim 2)

[0919] The system according to claim 1, characterized in that the individual data includes user behavior information.

[0920] (Claim 3)

[0921] The system according to claim 1, characterized in that the learning model is periodically retrained. [Explanation of Symbols]

[0922] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of acquiring personal information, A means for training a machine learning model based on the aforementioned personal information, A means for generating individual responses using the machine learning model based on user inquiries, Means for providing the aforementioned individual responses to the user, Methods for selecting partner services, A means for notifying the user of the aforementioned partner service, A system that includes this.

2. The system according to claim 1, characterized in that the personal information includes user behavior data.

3. The system according to claim 1, characterized in that the machine learning model is retrained periodically.

Images