system

Abstract

We provide the system. [Solution] A means of recording the user's normal lifestyle data, Means for providing findings generated based on the aforementioned lifestyle data, A means of providing the aforementioned findings through an information function that presents them in an accessible manner, A means for analyzing the aforementioned lifestyle data, evaluating health status using artificial intelligence, and generating an individualized improvement plan, A means for monitoring a user's daily activities in real time using a voice recognition device and an image recognition device, and for transmitting the collected data to a server, A means of awarding a certain number of reward points based on the user's achievement status and reporting this to the user through the aforementioned information function, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern times, it is not easy for users to continuously maintain daily health management such as diet, exercise, and sleep without neglecting it. In particular, there are major challenges in selecting a nutritionally balanced diet and maintaining motivation. Regarding these issues, existing health management methods lack freshness and continuity, and there is a problem that many users give up halfway.

Means for Solving the Problems

[0005] This invention provides a system for managing health data with a user-friendly character. Specifically, it includes means for recording the user's activity data, such as data on diet, exercise, and sleep, and means for providing personalized advice based on that data using a generated AI. Furthermore, by displaying the advice in a friendly manner through the character and, in some cases, providing rewards to reinforce behavior, the system can improve the user's motivation. In this way, users can manage their health continuously and enjoyably.

[0006] "Activity data" refers to various types of information obtained from the user's daily life, including their diet, exercise, and sleep.

[0007] "Generative AI" is an artificial intelligence system that automatically analyzes input data and generates advice and information.

[0008] "Advice" refers to suggestions and recommendations provided to users regarding their activities through generative AI.

[0009] A "character" is a personified interface designed to convey information to users in an approachable way.

[0010] "Rewards" are incentives provided to encourage specific behaviors in users and to increase their motivation. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the labeled communication I / F (Interface) is an interface that includes a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] This invention describes a specific embodiment of a system that assists in health management through interaction between a user and a character. The system aims to promote healthy lifestyle habits by collecting and analyzing user activity data.

[0033] The system mainly consists of terminals that handle data, servers that analyze data, and characters that interact with users.

[0034] Data collection and input

[0035] Users input information about their daily diet, exercise, and sleep through a terminal. The terminal receives the user's input, formats it, and sends it to the server. This information includes details about meals, type and duration of exercise, and sleep quality.

[0036] Data analysis and advisory generation

[0037] The server records the received activity data in a database and performs analysis using a generating AI. Based on the analysis results, it generates advice to improve the user's health. This advice is based on the user's past data and lifestyle patterns.

[0038] Advice provided by characters

[0039] The generated advice is displayed on the device through a character. On the device screen, a user-friendly character delivers the generated advice with humor and encouraging words. In this way, users can manage their health in an enjoyable way.

[0040] Maintaining Motivation

[0041] Furthermore, the system includes a reward function to maintain and improve user motivation. The server awards points based on increased activity levels and achieved goals, and notifies the user through a character reporting this via the terminal. For example, users who achieve a certain amount of exercise will receive encouraging words from the character such as, "Great! Let's work hard together towards your next goal!"

[0042] In this way, the present invention makes it possible to make users' health management fun and sustainable through characters, and to contribute to extending healthy life expectancy.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The user launches the app and accesses an interface for entering data on meals, exercise, and sleep. The device displays this input screen, allowing the user to enter the necessary information.

[0046] Step 2:

[0047] After the user completes the input and confirms the content, they press the submit button. The terminal converts the entered data into a predetermined format (e.g., JSON) and sends it to the server via the internet.

[0048] Step 3:

[0049] The server receives data sent from the terminal and stores it in the database. This data is recorded as an individual user's activity history and is ready for later analysis.

[0050] Step 4:

[0051] The server uses AI to analyze activity data stored in the database. This analysis compares the user's past patterns with current data to generate appropriate health management advice for the user.

[0052] Step 5:

[0053] The server sends the generated advice to the terminal. This prepares the terminal to display a character that will present the advice to the user in a user-friendly way.

[0054] Step 6:

[0055] The device displays character graphics and generated advice on the screen. The character offers specific action suggestions and words of encouragement to the user, providing a trigger for them to take the next step.

[0056] Step 7:

[0057] After receiving advice from the character, users adjust and improve their lifestyle habits based on that advice. Furthermore, users can set their next activity goals by receiving feedback from the app.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] In modern times, many people want to manage their health in a more enjoyable and effective way in their daily lives. However, conventional health management systems have problems with maintaining user motivation and sustained behavior, resulting in monotonous and difficult-to-continue systems. This invention aims to provide a system that allows users to manage their health in an enjoyable and sustainable way.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes a device for recording user activity information, a device for providing suggestions generated using a generative AI model, a device for presenting suggestions through a character that displays them in a friendly manner, and a device for calculating and notifying points to reinforce user behavior. This enables users to enjoy daily health management and make sustainable improvements to their lives.

[0063] "User activity information" refers to data about the user's daily activities, such as eating, exercising, and sleeping.

[0064] A "recording device" is hardware or software used to collect user activity information and store it as data.

[0065] A "generative AI model" is an artificial intelligence algorithm that analyzes user activity information and generates personalized suggestions.

[0066] A "proposal-providing device" refers to a device or software that presents suggestions generated by a generative AI model to a user.

[0067] "Approachable language" refers to a method of conveying information in a format that is easy for users to understand and that easily captures their interest.

[0068] A "device that uses characters" is a device that displays characters using on-screen animation or speech synthesis technology in order to interact with the user.

[0069] A "device that calculates and notifies points to reinforce behavior" is a system or application that calculates points based on the user's activity record and notifies the user of the results.

[0070] This invention is a system that enables users to manage their health in an enjoyable and sustainable way. The system primarily uses terminals, servers, and characters to manage and analyze user activity information and provide feedback in a user-friendly format.

[0071] The device collects activity information entered by the user. Digital devices and corresponding applications are used, allowing users to manually or through sensor technology input information about their daily diet, exercise, and sleep. For example, steps taken and calories burned can be recorded through a smartphone application. The device organizes this input information and transmits it to a server using a secure communication protocol.

[0072] The server records activity information received from the terminal in a database. The received data is analyzed using a generative AI model. Based on prompts, the generative AI model analyzes the user's health status and generates suggestions for improvement. The model processes prompts such as, "Generate suggestions for health promotion based on the user's exercise data from the past week." The server also calculates points based on the user's activity and prepares rewards to improve motivation.

[0073] The device receives suggestions and reward information generated from the server and displays it on the screen. A friendly character plays a crucial role in this display. This character communicates suggestions to the user in an approachable way through voice and text. For example, the character can encourage the user by saying, "You did a great job today!" and then present the next goal in a fun way. The feedback provided by the character helps maintain the user's motivation and is expected to help them establish healthy lifestyle habits.

[0074] With the above configuration, this invention can provide users with a health management method that combines enjoyment and sustainability in their daily lives.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] Users input daily health-related information using a device. This input data includes details of meals, calorie intake, type and duration of exercise, and sleep duration and quality. The device organizes this information and formats it as digital data. Specifically, it collects data via text input or a touch interface and converts it into a digital format such as JSON.

[0078] Step 2:

[0079] The terminal sends the formatted data to the server via a secure communication channel. The input information is transmitted using an encryption protocol (e.g., HTTPS), ensuring data security. The output from the terminal is an encrypted data stream, which the server receives.

[0080] Step 3:

[0081] The server records the received data in a database. The database is designed to store data chronologically. The input here is a data stream from the terminal, which the server converts into a parseable format and records.

[0082] Step 4:

[0083] The server analyzes the data using a generative AI model. This analysis is performed according to the prompt "Generate health promotion suggestions based on the user's exercise data from the past week." The output of the analysis is specific suggestions and advice tailored to the user's health condition.

[0084] Step 5:

[0085] The server sends the generated suggestions to the terminal. Furthermore, it calculates points based on activity data and prepares reward information to improve motivation. The data sent includes the generated advice and the number of points.

[0086] Step 6:

[0087] The device displays information received from the server to the user. During this process, a character appears on the screen and provides advice and reward information to the user in an engaging format. Specifically, it provides feedback to the user through animation and audio playback.

[0088] Step 7:

[0089] Users review the advice and rewards presented through their device and use them to guide their next health behaviors. They then implement new healthy habits based on the advice and prepare for the next data entry. This restarts the system cycle, enabling continuous health management.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] In modern society, while health management is becoming increasingly important, many people find it difficult to adequately manage their health amidst their busy lifestyles. In particular, there is a lack of systems that provide appropriate advice tailored to individual lifestyles. Furthermore, there is a need for systems that continuously provide such advice and that allow users to continue managing their health in an enjoyable way.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes means for recording the user's normal lifestyle data, means for analyzing the lifestyle data, evaluating the user's health status using artificial intelligence, and generating a personalized improvement plan, and means for monitoring the user's lifestyle behavior in real time using a voice recognition device and an image recognition device, and transmitting the collected data to the server. This allows the user to easily understand their own health status and receive appropriate advice. Furthermore, by providing advice through user-friendly information functions, the user can manage their health in an enjoyable way.

[0095] The term "user" refers to an individual who uses the system to manage their health.

[0096] "Normal lifestyle data" refers to data that includes information about the user's daily life, such as diet, exercise, and sleep.

[0097] "Findings" refer to advice and suggestions provided based on analyzed data to improve the user's health.

[0098] An "information function" is a mechanism that presents advice and information to users visually or audibly using friendly characters and display methods.

[0099] "Artificial intelligence" refers to computer programs and algorithms used to analyze a user's lifestyle data and generate personalized health advice.

[0100] A "speech recognition device" is a device that detects a user's speech, digitizes its content, and converts it into a format that the system can understand.

[0101] An "image recognition device" is a device that acquires a user's image information, analyzes it, and converts it into digital data.

[0102] "Reward points" are incentives awarded to users when they achieve their health management goals, and they serve to encourage user behavior.

[0103] This invention is designed as a system that comprehensively supports users' daily lives and promotes health management. The system aims to collect users' lifestyle data in real time and provide appropriate health advice based on that data.

[0104] The server is the core of the health management system. First, the server receives normal lifestyle data collected from the user's terminal. This data includes details of meals, exercise levels, sleep duration, etc. The data is stored in a cloud-based database and analyzed using a generative AI model. This analysis evaluates the user's health status and generates personalized recommendations for lifestyle improvements.

[0105] The terminal functions as an interface between the user and the server and is equipped with speech recognition and image recognition devices. For example, by using common speech recognition software or cloud-based image recognition services, it is possible to input the contents of a meal using voice or images.

[0106] Users are naturally motivated because they are awarded reward points based on their progress towards achieving their daily health goals. This information is provided through user-friendly features, allowing users to enjoy practicing a healthy lifestyle.

[0107] For example, if you speak into the device's microphone after breakfast, saying "I ate yogurt and fruit this morning," the speech recognition will digitize the content and send it to the server. The server will analyze the data and generate a suggestion such as "Try to eat a meal rich in vitamin C today," which will then be displayed on the device's screen through its information function.

[0108] An example of a prompt message might be, "Please evaluate the nutritional balance of the day based on the user's dietary data." In this way, specific and effective health advice based on data can be obtained.

[0109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0110] Step 1:

[0111] The device acquires information about the user's diet, exercise, and sleep via voice recognition and image recognition. For example, if the user says, "Today's breakfast is eggs and toast," the voice recognition software converts that information into text data. The input is voice or image, and the output is lifestyle data in text format.

[0112] Step 2:

[0113] The terminal formats the converted lifestyle data and sends it to the server. Specifically, it is converted into a specific key-value format, such as "breakfast," "eggs," and "toast." The input is the text data obtained in the previous step, and the output is the formatted data.

[0114] Step 3:

[0115] The server stores received normal life data in a cloud-based database. The data is permanently stored and used for later analysis. The input is formatted data, and the output is the stored data.

[0116] Step 4:

[0117] The server retrieves data from the database and analyzes it using a generative AI model. The analysis evaluates the user's current lifestyle and health status to identify areas for improvement. The input is lifestyle data retrieved from the database, and the output is the analysis results.

[0118] Step 5:

[0119] The server generates personalized findings based on the analysis results. These findings are actionable for the user and provide concrete advice to help improve their life. The input is the analysis results, and the output is the advice, including the findings.

[0120] Step 6:

[0121] The server sends the generated findings to the terminal. The findings are presented to the user through user-friendly information functions. The input is the generated advice, and the output is user-oriented information displayed on the terminal.

[0122] Step 7:

[0123] The user reviews the presented findings and chooses whether or not to perform a health behavior. Based on this choice, the reward function, which is the next step, is activated. The input is the information displayed on the device, and the output is the user's behavioral choice.

[0124] Step 8:

[0125] The server calculates reward points based on the user's action choices and notifies the terminal. For example, if the user achieves their daily goal, it displays the message, "You earned 100 points today!" The input is the user's action choices, and the output is reward information.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] This invention aims to achieve more personalized interactions by incorporating an emotion engine into a system that supports user activity management. This system collects user activity data and provides appropriate advice, while simultaneously using the emotion engine to recognize the user's emotions and respond accordingly.

[0128] The system mainly consists of terminals, servers, emotion engines, and characters.

[0129] Input and recording of activity data

[0130] Users input data on their daily activities, such as information about meals, exercise, and sleep, via a terminal. The terminal receives, formats, and sends the input to a server. The server records and stores this data in a database for later analysis and advice generation.

[0131] Emotion recognition and analysis

[0132] The device is equipped with an emotion engine that recognizes the user's emotions from their facial expressions, tone of voice, and entered text. This information is sent to a server and analyzed along with the user's activity data. The server's generative AI is used to generate appropriate advice based on these emotions.

[0133] Advice provided by characters

[0134] The generated advice is sent from the server to the terminal. The terminal adjusts the character's expression based on the user's emotions recognized by the emotion engine, providing advice in an appropriate tone and content. For example, if the user is feeling down, the advice will emphasize words of encouragement and be displayed with a cheerful character.

[0135] Motivation enhancement and reward function

[0136] The system has a function that provides rewards based on the user's activity level. The server evaluates the user's emotions and activity data, generates rewards such as points and badges, and sends them to the terminal. The character enjoys and reports these rewards, actively reinforcing the user's behavior.

[0137] For example, if a user engages in active participation and their joyful expression is recognized, the character will offer an additional reward along with a congratulatory message such as, "That's great! Keep it up!" This encourages users to continue participating.

[0138] As described above, the present invention utilizes an emotion engine to support users' lifestyles in a more personalized way, making it possible to continue health management in an enjoyable manner.

[0139] The following describes the processing flow.

[0140] Step 1:

[0141] The user starts up the device and opens the interface for entering daily activity data. They prepare to enter daily information such as meals, exercise, and sleep.

[0142] Step 2:

[0143] The terminal displays an input form, and the user enters detailed activity data (e.g., diet, type of exercise). Once the user has finished entering the data, the terminal formats it into a predetermined format.

[0144] Step 3:

[0145] The formatted data is sent to a server via the internet. The server stores the received data in a database, making it available for subsequent analysis.

[0146] Step 4:

[0147] The device's built-in emotion engine analyzes the user's facial expressions and voice to detect their current emotional state (e.g., joy, sadness). This information is immediately sent to the server.

[0148] Step 5:

[0149] The server integrates activity data and emotional data, and uses generative AI for analysis. It then generates optimal health advice tailored to the user's emotions.

[0150] Step 6:

[0151] The generated advice is sent from the server to the terminal. The terminal adjusts the character's facial expressions and messages based on the received information.

[0152] Step 7:

[0153] The device presents the user with a customized character and advice. For example, if the emotion engine determines that the user is feeling down, the character will offer words of encouragement with a gentle expression.

[0154] Step 8:

[0155] The user reviews the advice provided and incorporates it into their actions. The device records the user's response to help improve future advice.

[0156] Step 9:

[0157] The server provides incentives (e.g., points, badges) through a reward system based on the user's actions and emotions. Users receive these incentives and maintain their motivation.

[0158] (Example 2)

[0159] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0160] In managing users' health and improving their lifestyles, conventional systems struggle to provide advice tailored to individual emotional states and maintain motivation, making it challenging to continuously motivate users. Furthermore, there is a need for a system that can provide advice that considers not only activity data but also emotions.

[0161] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0162] In this invention, the server includes means for receiving user activity information from an input device, formatting it, and storing it on a recording medium; means for analyzing the user's emotions using an emotion recognition device and generating advice using a generative AI model based on that information; and means for displaying the generated advice in an expression method appropriate to the user's emotions and providing it through a friendly character. This enables users to continue working on health management and improving their lifestyle habits by using advice that takes into account their individual emotional state and an appropriate reward system.

[0163] "Activity information" refers to information about the user's actions and behaviors in their daily life, including data on meals, exercise, sleep, etc.

[0164] An "input device" refers to a device used by users to record activity information, such as a smartphone or personal computer.

[0165] A "recording medium" is a device or medium used to store received information, and includes databases and storage systems.

[0166] An "emotion recognition device" is a hardware and software system that analyzes emotions from a user's facial expressions, voice tone, and text data.

[0167] A "generative AI model" is an artificial intelligence model that generates advice in natural language based on received data.

[0168] A "character" is a representation of a virtual person or animal that provides information to the user visually or aurally, and is intended to present advice or rewards in an approachable way.

[0169] "Rewards" refer to incentives such as points or badges provided based on the evaluation of users' activities, and are means of reinforcing user behavior.

[0170] The system of this invention assists in managing user activity and provides feedback based on individual emotional states. The system mainly consists of a terminal, a server, an emotion recognition device, a generative AI model, and a friendly character.

[0171] Users input daily activity information using a terminal. The terminal is equipped with an input device, which is used to input activity information such as meals, exercise, and sleep. The input information is formatted by the terminal and sent to the server for storage on a recording medium. The server stores this data and uses it for later analysis.

[0172] The device is equipped with an emotion recognition device that analyzes the user's emotions from their facial expressions, voice tone, and entered text data. This analysis uses hardware such as a camera and microphone, as well as emotion analysis software. The analyzed emotion data is sent to a server along with activity information, where a generative AI model is used.

[0173] The server uses a generative AI model to generate personalized advice based on the received activity and emotion data. This advice is generated using prompts that correspond to the user's emotional state. For example, a prompt might be "Generate the most appropriate encouraging message based on the user's emotions."

[0174] The generated advice is sent from the server to the terminal, which then displays the advice using a character. The character uses a bright and friendly expression, providing advice that adjusts in tone and content according to the user's emotions. For example, if the user is feeling down, the character will offer encouraging words.

[0175] Furthermore, the server evaluates the user's activity and emotions, generating rewards such as points and badges. These rewards are sent to the device, and a character reports them to the user in an entertaining way. This helps users continue their actions and maintain motivation for health management and lifestyle improvements.

[0176] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0177] Step 1:

[0178] Users operate the terminal to input information about their daily activities. This information includes details about meals, exercise time, and sleep quality. This activity information is received by an input device on the terminal and converted into digital data. The input data is formatted in JSON format and prepared to be sent to the server.

[0179] Step 2:

[0180] The terminal sends formatted activity information to the server. The server receives the data sent via HTTP request and stores it in a database. This stored data is later parsed using SQL queries. In this case, the input is the activity information, and the output is the stored data.

[0181] Step 3:

[0182] The device analyzes the user's emotions using an emotion recognition device. This analysis collects the user's facial expressions and voice using a camera and microphone. This data is analyzed by emotion recognition software and converted into digital data representing the user's emotional state. The resulting emotional data is then formatted and ready to be sent to the server.

[0183] Step 4:

[0184] The device sends formatted emotion data to the server. The server stores the received emotion data in a database. The stored activity information and emotion data are then used by a generative AI model to generate advice. In this process, the input is the emotion data, and the output is the state in which that data is stored in the database.

[0185] Step 5:

[0186] The server generates advice using a generative AI model. This advice generation process uses the prompt "Generate the most appropriate encouraging message based on the user's emotions." The generative AI model receives stored activity information and emotion data as input and generates advice tailored to the user's emotions. The output is the generated advice message.

[0187] Step 6:

[0188] The server sends a generated advice message to the terminal. The terminal displays the received message to the user through a friendly character. The character delivers the advice in a tone that matches the user's emotions. The input is the advice message, and the output is the display of that message.

[0189] Step 7:

[0190] The server evaluates user activity and emotional data and generates rewards. This process generates reward data such as points and badges and sends it to the device. The generated reward data includes incentives that are tailored to the user's level of achievement and emotional state.

[0191] Step 8:

[0192] The device presents the user with reward data generated using a character. The character introduces the rewards in an enjoyable way, encouraging the user to continue their engagement. The input is the reward data, and the output is the visual presentation of that data to the user.

[0193] (Application Example 2)

[0194] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0195] Conventional activity management systems can only provide users with fixed advice, making it difficult to provide appropriate guidance that takes into account individual emotional states. Furthermore, there is a need to achieve more sophisticated interactions in providing advice in a friendly manner and in reward functions that enhance user motivation.

[0196] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0197] In this invention, the server includes a device for recording user behavior data, a device for providing instruction generated based on the behavior data, a device for delivering the instruction through a virtual character that presents the instruction in a friendly manner, and a device for recognizing the user's emotions using an emotion engine and adjusting the instruction content accordingly. This enables flexible and friendly instruction based on the individual emotions of each user.

[0198] "User behavioral data" refers to information about the activities that users engage in in their daily lives, and this includes information such as exercise, diet, and sleep.

[0199] A "device that provides guidance" is a device that has the function of generating and presenting appropriate guidance and advice to the user based on recorded behavioral data.

[0200] A "virtual character" is a visual entity with a personality generated digitally, and its role is to provide information and guidance through interaction with users.

[0201] An "emotion engine" refers to a technology or system that analyzes a user's facial expressions, voice, text input, etc., to recognize the user's emotions.

[0202] A "reward-providing device" is a device that generates and presents points or badges based on the user's behavioral data and emotional state, with the aim of improving the user's motivation.

[0203] This invention realizes a system for collecting and analyzing user activity data and emotions to provide personalized guidance. The system mainly consists of a terminal, a server, an emotion engine, and a virtual character.

[0204] The terminal collects and records data about the user's daily activities, such as information on diet, exercise, and sleep. Users can input this information using mobile devices such as smartphones and tablets. The terminal is responsible for formatting the input information and sending it to the server.

[0205] The server receives activity data provided by the user and stores it in a database. Furthermore, the server is equipped with a generative AI model that analyzes the accumulated behavioral data and emotional information to generate appropriate guidance for the user. In this analysis process, speech synthesis APIs such as Amazon Polly and image recognition APIs such as AWS® Rekognition are used to perform emotion recognition from voice input and image data.

[0206] The emotion engine is integrated into the device and instantly identifies emotions from the user's tone of voice, facial expressions, and input text. This information is sent to a server and used to provide specific guidance. The emotion engine helps understand the user's mental state, enabling more effective communication.

[0207] The virtual character receives instructional content from the server and presents it using expressions based on the user's emotions, as recognized by the emotion engine. The character plays a role in conveying information in a visually appealing and easy-to-understand manner for the user.

[0208] Furthermore, the reward system generates and displays points and badges on the device to reinforce user behavior. This allows users to be further motivated by the visual recognition of their efforts.

[0209] For example, if a user sends a message to the robot saying, "I feel great after my workout today," the emotion engine recognizes the user's "joy," and the server generates a response saying, "That's wonderful! Keep it up!" which is then conveyed to the user through a virtual character in an enthusiastic manner. This encourages users to take a more positive stance towards maintaining a healthy lifestyle.

[0210] An example of a prompt to input into the generation AI model is, "Based on my activity data, please generate an appropriate response when the user is happy."

[0211] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0212] Step 1:

[0213] Users input data on their daily activities using devices such as smartphones and tablets. This input includes information such as exercise levels, diet, and sleep duration. The device receives this data, formats it, and sends it to the server. The input data is primarily text-based and is appropriately structured on the device.

[0214] Step 2:

[0215] The server receives activity data sent from terminals and records it in the database. This activity data is used for later analysis and instruction generation, so the server stores it efficiently. Furthermore, data formatting processes remove unnecessary data and correct input errors, creating accurate database entries.

[0216] Step 3:

[0217] The device analyzes the user's voice tone and facial expressions via an emotion engine and generates emotion data. For example, when the user shows their facial expressions using the camera or speaks through the microphone, the device executes an emotion recognition algorithm and sends the results to a server. The input data is then classified into categories such as "joy" or "sadness" by an emotion analysis model within the program.

[0218] Step 4:

[0219] The server uses a generative AI model to generate appropriate guidance based on recorded activity and emotion data. The AI ​​model leverages a large amount of historical data and generates personalized guidance according to defined prompts (e.g., "Generate an appropriate response when the user is happy"). This process utilizes a machine learning model, and optimal guidance is generated through data computation.

[0220] Step 5:

[0221] The server sends the generated instructions to the terminal. The terminal, in accordance with the user's emotions recognized by the emotion engine, provides the instructions visually and audibly through a virtual character. Here, the character displays feedback through animation and sound, providing information in a way that is easy for the user to understand and relate to.

[0222] Step 6:

[0223] The device rewards users with points or badges using a reward system when their activity meets certain criteria. The server evaluates the user's activity and emotional data, generates rewards based on this evaluation, and presents them to the user through the device. This actively reinforces the user's behavior and promotes sustainable motivation.

[0224] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0225] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0226] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0227] [Second Embodiment]

[0228] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0229] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0230] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0231] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0232] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0233] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0234] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0235] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0236] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0237] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0238] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0239] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0240] This invention describes a specific embodiment of a system that assists in health management through interaction between a user and a character. The system aims to promote healthy lifestyle habits by collecting and analyzing user activity data.

[0241] The system mainly consists of terminals that handle data, servers that analyze data, and characters that interact with users.

[0242] Data collection and input

[0243] Users input information about their daily diet, exercise, and sleep through a terminal. The terminal receives the user's input, formats it, and sends it to the server. This information includes details about meals, type and duration of exercise, and sleep quality.

[0244] Data analysis and advisory generation

[0245] The server records the received activity data in a database and performs analysis using a generating AI. Based on the analysis results, it generates advice to improve the user's health. This advice is based on the user's past data and lifestyle patterns.

[0246] Advice provided by characters

[0247] The generated advice is displayed on the device through a character. On the device screen, a user-friendly character delivers the generated advice with humor and encouraging words. In this way, users can manage their health in an enjoyable way.

[0248] Maintaining Motivation

[0249] Furthermore, the system includes a reward function to maintain and improve user motivation. The server awards points based on increased activity levels and achieved goals, and notifies the user through a character reporting this via the terminal. For example, users who achieve a certain amount of exercise will receive encouraging words from the character such as, "Great! Let's work hard together towards your next goal!"

[0250] In this way, the present invention makes it possible to make users' health management fun and sustainable through characters, and to contribute to extending healthy life expectancy.

[0251] The following describes the processing flow.

[0252] Step 1:

[0253] The user launches the app and accesses an interface for entering data on meals, exercise, and sleep. The device displays this input screen, allowing the user to enter the necessary information.

[0254] Step 2:

[0255] After the user completes the input and confirms the content, they press the submit button. The terminal converts the entered data into a predetermined format (e.g., JSON) and sends it to the server via the internet.

[0256] Step 3:

[0257] The server receives data sent from the terminal and stores it in the database. This data is recorded as an individual user's activity history and is ready for later analysis.

[0258] Step 4:

[0259] The server uses AI to analyze activity data stored in the database. This analysis compares the user's past patterns with current data to generate appropriate health management advice for the user.

[0260] Step 5:

[0261] The server sends the generated advice to the terminal. This prepares the terminal to display a character that will present the advice to the user in a user-friendly way.

[0262] Step 6:

[0263] The device displays character graphics and generated advice on the screen. The character offers specific action suggestions and words of encouragement to the user, providing a trigger for them to take the next step.

[0264] Step 7:

[0265] After receiving advice from the character, users adjust and improve their lifestyle habits based on that advice. Furthermore, users can set their next activity goals by receiving feedback from the app.

[0266] (Example 1)

[0267] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0268] In modern times, many people want to manage their health in a more enjoyable and effective way in their daily lives. However, conventional health management systems have problems with maintaining user motivation and sustained behavior, resulting in monotonous and difficult-to-continue systems. This invention aims to provide a system that allows users to manage their health in an enjoyable and sustainable way.

[0269] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0270] In this invention, the server includes a device for recording user activity information, a device for providing suggestions generated using a generative AI model, a device for presenting suggestions through a character that displays them in a friendly manner, and a device for calculating and notifying points to reinforce user behavior. This enables users to enjoy daily health management and make sustainable improvements to their lives.

[0271] "User activity information" refers to data about the user's daily activities, such as eating, exercising, and sleeping.

[0272] A "recording device" is hardware or software used to collect user activity information and store it as data.

[0273] A "generative AI model" is an artificial intelligence algorithm that analyzes user activity information and generates personalized suggestions.

[0274] A "proposal-providing device" refers to a device or software that presents suggestions generated by a generative AI model to a user.

[0275] "Approachable language" refers to a method of conveying information in a format that is easy for users to understand and that easily captures their interest.

[0276] A "device that uses characters" is a device that displays characters using on-screen animation or speech synthesis technology in order to interact with the user.

[0277] A "device that calculates and notifies points to reinforce behavior" is a system or application that calculates points based on the user's activity record and notifies the user of the results.

[0278] This invention is a system that enables users to manage their health in an enjoyable and sustainable way. The system primarily uses terminals, servers, and characters to manage and analyze user activity information and provide feedback in a user-friendly format.

[0279] The device collects activity information entered by the user. Digital devices and corresponding applications are used, allowing users to manually or through sensor technology input information about their daily diet, exercise, and sleep. For example, steps taken and calories burned can be recorded through a smartphone application. The device organizes this input information and transmits it to a server using a secure communication protocol.

[0280] The server records the activity information received from the terminal in the database. The received data is analyzed using a generative AI model. The generative AI model analyzes the user's health status and generates improvement suggestions based on the prompt text. The model processes according to a prompt text such as "Please generate suggestions for health promotion based on the user's exercise data for the past week". Also, the server calculates points according to the user's activities and prepares rewards for motivation improvement.

[0281] The terminal receives the suggestions and reward information generated by the server and displays them on the screen. When displaying, friendly characters play an important role. This character conveys the suggestions to the user in a friendly manner through voice or text. As a specific example, it is possible for the character to present the next goal cheerfully while encouraging with "You did a great job today!". Through the feedback provided by the character, the user's motivation is continued, and the establishment of healthy lifestyle habits can be expected.

[0282] With the above configuration, the present invention can provide the user with a health management means that combines fun and sustainability in daily life.

[0283] The flow of the specific process in Example 1 will be described using FIG. 11.

[0284] Step 1:

[0285] The user inputs daily health-related information using the terminal. The input data includes the content of meals, calorie intake, type and duration of exercise, and sleep time and quality. The terminal organizes this information and formats it as digital data. As a specific operation, data is collected through text input or a touch interface and converted into a digital format such as JSON.

[0286] Step 2:

[0287] The terminal sends the formatted data to the server via a secure communication channel. The input information is transmitted using an encryption protocol (e.g., HTTPS), ensuring data security. The output from the terminal is an encrypted data stream, which the server receives.

[0288] Step 3:

[0289] The server records the received data in a database. The database is designed to store data chronologically. The input here is a data stream from the terminal, which the server converts into a parseable format and records.

[0290] Step 4:

[0291] The server analyzes the data using a generative AI model. This analysis is performed according to the prompt "Generate health promotion suggestions based on the user's exercise data from the past week." The output of the analysis is specific suggestions and advice tailored to the user's health condition.

[0292] Step 5:

[0293] The server sends the generated suggestions to the terminal. Furthermore, it calculates points based on activity data and prepares reward information to improve motivation. The data sent includes the generated advice and the number of points.

[0294] Step 6:

[0295] The device displays information received from the server to the user. During this process, a character appears on the screen and provides advice and reward information to the user in an engaging format. Specifically, it provides feedback to the user through animation and audio playback.

[0296] Step 7:

[0297] Users review the advice and rewards presented through their device and use them to guide their next health behaviors. They then implement new healthy habits based on the advice and prepare for the next data entry. This restarts the system cycle, enabling continuous health management.

[0298] (Application Example 1)

[0299] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0300] In modern society, while health management is becoming increasingly important, many people find it difficult to adequately manage their health amidst their busy lifestyles. In particular, there is a lack of systems that provide appropriate advice tailored to individual lifestyles. Furthermore, there is a need for systems that continuously provide such advice and that allow users to continue managing their health in an enjoyable way.

[0301] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0302] In this invention, the server includes means for recording the user's normal lifestyle data, means for analyzing the lifestyle data, evaluating the user's health status using artificial intelligence, and generating a personalized improvement plan, and means for monitoring the user's lifestyle behavior in real time using a voice recognition device and an image recognition device, and transmitting the collected data to the server. This allows the user to easily understand their own health status and receive appropriate advice. Furthermore, by providing advice through user-friendly information functions, the user can manage their health in an enjoyable way.

[0303] The term "user" refers to an individual who uses the system to manage their health.

[0304] "Normal life data" refers to data that includes information such as diet, exercise, and sleep in the user's daily life.

[0305] "Findings" refer to advice and suggestions provided to improve the user's health based on the analyzed data.

[0306] "Information function" is a mechanism that visually or auditorily presents advice and information to the user using friendly characters and display methods.

[0307] "Artificial intelligence" refers to technologies such as computer programs and algorithms used to analyze the user's life data and generate personalized health advice.

[0308] "Voice recognition device" is a device that senses the user's speech, digitizes its content, and converts it into a form that the system can understand.

[0309] "Image recognition device" is a device that acquires the user's image information, analyzes it, and converts it into digital data.

[0310] "Reward points" are incentives given when the user achieves the goal of health management and play a role in promoting the user's behavior.

[0311] This invention is configured as a system that comprehensively supports the user's daily life and promotes health management. The system aims to collect the user's life data in real time and provide appropriate health advice based on it.

[0312] The server is the core of the health management system. First, the server receives normal life data collected from the user's terminal. This data includes details of diet, exercise volume, sleep time, etc. The data is stored in a database on the cloud and analyzed using a generated AI model. Through this analysis, the user's health status is evaluated and findings for improving individual lifestyle habits are generated.

[0313] The terminal functions as an interface between the user and the server and is equipped with speech recognition and image recognition devices. For example, by using common speech recognition software or cloud-based image recognition services, it is possible to input the contents of a meal using voice or images.

[0314] Users are naturally motivated because they are awarded reward points based on their progress towards achieving their daily health goals. This information is provided through user-friendly features, allowing users to enjoy practicing a healthy lifestyle.

[0315] For example, if you speak into the device's microphone after breakfast, saying "I ate yogurt and fruit this morning," the speech recognition will digitize the content and send it to the server. The server will analyze the data and generate a suggestion such as "Try to eat a meal rich in vitamin C today," which will then be displayed on the device's screen through its information function.

[0316] An example of a prompt message might be, "Please evaluate the nutritional balance of the day based on the user's dietary data." In this way, specific and effective health advice based on data can be obtained.

[0317] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0318] Step 1:

[0319] The device acquires information about the user's diet, exercise, and sleep via voice recognition and image recognition. For example, if the user says, "Today's breakfast is eggs and toast," the voice recognition software converts that information into text data. The input is voice or image, and the output is lifestyle data in text format.

[0320] Step 2:

[0321] The terminal formats the converted lifestyle data and sends it to the server. Specifically, it is converted into a specific key-value format, such as "breakfast," "eggs," and "toast." The input is the text data obtained in the previous step, and the output is the formatted data.

[0322] Step 3:

[0323] The server stores received normal life data in a cloud-based database. The data is permanently stored and used for later analysis. The input is formatted data, and the output is the stored data.

[0324] Step 4:

[0325] The server retrieves data from the database and analyzes it using a generative AI model. The analysis evaluates the user's current lifestyle and health status to identify areas for improvement. The input is lifestyle data retrieved from the database, and the output is the analysis results.

[0326] Step 5:

[0327] The server generates personalized findings based on the analysis results. These findings are actionable for the user and provide concrete advice to help improve their life. The input is the analysis results, and the output is the advice, including the findings.

[0328] Step 6:

[0329] The server sends the generated findings to the terminal. The findings are presented to the user through user-friendly information functions. The input is the generated advice, and the output is user-oriented information displayed on the terminal.

[0330] Step 7:

[0331] The user reviews the presented findings and chooses whether or not to perform a health behavior. Based on this choice, the reward function, which is the next step, is activated. The input is the information displayed on the device, and the output is the user's behavioral choice.

[0332] Step 8:

[0333] The server calculates reward points based on the user's action choices and notifies the terminal. For example, if the user achieves their daily goal, it displays the message, "You earned 100 points today!" The input is the user's action choices, and the output is reward information.

[0334] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0335] This invention aims to achieve more personalized interactions by incorporating an emotion engine into a system that supports user activity management. This system collects user activity data and provides appropriate advice, while simultaneously using the emotion engine to recognize the user's emotions and respond accordingly.

[0336] The system mainly consists of terminals, servers, emotion engines, and characters.

[0337] Input and recording of activity data

[0338] Users input data on their daily activities, such as information about meals, exercise, and sleep, via a terminal. The terminal receives, formats, and sends the input to a server. The server records and stores this data in a database for later analysis and advice generation.

[0339] Emotion recognition and analysis

[0340] The device is equipped with an emotion engine that recognizes the user's emotions from their facial expressions, tone of voice, and entered text. This information is sent to a server and analyzed along with the user's activity data. The server's generative AI is used to generate appropriate advice based on these emotions.

[0341] Advice provided by characters

[0342] The generated advice is sent from the server to the terminal. The terminal adjusts the character's expression based on the user's emotions recognized by the emotion engine, providing advice in an appropriate tone and content. For example, if the user is feeling down, the advice will emphasize words of encouragement and be displayed with a cheerful character.

[0343] Motivation enhancement and reward function

[0344] The system has a function that provides rewards based on the user's activity level. The server evaluates the user's emotions and activity data, generates rewards such as points and badges, and sends them to the terminal. The character enjoys and reports these rewards, actively reinforcing the user's behavior.

[0345] For example, if a user engages in active participation and their joyful expression is recognized, the character will offer an additional reward along with a congratulatory message such as, "That's great! Keep it up!" This encourages users to continue participating.

[0346] As described above, the present invention utilizes an emotion engine to support users' lifestyles in a more personalized way, making it possible to continue health management in an enjoyable manner.

[0347] The following describes the processing flow.

[0348] Step 1:

[0349] The user starts up the device and opens the interface for entering daily activity data. They prepare to enter daily information such as meals, exercise, and sleep.

[0350] Step 2:

[0351] The terminal displays an input form, and the user enters detailed activity data (e.g., diet, type of exercise). Once the user has finished entering the data, the terminal formats it into a predetermined format.

[0352] Step 3:

[0353] The formatted data is sent to a server via the internet. The server stores the received data in a database, making it available for subsequent analysis.

[0354] Step 4:

[0355] The device's built-in emotion engine analyzes the user's facial expressions and voice to detect their current emotional state (e.g., joy, sadness). This information is immediately sent to the server.

[0356] Step 5:

[0357] The server integrates activity data and emotional data, and uses generative AI for analysis. It then generates optimal health advice tailored to the user's emotions.

[0358] Step 6:

[0359] The generated advice is sent from the server to the terminal. The terminal adjusts the character's facial expressions and messages based on the received information.

[0360] Step 7:

[0361] The device presents the user with a customized character and advice. For example, if the emotion engine determines that the user is feeling down, the character will offer words of encouragement with a gentle expression.

[0362] Step 8:

[0363] The user reviews the advice provided and incorporates it into their actions. The device records the user's response to help improve future advice.

[0364] Step 9:

[0365] The server provides incentives (e.g., points, badges) through a reward system based on the user's actions and emotions. Users receive these incentives and maintain their motivation.

[0366] (Example 2)

[0367] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0368] In managing users' health and improving their lifestyles, conventional systems struggle to provide advice tailored to individual emotional states and maintain motivation, making it challenging to continuously motivate users. Furthermore, there is a need for a system that can provide advice that considers not only activity data but also emotions.

[0369] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0370] In this invention, the server includes means for receiving user activity information from an input device, formatting it, and storing it on a recording medium; means for analyzing the user's emotions using an emotion recognition device and generating advice using a generative AI model based on that information; and means for displaying the generated advice in an expression method appropriate to the user's emotions and providing it through a friendly character. This enables users to continue working on health management and improving their lifestyle habits by using advice that takes into account their individual emotional state and an appropriate reward system.

[0371] "Activity information" refers to information about the user's actions and behaviors in their daily life, including data on meals, exercise, sleep, etc.

[0372] An "input device" refers to a device used by users to record activity information, such as a smartphone or personal computer.

[0373] A "recording medium" is a device or medium used to store received information, and includes databases and storage systems.

[0374] An "emotion recognition device" is a hardware and software system that analyzes emotions from a user's facial expressions, voice tone, and text data.

[0375] A "generative AI model" is an artificial intelligence model that generates advice in natural language based on received data.

[0376] A "character" is a representation of a virtual person or animal that provides information to the user visually or aurally, and is intended to present advice or rewards in an approachable way.

[0377] "Rewards" refer to incentives such as points or badges provided based on the evaluation of users' activities, and are means of reinforcing user behavior.

[0378] The system of this invention assists in managing user activity and provides feedback based on individual emotional states. The system mainly consists of a terminal, a server, an emotion recognition device, a generative AI model, and a friendly character.

[0379] Users input daily activity information using a terminal. The terminal is equipped with an input device, which is used to input activity information such as meals, exercise, and sleep. The input information is formatted by the terminal and sent to the server for storage on a recording medium. The server stores this data and uses it for later analysis.

[0380] The device is equipped with an emotion recognition device that analyzes the user's emotions from their facial expressions, voice tone, and entered text data. This analysis uses hardware such as a camera and microphone, as well as emotion analysis software. The analyzed emotion data is sent to a server along with activity information, where a generative AI model is used.

[0381] The server uses a generative AI model to generate personalized advice based on the received activity and emotion data. This advice is generated using prompts that correspond to the user's emotional state. For example, a prompt might be "Generate the most appropriate encouraging message based on the user's emotions."

[0382] The generated advice is sent from the server to the terminal, which then displays the advice using a character. The character uses a bright and friendly expression, providing advice that adjusts in tone and content according to the user's emotions. For example, if the user is feeling down, the character will offer encouraging words.

[0383] Furthermore, the server evaluates the user's activity and emotions, generating rewards such as points and badges. These rewards are sent to the device, and a character reports them to the user in an entertaining way. This helps users continue their actions and maintain motivation for health management and lifestyle improvements.

[0384] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0385] Step 1:

[0386] Users operate the terminal to input information about their daily activities. This information includes details about meals, exercise time, and sleep quality. This activity information is received by an input device on the terminal and converted into digital data. The input data is formatted in JSON format and prepared to be sent to the server.

[0387] Step 2:

[0388] The terminal sends formatted activity information to the server. The server receives the data sent via HTTP request and stores it in a database. This stored data is later parsed using SQL queries. In this case, the input is the activity information, and the output is the stored data.

[0389] Step 3:

[0390] The device analyzes the user's emotions using an emotion recognition device. This analysis collects the user's facial expressions and voice using a camera and microphone. This data is analyzed by emotion recognition software and converted into digital data representing the user's emotional state. The resulting emotional data is then formatted and ready to be sent to the server.

[0391] Step 4:

[0392] The device sends formatted emotion data to the server. The server stores the received emotion data in a database. The stored activity information and emotion data are then used by a generative AI model to generate advice. In this process, the input is the emotion data, and the output is the state in which that data is stored in the database.

[0393] Step 5:

[0394] The server generates advice using a generative AI model. This advice generation process uses the prompt "Generate the most appropriate encouraging message based on the user's emotions." The generative AI model receives stored activity information and emotion data as input and generates advice tailored to the user's emotions. The output is the generated advice message.

[0395] Step 6:

[0396] The server sends a generated advice message to the terminal. The terminal displays the received message to the user through a friendly character. The character delivers the advice in a tone that matches the user's emotions. The input is the advice message, and the output is the display of that message.

[0397] Step 7:

[0398] The server evaluates user activity and emotional data and generates rewards. This process generates reward data such as points and badges and sends it to the device. The generated reward data includes incentives that are tailored to the user's level of achievement and emotional state.

[0399] Step 8:

[0400] The device presents the user with reward data generated using a character. The character introduces the rewards in an enjoyable way, encouraging the user to continue their engagement. The input is the reward data, and the output is the visual presentation of that data to the user.

[0401] (Application Example 2)

[0402] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0403] Conventional activity management systems can only provide users with fixed advice, making it difficult to provide appropriate guidance that takes into account individual emotional states. Furthermore, there is a need to achieve more sophisticated interactions in providing advice in a friendly manner and in reward functions that enhance user motivation.

[0404] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0405] In this invention, the server includes a device for recording user behavior data, a device for providing instruction generated based on the behavior data, a device for delivering the instruction through a virtual character that presents the instruction in a friendly manner, and a device for recognizing the user's emotions using an emotion engine and adjusting the instruction content accordingly. This enables flexible and friendly instruction based on the individual emotions of each user.

[0406] "User behavioral data" refers to information about the activities that users engage in in their daily lives, and this includes information such as exercise, diet, and sleep.

[0407] A "device that provides guidance" is a device that has the function of generating and presenting appropriate guidance and advice to the user based on recorded behavioral data.

[0408] A "virtual character" is a visual entity with a personality generated digitally, and its role is to provide information and guidance through interaction with users.

[0409] An "emotion engine" refers to a technology or system that analyzes a user's facial expressions, voice, text input, etc., to recognize the user's emotions.

[0410] A "reward-providing device" is a device that generates and presents points or badges based on the user's behavioral data and emotional state, with the aim of improving the user's motivation.

[0411] This invention realizes a system for collecting and analyzing user activity data and emotions to provide personalized guidance. The system mainly consists of a terminal, a server, an emotion engine, and a virtual character.

[0412] The terminal collects and records data about the user's daily activities, such as information on diet, exercise, and sleep. Users can input this information using mobile devices such as smartphones and tablets. The terminal is responsible for formatting the input information and sending it to the server.

[0413] The server receives activity data provided by the user and stores it in a database. Furthermore, the server is equipped with a generative AI model that analyzes the accumulated behavioral data and emotional information to generate appropriate guidance for the user. In this analysis process, speech synthesis APIs such as Amazon Polly and image recognition APIs such as AWS Rekognition are used to perform emotion recognition from voice input and image data.

[0414] The emotion engine is integrated into the device and instantly identifies emotions from the user's tone of voice, facial expressions, and input text. This information is sent to a server and used to provide specific guidance. The emotion engine helps understand the user's mental state, enabling more effective communication.

[0415] The virtual character receives instructional content from the server and presents it using expressions based on the user's emotions, as recognized by the emotion engine. The character plays a role in conveying information in a visually appealing and easy-to-understand manner for the user.

[0416] Furthermore, the reward system generates and displays points and badges on the device to reinforce user behavior. This allows users to be further motivated by the visual recognition of their efforts.

[0417] For example, if a user sends a message to the robot saying, "I feel great after my workout today," the emotion engine recognizes the user's "joy," and the server generates a response saying, "That's wonderful! Keep it up!" which is then conveyed to the user through a virtual character in an enthusiastic manner. This encourages users to take a more positive stance towards maintaining a healthy lifestyle.

[0418] An example of a prompt to input into the generation AI model is, "Based on my activity data, please generate an appropriate response when the user is happy."

[0419] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0420] Step 1:

[0421] Users input data on their daily activities using devices such as smartphones and tablets. This input includes information such as exercise levels, diet, and sleep duration. The device receives this data, formats it, and sends it to the server. The input data is primarily text-based and is appropriately structured on the device.

[0422] Step 2:

[0423] The server receives activity data sent from terminals and records it in the database. This activity data is used for later analysis and instruction generation, so the server stores it efficiently. Furthermore, data formatting processes remove unnecessary data and correct input errors, creating accurate database entries.

[0424] Step 3:

[0425] The device analyzes the user's voice tone and facial expressions via an emotion engine and generates emotion data. For example, when the user shows their facial expressions using the camera or speaks through the microphone, the device executes an emotion recognition algorithm and sends the results to a server. The input data is then classified into categories such as "joy" or "sadness" by an emotion analysis model within the program.

[0426] Step 4:

[0427] The server uses a generative AI model to generate appropriate guidance based on recorded activity and emotion data. The AI ​​model leverages a large amount of historical data and generates personalized guidance according to defined prompts (e.g., "Generate an appropriate response when the user is happy"). This process utilizes a machine learning model, and optimal guidance is generated through data computation.

[0428] Step 5:

[0429] The server sends the generated instructions to the terminal. The terminal, in accordance with the user's emotions recognized by the emotion engine, provides the instructions visually and audibly through a virtual character. Here, the character displays feedback through animation and sound, providing information in a way that is easy for the user to understand and relate to.

[0430] Step 6:

[0431] The device rewards users with points or badges using a reward system when their activity meets certain criteria. The server evaluates the user's activity and emotional data, generates rewards based on this evaluation, and presents them to the user through the device. This actively reinforces the user's behavior and promotes sustainable motivation.

[0432] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0433] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0434] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0435] [Third Embodiment]

[0436] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0437] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0438] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0439] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0440] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0441] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0442] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0443] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0444] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0445] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0446] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0447] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0448] This invention describes a specific embodiment of a system that assists in health management through interaction between a user and a character. The system aims to promote healthy lifestyle habits by collecting and analyzing user activity data.

[0449] The system mainly consists of terminals that handle data, servers that analyze data, and characters that interact with users.

[0450] Data collection and input

[0451] Users input information about their daily diet, exercise, and sleep through a terminal. The terminal receives the user's input, formats it, and sends it to the server. This information includes details about meals, type and duration of exercise, and sleep quality.

[0452] Data analysis and advisory generation

[0453] The server records the received activity data in a database and performs analysis using a generating AI. Based on the analysis results, it generates advice to improve the user's health. This advice is based on the user's past data and lifestyle patterns.

[0454] Advice provided by characters

[0455] The generated advice is displayed on the device through a character. On the device screen, a user-friendly character delivers the generated advice with humor and encouraging words. In this way, users can manage their health in an enjoyable way.

[0456] Maintaining Motivation

[0457] Furthermore, the system includes a reward function to maintain and improve user motivation. The server awards points based on increased activity levels and achieved goals, and notifies the user through a character reporting this via the terminal. For example, users who achieve a certain amount of exercise will receive encouraging words from the character such as, "Great! Let's work hard together towards your next goal!"

[0458] In this way, the present invention makes it possible to make users' health management fun and sustainable through characters, and to contribute to extending healthy life expectancy.

[0459] The following describes the processing flow.

[0460] Step 1:

[0461] The user launches the app and accesses an interface for entering data on meals, exercise, and sleep. The device displays this input screen, allowing the user to enter the necessary information.

[0462] Step 2:

[0463] After the user completes the input and confirms the content, they press the submit button. The terminal converts the entered data into a predetermined format (e.g., JSON) and sends it to the server via the internet.

[0464] Step 3:

[0465] The server receives data sent from the terminal and stores it in the database. This data is recorded as an individual user's activity history and is ready for later analysis.

[0466] Step 4:

[0467] The server uses AI to analyze activity data stored in the database. This analysis compares the user's past patterns with current data to generate appropriate health management advice for the user.

[0468] Step 5:

[0469] The server sends the generated advice to the terminal. This prepares the terminal to display a character that will present the advice to the user in a user-friendly way.

[0470] Step 6:

[0471] The device displays character graphics and generated advice on the screen. The character offers specific action suggestions and words of encouragement to the user, providing a trigger for them to take the next step.

[0472] Step 7:

[0473] After receiving advice from the character, users adjust and improve their lifestyle habits based on that advice. Furthermore, users can set their next activity goals by receiving feedback from the app.

[0474] (Example 1)

[0475] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0476] In modern times, many people want to manage their health in a more enjoyable and effective way in their daily lives. However, conventional health management systems have problems with maintaining user motivation and sustained behavior, resulting in monotonous and difficult-to-continue systems. This invention aims to provide a system that allows users to manage their health in an enjoyable and sustainable way.

[0477] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0478] In this invention, the server includes a device for recording user activity information, a device for providing suggestions generated using a generative AI model, a device for presenting suggestions through a character that displays them in a friendly manner, and a device for calculating and notifying points to reinforce user behavior. This enables users to enjoy daily health management and make sustainable improvements to their lives.

[0479] "User activity information" refers to data about the user's daily activities, such as eating, exercising, and sleeping.

[0480] A "recording device" is hardware or software used to collect user activity information and store it as data.

[0481] A "generative AI model" is an artificial intelligence algorithm that analyzes user activity information and generates personalized suggestions.

[0482] A "proposal-providing device" refers to a device or software that presents suggestions generated by a generative AI model to a user.

[0483] "Approachable language" refers to a method of conveying information in a format that is easy for users to understand and that easily captures their interest.

[0484] A "device that uses characters" is a device that displays characters using on-screen animation or speech synthesis technology in order to interact with the user.

[0485] A "device that calculates and notifies points to reinforce behavior" is a system or application that calculates points based on the user's activity record and notifies the user of the results.

[0486] This invention is a system that enables users to manage their health in an enjoyable and sustainable way. The system primarily uses terminals, servers, and characters to manage and analyze user activity information and provide feedback in a user-friendly format.

[0487] The device collects activity information entered by the user. Digital devices and corresponding applications are used, allowing users to manually or through sensor technology input information about their daily diet, exercise, and sleep. For example, steps taken and calories burned can be recorded through a smartphone application. The device organizes this input information and transmits it to a server using a secure communication protocol.

[0488] The server records activity information received from the terminal in a database. The received data is analyzed using a generative AI model. Based on prompts, the generative AI model analyzes the user's health status and generates suggestions for improvement. The model processes prompts such as, "Generate suggestions for health promotion based on the user's exercise data from the past week." The server also calculates points based on the user's activity and prepares rewards to improve motivation.

[0489] The device receives suggestions and reward information generated from the server and displays it on the screen. A friendly character plays a crucial role in this display. This character communicates suggestions to the user in an approachable way through voice and text. For example, the character can encourage the user by saying, "You did a great job today!" and then present the next goal in a fun way. The feedback provided by the character helps maintain the user's motivation and is expected to help them establish healthy lifestyle habits.

[0490] With the above configuration, this invention can provide users with a health management method that combines enjoyment and sustainability in their daily lives.

[0491] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0492] Step 1:

[0493] Users input daily health-related information using a device. This input data includes details of meals, calorie intake, type and duration of exercise, and sleep duration and quality. The device organizes this information and formats it as digital data. Specifically, it collects data via text input or a touch interface and converts it into a digital format such as JSON.

[0494] Step 2:

[0495] The terminal sends the formatted data to the server via a secure communication channel. The input information is transmitted using an encryption protocol (e.g., HTTPS), ensuring data security. The output from the terminal is an encrypted data stream, which the server receives.

[0496] Step 3:

[0497] The server records the received data in a database. The database is designed to store data chronologically. The input here is a data stream from the terminal, which the server converts into a parseable format and records.

[0498] Step 4:

[0499] The server analyzes the data using a generative AI model. This analysis is performed according to the prompt "Generate health promotion suggestions based on the user's exercise data from the past week." The output of the analysis is specific suggestions and advice tailored to the user's health condition.

[0500] Step 5:

[0501] The server sends the generated suggestions to the terminal. Furthermore, it calculates points based on activity data and prepares reward information to improve motivation. The data sent includes the generated advice and the number of points.

[0502] Step 6:

[0503] The device displays information received from the server to the user. During this process, a character appears on the screen and provides advice and reward information to the user in an engaging format. Specifically, it provides feedback to the user through animation and audio playback.

[0504] Step 7:

[0505] Users review the advice and rewards presented through their device and use them to guide their next health behaviors. They then implement new healthy habits based on the advice and prepare for the next data entry. This restarts the system cycle, enabling continuous health management.

[0506] (Application Example 1)

[0507] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0508] In modern society, while health management is becoming increasingly important, many people find it difficult to adequately manage their health amidst their busy lifestyles. In particular, there is a lack of systems that provide appropriate advice tailored to individual lifestyles. Furthermore, there is a need for systems that continuously provide such advice and that allow users to continue managing their health in an enjoyable way.

[0509] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0510] In this invention, the server includes means for recording the user's normal lifestyle data, means for analyzing the lifestyle data, evaluating the user's health status using artificial intelligence, and generating a personalized improvement plan, and means for monitoring the user's lifestyle behavior in real time using a voice recognition device and an image recognition device, and transmitting the collected data to the server. This allows the user to easily understand their own health status and receive appropriate advice. Furthermore, by providing advice through user-friendly information functions, the user can manage their health in an enjoyable way.

[0511] The term "user" refers to an individual who uses the system to manage their health.

[0512] "Normal lifestyle data" refers to data that includes information about the user's daily life, such as diet, exercise, and sleep.

[0513] "Findings" refer to advice and suggestions provided based on analyzed data to improve the user's health.

[0514] An "information function" is a mechanism that presents advice and information to users visually or audibly using friendly characters and display methods.

[0515] "Artificial intelligence" refers to computer programs and algorithms used to analyze a user's lifestyle data and generate personalized health advice.

[0516] A "speech recognition device" is a device that detects a user's speech, digitizes its content, and converts it into a format that the system can understand.

[0517] An "image recognition device" is a device that acquires a user's image information, analyzes it, and converts it into digital data.

[0518] "Reward points" are incentives awarded to users when they achieve their health management goals, and they serve to encourage user behavior.

[0519] This invention is designed as a system that comprehensively supports users' daily lives and promotes health management. The system aims to collect users' lifestyle data in real time and provide appropriate health advice based on that data.

[0520] The server is the core of the health management system. First, the server receives normal lifestyle data collected from the user's terminal. This data includes details of meals, exercise levels, sleep duration, etc. The data is stored in a cloud-based database and analyzed using a generative AI model. This analysis evaluates the user's health status and generates personalized recommendations for lifestyle improvements.

[0521] The terminal functions as an interface between the user and the server and is equipped with speech recognition and image recognition devices. For example, by using common speech recognition software or cloud-based image recognition services, it is possible to input the contents of a meal using voice or images.

[0522] Users are naturally motivated because they are awarded reward points based on their progress towards achieving their daily health goals. This information is provided through user-friendly features, allowing users to enjoy practicing a healthy lifestyle.

[0523] For example, if you speak into the device's microphone after breakfast, saying "I ate yogurt and fruit this morning," the speech recognition will digitize the content and send it to the server. The server will analyze the data and generate a suggestion such as "Try to eat a meal rich in vitamin C today," which will then be displayed on the device's screen through its information function.

[0524] An example of a prompt message might be, "Please evaluate the nutritional balance of the day based on the user's dietary data." In this way, specific and effective health advice based on data can be obtained.

[0525] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0526] Step 1:

[0527] The device acquires information about the user's diet, exercise, and sleep via voice recognition and image recognition. For example, if the user says, "Today's breakfast is eggs and toast," the voice recognition software converts that information into text data. The input is voice or image, and the output is lifestyle data in text format.

[0528] Step 2:

[0529] The terminal formats the converted lifestyle data and sends it to the server. Specifically, it is converted into a specific key-value format, such as "breakfast," "eggs," and "toast." The input is the text data obtained in the previous step, and the output is the formatted data.

[0530] Step 3:

[0531] The server stores received normal life data in a cloud-based database. The data is permanently stored and used for later analysis. The input is formatted data, and the output is the stored data.

[0532] Step 4:

[0533] The server retrieves data from the database and analyzes it using a generative AI model. The analysis evaluates the user's current lifestyle and health status to identify areas for improvement. The input is lifestyle data retrieved from the database, and the output is the analysis results.

[0534] Step 5:

[0535] The server generates personalized findings based on the analysis results. These findings are actionable for the user and provide concrete advice to help improve their life. The input is the analysis results, and the output is the advice, including the findings.

[0536] Step 6:

[0537] The server sends the generated findings to the terminal. The findings are presented to the user through user-friendly information functions. The input is the generated advice, and the output is user-oriented information displayed on the terminal.

[0538] Step 7:

[0539] The user reviews the presented findings and chooses whether or not to perform a health behavior. Based on this choice, the reward function, which is the next step, is activated. The input is the information displayed on the device, and the output is the user's behavioral choice.

[0540] Step 8:

[0541] The server calculates reward points based on the user's action choices and notifies the terminal. For example, if the user achieves their daily goal, it displays the message, "You earned 100 points today!" The input is the user's action choices, and the output is reward information.

[0542] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0543] This invention aims to achieve more personalized interactions by incorporating an emotion engine into a system that supports user activity management. This system collects user activity data and provides appropriate advice, while simultaneously using the emotion engine to recognize the user's emotions and respond accordingly.

[0544] The system mainly consists of terminals, servers, emotion engines, and characters.

[0545] Input and recording of activity data

[0546] Users input data on their daily activities, such as information about meals, exercise, and sleep, via a terminal. The terminal receives, formats, and sends the input to a server. The server records and stores this data in a database for later analysis and advice generation.

[0547] Emotion recognition and analysis

[0548] The device is equipped with an emotion engine that recognizes the user's emotions from their facial expressions, tone of voice, and entered text. This information is sent to a server and analyzed along with the user's activity data. The server's generative AI is used to generate appropriate advice based on these emotions.

[0549] Advice provided by characters

[0550] The generated advice is sent from the server to the terminal. The terminal adjusts the character's expression based on the user's emotions recognized by the emotion engine, providing advice in an appropriate tone and content. For example, if the user is feeling down, the advice will emphasize words of encouragement and be displayed with a cheerful character.

[0551] Motivation enhancement and reward function

[0552] The system has a function that provides rewards based on the user's activity level. The server evaluates the user's emotions and activity data, generates rewards such as points and badges, and sends them to the terminal. The character enjoys and reports these rewards, actively reinforcing the user's behavior.

[0553] For example, if a user engages in active participation and their joyful expression is recognized, the character will offer an additional reward along with a congratulatory message such as, "That's great! Keep it up!" This encourages users to continue participating.

[0554] As described above, the present invention utilizes an emotion engine to support users' lifestyles in a more personalized way, making it possible to continue health management in an enjoyable manner.

[0555] The following describes the processing flow.

[0556] Step 1:

[0557] The user starts up the device and opens the interface for entering daily activity data. They prepare to enter daily information such as meals, exercise, and sleep.

[0558] Step 2:

[0559] The terminal displays an input form, and the user enters detailed activity data (e.g., diet, type of exercise). Once the user has finished entering the data, the terminal formats it into a predetermined format.

[0560] Step 3:

[0561] The formatted data is sent to a server via the internet. The server stores the received data in a database, making it available for subsequent analysis.

[0562] Step 4:

[0563] The device's built-in emotion engine analyzes the user's facial expressions and voice to detect their current emotional state (e.g., joy, sadness). This information is immediately sent to the server.

[0564] Step 5:

[0565] The server integrates activity data and emotional data, and uses generative AI for analysis. It then generates optimal health advice tailored to the user's emotions.

[0566] Step 6:

[0567] The generated advice is sent from the server to the terminal. The terminal adjusts the character's facial expressions and messages based on the received information.

[0568] Step 7:

[0569] The device presents the user with a customized character and advice. For example, if the emotion engine determines that the user is feeling down, the character will offer words of encouragement with a gentle expression.

[0570] Step 8:

[0571] The user reviews the advice provided and incorporates it into their actions. The device records the user's response to help improve future advice.

[0572] Step 9:

[0573] The server provides incentives (e.g., points, badges) through a reward system based on the user's actions and emotions. Users receive these incentives and maintain their motivation.

[0574] (Example 2)

[0575] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0576] In managing users' health and improving their lifestyles, conventional systems struggle to provide advice tailored to individual emotional states and maintain motivation, making it challenging to continuously motivate users. Furthermore, there is a need for a system that can provide advice that considers not only activity data but also emotions.

[0577] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0578] In this invention, the server includes means for receiving user activity information from an input device, formatting it, and storing it on a recording medium; means for analyzing the user's emotions using an emotion recognition device and generating advice using a generative AI model based on that information; and means for displaying the generated advice in an expression method appropriate to the user's emotions and providing it through a friendly character. This enables users to continue working on health management and improving their lifestyle habits by using advice that takes into account their individual emotional state and an appropriate reward system.

[0579] "Activity information" refers to information about the user's actions and behaviors in their daily life, including data on meals, exercise, sleep, etc.

[0580] An "input device" refers to a device used by users to record activity information, such as a smartphone or personal computer.

[0581] A "recording medium" is a device or medium used to store received information, and includes databases and storage systems.

[0582] An "emotion recognition device" is a hardware and software system that analyzes emotions from a user's facial expressions, voice tone, and text data.

[0583] A "generative AI model" is an artificial intelligence model that generates advice in natural language based on received data.

[0584] A "character" is a representation of a virtual person or animal that provides information to the user visually or aurally, and is intended to present advice or rewards in an approachable way.

[0585] "Rewards" refer to incentives such as points or badges provided based on the evaluation of users' activities, and are means of reinforcing user behavior.

[0586] The system of this invention assists in managing user activity and provides feedback based on individual emotional states. The system mainly consists of a terminal, a server, an emotion recognition device, a generative AI model, and a friendly character.

[0587] Users input daily activity information using a terminal. The terminal is equipped with an input device, which is used to input activity information such as meals, exercise, and sleep. The input information is formatted by the terminal and sent to the server for storage on a recording medium. The server stores this data and uses it for later analysis.

[0588] The device is equipped with an emotion recognition device that analyzes the user's emotions from their facial expressions, voice tone, and entered text data. This analysis uses hardware such as a camera and microphone, as well as emotion analysis software. The analyzed emotion data is sent to a server along with activity information, where a generative AI model is used.

[0589] The server uses a generative AI model to generate personalized advice based on the received activity and emotion data. This advice is generated using prompts that correspond to the user's emotional state. For example, a prompt might be "Generate the most appropriate encouraging message based on the user's emotions."

[0590] The generated advice is sent from the server to the terminal, which then displays the advice using a character. The character uses a bright and friendly expression, providing advice that adjusts in tone and content according to the user's emotions. For example, if the user is feeling down, the character will offer encouraging words.

[0591] Furthermore, the server evaluates the user's activity and emotions, generating rewards such as points and badges. These rewards are sent to the device, and a character reports them to the user in an entertaining way. This helps users continue their actions and maintain motivation for health management and lifestyle improvements.

[0592] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0593] Step 1:

[0594] Users operate the terminal to input information about their daily activities. This information includes details about meals, exercise time, and sleep quality. This activity information is received by an input device on the terminal and converted into digital data. The input data is formatted in JSON format and prepared to be sent to the server.

[0595] Step 2:

[0596] The terminal sends formatted activity information to the server. The server receives the data sent via HTTP request and stores it in a database. This stored data is later parsed using SQL queries. In this case, the input is the activity information, and the output is the stored data.

[0597] Step 3:

[0598] The device analyzes the user's emotions using an emotion recognition device. This analysis collects the user's facial expressions and voice using a camera and microphone. This data is analyzed by emotion recognition software and converted into digital data representing the user's emotional state. The resulting emotional data is then formatted and ready to be sent to the server.

[0599] Step 4:

[0600] The device sends formatted emotion data to the server. The server stores the received emotion data in a database. The stored activity information and emotion data are then used by a generative AI model to generate advice. In this process, the input is the emotion data, and the output is the state in which that data is stored in the database.

[0601] Step 5:

[0602] The server generates advice using a generative AI model. This advice generation process uses the prompt "Generate the most appropriate encouraging message based on the user's emotions." The generative AI model receives stored activity information and emotion data as input and generates advice tailored to the user's emotions. The output is the generated advice message.

[0603] Step 6:

[0604] The server sends a generated advice message to the terminal. The terminal displays the received message to the user through a friendly character. The character delivers the advice in a tone that matches the user's emotions. The input is the advice message, and the output is the display of that message.

[0605] Step 7:

[0606] The server evaluates user activity and emotional data and generates rewards. This process generates reward data such as points and badges and sends it to the device. The generated reward data includes incentives that are tailored to the user's level of achievement and emotional state.

[0607] Step 8:

[0608] The device presents the user with reward data generated using a character. The character introduces the rewards in an enjoyable way, encouraging the user to continue their engagement. The input is the reward data, and the output is the visual presentation of that data to the user.

[0609] (Application Example 2)

[0610] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0611] Conventional activity management systems can only provide users with fixed advice, making it difficult to provide appropriate guidance that takes into account individual emotional states. Furthermore, there is a need to achieve more sophisticated interactions in providing advice in a friendly manner and in reward functions that enhance user motivation.

[0612] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0613] In this invention, the server includes a device for recording user behavior data, a device for providing instruction generated based on the behavior data, a device for delivering the instruction through a virtual character that presents the instruction in a friendly manner, and a device for recognizing the user's emotions using an emotion engine and adjusting the instruction content accordingly. This enables flexible and friendly instruction based on the individual emotions of each user.

[0614] "User behavioral data" refers to information about the activities that users engage in in their daily lives, and this includes information such as exercise, diet, and sleep.

[0615] A "device that provides guidance" is a device that has the function of generating and presenting appropriate guidance and advice to the user based on recorded behavioral data.

[0616] A "virtual character" is a visual entity with a personality generated digitally, and its role is to provide information and guidance through interaction with users.

[0617] An "emotion engine" refers to a technology or system that analyzes a user's facial expressions, voice, text input, etc., to recognize the user's emotions.

[0618] A "reward-providing device" is a device that generates and presents points or badges based on the user's behavioral data and emotional state, with the aim of improving the user's motivation.

[0619] This invention realizes a system for collecting and analyzing user activity data and emotions to provide personalized guidance. The system mainly consists of a terminal, a server, an emotion engine, and a virtual character.

[0620] The terminal collects and records data about the user's daily activities, such as information on diet, exercise, and sleep. Users can input this information using mobile devices such as smartphones and tablets. The terminal is responsible for formatting the input information and sending it to the server.

[0621] The server receives activity data provided by the user and stores it in a database. Furthermore, the server is equipped with a generative AI model that analyzes the accumulated behavioral data and emotional information to generate appropriate guidance for the user. In this analysis process, speech synthesis APIs such as Amazon Polly and image recognition APIs such as AWS Rekognition are used to perform emotion recognition from voice input and image data.

[0622] The emotion engine is integrated into the device and instantly identifies emotions from the user's tone of voice, facial expressions, and input text. This information is sent to a server and used to provide specific guidance. The emotion engine helps understand the user's mental state, enabling more effective communication.

[0623] The virtual character receives instructional content from the server and presents it using expressions based on the user's emotions, as recognized by the emotion engine. The character plays a role in conveying information in a visually appealing and easy-to-understand manner for the user.

[0624] Furthermore, the reward system generates and displays points and badges on the device to reinforce user behavior. This allows users to be further motivated by the visual recognition of their efforts.

[0625] For example, if a user sends a message to the robot saying, "I feel great after my workout today," the emotion engine recognizes the user's "joy," and the server generates a response saying, "That's wonderful! Keep it up!" which is then conveyed to the user through a virtual character in an enthusiastic manner. This encourages users to take a more positive stance towards maintaining a healthy lifestyle.

[0626] An example of a prompt to input into the generation AI model is, "Based on my activity data, please generate an appropriate response when the user is happy."

[0627] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0628] Step 1:

[0629] Users input data on their daily activities using devices such as smartphones and tablets. This input includes information such as exercise levels, diet, and sleep duration. The device receives this data, formats it, and sends it to the server. The input data is primarily text-based and is appropriately structured on the device.

[0630] Step 2:

[0631] The server receives activity data sent from terminals and records it in the database. This activity data is used for later analysis and instruction generation, so the server stores it efficiently. Furthermore, data formatting processes remove unnecessary data and correct input errors, creating accurate database entries.

[0632] Step 3:

[0633] The device analyzes the user's voice tone and facial expressions via an emotion engine and generates emotion data. For example, when the user shows their facial expressions using the camera or speaks through the microphone, the device executes an emotion recognition algorithm and sends the results to a server. The input data is then classified into categories such as "joy" or "sadness" by an emotion analysis model within the program.

[0634] Step 4:

[0635] The server uses a generative AI model to generate appropriate guidance based on recorded activity and emotion data. The AI ​​model leverages a large amount of historical data and generates personalized guidance according to defined prompts (e.g., "Generate an appropriate response when the user is happy"). This process utilizes a machine learning model, and optimal guidance is generated through data computation.

[0636] Step 5:

[0637] The server sends the generated instructions to the terminal. The terminal, in accordance with the user's emotions recognized by the emotion engine, provides the instructions visually and audibly through a virtual character. Here, the character displays feedback through animation and sound, providing information in a way that is easy for the user to understand and relate to.

[0638] Step 6:

[0639] The device rewards users with points or badges using a reward system when their activity meets certain criteria. The server evaluates the user's activity and emotional data, generates rewards based on this evaluation, and presents them to the user through the device. This actively reinforces the user's behavior and promotes sustainable motivation.

[0640] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0641] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0642] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0643] [Fourth Embodiment]

[0644] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0645] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0646] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0647] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0648] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0649] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0650] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0651] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0652] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0653] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0654] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0655] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0656] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0657] This invention describes a specific embodiment of a system that assists in health management through interaction between a user and a character. The system aims to promote healthy lifestyle habits by collecting and analyzing user activity data.

[0658] The system mainly consists of terminals that handle data, servers that analyze data, and characters that interact with users.

[0659] Data collection and input

[0660] Users input information about their daily diet, exercise, and sleep through a terminal. The terminal receives the user's input, formats it, and sends it to the server. This information includes details about meals, type and duration of exercise, and sleep quality.

[0661] Data analysis and advisory generation

[0662] The server records the received activity data in a database and performs analysis using a generating AI. Based on the analysis results, it generates advice to improve the user's health. This advice is based on the user's past data and lifestyle patterns.

[0663] Advice provided by characters

[0664] The generated advice is displayed on the device through a character. On the device screen, a user-friendly character delivers the generated advice with humor and encouraging words. In this way, users can manage their health in an enjoyable way.

[0665] Maintaining Motivation

[0666] Furthermore, the system includes a reward function to maintain and improve user motivation. The server awards points based on increased activity levels and achieved goals, and notifies the user through a character reporting this via the terminal. For example, users who achieve a certain amount of exercise will receive encouraging words from the character such as, "Great! Let's work hard together towards your next goal!"

[0667] In this way, the present invention makes it possible to make users' health management fun and sustainable through characters, and to contribute to extending healthy life expectancy.

[0668] The following describes the processing flow.

[0669] Step 1:

[0670] The user launches the app and accesses an interface for entering data on meals, exercise, and sleep. The device displays this input screen, allowing the user to enter the necessary information.

[0671] Step 2:

[0672] After the user completes the input and confirms the content, they press the submit button. The terminal converts the entered data into a predetermined format (e.g., JSON) and sends it to the server via the internet.

[0673] Step 3:

[0674] The server receives data sent from the terminal and stores it in the database. This data is recorded as an individual user's activity history and is ready for later analysis.

[0675] Step 4:

[0676] The server uses AI to analyze activity data stored in the database. This analysis compares the user's past patterns with current data to generate appropriate health management advice for the user.

[0677] Step 5:

[0678] The server sends the generated advice to the terminal. This prepares the terminal to display a character that will present the advice to the user in a user-friendly way.

[0679] Step 6:

[0680] The device displays character graphics and generated advice on the screen. The character offers specific action suggestions and words of encouragement to the user, providing a trigger for them to take the next step.

[0681] Step 7:

[0682] After receiving advice from the character, users adjust and improve their lifestyle habits based on that advice. Furthermore, users can set their next activity goals by receiving feedback from the app.

[0683] (Example 1)

[0684] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0685] In modern times, many people want to manage their health in a more enjoyable and effective way in their daily lives. However, conventional health management systems have problems with maintaining user motivation and sustained behavior, resulting in monotonous and difficult-to-continue systems. This invention aims to provide a system that allows users to manage their health in an enjoyable and sustainable way.

[0686] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0687] In this invention, the server includes a device for recording user activity information, a device for providing suggestions generated using a generative AI model, a device for presenting suggestions through a character that displays them in a friendly manner, and a device for calculating and notifying points to reinforce user behavior. This enables users to enjoy daily health management and make sustainable improvements to their lives.

[0688] "User activity information" refers to data about the user's daily activities, such as eating, exercising, and sleeping.

[0689] A "recording device" is hardware or software used to collect user activity information and store it as data.

[0690] A "generative AI model" is an artificial intelligence algorithm that analyzes user activity information and generates personalized suggestions.

[0691] A "proposal-providing device" refers to a device or software that presents suggestions generated by a generative AI model to a user.

[0692] "Approachable language" refers to a method of conveying information in a format that is easy for users to understand and that easily captures their interest.

[0693] A "device that uses characters" is a device that displays characters using on-screen animation or speech synthesis technology in order to interact with the user.

[0694] A "device that calculates and notifies points to reinforce behavior" is a system or application that calculates points based on the user's activity record and notifies the user of the results.

[0695] This invention is a system that enables users to manage their health in an enjoyable and sustainable way. The system primarily uses terminals, servers, and characters to manage and analyze user activity information and provide feedback in a user-friendly format.

[0696] The device collects activity information entered by the user. Digital devices and corresponding applications are used, allowing users to manually or through sensor technology input information about their daily diet, exercise, and sleep. For example, steps taken and calories burned can be recorded through a smartphone application. The device organizes this input information and transmits it to a server using a secure communication protocol.

[0697] The server records activity information received from the terminal in a database. The received data is analyzed using a generative AI model. Based on prompts, the generative AI model analyzes the user's health status and generates suggestions for improvement. The model processes prompts such as, "Generate suggestions for health promotion based on the user's exercise data from the past week." The server also calculates points based on the user's activity and prepares rewards to improve motivation.

[0698] The device receives suggestions and reward information generated from the server and displays it on the screen. A friendly character plays a crucial role in this display. This character communicates suggestions to the user in an approachable way through voice and text. For example, the character can encourage the user by saying, "You did a great job today!" and then present the next goal in a fun way. The feedback provided by the character helps maintain the user's motivation and is expected to help them establish healthy lifestyle habits.

[0699] With the above configuration, this invention can provide users with a health management method that combines enjoyment and sustainability in their daily lives.

[0700] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0701] Step 1:

[0702] Users input daily health-related information using a device. This input data includes details of meals, calorie intake, type and duration of exercise, and sleep duration and quality. The device organizes this information and formats it as digital data. Specifically, it collects data via text input or a touch interface and converts it into a digital format such as JSON.

[0703] Step 2:

[0704] The terminal sends the formatted data to the server via a secure communication channel. The input information is transmitted using an encryption protocol (e.g., HTTPS), ensuring data security. The output from the terminal is an encrypted data stream, which the server receives.

[0705] Step 3:

[0706] The server records the received data in a database. The database is designed to store data chronologically. The input here is a data stream from the terminal, which the server converts into a parseable format and records.

[0707] Step 4:

[0708] The server analyzes the data using a generative AI model. This analysis is performed according to the prompt "Generate health promotion suggestions based on the user's exercise data from the past week." The output of the analysis is specific suggestions and advice tailored to the user's health condition.

[0709] Step 5:

[0710] The server sends the generated suggestions to the terminal. Furthermore, it calculates points based on activity data and prepares reward information to improve motivation. The data sent includes the generated advice and the number of points.

[0711] Step 6:

[0712] The device displays information received from the server to the user. During this process, a character appears on the screen and provides advice and reward information to the user in an engaging format. Specifically, it provides feedback to the user through animation and audio playback.

[0713] Step 7:

[0714] Users review the advice and rewards presented through their device and use them to guide their next health behaviors. They then implement new healthy habits based on the advice and prepare for the next data entry. This restarts the system cycle, enabling continuous health management.

[0715] (Application Example 1)

[0716] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0717] In modern society, while health management is becoming increasingly important, many people find it difficult to adequately manage their health amidst their busy lifestyles. In particular, there is a lack of systems that provide appropriate advice tailored to individual lifestyles. Furthermore, there is a need for systems that continuously provide such advice and that allow users to continue managing their health in an enjoyable way.

[0718] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0719] In this invention, the server includes means for recording the user's normal lifestyle data, means for analyzing the lifestyle data, evaluating the user's health status using artificial intelligence, and generating a personalized improvement plan, and means for monitoring the user's lifestyle behavior in real time using a voice recognition device and an image recognition device, and transmitting the collected data to the server. This allows the user to easily understand their own health status and receive appropriate advice. Furthermore, by providing advice through user-friendly information functions, the user can manage their health in an enjoyable way.

[0720] The term "user" refers to an individual who uses the system to manage their health.

[0721] "Normal lifestyle data" refers to data that includes information about the user's daily life, such as diet, exercise, and sleep.

[0722] "Findings" refer to advice and suggestions provided based on analyzed data to improve the user's health.

[0723] An "information function" is a mechanism that presents advice and information to users visually or audibly using friendly characters and display methods.

[0724] "Artificial intelligence" refers to computer programs and algorithms used to analyze a user's lifestyle data and generate personalized health advice.

[0725] A "speech recognition device" is a device that detects a user's speech, digitizes its content, and converts it into a format that the system can understand.

[0726] An "image recognition device" is a device that acquires a user's image information, analyzes it, and converts it into digital data.

[0727] "Reward points" are incentives awarded to users when they achieve their health management goals, and they serve to encourage user behavior.

[0728] This invention is designed as a system that comprehensively supports users' daily lives and promotes health management. The system aims to collect users' lifestyle data in real time and provide appropriate health advice based on that data.

[0729] The server is the core of the health management system. First, the server receives normal lifestyle data collected from the user's terminal. This data includes details of meals, exercise levels, sleep duration, etc. The data is stored in a cloud-based database and analyzed using a generative AI model. This analysis evaluates the user's health status and generates personalized recommendations for lifestyle improvements.

[0730] The terminal functions as an interface between the user and the server and is equipped with speech recognition and image recognition devices. For example, by using common speech recognition software or cloud-based image recognition services, it is possible to input the contents of a meal using voice or images.

[0731] Users are naturally motivated because they are awarded reward points based on their progress towards achieving their daily health goals. This information is provided through user-friendly features, allowing users to enjoy practicing a healthy lifestyle.

[0732] For example, if you speak into the device's microphone after breakfast, saying "I ate yogurt and fruit this morning," the speech recognition will digitize the content and send it to the server. The server will analyze the data and generate a suggestion such as "Try to eat a meal rich in vitamin C today," which will then be displayed on the device's screen through its information function.

[0733] An example of a prompt message might be, "Please evaluate the nutritional balance of the day based on the user's dietary data." In this way, specific and effective health advice based on data can be obtained.

[0734] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0735] Step 1:

[0736] The device acquires information about the user's diet, exercise, and sleep via voice recognition and image recognition. For example, if the user says, "Today's breakfast is eggs and toast," the voice recognition software converts that information into text data. The input is voice or image, and the output is lifestyle data in text format.

[0737] Step 2:

[0738] The terminal formats the converted lifestyle data and sends it to the server. Specifically, it is converted into a specific key-value format, such as "breakfast," "eggs," and "toast." The input is the text data obtained in the previous step, and the output is the formatted data.

[0739] Step 3:

[0740] The server stores received normal life data in a cloud-based database. The data is permanently stored and used for later analysis. The input is formatted data, and the output is the stored data.

[0741] Step 4:

[0742] The server retrieves data from the database and analyzes it using a generative AI model. The analysis evaluates the user's current lifestyle and health status to identify areas for improvement. The input is lifestyle data retrieved from the database, and the output is the analysis results.

[0743] Step 5:

[0744] The server generates personalized findings based on the analysis results. These findings are actionable for the user and provide concrete advice to help improve their life. The input is the analysis results, and the output is the advice, including the findings.

[0745] Step 6:

[0746] The server sends the generated findings to the terminal. The findings are presented to the user through user-friendly information functions. The input is the generated advice, and the output is user-oriented information displayed on the terminal.

[0747] Step 7:

[0748] The user reviews the presented findings and chooses whether or not to perform a health behavior. Based on this choice, the reward function, which is the next step, is activated. The input is the information displayed on the device, and the output is the user's behavioral choice.

[0749] Step 8:

[0750] The server calculates reward points based on the user's action choices and notifies the terminal. For example, if the user achieves their daily goal, it displays the message, "You earned 100 points today!" The input is the user's action choices, and the output is reward information.

[0751] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0752] This invention aims to achieve more personalized interactions by incorporating an emotion engine into a system that supports user activity management. This system collects user activity data and provides appropriate advice, while simultaneously using the emotion engine to recognize the user's emotions and respond accordingly.

[0753] The system mainly consists of terminals, servers, emotion engines, and characters.

[0754] Input and recording of activity data

[0755] Users input data on their daily activities, such as information about meals, exercise, and sleep, via a terminal. The terminal receives, formats, and sends the input to a server. The server records and stores this data in a database for later analysis and advice generation.

[0756] Emotion recognition and analysis

[0757] The device is equipped with an emotion engine that recognizes the user's emotions from their facial expressions, tone of voice, and entered text. This information is sent to a server and analyzed along with the user's activity data. The server's generative AI is used to generate appropriate advice based on these emotions.

[0758] Advice provided by characters

[0759] The generated advice is sent from the server to the terminal. The terminal adjusts the character's expression based on the user's emotions recognized by the emotion engine, providing advice in an appropriate tone and content. For example, if the user is feeling down, the advice will emphasize words of encouragement and be displayed with a cheerful character.

[0760] Motivation enhancement and reward function

[0761] The system has a function that provides rewards based on the user's activity level. The server evaluates the user's emotions and activity data, generates rewards such as points and badges, and sends them to the terminal. The character enjoys and reports these rewards, actively reinforcing the user's behavior.

[0762] For example, if a user engages in active participation and their joyful expression is recognized, the character will offer an additional reward along with a congratulatory message such as, "That's great! Keep it up!" This encourages users to continue participating.

[0763] As described above, the present invention utilizes an emotion engine to support users' lifestyles in a more personalized way, making it possible to continue health management in an enjoyable manner.

[0764] The following describes the processing flow.

[0765] Step 1:

[0766] The user starts up the device and opens the interface for entering daily activity data. They prepare to enter daily information such as meals, exercise, and sleep.

[0767] Step 2:

[0768] The terminal displays an input form, and the user enters detailed activity data (e.g., diet, type of exercise). Once the user has finished entering the data, the terminal formats it into a predetermined format.

[0769] Step 3:

[0770] The formatted data is sent to a server via the internet. The server stores the received data in a database, making it available for subsequent analysis.

[0771] Step 4:

[0772] The device's built-in emotion engine analyzes the user's facial expressions and voice to detect their current emotional state (e.g., joy, sadness). This information is immediately sent to the server.

[0773] Step 5:

[0774] The server integrates activity data and emotional data, and uses generative AI for analysis. It then generates optimal health advice tailored to the user's emotions.

[0775] Step 6:

[0776] The generated advice is sent from the server to the terminal. The terminal adjusts the character's facial expressions and messages based on the received information.

[0777] Step 7:

[0778] The device presents the user with a customized character and advice. For example, if the emotion engine determines that the user is feeling down, the character will offer words of encouragement with a gentle expression.

[0779] Step 8:

[0780] The user reviews the advice provided and incorporates it into their actions. The device records the user's response to help improve future advice.

[0781] Step 9:

[0782] The server provides incentives (e.g., points, badges) through a reward system based on the user's actions and emotions. Users receive these incentives and maintain their motivation.

[0783] (Example 2)

[0784] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0785] In managing users' health and improving their lifestyles, conventional systems struggle to provide advice tailored to individual emotional states and maintain motivation, making it challenging to continuously motivate users. Furthermore, there is a need for a system that can provide advice that considers not only activity data but also emotions.

[0786] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0787] In this invention, the server includes means for receiving user activity information from an input device, formatting it, and storing it on a recording medium; means for analyzing the user's emotions using an emotion recognition device and generating advice using a generative AI model based on that information; and means for displaying the generated advice in an expression method appropriate to the user's emotions and providing it through a friendly character. This enables users to continue working on health management and improving their lifestyle habits by using advice that takes into account their individual emotional state and an appropriate reward system.

[0788] "Activity information" refers to information about the user's actions and behaviors in their daily life, including data on meals, exercise, sleep, etc.

[0789] An "input device" refers to a device used by users to record activity information, such as a smartphone or personal computer.

[0790] A "recording medium" is a device or medium used to store received information, and includes databases and storage systems.

[0791] An "emotion recognition device" is a hardware and software system that analyzes emotions from a user's facial expressions, voice tone, and text data.

[0792] A "generative AI model" is an artificial intelligence model that generates advice in natural language based on received data.

[0793] A "character" is a representation of a virtual person or animal that provides information to the user visually or aurally, and is intended to present advice or rewards in an approachable way.

[0794] "Rewards" refer to incentives such as points or badges provided based on the evaluation of users' activities, and are means of reinforcing user behavior.

[0795] The system of this invention assists in managing user activity and provides feedback based on individual emotional states. The system mainly consists of a terminal, a server, an emotion recognition device, a generative AI model, and a friendly character.

[0796] Users input daily activity information using a terminal. The terminal is equipped with an input device, which is used to input activity information such as meals, exercise, and sleep. The input information is formatted by the terminal and sent to the server for storage on a recording medium. The server stores this data and uses it for later analysis.

[0797] The device is equipped with an emotion recognition device that analyzes the user's emotions from their facial expressions, voice tone, and entered text data. This analysis uses hardware such as a camera and microphone, as well as emotion analysis software. The analyzed emotion data is sent to a server along with activity information, where a generative AI model is used.

[0798] The server uses a generative AI model to generate personalized advice based on the received activity and emotion data. This advice is generated using prompts that correspond to the user's emotional state. For example, a prompt might be "Generate the most appropriate encouraging message based on the user's emotions."

[0799] The generated advice is sent from the server to the terminal, which then displays the advice using a character. The character uses a bright and friendly expression, providing advice that adjusts in tone and content according to the user's emotions. For example, if the user is feeling down, the character will offer encouraging words.

[0800] Furthermore, the server evaluates the user's activity and emotions, generating rewards such as points and badges. These rewards are sent to the device, and a character reports them to the user in an entertaining way. This helps users continue their actions and maintain motivation for health management and lifestyle improvements.

[0801] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0802] Step 1:

[0803] Users operate the terminal to input information about their daily activities. This information includes details about meals, exercise time, and sleep quality. This activity information is received by an input device on the terminal and converted into digital data. The input data is formatted in JSON format and prepared to be sent to the server.

[0804] Step 2:

[0805] The terminal sends formatted activity information to the server. The server receives the data sent via HTTP request and stores it in a database. This stored data is later parsed using SQL queries. In this case, the input is the activity information, and the output is the stored data.

[0806] Step 3:

[0807] The device analyzes the user's emotions using an emotion recognition device. This analysis collects the user's facial expressions and voice using a camera and microphone. This data is analyzed by emotion recognition software and converted into digital data representing the user's emotional state. The resulting emotional data is then formatted and ready to be sent to the server.

[0808] Step 4:

[0809] The device sends formatted emotion data to the server. The server stores the received emotion data in a database. The stored activity information and emotion data are then used by a generative AI model to generate advice. In this process, the input is the emotion data, and the output is the state in which that data is stored in the database.

[0810] Step 5:

[0811] The server generates advice using a generative AI model. This advice generation process uses the prompt "Generate the most appropriate encouraging message based on the user's emotions." The generative AI model receives stored activity information and emotion data as input and generates advice tailored to the user's emotions. The output is the generated advice message.

[0812] Step 6:

[0813] The server sends a generated advice message to the terminal. The terminal displays the received message to the user through a friendly character. The character delivers the advice in a tone that matches the user's emotions. The input is the advice message, and the output is the display of that message.

[0814] Step 7:

[0815] The server evaluates user activity and emotional data and generates rewards. This process generates reward data such as points and badges and sends it to the device. The generated reward data includes incentives that are tailored to the user's level of achievement and emotional state.

[0816] Step 8:

[0817] The device presents the user with reward data generated using a character. The character introduces the rewards in an enjoyable way, encouraging the user to continue their engagement. The input is the reward data, and the output is the visual presentation of that data to the user.

[0818] (Application Example 2)

[0819] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0820] Conventional activity management systems can only provide users with fixed advice, making it difficult to provide appropriate guidance that takes into account individual emotional states. Furthermore, there is a need to achieve more sophisticated interactions in providing advice in a friendly manner and in reward functions that enhance user motivation.

[0821] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0822] In this invention, the server includes a device for recording user behavior data, a device for providing instruction generated based on the behavior data, a device for delivering the instruction through a virtual character that presents the instruction in a friendly manner, and a device for recognizing the user's emotions using an emotion engine and adjusting the instruction content accordingly. This enables flexible and friendly instruction based on the individual emotions of each user.

[0823] "User behavioral data" refers to information about the activities that users engage in in their daily lives, and this includes information such as exercise, diet, and sleep.

[0824] A "device that provides guidance" is a device that has the function of generating and presenting appropriate guidance and advice to the user based on recorded behavioral data.

[0825] A "virtual character" is a visual entity with a personality generated digitally, and its role is to provide information and guidance through interaction with users.

[0826] An "emotion engine" refers to a technology or system that analyzes a user's facial expressions, voice, text input, etc., to recognize the user's emotions.

[0827] A "reward-providing device" is a device that generates and presents points or badges based on the user's behavioral data and emotional state, with the aim of improving the user's motivation.

[0828] This invention realizes a system for collecting and analyzing user activity data and emotions to provide personalized guidance. The system mainly consists of a terminal, a server, an emotion engine, and a virtual character.

[0829] The terminal collects and records data about the user's daily activities, such as information on diet, exercise, and sleep. Users can input this information using mobile devices such as smartphones and tablets. The terminal is responsible for formatting the input information and sending it to the server.

[0830] The server receives activity data provided by the user and stores it in a database. Furthermore, the server is equipped with a generative AI model that analyzes the accumulated behavioral data and emotional information to generate appropriate guidance for the user. In this analysis process, speech synthesis APIs such as Amazon Polly and image recognition APIs such as AWS Rekognition are used to perform emotion recognition from voice input and image data.

[0831] The emotion engine is integrated into the device and instantly identifies emotions from the user's tone of voice, facial expressions, and input text. This information is sent to a server and used to provide specific guidance. The emotion engine helps understand the user's mental state, enabling more effective communication.

[0832] The virtual character receives instructional content from the server and presents it using expressions based on the user's emotions, as recognized by the emotion engine. The character plays a role in conveying information in a visually appealing and easy-to-understand manner for the user.

[0833] Furthermore, the reward system generates and displays points and badges on the device to reinforce user behavior. This allows users to be further motivated by the visual recognition of their efforts.

[0834] For example, if a user sends a message to the robot saying, "I feel great after my workout today," the emotion engine recognizes the user's "joy," and the server generates a response saying, "That's wonderful! Keep it up!" which is then conveyed to the user through a virtual character in an enthusiastic manner. This encourages users to take a more positive stance towards maintaining a healthy lifestyle.

[0835] An example of a prompt to input into the generation AI model is, "Based on my activity data, please generate an appropriate response when the user is happy."

[0836] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0837] Step 1:

[0838] Users input data on their daily activities using devices such as smartphones and tablets. This input includes information such as exercise levels, diet, and sleep duration. The device receives this data, formats it, and sends it to the server. The input data is primarily text-based and is appropriately structured on the device.

[0839] Step 2:

[0840] The server receives activity data sent from terminals and records it in the database. This activity data is used for later analysis and instruction generation, so the server stores it efficiently. Furthermore, data formatting processes remove unnecessary data and correct input errors, creating accurate database entries.

[0841] Step 3:

[0842] The device analyzes the user's voice tone and facial expressions via an emotion engine and generates emotion data. For example, when the user shows their facial expressions using the camera or speaks through the microphone, the device executes an emotion recognition algorithm and sends the results to a server. The input data is then classified into categories such as "joy" or "sadness" by an emotion analysis model within the program.

[0843] Step 4:

[0844] The server uses a generative AI model to generate appropriate guidance based on recorded activity and emotion data. The AI ​​model leverages a large amount of historical data and generates personalized guidance according to defined prompts (e.g., "Generate an appropriate response when the user is happy"). This process utilizes a machine learning model, and optimal guidance is generated through data computation.

[0845] Step 5:

[0846] The server sends the generated instructions to the terminal. The terminal, in accordance with the user's emotions recognized by the emotion engine, provides the instructions visually and audibly through a virtual character. Here, the character displays feedback through animation and sound, providing information in a way that is easy for the user to understand and relate to.

[0847] Step 6:

[0848] The device rewards users with points or badges using a reward system when their activity meets certain criteria. The server evaluates the user's activity and emotional data, generates rewards based on this evaluation, and presents them to the user through the device. This actively reinforces the user's behavior and promotes sustainable motivation.

[0849] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0850] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0851] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0852] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0853] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0854] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0855] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0856] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0857] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0858] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0859] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0860] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0861] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0862] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0863] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0864] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0865] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0866] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0867] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0868] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0869] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0870] The following is further disclosed regarding the embodiments described above.

[0871] (Claim 1)

[0872] A means of recording user activity data,

[0873] A means for providing advice generated based on the aforementioned activity data,

[0874] The means of providing the aforementioned advice through a character that displays it in a friendly manner,

[0875] A system that includes this.

[0876] (Claim 2)

[0877] The system according to claim 1, comprising means for analyzing nutritional information in the activity data and generating advice based thereon.

[0878] (Claim 3)

[0879] The system according to claim 1, comprising means for providing rewards to reinforce user behavior through the aforementioned character.

[0880] "Example 1"

[0881] (Claim 1)

[0882] A device for recording user activity information,

[0883] A device that provides proposals generated using a generative AI model based on the aforementioned activity information,

[0884] A device that implements the aforementioned proposal through a character that displays it in a friendly manner,

[0885] A device that calculates and notifies users of points in order to reinforce their behavior,

[0886] A system that includes this.

[0887] (Claim 2)

[0888] The system according to claim 1, comprising a device that analyzes nutrient information in the activity information and generates suggestions based thereon.

[0889] (Claim 3)

[0890] The system according to claim 1, further comprising a device that provides animated or audio notifications to the user through the aforementioned character.

[0891] "Application Example 1"

[0892] (Claim 1)

[0893] A means of recording the user's normal lifestyle data,

[0894] Means for providing findings generated based on the aforementioned lifestyle data,

[0895] A means of providing the aforementioned findings through an information function that presents them in an accessible manner,

[0896] A means for analyzing the aforementioned lifestyle data, evaluating health status using artificial intelligence, and generating an individualized improvement plan,

[0897] A means for monitoring a user's daily activities in real time using a voice recognition device and an image recognition device, and for transmitting the collected data to a server,

[0898] A means of awarding a certain number of reward points based on the user's achievement status and reporting this to the user through the aforementioned information function,

[0899] A system that includes this.

[0900] (Claim 2)

[0901] The system according to claim 1, comprising means for analyzing nutritional information in the aforementioned lifestyle data and generating findings based thereon.

[0902] (Claim 3)

[0903] The system according to claim 1, comprising means for providing rewards to enhance user behavior through the aforementioned information functions.

[0904] "Example 2 of combining an emotion engine"

[0905] (Claim 1)

[0906] A means for receiving user activity information from an input device, formatting it, and saving it to a recording medium,

[0907] A means of analyzing the user's emotions using an emotion recognition device and generating advice using an AI model based on that information,

[0908] A means of displaying generated advice in an expression that responds to the user's emotions and providing it through a friendly character,

[0909] A means for evaluating user activity information and sentiment analysis results, and for generating and notifying rewards,

[0910] A system that includes this.

[0911] (Claim 2)

[0912] The system according to claim 1, comprising means for generating statistical information based on the activity information and sentiment analysis results and providing it to the user.

[0913] (Claim 3)

[0914] The system according to claim 1, which includes means for enjoying and reporting rewards generated through the aforementioned character, thereby reinforcing the user's behavior.

[0915] "Application example 2 when combining with an emotional engine"

[0916] (Claim 1)

[0917] A device that records user behavior data,

[0918] A device that provides guidance generated based on the aforementioned behavioral data,

[0919] A device that provides the aforementioned instruction through a virtual character that presents it in a friendly manner,

[0920] A device that uses an emotion engine to recognize the user's emotions and adjusts the instruction content accordingly,

[0921] A system that includes this.

[0922] (Claim 2)

[0923] The system according to claim 1, comprising a device that analyzes nutritional information in the aforementioned behavioral data and generates guidance based thereon.

[0924] (Claim 3)

[0925] The system according to claim 1, comprising a device that provides rewards to enhance user behavior through the virtual character. [Explanation of Symbols]

[0926] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of recording the user's normal lifestyle data, Means for providing findings generated based on the aforementioned lifestyle data, A means of providing the aforementioned findings through an information function that presents them in an accessible manner, A means for analyzing the aforementioned lifestyle data, evaluating health status using artificial intelligence, and generating an individualized improvement plan, A means for monitoring a user's daily activities in real time using a voice recognition device and an image recognition device, and for transmitting the collected data to a server, A means of awarding a certain number of reward points based on the user's achievement status and reporting this to the user through the aforementioned information function, A system that includes this.

2. The system according to claim 1, comprising means for analyzing nutritional information in the aforementioned lifestyle data and generating findings based thereon.

3. The system according to claim 1, comprising means for providing rewards to enhance user behavior through the aforementioned information function.

Images