System

Abstract

A system is provided.SOLUTION: A system comprising: means for selecting a person or character trusted by a user; means for displaying the selected person or character as an avatar; means for inputting health data and background information of the user; means for analyzing the input health data and background information to generate health advice; and means for providing the generated health advice to the user through the avatar.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Lifestyle-related diseases are difficult to treat for many patients, and in many cases, treatment is not effective with doctor's instructions alone. Patients are expected to continue treatment voluntarily, but this is not always possible, which is a problem. Increasing medical costs are also a social issue. With conventional systems, it is difficult for patients to receive feedback from trusted individuals or characters and be motivated to improve their lifestyle habits. Against this background, there is a need for a system that encourages patients to make voluntary behavioral changes. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system that allows a user to select a person or character they trust and displays that person's avatar. The system also includes a means for the user to input health data and lifestyle background information, analyze the data, and generate health advice. Providing the generated advice to the user through the avatar increases the user's motivation to voluntarily change their behavior. By adding a means for storing and analyzing health data and lifestyle background information on a server, more accurate advice can be provided. Furthermore, the avatar can converse with the user in real time and provide appropriate feedback, supporting sustainable health management. This can improve the effectiveness of treatment for lifestyle-related diseases and contribute to reducing medical costs.

[0006] "User" refers to a person who uses this system to input health data and receive advice from an avatar.

[0007] A "trusted person" refers to a real person or fictional character in whom a user places personal trust.

[0008] A "character" is a fictional entity that a user can trust and is displayed in the system as an avatar.

[0009] "Avatar" refers to a digital representation of a person or character chosen by the user and trusted by the user, who interacts with the user through the system.

[0010] "Health data" refers to health-related information (e.g., blood pressure, weight, amount of exercise, dietary details, etc.) that a user records on a daily basis.

[0011] "Lifestyle background information" refers to information such as the user's lifestyle habits, preferences, and daily rhythm.

[0012] "Analysis" refers to the act of evaluating and analyzing a user's behavioral patterns based on health data and lifestyle background information collected by the server.

[0013] "Health advice" refers to instructions or suggestions generated based on the analysis results and provided to the user to maintain or improve their health.

[0014] "Providing" refers to the act of showing health advice to a user through an avatar.

[0015] "Server" refers to a central device that stores health data and lifestyle background information, analyzes them, and generates advice.

[0016] "Saving" refers to the act of recording data entered by a user in a database.

[0017] "Conversation" refers to the act of an avatar exchanging words and information with a user in real time.

[0018] "Feedback" refers to appropriate reactions and advice provided by an avatar in response to a user's actions and input data. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention is a system for promoting behavioral change in patients with lifestyle-related diseases. The system allows users to set a person or character they trust as an avatar, and aims to encourage users to improve their lifestyle habits based on health advice provided by the avatar.

[0041] User Action

[0042] First, users install the app and create an account. They choose a person or character they trust and set their avatar. Then, they set their health goals and enter information about their personality and lifestyle.

[0043] Next, users input their daily health data (e.g., blood pressure, weight, exercise, and diet) into the application. Through conversations with the avatar, they receive specific health advice. Based on this advice, users can make changes to their daily behavior.

[0044] Terminal handling

[0045] The device transmits the user's account information and health data to the server, receives the data entered by the user in real time and uploads it to the server, and interacts with the avatar to provide health advice and feedback to the user.

[0046] Server Processing

[0047] The server stores and analyzes the user's account information and daily health data. The server uses an AI model to generate advice based on the user's health data and lifestyle information. This advice is sent to the device in real time and provided to the user through an avatar.

[0048] Specifically, when a user measures their blood pressure and enters the results into the application, the device sends the data to the server. The server analyzes this data and generates advice, such as "Your blood pressure is high. Try some light aerobic exercise." This advice is sent to the device and displayed to the user via an avatar.

[0049] Specific examples

[0050] User: Mr. Tanaka (50 years old, male), Trusted person: Mr. Yamakawa, a sports instructor

[0051] 1. Tanaka installs the application, creates an account, and sets Yamakawa as his avatar.

[0052] 2. Mr. Tanaka set a goal of "keeping his blood pressure below 120 / 80 mmHg" and entered his lifestyle, which mainly involves desk work.

[0053] 3. Tanaka measures his blood pressure every morning and enters the results into the application. He also records his diet and exercise.

[0054] 4. The avatar, Mr. Yamakawa, offers advice: "You should exercise a little more."

[0055] 5. Tanaka follows the advice and starts walking daily, continuing while monitoring fluctuations in his blood pressure.

[0056] As described above, users can voluntarily review their behavior and make sustainable lifestyle improvements. This system is expected to improve the effectiveness of treatments for lifestyle-related diseases and contribute to reducing medical costs.

[0057] The processing flow will be explained below.

[0058] User Action

[0059] Step 1:

[0060] The user installs the application and launches the app.

[0061] Step 2:

[0062] The user enters their name, email address, and password on the account creation screen.

[0063] Step 3:

[0064] The user selects a person or character they trust and sets their avatar.

[0065] Step 4:

[0066] The user inputs health goals (e.g., blood pressure goal, weight loss goal).

[0067] Step 5:

[0068] The user enters information about their personality and background.

[0069] Step 6:

[0070] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details).

[0071] Step 7:

[0072] The user converses with the avatar and receives specific health advice from the avatar.

[0073] Step 8:

[0074] Users can view their progress and set new goals in the app.

[0075] Terminal handling

[0076] Step 1:

[0077] The terminal transmits the user's account information to the server.

[0078] Step 2:

[0079] The terminal stores the avatar information selected by the user.

[0080] Step 3:

[0081] The terminal transmits the health data entered by the user to the server.

[0082] Step 4:

[0083] The advice received by the terminal from the server is displayed as an avatar.

[0084] Step 5:

[0085] The device uploads the conversation log with the avatar to the server.

[0086] Server Processing

[0087] Step 1:

[0088] The server stores the received user account information in a database.

[0089] Step 2:

[0090] The server stores the user's selected avatar information in a database.

[0091] Step 3:

[0092] The server analyzes the health data entered by the user.

[0093] Step 4:

[0094] The server generates health advice based on the analysis results.

[0095] Step 5:

[0096] The server sends the generated advice to the terminal.

[0097] Step 6:

[0098] The server stores the conversation log with the avatar in a database.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] Conventional lifestyle-related disease management systems simply record users' health data and do not adequately support users in making specific behavioral changes. As a result, users find it difficult to grasp the specific steps they need to take to achieve their health goals, which hinders the improvement of lifestyle-related diseases.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes means for selecting a character trusted by the user, means for displaying the selected character as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information and generating health advice using an AI model, and means for providing the generated health advice to the user via the avatar, thereby enabling the user to easily understand specific behaviors and voluntarily change their behavior.

[0104] A "user" is an individual who uses this system and inputs health data and lifestyle information and receives advice from an avatar.

[0105] A "character" is a person or fictional entity that a user trusts, displayed as an avatar, and whose role is to provide health advice through interactions with the user.

[0106] An "avatar" is a virtual representation of a character that a user trusts, and serves as a means of providing health advice to the user.

[0107] "Health data" refers to health-related information such as blood pressure, weight, amount of exercise, and dietary details that a user inputs into the application.

[0108] "Lifestyle background information" is data that may affect the achievement of health goals, such as the user's occupation, lifestyle, and home environment.

[0109] The "server" is a central computer system that analyzes users' health data and lifestyle background information and generates health advice using AI models.

[0110] An "AI model" is an artificial intelligence algorithm that analyzes a user's health data and lifestyle background information and generates appropriate health advice based on that data.

[0111] "Health advice" is specific instructions and suggestions for improving health that are generated by the server using an AI model and provided to the user through an avatar.

[0112] "Interaction" refers to a two-way exchange between the user and the avatar in real time, including advice and feedback in response to the user's questions.

[0113] The present invention provides a system in which a user sets a character that the user trusts as an avatar and improves their lifestyle based on health advice provided by the avatar. An embodiment of this system will be described in detail below.

[0114] User Action

[0115] First, users install the dedicated application and create an account. This application is installed on a mobile device such as a smartphone or tablet. The user selects a trusted character and sets that avatar. Next, the user sets health goals and enters information about their personality and lifestyle.

[0116] As a concrete example, consider a case where a user sets a goal of "keeping blood pressure below 120 / 80 mmHg" and leads a desk-based lifestyle. The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, diet) into the application. The user also receives specific health advice through conversations with an avatar. Based on this advice, the user can change their behavior in their daily life.

[0117] Terminal handling

[0118] The device transmits the user's account information and health data to the server in real time, allowing the server to receive the latest health data and analyze it immediately. The device also provides an interface for interacting with the avatar and providing health advice and feedback to the user.

[0119] Server Processing

[0120] The server stores and analyzes the user's account information and daily health data. A generative AI model (e.g., a machine learning algorithm) is used for the analysis. Real-time advice based on the user's health data and lifestyle information is generated and sent to the device.

[0121] Specifically, when a user measures their blood pressure and enters the results into the application, the device sends the data to the server. The server analyzes the data and generates advice such as "Your blood pressure is high. Try some light aerobic exercise." This advice is sent to the device and displayed to the user via an avatar.

[0122] Specific examples

[0123] A middle-aged man installs the application and creates an account. He selects the character of a trusted sports instructor as his avatar. He then sets a goal of "keeping his blood pressure below 120 / 80 mmHg" and enters his desk-bound lifestyle. Every morning, he measures his blood pressure and enters the results into the application. He also records his diet and exercise. The avatar offers specific advice, such as "Try exercising a little more," and he begins walking daily. After a few weeks, he can see that his blood pressure is gradually approaching his target value.

[0124] This system encourages users to make specific behavioral changes and make sustainable lifestyle improvements. By utilizing generative AI models, it is possible to provide personalized health advice and effectively support users' health management.

[0125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0126] Step 1: Initial User Setup

[0127] The user installs a dedicated application and creates an account, selects a trusted character, and configures its avatar. Input includes a username, password, and the selection of a trusted character. Output is account information and avatar configuration information. Specific actions include character selection and goal setting using drop-down menus and sliders.

[0128] Step 2: Set your health goals

[0129] The user sets health goals and inputs information about their personality and lifestyle. The input includes information such as blood pressure goals, exercise goals, diet, and living environment. The output is the user's goals and lifestyle information. Specific actions involve filling out a form and operating a slider to set the goal value.

[0130] Step 3: Enter your health data

[0131] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details) into the application. The input includes daily blood pressure, dietary details, exercise time, etc. The output is the input health data. Specific operations include measuring blood pressure and inputting the results into the application. The application also records meals and exercise.

[0132] Step 4: Sending data

[0133] The device transmits the health data entered by the user to the server in real time. At this time, the input data is encrypted before transmission. The input includes the health data entered by the user. The output is the data transmitted to the server. Specifically, the data is transmitted via the Internet, and a notification of successful transmission is displayed to the user.

[0134] Step 5: Save and analyze data

[0135] The server stores the received health data and analyzes it using a generative AI model. The input includes the health data sent from the device. The output is a trend analysis based on the analysis results and the user's health status. Specifically, the data is stored in a database and analyzed using machine learning algorithms.

[0136] Step 6: Generating Advice

[0137] The server generates health advice using an AI model based on the analysis results. The input includes the analysis results of the health data. The output is personalized health advice. Specifically, the AI ​​model generates appropriate health advice for each user in text format.

[0138] Step 7: Providing advice

[0139] The terminal provides the user with health advice received from the server. The input includes the health advice sent from the server. The output is the health advice displayed to the user through an avatar. Specifically, the avatar presents the advice to the user in text or voice on the application screen.

[0140] Step 8: Changing user behavior

[0141] The user changes their daily behavior based on advice from the avatar. The input includes health advice from the avatar. The output is a specific change in the user's behavior, which is expected to improve their health. As a specific action, the user performs and continues specific health behaviors such as walking or dietary restrictions.

[0142] (Application example 1)

[0143] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0144] Currently, there are several health management systems on the market for patients with lifestyle-related diseases, but few systems include specific dietary suggestions to comprehensively improve users' lifestyles. As a result, lifestyle-related disease prevention and management are often ineffective. Furthermore, conventional systems have the problem of difficulty in providing real-time dietary suggestions linked to health data. Conventional technologies make it difficult for users to select appropriate diets based on their own health data, which hinders lifestyle improvement.

[0145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0146] In this invention, the server includes means for selecting a person or character trusted by the user, means for displaying the selected person or character as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information and generating health advice, means for providing the generated health advice to the user via the avatar, and means for suggesting an appropriate diet based on the generated health advice, thereby enabling the user to select an appropriate diet based on their own health condition and enabling more effective prevention and management of lifestyle-related diseases.

[0147] "User" refers to an individual who uses the system.

[0148] A "trusted person or character" refers to a real person or virtual character with whom a user feels a sense of affinity or trust.

[0149] An "avatar" is a digital representation of a person or character that a user trusts, and refers to a virtual presence that is used to interact with the user on the system.

[0150] "Health data" refers to information about a user's health status, such as physiological values ​​such as blood pressure, weight, and blood sugar levels, and diagnostic results.

[0151] "Lifestyle background information" refers to information about the user's living environment and habits that affect health, such as the user's occupation, daily activities, and eating habits.

[0152] "Analysis" refers to the process of evaluating a user's health status and identifying areas for improvement and risks based on the health data and lifestyle background information provided by the user.

[0153] "Health Advice" refers to specific advice or suggestions for improving or maintaining a user's health, generated based on the analysis.

[0154] "Means for suggesting meals" refers to a system that analyzes a user's health data and lifestyle background information and provides appropriate meal content and nutritional balance based on the results.

[0155] To implement the present invention, the following system configuration is required.

[0156] User Action

[0157] The first thing a user does is install a dedicated health management application on their smartphone. The user creates an account and selects a person or character they trust. For example, if the user selects a health and sports advisor as a trusted person, that person will appear as an avatar. The user then sets their own health goals and enters background information about their lifestyle, such as their occupation and eating habits. They enter their daily health data (e.g., blood pressure, weight, amount of exercise, and dietary details) into the app and receive health advice through interactions with the avatar.

[0158] Terminal handling

[0159] The program running on the device sends the user's account information and health data to a server in real time, and provides feedback to the user based on the analysis results returned from the server. An avatar within the application interacts with the user, offering advice such as, "Your blood pressure is high today. We recommend you eat a diet low in carbohydrates."

[0160] Server Processing

[0161] The server is built using programming languages ​​such as Python and stores and analyzes users' account information and daily health data. A generative AI model is used to analyze the health data, and appropriate health advice is generated based on the analysis results. Health advice and dietary suggestions based on the analysis are sent to the device in real time and provided to the user through an avatar.

[0162] Specifically, for example, when a user inputs their blood glucose level and BMI, the following prompt sentence is parsed by the generative AI model:

[0163] text

[0164] My blood sugar level is 150 mg / dL and my BMI is 26. What are some dietary suggestions and advice based on this health condition?

[0165] The generative AI model then generates appropriate advice and dietary suggestions, such as "Your blood sugar level is high, so eat a diet low in carbohydrates."

[0166] In this way, users can receive specific advice based on their own health status and are encouraged to take practical actions to prevent and manage lifestyle-related diseases. This system encourages users to autonomously change their health management behavior and enable them to maintain a healthier lifestyle.

[0167] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0168] Step 1:

[0169] The user installs a dedicated health management application on their smartphone and creates an account. They then select a trusted person or character to set up their avatar. During this initial setup step, the user also enters their health goals and lifestyle background information (e.g., occupation and dietary habits). The input data is collected by the device and sent to the server. The input for this step is the user's profile information and lifestyle background information, and the output is user profile data that stores this information.

[0170] Step 2:

[0171] The user inputs daily health data (e.g., blood pressure, weight, exercise, and dietary information) into the application. The device transmits this data to the server in real time. The input data is health-related information, and the output data is updated health data that is uploaded to the server.

[0172] Step 3:

[0173] The server performs data analysis based on the received health data and user profile. First, it requests an analysis by passing a prompt to the generative AI model. As a specific example, it generates a prompt such as, "Your blood pressure is 150 / 90 mmHg and your BMI is 28. Please give me some dietary suggestions and advice based on these health conditions." The input to this prompt is the health data to be analyzed on the server, and the output is the health advice generated as a result of the analysis.

[0174] Step 4:

[0175] The generative AI model analyzes the prompt text and generates appropriate health advice and dietary suggestions. For example, the generated advice might be, "Your blood pressure is high, so eat a low-salt diet." The input data is the prompt text, and the output data is the generated advice.

[0176] Step 5:

[0177] The server sends the generated health advice to the device. The device then provides the received advice to the user. An avatar within the application provides the advice in the form of voice or text through dialogue with the user. For example, the advice might be displayed in the form of "Your blood pressure is high today. We recommend a low-carbohydrate diet." The input of this step is the generated advice, and the output is feedback advice for the user.

[0178] Step 6:

[0179] The user takes action to improve their lifestyle habits according to the advice provided by the avatar. The user then enters the results into the app again to check the progress of their improvement. This data is also sent from the device to the server and reflected in the next analysis. The input for this step is new health data, and the output data is updated health data and the progress of improvement.

[0180] Through this series of processing steps, users receive specific advice based on their own health condition, enabling them to effectively improve their lifestyle habits.

[0181] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0182] This invention is a system for promoting behavioral change in patients with lifestyle-related diseases, which displays people or characters trusted by the user as avatars and provides health advice generated by analyzing the user's health data and lifestyle background information.The invention also includes an emotion engine that recognizes the user's emotions and responds appropriately according to the recognized emotions, thereby increasing the user's motivation and supporting sustainable health management.

[0183] User Action

[0184] Users install a dedicated application and create an account. When creating an account, they enter the required information (e.g., name, email address, password) and select a trusted person or character to set as their avatar. Next, users set health goals and enter information about their personality and lifestyle. After that, they enter daily health data (e.g., blood pressure, weight, amount of exercise, dietary details). This data is managed within the application, and users receive specific health advice through conversations with their avatar.

[0185] Furthermore, this invention incorporates an emotion engine that can recognize emotions through the user's voice input and facial expression analysis. Based on the recognized emotions, the avatar provides the user with appropriate health advice and words of encouragement. This motivates the user to take better care of themselves and encourages them to continue managing their health.

[0186] Terminal handling

[0187] The device sends the user's account information and health data to a server. The data entered by the user is uploaded to the server in real time. The device also interacts with the avatar and provides the user with health advice and feedback. At the same time, an emotion engine monitors the user's voice and facial expressions to recognize emotions. This information is also sent to the server and used for analysis.

[0188] Server Processing

[0189] The server stores and analyzes the user's account information and daily health data. Using an AI model, the server generates health advice based on the user's health data and lifestyle background information. It also analyzes emotional data recognized by an emotion engine and generates appropriate feedback based on the user's feelings. The generated advice and feedback are sent to the device and provided in real time via an avatar.

[0190] Specific examples

[0191] User: Ms. Sato (45 years old, female), Trusted person: Nutritionist Ms. Yamada

[0192] 1. Mr. Sato installs the application, creates an account, and sets his trusted nutritionist, Ms. Yamada, as his avatar.

[0193] 2. Mr. Sato sets a goal of "losing 5 kg" and enters his lifestyle information (mainly desk work, mostly eating out).

[0194] 3. Mr. Sato records his daily weight and dietary habits in the application.

[0195] 4. The server analyzes this data and generates advice such as, "We recommend that you change to a balanced diet and take a walk after dinner."

[0196] 5. The avatar Yamada provides advice based on the analysis results and encourages Sato to act.

[0197] 6. If Sato-san shows a depressed expression, the emotion engine will recognize this and Yamada-san's avatar will offer words of encouragement such as, "You're doing well, if you keep going you'll see results."

[0198] In this way, the system of the present invention allows users to receive accurate advice from trusted individuals, enabling them to continuously improve their lifestyle habits. Furthermore, receiving feedback based on their emotions can increase motivation for behavioral change.

[0199] The processing flow will be explained below.

[0200] User Action

[0201] Step 1:

[0202] The user installs the application and launches the app.

[0203] Step 2:

[0204] The user enters their name, email address, and password on the account creation screen.

[0205] Step 3:

[0206] The user selects a person or character they trust and sets their avatar.

[0207] Step 4:

[0208] The user inputs health goals (e.g., blood pressure goal, weight loss goal).

[0209] Step 5:

[0210] The user enters information about their personality and background.

[0211] Step 6:

[0212] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details).

[0213] Step 7:

[0214] The user converses with the avatar and receives specific health advice from the avatar.

[0215] Step 8:

[0216] The device's camera and microphone recognize the user's voice input and facial expressions to collect emotional information.

[0217] Step 9:

[0218] Users can view their progress and set new goals in the app.

[0219] Terminal handling

[0220] Step 1:

[0221] The terminal transmits the user's account information to the server.

[0222] Step 2:

[0223] The terminal stores the avatar information selected by the user.

[0224] Step 3:

[0225] The terminal transmits the health data entered by the user to the server.

[0226] Step 4:

[0227] The advice received by the terminal from the server is displayed as an avatar.

[0228] Step 5:

[0229] The terminal processes the user's voice input and facial expressions with an emotion engine, and transmits the emotion information to the server.

[0230] Step 6:

[0231] The device uploads the conversation log with the avatar to the server.

[0232] Server Processing

[0233] Step 1:

[0234] The server stores the received user account information in a database.

[0235] Step 2:

[0236] The server stores the user's selected avatar information in a database.

[0237] Step 3:

[0238] The server analyzes the health data entered by the user.

[0239] Step 4:

[0240] The server generates health advice based on the analysis results.

[0241] Step 5:

[0242] Based on the emotional information received from the emotion engine, the server modifies the analysis results and generates advice and feedback according to the emotion.

[0243] Step 6:

[0244] The server sends generated advice and feedback to the terminal.

[0245] Step 7:

[0246] The server stores the conversation log with the avatar in a database.

[0247] Specific examples

[0248] User: Ms. Sato (45 years old, female), Trusted person: Nutritionist Ms. Yamada

[0249] User Steps

[0250] Step 1:

[0251] Mr. Sato installs the application and launches it.

[0252] Step 2:

[0253] On the account creation screen, Sato enters his name, email address, and password.

[0254] Step 3:

[0255] Sato sets Yamada, a nutritionist he trusts, as his avatar.

[0256] Step 4:

[0257] Sato enters his health goal of "lose 5 kg."

[0258] Step 5:

[0259] Sato enters background information about his lifestyle, such as the fact that he mainly does desk work and eats out most of the time.

[0260] Step 6:

[0261] Sato enters his daily weight and dietary information into the application.

[0262] Step 7:

[0263] Talk to the avatar Yamada and receive health advice.

[0264] Step 8:

[0265] Sato's voice and facial expressions are recognized by the device's emotion engine, and emotional information is collected.

[0266] Step 9:

[0267] Sato checks the progress of his goals and sets new goals in the app.

[0268] Terminal Steps

[0269] Step 1:

[0270] The device sends Sato's account information to the server.

[0271] Step 2:

[0272] The device saves the avatar information selected by Sato.

[0273] Step 3:

[0274] The device sends the health data entered by Sato to the server.

[0275] Step 4:

[0276] The advice received by the terminal from the server is displayed as an avatar.

[0277] Step 5:

[0278] The device's emotion engine processes Sato's voice input and facial expressions, and sends the emotional information to the server.

[0279] Step 6:

[0280] The device uploads the conversation log with the avatar to the server.

[0281] Server Steps

[0282] Step 1:

[0283] The server stores Mr. Sato's account information in a database.

[0284] Step 2:

[0285] The server saves Sato's selected avatar information in a database.

[0286] Step 3:

[0287] The server analyzes the health data entered by Sato.

[0288] Step 4:

[0289] The server generates appropriate health advice based on the analysis results.

[0290] Step 5:

[0291] The server modifies the analysis results based on the emotional information from the emotion engine and generates advice and feedback according to the emotion.

[0292] Step 6:

[0293] The server sends generated advice and feedback to the terminal.

[0294] Step 7:

[0295] The server stores the conversation log with the avatar in a database.

[0296] Example 2

[0297] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0298] In modern society, lifestyle-related diseases are on the rise, making their prevention and management a critical issue. Patients with lifestyle-related diseases, who are known to have difficulty managing their conditions themselves, are particularly required to receive appropriate health management advice on an ongoing basis while maintaining their motivation. However, there are very few existing systems that provide customized advice tailored to the user's psychological state and individual lifestyle. Furthermore, there is a lack of systems that recognize the user's emotions in real time and provide appropriate feedback based on that information. This poses a challenge, making it difficult for users to motivate themselves to continuously manage their health.

[0299] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0300] In this invention, the server includes means for analyzing a user's health data and lifestyle background information to generate health advice, means for acquiring the user's voice input and facial expression data to analyze emotions, and means for providing feedback based on the analyzed emotions via the avatar, thereby making it possible to provide health advice and feedback customized to the user's individual lifestyle background and psychological state in real time.

[0301] A "user" is an individual who uses this system to manage their health.

[0302] A "trusted person or character" is a virtual person or character that a user psychologically trusts and that is displayed when receiving health advice.

[0303] An "avatar" is a virtual representation of a trusted person or character, and is a way to provide health advice and feedback to users.

[0304] "Health data" refers to information about a user's physical condition (e.g., weight, blood pressure, amount of exercise, dietary details).

[0305] "Lifestyle background information" is information about the user's lifestyle and environment (e.g., occupation, eating habits, sleeping habits).

[0306] "Analysis" is the process of generating appropriate health advice based on the input health data and lifestyle background information.

[0307] "Health advice" refers to health recommendations or instructions provided to the user based on the analysis results.

[0308] "Voice input" refers to voice information uttered by the user through the microphone of the terminal.

[0309] "Facial expression data" is information about facial expressions that express the user's emotions.

[0310] "Means for analyzing emotions" refers to technology for recognizing and analyzing a user's emotional state based on voice input and facial expression data.

[0311] "Feedback" is a response, such as encouragement or instruction, provided to the user based on the analyzed emotional state.

[0312] The "server" is a central processing unit that receives, stores, and analyzes health data, lifestyle background information, and emotional data, and then generates advice and feedback and sends it to the terminal.

[0313] "Real-time" is a concept that refers to the state in which input information is processed and reflected immediately without delay.

[0314] This invention is a system that analyzes a user's unique health data and lifestyle background information and provides customized health advice through a trusted avatar. It also analyzes the user's voice input and facial expression data and provides feedback according to their emotions, supporting continuous health management.

[0315] System configuration

[0316] The system mainly consists of the following parts:

[0317] 1. User Device

[0318] 2. Server

[0319] 3. Avatar display method

[0320] 4. Voice input and facial expression recognition

[0321] User device functions

[0322] User devices are smart devices (e.g., smartphones, tablets) on which users install and use applications. These devices use the following hardware and software:

[0323] Camera and microphone: Captures your facial expressions and voice.

[0324] Application software (e.g. iOS app, Android app): Enters health data and interacts with the avatar.

[0325] Network communication function: Sends user data to the server and receives data from the server.

[0326] Server Features

[0327] The server acts as a central processing unit and is responsible for:

[0328] Database (e.g. MySQL): Stores user health data and lifestyle information.

[0329] AI module (e.g., Python, TensorFlow): Analyzes health data and lifestyle background information and generates health advice.

[0330] Emotion engine: Analyzes the user's voice input and facial expression data to generate emotion-based feedback.

[0331] Avatar display method

[0332] The avatar display unit displays a trusted person or character selected by the user as a virtual avatar, which provides real-time health advice and emotional feedback to the user.

[0333] Example of operation

[0334] 1. User creates an account:

[0335] Users install the application and create an account by entering basic information such as their name, email address, and password. They then select a trusted person or character to set as their avatar.

[0336] 2. Enter your health goals and lifestyle information:

[0337] The user inputs background information such as their weight loss goal, occupation, diet, etc. For example, they can set a goal such as "lose 5 kg."

[0338] 3. Enter your daily health data:

[0339] Users record their daily health data, such as weight, exercise, and diet, in the application, which is then sent to the server in real time.

[0340] 4. Health advice generation:

[0341] The server analyzes the received health data and generates appropriate health advice, such as "We recommend that you change to a balanced diet and take a walk after dinner."

[0342] 5. Offer advice through an avatar:

[0343] The device receives health advice from the server and provides it to the user through an avatar, which communicates the advice both audibly and visually.

[0344] 6. Sentiment analysis and feedback provision:

[0345] The system analyzes the user's voice input and facial expression data to generate appropriate feedback based on their emotions. For example, if the user is feeling down, the avatar will provide an encouraging message such as, "You're doing well. If you keep going, you'll see results."

[0346] Prompt Sentence Examples

[0347] "I'm a 45-year-old woman who works a desk job. I eat out a lot and would like to lose 5kg. What health advice would you give me?"

[0348] In this way, the present invention is a system that provides health management tailored to the individual needs of the user and supports the formation of sustainable healthy habits.

[0349] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0350] Step 1:

[0351] A user creates an account and enters basic information.

[0352] Enter your name, email address, and password.

[0353] Specific operation: The user installs the application and taps the "New Registration" button. They enter their name, email address, and password in the form displayed on the screen and press the "Register" button.

[0354] Output: The user account information is sent to the server and the account is created.

[0355] Step 2:

[0356] Users select an avatar they trust and set their health goals and lifestyle background information.

[0357] Input: Select a trusted person or character, health goals, and background information (e.g., occupation, diet).

[0358] How it works: After creating an account, users are directed to an avatar selection screen where they can tap to select a trusted person or character. Next, they enter their health goal (e.g., "lose 5kg") and select background information from options.

[0359] Output: Avatar settings, health goals, and lifestyle background information are sent to the server.

[0360] Step 3:

[0361] The user inputs daily health data (weight, amount of exercise, diet, etc.).

[0362] Input: Daily health data such as weight, exercise, and diet.

[0363] Specific actions: The user accesses the health data entry screen and uses sliders and text fields to enter weight, exercise volume, dietary information, etc. Then taps the "Save" button.

[0364] Output: The entered health data is sent to the server and stored.

[0365] Step 4:

[0366] The server stores and analyzes health data and lifestyle background information.

[0367] Input: Health data, lifestyle background information.

[0368] How it works: The server stores the data in a MySQL database and uses Python and TensorFlow to analyze it, using algorithms to generate health advice.

[0369] Output: The generated health advice.

[0370] Step 5:

[0371] The server sends the generated health advice to the terminal.

[0372] Input: Generated health advice.

[0373] Specific operation: The server sends the analysis results, which are health advice, in JSON format to the terminal.

[0374] Output: The device receives the health advice.

[0375] Step 6:

[0376] The device provides advice to the user through an avatar.

[0377] Enter: health advice.

[0378] Specific operation: The device opens the avatar display screen, and the avatar conveys the received advice to the user audibly and visually. An animation is played, and the avatar says, "I recommend you change to a balanced diet and take a walk after dinner."

[0379] Output: The user receives the health advice.

[0380] Step 7:

[0381] The user expresses their emotions through voice and facial expressions.

[0382] Input: Voice input, facial expression data.

[0383] Specific actions: The user looks at the device's camera, makes facial expressions, and speaks into the microphone.

[0384] Output: Voice and facial expression data are input to the device.

[0385] Step 8:

[0386] The device transmits the emotion data to the server.

[0387] Input: speech and facial expression data.

[0388] Specific operation: The device transmits the captured emotion data to the server in real time.

[0389] Output: Emotion data is sent to the server.

[0390] Step 9:

[0391] The server analyzes the emotional data and generates appropriate feedback.

[0392] Input: Emotion data.

[0393] Specific operation: The server analyzes the received voice and facial expression data using an emotion engine to recognize the user's emotional state, and generates appropriate feedback based on the results.

[0394] Output: The generated feedback.

[0395] Step 10:

[0396] The device provides feedback to the user through an avatar.

[0397] Input: Generated feedback.

[0398] Specific operation: The device reflects the feedback on the avatar display screen and plays an animation in which the avatar says, "You're doing well. If you keep at it, you'll definitely see results."

[0399] Output: The user receives feedback.

[0400] (Application example 2)

[0401] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0402] In modern society, the prevention and management of lifestyle-related diseases is an important issue. However, many patients find it difficult to maintain motivation for self-management and do not have easy access to professional health advice. Furthermore, in order to receive regular health checks and lifestyle improvement methods, patients themselves must habitually record their health data and properly analyze it. This process is complex, making sustained management difficult, which presents a challenge.

[0403] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for displaying a person or character trusted by the user as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information to generate health advice, means for providing the user with the health advice generated through the avatar, and means having an emotion engine for analyzing the user's emotions in real time and generating appropriate feedback based on the analysis. This enables the user to continuously manage their health based on advice from the trusted avatar, and to maintain motivation through appropriate feedback according to their emotions.

[0404] A "trusted person or character" is a being that a user psychologically trusts, and is a virtual figure that is set up to provide health advice or increase motivation.

[0405] An "avatar" is a visual or audio representation of a trusted person or character that interacts with the user to provide health advice.

[0406] "Health data" refers to physical information accumulated by a user on a daily basis, specifically data including weight, blood pressure, amount of exercise, dietary details, and the like.

[0407] "Lifestyle background information" refers to information about the user's daily life, and specifically includes data such as occupation, diet, exercise habits, and home environment.

[0408] An "emotion engine" is a software or hardware mechanism that analyzes emotions from a user's facial expressions and voice and generates appropriate feedback based on the results.

[0409] "Analysis" refers to data processing to generate health advice using algorithms and AI technology based on input health data and lifestyle background information.

[0410] "Feedback" refers to appropriate responses and words of encouragement generated based on the user's emotions analyzed by the emotion engine.

[0411] A "server" refers to a computer system that stores and analyzes input data, and this system functions in conjunction with the user's terminal via a network.

[0412] This invention is a system that allows users to prevent and manage lifestyle-related diseases in a physical store. This system displays people or characters that the user trusts as avatars and provides health advice generated by analyzing the user's health data and lifestyle background information. It also has an emotion engine that recognizes the user's emotions in real time and provides appropriate feedback to increase the user's motivation and support continuous health management.

[0413] Hardware and Software

[0414] The user interacts with the device using a smartphone or smart glasses. These devices are equipped with a camera and microphone to capture the user's facial expressions and voice in real time. The camera detects the user's face using OpenCV and analyzes emotions using TensorFlow. Speech recognition is performed using the speech_recognition library.

[0415] Smartphone or smart glasses: A device that has a built-in camera and microphone and is used to run applications.

[0416] OpenCV: An image processing library used to detect faces in camera footage.

[0417] TensorFlow: A machine learning framework that analyzes user emotions using emotion recognition models.

[0418] speech_recognition: A library for speech recognition, converting user speech into text.

[0419] System processing flow

[0420] The server receives, stores, and analyzes the user's health data and lifestyle information. Data entered by the user through a dedicated application is uploaded to the server in real time. The emotion engine also recognizes the user's emotions from facial expressions and voice, which are also sent to the server for analysis.

[0421] Specifically, the following steps are performed:

[0422] 1. Data Collection:

[0423] Users use a smartphone or smart glasses to input health data (weight, blood pressure, exercise, diet, etc.) and lifestyle background information.

[0424] Using a camera and microphone, the user's facial expressions and voice data are also captured in real time.

[0425] 2. Data Analysis:

[0426] The server receives and stores this data.

[0427] The AI ​​model generates health advice based on health data and lifestyle background information.

[0428] The emotion engine analyzes facial and voice data to identify the user's emotional state.

[0429] 3. Providing advice:

[0430] The avatar provides generated health advice to the user.

[0431] At the same time, the avatar provides appropriate feedback and words of encouragement based on the results of emotion analysis.

[0432] Specific examples

[0433] For example, suppose a user says, "I've been finding it difficult to manage my diet lately. Do you have any simple advice?" This voice input is converted into text using the speech_recognition library. The emotion engine analyzes the user's facial expressions and recognizes that the user is in a "happy" emotional state. Based on this data, the server generates advice such as, "Managing your diet is certainly difficult. But start by trying to eat a balanced diet a little at a time every day. If you make small improvements, you'll see big changes." This advice is provided to the user through an avatar.

[0434] Example prompt sentence:

[0435] plain

[0436] A user is using "Health Concierge" in a physical store. The user speaks to the camera in a relaxed manner, and the microphone picks up the voice input. The emotion recognition model recognizes "happy" from the user's facial expression.

[0437] [Voice Typing]: "I've been having trouble managing my diet lately. Do you have any quick tips?"

[0438] (Expected output)

[0439] Avatar: "Diet management can be difficult, but start by focusing on eating a balanced diet every day. Small improvements can make a big difference."

[0440] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0441] Step 1:

[0442] The user activates their smartphone or smart glasses and inputs their health data and lifestyle information. Using a camera and microphone, facial expressions and voice data are also collected in real time. This input data includes health data (e.g., weight, blood pressure, exercise volume, dietary details) and lifestyle information (e.g., occupation, dietary habits, exercise habits, home environment).

[0443] Input: Health data, lifestyle background information, facial expression data, voice data

[0444] Output: Collected health data, lifestyle background information, facial expression data, and voice data are sent to the server.

[0445] Step 2:

[0446] The devices upload the collected data to a server in real time, and the data transfer is done using a secure protocol.

[0447] Input: Collected health data, background information, facial expression data, and voice data

[0448] Output: Data transferred to the server

[0449] Step 3:

[0450] The server stores the received health data and lifestyle background information. At the same time, it analyzes this data and generates optimal health advice for the user. A generative AI model is used for the analysis, and data processing and calculations are performed to generate health advice based on the input data.

[0451] Input: Transferred health data, background information

[0452] Output: Health advice

[0453] Step 4:

[0454] The server inputs facial expression data and voice data into the emotion engine, which analyzes the user's emotions in real time. The emotion engine uses TensorFlow to analyze the facial expression data and converts the voice data into text using the speech_recognition library.

[0455] Input: facial expression data, voice data

[0456] Output: Emotion analysis results

[0457] Step 5:

[0458] The server generates appropriate feedback based on the generated health advice and emotion analysis results.

[0459] Input: Health advice, sentiment analysis results

[0460] Output: Feedback message

[0461] Step 6:

[0462] The device receives health advice and feedback sent from the server and provides it to the user through an avatar, which displays advice and encouragement in real time through conversation with the user.

[0463] Input: Health advice, feedback message

[0464] Output: Advice and feedback displayed through avatars

[0465] As a specific example of how the avatar works, it uses voice recognition to convert what the user says into text, and then uses emotion analysis to provide appropriate advice and feedback based on the user's feelings.

[0466] The above are the specific processing steps for carrying out the invention. This system allows users to receive accurate health advice from a trusted avatar in a physical store, enabling continuous health management.

[0467] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0468] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0469] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0470] [Second embodiment]

[0471] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0472] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0473] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0474] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0475] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0476] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0477] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0478] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0479] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0480] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0481] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0482] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0483] This invention is a system for promoting behavioral change in patients with lifestyle-related diseases. The system allows users to set a person or character they trust as an avatar, and aims to encourage users to improve their lifestyle habits based on health advice provided by the avatar.

[0484] User Action

[0485] First, users install the app and create an account. They choose a person or character they trust and set their avatar. Then, they set their health goals and enter information about their personality and lifestyle.

[0486] Next, users input their daily health data (e.g., blood pressure, weight, exercise, and diet) into the application. Through conversations with the avatar, they receive specific health advice. Based on this advice, users can make changes to their daily behavior.

[0487] Terminal handling

[0488] The device transmits the user's account information and health data to the server, receives the data entered by the user in real time and uploads it to the server, and interacts with the avatar to provide health advice and feedback to the user.

[0489] Server Processing

[0490] The server stores and analyzes the user's account information and daily health data. The server uses an AI model to generate advice based on the user's health data and lifestyle information. This advice is sent to the device in real time and provided to the user through an avatar.

[0491] Specifically, when a user measures their blood pressure and enters the results into the application, the device sends the data to the server. The server analyzes this data and generates advice, such as "Your blood pressure is high. Try some light aerobic exercise." This advice is sent to the device and displayed to the user via an avatar.

[0492] Specific examples

[0493] User: Mr. Tanaka (50 years old, male), Trusted person: Mr. Yamakawa, a sports instructor

[0494] 1. Tanaka installs the application, creates an account, and sets Yamakawa as his avatar.

[0495] 2. Mr. Tanaka set a goal of "keeping his blood pressure below 120 / 80 mmHg" and entered his lifestyle, which mainly involves desk work.

[0496] 3. Tanaka measures his blood pressure every morning and enters the results into the application. He also records his diet and exercise.

[0497] 4. The avatar, Mr. Yamakawa, offers advice: "You should exercise a little more."

[0498] 5. Tanaka follows the advice and starts walking daily, continuing while monitoring fluctuations in his blood pressure.

[0499] As described above, users can voluntarily review their behavior and make sustainable lifestyle improvements. This system is expected to improve the effectiveness of treatments for lifestyle-related diseases and contribute to reducing medical costs.

[0500] The processing flow will be explained below.

[0501] User Action

[0502] Step 1:

[0503] The user installs the application and launches the app.

[0504] Step 2:

[0505] The user enters their name, email address, and password on the account creation screen.

[0506] Step 3:

[0507] The user selects a person or character they trust and sets their avatar.

[0508] Step 4:

[0509] The user inputs health goals (e.g., blood pressure goal, weight loss goal).

[0510] Step 5:

[0511] The user enters information about their personality and background.

[0512] Step 6:

[0513] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details).

[0514] Step 7:

[0515] The user converses with the avatar and receives specific health advice from the avatar.

[0516] Step 8:

[0517] Users can view their progress and set new goals in the app.

[0518] Terminal handling

[0519] Step 1:

[0520] The terminal transmits the user's account information to the server.

[0521] Step 2:

[0522] The terminal stores the avatar information selected by the user.

[0523] Step 3:

[0524] The terminal transmits the health data entered by the user to the server.

[0525] Step 4:

[0526] The advice received by the terminal from the server is displayed as an avatar.

[0527] Step 5:

[0528] The device uploads the conversation log with the avatar to the server.

[0529] Server Processing

[0530] Step 1:

[0531] The server stores the received user account information in a database.

[0532] Step 2:

[0533] The server stores the user's selected avatar information in a database.

[0534] Step 3:

[0535] The server analyzes the health data entered by the user.

[0536] Step 4:

[0537] The server generates health advice based on the analysis results.

[0538] Step 5:

[0539] The server sends the generated advice to the terminal.

[0540] Step 6:

[0541] The server stores the conversation log with the avatar in a database.

[0542] Example 1

[0543] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0544] Conventional lifestyle-related disease management systems simply record users' health data and do not adequately support users in making specific behavioral changes. As a result, users find it difficult to grasp the specific steps they need to take to achieve their health goals, which hinders the improvement of lifestyle-related diseases.

[0545] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0546] In this invention, the server includes means for selecting a character trusted by the user, means for displaying the selected character as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information and generating health advice using an AI model, and means for providing the generated health advice to the user via the avatar, thereby enabling the user to easily understand specific behaviors and voluntarily change their behavior.

[0547] A "user" is an individual who uses this system and inputs health data and lifestyle information and receives advice from an avatar.

[0548] A "character" is a person or fictional entity that a user trusts, displayed as an avatar, and whose role is to provide health advice through interactions with the user.

[0549] An "avatar" is a virtual representation of a character that a user trusts, and serves as a means of providing health advice to the user.

[0550] "Health data" refers to health-related information such as blood pressure, weight, amount of exercise, and dietary details that a user inputs into the application.

[0551] "Lifestyle background information" is data that may affect the achievement of health goals, such as the user's occupation, lifestyle, and home environment.

[0552] The "server" is a central computer system that analyzes users' health data and lifestyle background information and generates health advice using AI models.

[0553] An "AI model" is an artificial intelligence algorithm that analyzes a user's health data and lifestyle background information and generates appropriate health advice based on that data.

[0554] "Health advice" is specific instructions and suggestions for improving health that are generated by the server using an AI model and provided to the user through an avatar.

[0555] "Interaction" refers to a two-way exchange between the user and the avatar in real time, including advice and feedback in response to the user's questions.

[0556] The present invention provides a system in which a user sets a character that the user trusts as an avatar and improves their lifestyle based on health advice provided by the avatar. An embodiment of this system will be described in detail below.

[0557] User Action

[0558] First, users install the dedicated application and create an account. This application is installed on a mobile device such as a smartphone or tablet. The user selects a trusted character and sets that avatar. Next, the user sets health goals and enters information about their personality and lifestyle.

[0559] As a concrete example, consider a case where a user sets a goal of "keeping blood pressure below 120 / 80 mmHg" and leads a desk-based lifestyle. The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, diet) into the application. The user also receives specific health advice through conversations with an avatar. Based on this advice, the user can change their behavior in their daily life.

[0560] Terminal handling

[0561] The device transmits the user's account information and health data to the server in real time, allowing the server to receive the latest health data and analyze it immediately. The device also provides an interface for interacting with the avatar and providing health advice and feedback to the user.

[0562] Server Processing

[0563] The server stores and analyzes the user's account information and daily health data. A generative AI model (e.g., a machine learning algorithm) is used for the analysis. Real-time advice based on the user's health data and lifestyle information is generated and sent to the device.

[0564] Specifically, when a user measures their blood pressure and enters the results into the application, the device sends the data to the server. The server analyzes the data and generates advice such as "Your blood pressure is high. Try some light aerobic exercise." This advice is sent to the device and displayed to the user via an avatar.

[0565] Specific examples

[0566] A middle-aged man installs the application and creates an account. He selects the character of a trusted sports instructor as his avatar. He then sets a goal of "keeping his blood pressure below 120 / 80 mmHg" and enters his desk-bound lifestyle. Every morning, he measures his blood pressure and enters the results into the application. He also records his diet and exercise. The avatar offers specific advice, such as "Try exercising a little more," and he begins walking daily. After a few weeks, he can see that his blood pressure is gradually approaching his target value.

[0567] This system encourages users to make specific behavioral changes and make sustainable lifestyle improvements. By utilizing generative AI models, it is possible to provide personalized health advice and effectively support users' health management.

[0568] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0569] Step 1: Initial User Setup

[0570] The user installs a dedicated application and creates an account, selects a trusted character, and configures its avatar. Input includes a username, password, and the selection of a trusted character. Output is account information and avatar configuration information. Specific actions include character selection and goal setting using drop-down menus and sliders.

[0571] Step 2: Set your health goals

[0572] The user sets health goals and inputs information about their personality and lifestyle. The input includes information such as blood pressure goals, exercise goals, diet, and living environment. The output is the user's goals and lifestyle information. Specific actions involve filling out a form and operating a slider to set the goal value.

[0573] Step 3: Enter your health data

[0574] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details) into the application. The input includes daily blood pressure, dietary details, exercise time, etc. The output is the input health data. Specific operations include measuring blood pressure and inputting the results into the application. The application also records meals and exercise.

[0575] Step 4: Sending data

[0576] The device transmits the health data entered by the user to the server in real time. At this time, the input data is encrypted before transmission. The input includes the health data entered by the user. The output is the data transmitted to the server. Specifically, the data is transmitted via the Internet, and a notification of successful transmission is displayed to the user.

[0577] Step 5: Save and analyze data

[0578] The server stores the received health data and analyzes it using a generative AI model. The input includes the health data sent from the device. The output is a trend analysis based on the analysis results and the user's health status. Specifically, the data is stored in a database and analyzed using machine learning algorithms.

[0579] Step 6: Generating Advice

[0580] The server generates health advice using an AI model based on the analysis results. The input includes the analysis results of the health data. The output is personalized health advice. Specifically, the AI ​​model generates appropriate health advice for each user in text format.

[0581] Step 7: Providing advice

[0582] The terminal provides the user with health advice received from the server. The input includes the health advice sent from the server. The output is the health advice displayed to the user through an avatar. Specifically, the avatar presents the advice to the user in text or voice on the application screen.

[0583] Step 8: Changing user behavior

[0584] The user changes their daily behavior based on advice from the avatar. The input includes health advice from the avatar. The output is a specific change in the user's behavior, which is expected to improve their health. As a specific action, the user performs and continues specific health behaviors such as walking or dietary restrictions.

[0585] (Application example 1)

[0586] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0587] Currently, there are several health management systems on the market for patients with lifestyle-related diseases, but few systems include specific dietary suggestions to comprehensively improve users' lifestyles. As a result, lifestyle-related disease prevention and management are often ineffective. Furthermore, conventional systems have the problem of difficulty in providing real-time dietary suggestions linked to health data. Conventional technologies make it difficult for users to select appropriate diets based on their own health data, which hinders lifestyle improvement.

[0588] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0589] In this invention, the server includes means for selecting a person or character trusted by the user, means for displaying the selected person or character as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information and generating health advice, means for providing the generated health advice to the user via the avatar, and means for suggesting an appropriate diet based on the generated health advice, thereby enabling the user to select an appropriate diet based on their own health condition and enabling more effective prevention and management of lifestyle-related diseases.

[0590] "User" refers to an individual who uses the system.

[0591] A "trusted person or character" refers to a real person or virtual character with whom a user feels a sense of affinity or trust.

[0592] An "avatar" is a digital representation of a person or character that a user trusts, and refers to a virtual presence that is used to interact with the user on the system.

[0593] "Health data" refers to information about a user's health status, such as physiological values ​​such as blood pressure, weight, and blood sugar levels, and diagnostic results.

[0594] "Lifestyle background information" refers to information about the user's living environment and habits that affect health, such as the user's occupation, daily activities, and eating habits.

[0595] "Analysis" refers to the process of evaluating a user's health status and identifying areas for improvement and risks based on the health data and lifestyle background information provided by the user.

[0596] "Health Advice" refers to specific advice or suggestions for improving or maintaining a user's health, generated based on the analysis.

[0597] "Means for suggesting meals" refers to a system that analyzes a user's health data and lifestyle background information and provides appropriate meal content and nutritional balance based on the results.

[0598] To implement the present invention, the following system configuration is required.

[0599] User Action

[0600] The first thing a user does is install a dedicated health management application on their smartphone. The user creates an account and selects a person or character they trust. For example, if the user selects a health and sports advisor as a trusted person, that person will appear as an avatar. The user then sets their own health goals and enters background information about their lifestyle, such as their occupation and eating habits. They enter their daily health data (e.g., blood pressure, weight, amount of exercise, and dietary details) into the app and receive health advice through interactions with the avatar.

[0601] Terminal handling

[0602] The program running on the device sends the user's account information and health data to a server in real time, and provides feedback to the user based on the analysis results returned from the server. An avatar within the application interacts with the user, offering advice such as, "Your blood pressure is high today. We recommend you eat a diet low in carbohydrates."

[0603] Server Processing

[0604] The server is built using programming languages ​​such as Python and stores and analyzes users' account information and daily health data. A generative AI model is used to analyze the health data, and appropriate health advice is generated based on the analysis results. Health advice and dietary suggestions based on the analysis are sent to the device in real time and provided to the user through an avatar.

[0605] Specifically, for example, when a user inputs their blood glucose level and BMI, the following prompt sentence is parsed by the generative AI model:

[0606] text

[0607] My blood sugar level is 150 mg / dL and my BMI is 26. What are some dietary suggestions and advice based on this health condition?

[0608] The generative AI model then generates appropriate advice and dietary suggestions, such as "Your blood sugar level is high, so eat a diet low in carbohydrates."

[0609] In this way, users can receive specific advice based on their own health status and are encouraged to take practical actions to prevent and manage lifestyle-related diseases. This system encourages users to autonomously change their health management behavior and enable them to maintain a healthier lifestyle.

[0610] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0611] Step 1:

[0612] The user installs a dedicated health management application on their smartphone and creates an account. They then select a trusted person or character to set up their avatar. During this initial setup step, the user also enters their health goals and lifestyle background information (e.g., occupation and dietary habits). The input data is collected by the device and sent to the server. The input for this step is the user's profile information and lifestyle background information, and the output is user profile data that stores this information.

[0613] Step 2:

[0614] The user inputs daily health data (e.g., blood pressure, weight, exercise, and dietary information) into the application. The device transmits this data to the server in real time. The input data is health-related information, and the output data is updated health data that is uploaded to the server.

[0615] Step 3:

[0616] The server performs data analysis based on the received health data and user profile. First, it requests an analysis by passing a prompt to the generative AI model. As a specific example, it generates a prompt such as, "Your blood pressure is 150 / 90 mmHg and your BMI is 28. Please give me some dietary suggestions and advice based on these health conditions." The input to this prompt is the health data to be analyzed on the server, and the output is the health advice generated as a result of the analysis.

[0617] Step 4:

[0618] The generative AI model analyzes the prompt text and generates appropriate health advice and dietary suggestions. For example, the generated advice might be, "Your blood pressure is high, so eat a low-salt diet." The input data is the prompt text, and the output data is the generated advice.

[0619] Step 5:

[0620] The server sends the generated health advice to the device. The device then provides the received advice to the user. An avatar within the application provides the advice in the form of voice or text through dialogue with the user. For example, the advice might be displayed in the form of "Your blood pressure is high today. We recommend a low-carbohydrate diet." The input of this step is the generated advice, and the output is feedback advice for the user.

[0621] Step 6:

[0622] The user takes action to improve their lifestyle habits according to the advice provided by the avatar. The user then enters the results into the app again to check the progress of their improvement. This data is also sent from the device to the server and reflected in the next analysis. The input for this step is new health data, and the output data is updated health data and the progress of improvement.

[0623] Through this series of processing steps, users receive specific advice based on their own health condition, enabling them to effectively improve their lifestyle habits.

[0624] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0625] This invention is a system for promoting behavioral change in patients with lifestyle-related diseases, which displays people or characters trusted by the user as avatars and provides health advice generated by analyzing the user's health data and lifestyle background information.The invention also includes an emotion engine that recognizes the user's emotions and responds appropriately according to the recognized emotions, thereby increasing the user's motivation and supporting sustainable health management.

[0626] User Action

[0627] Users install a dedicated application and create an account. When creating an account, they enter the required information (e.g., name, email address, password) and select a trusted person or character to set as their avatar. Next, users set health goals and enter information about their personality and lifestyle. After that, they enter daily health data (e.g., blood pressure, weight, amount of exercise, dietary details). This data is managed within the application, and users receive specific health advice through conversations with their avatar.

[0628] Furthermore, this invention incorporates an emotion engine that can recognize emotions through the user's voice input and facial expression analysis. Based on the recognized emotions, the avatar provides the user with appropriate health advice and words of encouragement. This motivates the user to take better care of themselves and encourages them to continue managing their health.

[0629] Terminal handling

[0630] The device sends the user's account information and health data to a server. The data entered by the user is uploaded to the server in real time. The device also interacts with the avatar and provides the user with health advice and feedback. At the same time, an emotion engine monitors the user's voice and facial expressions to recognize emotions. This information is also sent to the server and used for analysis.

[0631] Server Processing

[0632] The server stores and analyzes the user's account information and daily health data. Using an AI model, the server generates health advice based on the user's health data and lifestyle background information. It also analyzes emotional data recognized by an emotion engine and generates appropriate feedback based on the user's feelings. The generated advice and feedback are sent to the device and provided in real time via an avatar.

[0633] Specific examples

[0634] User: Ms. Sato (45 years old, female), Trusted person: Nutritionist Ms. Yamada

[0635] 1. Mr. Sato installs the application, creates an account, and sets his trusted nutritionist, Ms. Yamada, as his avatar.

[0636] 2. Mr. Sato sets a goal of "losing 5 kg" and enters his lifestyle information (mainly desk work, mostly eating out).

[0637] 3. Mr. Sato records his daily weight and dietary habits in the application.

[0638] 4. The server analyzes this data and generates advice such as, "We recommend that you change to a balanced diet and take a walk after dinner."

[0639] 5. The avatar Yamada provides advice based on the analysis results and encourages Sato to act.

[0640] 6. If Sato-san shows a depressed expression, the emotion engine will recognize this and Yamada-san's avatar will offer words of encouragement such as, "You're doing well, if you keep going you'll see results."

[0641] In this way, the system of the present invention allows users to receive accurate advice from trusted individuals, enabling them to continuously improve their lifestyle habits. Furthermore, receiving feedback based on their emotions can increase motivation for behavioral change.

[0642] The processing flow will be explained below.

[0643] User Action

[0644] Step 1:

[0645] The user installs the application and launches the app.

[0646] Step 2:

[0647] The user enters their name, email address, and password on the account creation screen.

[0648] Step 3:

[0649] The user selects a person or character they trust and sets their avatar.

[0650] Step 4:

[0651] The user inputs health goals (e.g., blood pressure goal, weight loss goal).

[0652] Step 5:

[0653] The user enters information about their personality and background.

[0654] Step 6:

[0655] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details).

[0656] Step 7:

[0657] The user converses with the avatar and receives specific health advice from the avatar.

[0658] Step 8:

[0659] The device's camera and microphone recognize the user's voice input and facial expressions to collect emotional information.

[0660] Step 9:

[0661] Users can view their progress and set new goals in the app.

[0662] Terminal handling

[0663] Step 1:

[0664] The terminal transmits the user's account information to the server.

[0665] Step 2:

[0666] The terminal stores the avatar information selected by the user.

[0667] Step 3:

[0668] The terminal transmits the health data entered by the user to the server.

[0669] Step 4:

[0670] The advice received by the terminal from the server is displayed as an avatar.

[0671] Step 5:

[0672] The terminal processes the user's voice input and facial expressions with an emotion engine, and transmits the emotion information to the server.

[0673] Step 6:

[0674] The device uploads the conversation log with the avatar to the server.

[0675] Server Processing

[0676] Step 1:

[0677] The server stores the received user account information in a database.

[0678] Step 2:

[0679] The server stores the user's selected avatar information in a database.

[0680] Step 3:

[0681] The server analyzes the health data entered by the user.

[0682] Step 4:

[0683] The server generates health advice based on the analysis results.

[0684] Step 5:

[0685] Based on the emotional information received from the emotion engine, the server modifies the analysis results and generates advice and feedback according to the emotion.

[0686] Step 6:

[0687] The server sends generated advice and feedback to the terminal.

[0688] Step 7:

[0689] The server stores the conversation log with the avatar in a database.

[0690] Specific examples

[0691] User: Ms. Sato (45 years old, female), Trusted person: Nutritionist Ms. Yamada

[0692] User Steps

[0693] Step 1:

[0694] Mr. Sato installs the application and launches it.

[0695] Step 2:

[0696] On the account creation screen, Sato enters his name, email address, and password.

[0697] Step 3:

[0698] Sato sets Yamada, a nutritionist he trusts, as his avatar.

[0699] Step 4:

[0700] Sato enters his health goal of "lose 5 kg."

[0701] Step 5:

[0702] Sato enters background information about his lifestyle, such as the fact that he mainly does desk work and eats out most of the time.

[0703] Step 6:

[0704] Sato enters his daily weight and dietary information into the application.

[0705] Step 7:

[0706] Talk to the avatar Yamada and receive health advice.

[0707] Step 8:

[0708] Sato's voice and facial expressions are recognized by the device's emotion engine, and emotional information is collected.

[0709] Step 9:

[0710] Sato checks the progress of his goals and sets new goals in the app.

[0711] Terminal Steps

[0712] Step 1:

[0713] The device sends Sato's account information to the server.

[0714] Step 2:

[0715] The device saves the avatar information selected by Sato.

[0716] Step 3:

[0717] The device sends the health data entered by Sato to the server.

[0718] Step 4:

[0719] The advice received by the terminal from the server is displayed as an avatar.

[0720] Step 5:

[0721] The device's emotion engine processes Sato's voice input and facial expressions, and sends the emotional information to the server.

[0722] Step 6:

[0723] The device uploads the conversation log with the avatar to the server.

[0724] Server Steps

[0725] Step 1:

[0726] The server stores Mr. Sato's account information in a database.

[0727] Step 2:

[0728] The server saves Sato's selected avatar information in a database.

[0729] Step 3:

[0730] The server analyzes the health data entered by Sato.

[0731] Step 4:

[0732] The server generates appropriate health advice based on the analysis results.

[0733] Step 5:

[0734] The server modifies the analysis results based on the emotional information from the emotion engine and generates advice and feedback according to the emotion.

[0735] Step 6:

[0736] The server sends generated advice and feedback to the terminal.

[0737] Step 7:

[0738] The server stores the conversation log with the avatar in a database.

[0739] Example 2

[0740] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0741] In modern society, lifestyle-related diseases are on the rise, making their prevention and management a critical issue. Patients with lifestyle-related diseases, who are known to have difficulty managing their conditions themselves, are particularly required to receive appropriate health management advice on an ongoing basis while maintaining their motivation. However, there are very few existing systems that provide customized advice tailored to the user's psychological state and individual lifestyle. Furthermore, there is a lack of systems that recognize the user's emotions in real time and provide appropriate feedback based on that information. This poses a challenge, making it difficult for users to motivate themselves to continuously manage their health.

[0742] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0743] In this invention, the server includes means for analyzing a user's health data and lifestyle background information to generate health advice, means for acquiring the user's voice input and facial expression data to analyze emotions, and means for providing feedback based on the analyzed emotions via the avatar, thereby making it possible to provide health advice and feedback customized to the user's individual lifestyle background and psychological state in real time.

[0744] A "user" is an individual who uses this system to manage their health.

[0745] A "trusted person or character" is a virtual person or character that a user psychologically trusts and that is displayed when receiving health advice.

[0746] An "avatar" is a virtual representation of a trusted person or character, and is a way to provide health advice and feedback to users.

[0747] "Health data" refers to information about a user's physical condition (e.g., weight, blood pressure, amount of exercise, dietary details).

[0748] "Lifestyle background information" is information about the user's lifestyle and environment (e.g., occupation, eating habits, sleeping habits).

[0749] "Analysis" is the process of generating appropriate health advice based on the input health data and lifestyle background information.

[0750] "Health advice" refers to health recommendations or instructions provided to the user based on the analysis results.

[0751] "Voice input" refers to voice information uttered by the user through the microphone of the terminal.

[0752] "Facial expression data" is information about facial expressions that express the user's emotions.

[0753] "Means for analyzing emotions" refers to technology for recognizing and analyzing a user's emotional state based on voice input and facial expression data.

[0754] "Feedback" is a response, such as encouragement or instruction, provided to the user based on the analyzed emotional state.

[0755] The "server" is a central processing unit that receives, stores, and analyzes health data, lifestyle background information, and emotional data, and then generates advice and feedback and sends it to the terminal.

[0756] "Real-time" is a concept that refers to the state in which input information is processed and reflected immediately without delay.

[0757] This invention is a system that analyzes a user's unique health data and lifestyle background information and provides customized health advice through a trusted avatar. It also analyzes the user's voice input and facial expression data and provides feedback according to their emotions, supporting continuous health management.

[0758] System configuration

[0759] The system mainly consists of the following parts:

[0760] 1. User Device

[0761] 2. Server

[0762] 3. Avatar display method

[0763] 4. Voice input and facial expression recognition

[0764] User device functions

[0765] User devices are smart devices (e.g., smartphones, tablets) on which users install and use applications. These devices use the following hardware and software:

[0766] Camera and microphone: Captures your facial expressions and voice.

[0767] Application software (e.g. iOS app, Android app): Enters health data and interacts with the avatar.

[0768] Network communication function: Sends user data to the server and receives data from the server.

[0769] Server Features

[0770] The server acts as a central processing unit and is responsible for:

[0771] Database (e.g. MySQL): Stores user health data and lifestyle information.

[0772] AI module (e.g., Python, TensorFlow): Analyzes health data and lifestyle background information and generates health advice.

[0773] Emotion engine: Analyzes the user's voice input and facial expression data to generate emotion-based feedback.

[0774] Avatar display method

[0775] The avatar display unit displays a trusted person or character selected by the user as a virtual avatar, which provides real-time health advice and emotional feedback to the user.

[0776] Example of operation

[0777] 1. User creates an account:

[0778] Users install the application and create an account by entering basic information such as their name, email address, and password. They then select a trusted person or character to set as their avatar.

[0779] 2. Enter your health goals and lifestyle information:

[0780] The user inputs background information such as their weight loss goal, occupation, diet, etc. For example, they can set a goal such as "lose 5 kg."

[0781] 3. Enter your daily health data:

[0782] Users record their daily health data, such as weight, exercise, and diet, in the application, which is then sent to the server in real time.

[0783] 4. Health advice generation:

[0784] The server analyzes the received health data and generates appropriate health advice, such as "We recommend that you change to a balanced diet and take a walk after dinner."

[0785] 5. Offer advice through an avatar:

[0786] The device receives health advice from the server and provides it to the user through an avatar, which communicates the advice both audibly and visually.

[0787] 6. Sentiment analysis and feedback provision:

[0788] The system analyzes the user's voice input and facial expression data to generate appropriate feedback based on their emotions. For example, if the user is feeling down, the avatar will provide an encouraging message such as, "You're doing well. If you keep going, you'll see results."

[0789] Prompt Sentence Examples

[0790] "I'm a 45-year-old woman who works a desk job. I eat out a lot and would like to lose 5kg. What health advice would you give me?"

[0791] In this way, the present invention is a system that provides health management tailored to the individual needs of the user and supports the formation of sustainable healthy habits.

[0792] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0793] Step 1:

[0794] A user creates an account and enters basic information.

[0795] Enter your name, email address, and password.

[0796] Specific operation: The user installs the application and taps the "New Registration" button. They enter their name, email address, and password in the form displayed on the screen and press the "Register" button.

[0797] Output: The user account information is sent to the server and the account is created.

[0798] Step 2:

[0799] Users select an avatar they trust and set their health goals and lifestyle background information.

[0800] Input: Select a trusted person or character, health goals, and background information (e.g., occupation, diet).

[0801] How it works: After creating an account, users are directed to an avatar selection screen where they can tap to select a trusted person or character. Next, they enter their health goal (e.g., "lose 5kg") and select background information from options.

[0802] Output: Avatar settings, health goals, and lifestyle background information are sent to the server.

[0803] Step 3:

[0804] The user inputs daily health data (weight, amount of exercise, diet, etc.).

[0805] Input: Daily health data such as weight, exercise, and diet.

[0806] Specific actions: The user accesses the health data entry screen and uses sliders and text fields to enter weight, exercise volume, dietary information, etc. Then taps the "Save" button.

[0807] Output: The entered health data is sent to the server and stored.

[0808] Step 4:

[0809] The server stores and analyzes health data and lifestyle background information.

[0810] Input: Health data, lifestyle background information.

[0811] How it works: The server stores the data in a MySQL database and uses Python and TensorFlow to analyze it, using algorithms to generate health advice.

[0812] Output: The generated health advice.

[0813] Step 5:

[0814] The server sends the generated health advice to the terminal.

[0815] Input: Generated health advice.

[0816] Specific operation: The server sends the analysis results, which are health advice, in JSON format to the terminal.

[0817] Output: The device receives the health advice.

[0818] Step 6:

[0819] The device provides advice to the user through an avatar.

[0820] Enter: health advice.

[0821] Specific operation: The device opens the avatar display screen, and the avatar conveys the received advice to the user audibly and visually. An animation is played, and the avatar says, "I recommend you change to a balanced diet and take a walk after dinner."

[0822] Output: The user receives the health advice.

[0823] Step 7:

[0824] The user expresses their emotions through voice and facial expressions.

[0825] Input: Voice input, facial expression data.

[0826] Specific actions: The user looks at the device's camera, makes facial expressions, and speaks into the microphone.

[0827] Output: Voice and facial expression data are input to the device.

[0828] Step 8:

[0829] The device transmits the emotion data to the server.

[0830] Input: speech and facial expression data.

[0831] Specific operation: The device transmits the captured emotion data to the server in real time.

[0832] Output: Emotion data is sent to the server.

[0833] Step 9:

[0834] The server analyzes the emotional data and generates appropriate feedback.

[0835] Input: Emotion data.

[0836] Specific operation: The server analyzes the received voice and facial expression data using an emotion engine to recognize the user's emotional state, and generates appropriate feedback based on the results.

[0837] Output: The generated feedback.

[0838] Step 10:

[0839] The device provides feedback to the user through an avatar.

[0840] Input: Generated feedback.

[0841] Specific operation: The device reflects the feedback on the avatar display screen and plays an animation in which the avatar says, "You're doing well. If you keep at it, you'll definitely see results."

[0842] Output: The user receives feedback.

[0843] (Application example 2)

[0844] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0845] In modern society, the prevention and management of lifestyle-related diseases is an important issue. However, many patients find it difficult to maintain motivation for self-management and do not have easy access to professional health advice. Furthermore, in order to receive regular health checks and lifestyle improvement methods, patients themselves must habitually record their health data and properly analyze it. This process is complex, making sustained management difficult, which presents a challenge.

[0846] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for displaying a person or character trusted by the user as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information to generate health advice, means for providing the user with the health advice generated through the avatar, and means having an emotion engine for analyzing the user's emotions in real time and generating appropriate feedback based on the analysis. This enables the user to continuously manage their health based on advice from the trusted avatar, and to maintain motivation through appropriate feedback according to their emotions.

[0847] A "trusted person or character" is a being that a user psychologically trusts, and is a virtual figure that is set up to provide health advice or increase motivation.

[0848] An "avatar" is a visual or audio representation of a trusted person or character that interacts with the user to provide health advice.

[0849] "Health data" refers to physical information accumulated by a user on a daily basis, specifically data including weight, blood pressure, amount of exercise, dietary details, and the like.

[0850] "Lifestyle background information" refers to information about the user's daily life, and specifically includes data such as occupation, diet, exercise habits, and home environment.

[0851] An "emotion engine" is a software or hardware mechanism that analyzes emotions from a user's facial expressions and voice and generates appropriate feedback based on the results.

[0852] "Analysis" refers to data processing to generate health advice using algorithms and AI technology based on input health data and lifestyle background information.

[0853] "Feedback" refers to appropriate responses and words of encouragement generated based on the user's emotions analyzed by the emotion engine.

[0854] A "server" refers to a computer system that stores and analyzes input data, and this system functions in conjunction with the user's terminal via a network.

[0855] This invention is a system that allows users to prevent and manage lifestyle-related diseases in a physical store. This system displays people or characters that the user trusts as avatars and provides health advice generated by analyzing the user's health data and lifestyle background information. It also has an emotion engine that recognizes the user's emotions in real time and provides appropriate feedback to increase the user's motivation and support continuous health management.

[0856] Hardware and Software

[0857] The user interacts with the device using a smartphone or smart glasses. These devices are equipped with a camera and microphone to capture the user's facial expressions and voice in real time. The camera detects the user's face using OpenCV and analyzes emotions using TensorFlow. Speech recognition is performed using the speech_recognition library.

[0858] Smartphone or smart glasses: A device that has a built-in camera and microphone and is used to run applications.

[0859] OpenCV: An image processing library used to detect faces in camera footage.

[0860] TensorFlow: A machine learning framework that analyzes user emotions using emotion recognition models.

[0861] speech_recognition: A library for speech recognition, converting user speech into text.

[0862] System processing flow

[0863] The server receives, stores, and analyzes the user's health data and lifestyle information. Data entered by the user through a dedicated application is uploaded to the server in real time. The emotion engine also recognizes the user's emotions from facial expressions and voice, which are also sent to the server for analysis.

[0864] Specifically, the following steps are performed:

[0865] 1. Data Collection:

[0866] Users use a smartphone or smart glasses to input health data (weight, blood pressure, exercise, diet, etc.) and lifestyle background information.

[0867] Using a camera and microphone, the user's facial expressions and voice data are also captured in real time.

[0868] 2. Data Analysis:

[0869] The server receives and stores this data.

[0870] The AI ​​model generates health advice based on health data and lifestyle background information.

[0871] The emotion engine analyzes facial and voice data to identify the user's emotional state.

[0872] 3. Providing advice:

[0873] The avatar provides generated health advice to the user.

[0874] At the same time, the avatar provides appropriate feedback and words of encouragement based on the results of emotion analysis.

[0875] Specific examples

[0876] For example, suppose a user says, "I've been finding it difficult to manage my diet lately. Do you have any simple advice?" This voice input is converted into text using the speech_recognition library. The emotion engine analyzes the user's facial expressions and recognizes that the user is in a "happy" emotional state. Based on this data, the server generates advice such as, "Managing your diet is certainly difficult. But start by trying to eat a balanced diet a little at a time every day. If you make small improvements, you'll see big changes." and provides this to the user through an avatar.

[0877] Example prompt sentence:

[0878] plain

[0879] A user is using "Health Concierge" in a physical store. The user speaks to the camera in a relaxed manner, and the microphone picks up the voice input. The emotion recognition model recognizes "happy" from the user's facial expression.

[0880] [Voice Typing]: "I've been having trouble managing my diet lately. Do you have any quick tips?"

[0881] (Expected output)

[0882] Avatar: "Diet management can be difficult, but start by focusing on eating a balanced diet a little at a time every day. Small improvements can make a big difference."

[0883] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0884] Step 1:

[0885] The user activates their smartphone or smart glasses and inputs their health data and lifestyle information. Using a camera and microphone, facial expressions and voice data are also collected in real time. This input data includes health data (e.g., weight, blood pressure, exercise volume, dietary details) and lifestyle information (e.g., occupation, dietary habits, exercise habits, home environment).

[0886] Input: Health data, lifestyle background information, facial expression data, voice data

[0887] Output: Collected health data, lifestyle background information, facial expression data, and voice data are sent to the server.

[0888] Step 2:

[0889] The devices upload the collected data to a server in real time, and the data transfer is done using a secure protocol.

[0890] Input: Collected health data, background information, facial expression data, and voice data

[0891] Output: Data transferred to the server

[0892] Step 3:

[0893] The server stores the received health data and lifestyle background information. At the same time, it analyzes this data and generates optimal health advice for the user. A generative AI model is used for the analysis, and data processing and calculations are performed to generate health advice based on the input data.

[0894] Input: Transferred health data, background information

[0895] Output: Health advice

[0896] Step 4:

[0897] The server inputs facial expression data and voice data into the emotion engine, which analyzes the user's emotions in real time. The emotion engine uses TensorFlow to analyze the facial expression data and converts the voice data into text using the speech_recognition library.

[0898] Input: facial expression data, voice data

[0899] Output: Emotion analysis results

[0900] Step 5:

[0901] The server generates appropriate feedback based on the generated health advice and emotion analysis results.

[0902] Input: Health advice, sentiment analysis results

[0903] Output: Feedback message

[0904] Step 6:

[0905] The device receives health advice and feedback sent from the server and provides it to the user through an avatar, which displays advice and encouragement in real time through conversation with the user.

[0906] Input: Health advice, feedback message

[0907] Output: Advice and feedback displayed through avatars

[0908] As a specific example of how the avatar works, it uses voice recognition to convert what the user says into text, and then uses emotion analysis to provide appropriate advice and feedback based on the user's feelings.

[0909] The above are the specific processing steps for carrying out the invention. This system allows users to receive accurate health advice from a trusted avatar in a physical store, enabling continuous health management.

[0910] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0911] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0912] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0913] [Third embodiment]

[0914] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0915] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0916] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0917] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0918] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0919] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0920] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0921] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0922] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0923] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0924] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0925] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0926] This invention is a system for promoting behavioral change in patients with lifestyle-related diseases. The system allows users to set a person or character they trust as an avatar, and aims to encourage users to improve their lifestyle habits based on health advice provided by the avatar.

[0927] User Action

[0928] First, users install the app and create an account. They choose a person or character they trust and set their avatar. Then, they set their health goals and enter information about their personality and lifestyle.

[0929] Next, users input their daily health data (e.g., blood pressure, weight, exercise, and diet) into the application. Through conversations with the avatar, they receive specific health advice. Based on this advice, users can make changes to their daily behavior.

[0930] Terminal handling

[0931] The device transmits the user's account information and health data to the server, receives the data entered by the user in real time and uploads it to the server, and interacts with the avatar to provide health advice and feedback to the user.

[0932] Server Processing

[0933] The server stores and analyzes the user's account information and daily health data. The server uses an AI model to generate advice based on the user's health data and lifestyle information. This advice is sent to the device in real time and provided to the user through an avatar.

[0934] Specifically, when a user measures their blood pressure and enters the results into the application, the device sends the data to the server. The server analyzes this data and generates advice, such as "Your blood pressure is high. Try some light aerobic exercise." This advice is sent to the device and displayed to the user via an avatar.

[0935] Specific examples

[0936] User: Mr. Tanaka (50 years old, male), Trusted person: Mr. Yamakawa, a sports instructor

[0937] 1. Tanaka installs the application, creates an account, and sets Yamakawa as his avatar.

[0938] 2. Mr. Tanaka set a goal of "keeping his blood pressure below 120 / 80 mmHg" and entered his lifestyle, which mainly involves desk work.

[0939] 3. Tanaka measures his blood pressure every morning and enters the results into the application. He also records his diet and exercise.

[0940] 4. The avatar, Mr. Yamakawa, offers advice: "You should exercise a little more."

[0941] 5. Tanaka follows the advice and starts walking daily, continuing while monitoring fluctuations in his blood pressure.

[0942] As described above, users can voluntarily review their behavior and make sustainable lifestyle improvements. This system is expected to improve the effectiveness of treatments for lifestyle-related diseases and contribute to reducing medical costs.

[0943] The processing flow will be explained below.

[0944] User Action

[0945] Step 1:

[0946] The user installs the application and launches the app.

[0947] Step 2:

[0948] The user enters their name, email address, and password on the account creation screen.

[0949] Step 3:

[0950] The user selects a person or character they trust and sets their avatar.

[0951] Step 4:

[0952] The user inputs health goals (e.g., blood pressure goal, weight loss goal).

[0953] Step 5:

[0954] The user enters information about their personality and background.

[0955] Step 6:

[0956] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details).

[0957] Step 7:

[0958] The user converses with the avatar and receives specific health advice from the avatar.

[0959] Step 8:

[0960] Users can view their progress and set new goals in the app.

[0961] Terminal handling

[0962] Step 1:

[0963] The terminal transmits the user's account information to the server.

[0964] Step 2:

[0965] The terminal stores the avatar information selected by the user.

[0966] Step 3:

[0967] The terminal transmits the health data entered by the user to the server.

[0968] Step 4:

[0969] The advice received by the terminal from the server is displayed as an avatar.

[0970] Step 5:

[0971] The device uploads the conversation log with the avatar to the server.

[0972] Server Processing

[0973] Step 1:

[0974] The server stores the received user account information in a database.

[0975] Step 2:

[0976] The server stores the user's selected avatar information in a database.

[0977] Step 3:

[0978] The server analyzes the health data entered by the user.

[0979] Step 4:

[0980] The server generates health advice based on the analysis results.

[0981] Step 5:

[0982] The server sends the generated advice to the terminal.

[0983] Step 6:

[0984] The server stores the conversation log with the avatar in a database.

[0985] Example 1

[0986] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0987] Conventional lifestyle-related disease management systems simply record users' health data and do not adequately support users in making specific behavioral changes. As a result, users find it difficult to grasp the specific steps they need to take to achieve their health goals, which hinders the improvement of lifestyle-related diseases.

[0988] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0989] In this invention, the server includes means for selecting a character trusted by the user, means for displaying the selected character as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information and generating health advice using an AI model, and means for providing the generated health advice to the user via the avatar, thereby enabling the user to easily understand specific behaviors and voluntarily change their behavior.

[0990] A "user" is an individual who uses this system and inputs health data and lifestyle information and receives advice from an avatar.

[0991] A "character" is a person or fictional entity that a user trusts, displayed as an avatar, and whose role is to provide health advice through interactions with the user.

[0992] An "avatar" is a virtual representation of a character that a user trusts, and serves as a means of providing health advice to the user.

[0993] "Health data" refers to health-related information such as blood pressure, weight, amount of exercise, and dietary details that a user inputs into the application.

[0994] "Lifestyle background information" is data that may affect the achievement of health goals, such as the user's occupation, lifestyle, and home environment.

[0995] The "server" is a central computer system that analyzes users' health data and lifestyle background information and generates health advice using AI models.

[0996] An "AI model" is an artificial intelligence algorithm that analyzes a user's health data and lifestyle background information and generates appropriate health advice based on that data.

[0997] "Health advice" is specific instructions and suggestions for improving health that are generated by the server using an AI model and provided to the user through an avatar.

[0998] "Interaction" refers to a two-way exchange between the user and the avatar in real time, including advice and feedback in response to the user's questions.

[0999] The present invention provides a system in which a user sets a character that the user trusts as an avatar and improves their lifestyle based on health advice provided by the avatar. An embodiment of this system will be described in detail below.

[1000] User Action

[1001] First, users install the dedicated application and create an account. This application is installed on a mobile device such as a smartphone or tablet. The user selects a trusted character and sets that avatar. Next, the user sets health goals and enters information about their personality and lifestyle.

[1002] As a concrete example, consider a case where a user sets a goal of "keeping blood pressure below 120 / 80 mmHg" and leads a desk-based lifestyle. The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, diet) into the application. The user also receives specific health advice through conversations with an avatar. Based on this advice, the user can change their behavior in their daily life.

[1003] Terminal handling

[1004] The device transmits the user's account information and health data to the server in real time, allowing the server to receive the latest health data and analyze it immediately. The device also provides an interface for interacting with the avatar and providing health advice and feedback to the user.

[1005] Server Processing

[1006] The server stores and analyzes the user's account information and daily health data. A generative AI model (e.g., a machine learning algorithm) is used for the analysis. Real-time advice based on the user's health data and lifestyle information is generated and sent to the device.

[1007] Specifically, when a user measures their blood pressure and enters the results into the application, the device sends the data to the server. The server analyzes the data and generates advice such as "Your blood pressure is high. Try some light aerobic exercise." This advice is sent to the device and displayed to the user via an avatar.

[1008] Specific examples

[1009] A middle-aged man installs the application and creates an account. He selects the character of a trusted sports instructor as his avatar. He then sets a goal of "keeping his blood pressure below 120 / 80 mmHg" and enters his desk-bound lifestyle. Every morning, he measures his blood pressure and enters the results into the application. He also records his diet and exercise. The avatar offers specific advice, such as "Try exercising a little more," and he begins walking daily. After a few weeks, he can see that his blood pressure is gradually approaching his target value.

[1010] This system encourages users to make specific behavioral changes and make sustainable lifestyle improvements. By utilizing generative AI models, it is possible to provide personalized health advice and effectively support users' health management.

[1011] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1012] Step 1: Initial User Setup

[1013] The user installs a dedicated application and creates an account, selects a trusted character, and configures its avatar. Input includes a username, password, and the selection of a trusted character. Output is account information and avatar configuration information. Specific actions include character selection and goal setting using drop-down menus and sliders.

[1014] Step 2: Set your health goals

[1015] The user sets health goals and inputs information about their personality and lifestyle. The input includes information such as blood pressure goals, exercise goals, diet, and living environment. The output is the user's goals and lifestyle information. Specific actions involve filling out a form and operating a slider to set the goal value.

[1016] Step 3: Enter your health data

[1017] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details) into the application. The input includes daily blood pressure, dietary details, exercise time, etc. The output is the input health data. Specific operations include measuring blood pressure and inputting the results into the application. The application also records meals and exercise.

[1018] Step 4: Sending data

[1019] The device transmits the health data entered by the user to the server in real time. At this time, the input data is encrypted before transmission. The input includes the health data entered by the user. The output is the data transmitted to the server. Specifically, the data is transmitted via the Internet, and a notification of successful transmission is displayed to the user.

[1020] Step 5: Save and analyze data

[1021] The server stores the received health data and analyzes it using a generative AI model. The input includes the health data sent from the device. The output is a trend analysis based on the analysis results and the user's health status. Specifically, the data is stored in a database and analyzed using machine learning algorithms.

[1022] Step 6: Generating Advice

[1023] The server generates health advice using an AI model based on the analysis results. The input includes the analysis results of the health data. The output is personalized health advice. Specifically, the AI ​​model generates appropriate health advice for each user in text format.

[1024] Step 7: Providing advice

[1025] The terminal provides the user with health advice received from the server. The input includes the health advice sent from the server. The output is the health advice displayed to the user through an avatar. Specifically, the avatar presents the advice to the user in text or voice on the application screen.

[1026] Step 8: Changing user behavior

[1027] The user changes their daily behavior based on advice from the avatar. The input includes health advice from the avatar. The output is a specific change in the user's behavior, which is expected to improve their health. As a specific action, the user performs and continues specific health behaviors such as walking or dietary restrictions.

[1028] (Application example 1)

[1029] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1030] Currently, there are several health management systems on the market for patients with lifestyle-related diseases, but few systems include specific dietary suggestions to comprehensively improve users' lifestyles. As a result, lifestyle-related disease prevention and management are often ineffective. Furthermore, conventional systems have the problem of difficulty in providing real-time dietary suggestions linked to health data. Conventional technologies make it difficult for users to select appropriate diets based on their own health data, which hinders lifestyle improvement.

[1031] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1032] In this invention, the server includes means for selecting a person or character trusted by the user, means for displaying the selected person or character as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information and generating health advice, means for providing the generated health advice to the user via the avatar, and means for suggesting an appropriate diet based on the generated health advice, thereby enabling the user to select an appropriate diet based on their own health condition and enabling more effective prevention and management of lifestyle-related diseases.

[1033] "User" refers to an individual who uses the system.

[1034] A "trusted person or character" refers to a real person or virtual character with whom a user feels a sense of affinity or trust.

[1035] An "avatar" is a digital representation of a person or character that a user trusts, and refers to a virtual presence that is used to interact with the user on the system.

[1036] "Health data" refers to information about a user's health status, such as physiological values ​​such as blood pressure, weight, and blood sugar levels, and diagnostic results.

[1037] "Lifestyle background information" refers to information about the user's living environment and habits that affect health, such as the user's occupation, daily activities, and eating habits.

[1038] "Analysis" refers to the process of evaluating a user's health status and identifying areas for improvement and risks based on the health data and lifestyle background information provided by the user.

[1039] "Health Advice" refers to specific advice or suggestions for improving or maintaining a user's health, generated based on the analysis.

[1040] "Means for suggesting meals" refers to a system that analyzes a user's health data and lifestyle background information and provides appropriate meal content and nutritional balance based on the results.

[1041] To implement the present invention, the following system configuration is required.

[1042] User Action

[1043] The first thing a user does is install a dedicated health management application on their smartphone. The user creates an account and selects a person or character they trust. For example, if the user selects a health and sports advisor as a trusted person, that person will appear as an avatar. The user then sets their own health goals and enters background information about their lifestyle, such as their occupation and eating habits. They enter their daily health data (e.g., blood pressure, weight, amount of exercise, and dietary details) into the app and receive health advice through interactions with the avatar.

[1044] Terminal handling

[1045] The program running on the device sends the user's account information and health data to a server in real time, and provides feedback to the user based on the analysis results returned from the server. An avatar within the application interacts with the user, offering advice such as, "Your blood pressure is high today. We recommend you eat a diet low in carbohydrates."

[1046] Server Processing

[1047] The server is built using programming languages ​​such as Python and stores and analyzes users' account information and daily health data. A generative AI model is used to analyze the health data, and appropriate health advice is generated based on the analysis results. Health advice and dietary suggestions based on the analysis are sent to the device in real time and provided to the user through an avatar.

[1048] Specifically, for example, when a user inputs their blood glucose level and BMI, the following prompt sentence is parsed by the generative AI model:

[1049] text

[1050] My blood sugar level is 150 mg / dL and my BMI is 26. What are some dietary suggestions and advice based on this health condition?

[1051] The generative AI model then generates appropriate advice and dietary suggestions, such as "Your blood sugar level is high, so eat a diet low in carbohydrates."

[1052] In this way, users can receive specific advice based on their own health status and are encouraged to take practical actions to prevent and manage lifestyle-related diseases. This system encourages users to autonomously change their health management behavior and enable them to maintain a healthier lifestyle.

[1053] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1054] Step 1:

[1055] The user installs a dedicated health management application on their smartphone and creates an account. They then select a trusted person or character to set up their avatar. During this initial setup step, the user also enters their health goals and lifestyle background information (e.g., occupation and dietary habits). The input data is collected by the device and sent to the server. The input for this step is the user's profile information and lifestyle background information, and the output is user profile data that stores this information.

[1056] Step 2:

[1057] The user inputs daily health data (e.g., blood pressure, weight, exercise, and dietary information) into the application. The device transmits this data to the server in real time. The input data is health-related information, and the output data is updated health data that is uploaded to the server.

[1058] Step 3:

[1059] The server performs data analysis based on the received health data and user profile. First, it requests an analysis by passing a prompt to the generative AI model. As a specific example, it generates a prompt such as, "Your blood pressure is 150 / 90 mmHg and your BMI is 28. Please give me some dietary suggestions and advice based on these health conditions." The input to this prompt is the health data to be analyzed on the server, and the output is the health advice generated as a result of the analysis.

[1060] Step 4:

[1061] The generative AI model analyzes the prompt text and generates appropriate health advice and dietary suggestions. For example, the generated advice might be, "Your blood pressure is high, so eat a low-salt diet." The input data is the prompt text, and the output data is the generated advice.

[1062] Step 5:

[1063] The server sends the generated health advice to the device. The device then provides the received advice to the user. An avatar within the application provides the advice in the form of voice or text through dialogue with the user. For example, the advice might be displayed in the form of "Your blood pressure is high today. We recommend a low-carbohydrate diet." The input of this step is the generated advice, and the output is feedback advice for the user.

[1064] Step 6:

[1065] The user takes action to improve their lifestyle habits according to the advice provided by the avatar. The user then enters the results into the app again to check the progress of their improvement. This data is also sent from the device to the server and reflected in the next analysis. The input for this step is new health data, and the output data is updated health data and the progress of improvement.

[1066] Through this series of processing steps, users receive specific advice based on their own health condition, enabling them to effectively improve their lifestyle habits.

[1067] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1068] This invention is a system for promoting behavioral change in patients with lifestyle-related diseases, which displays people or characters trusted by the user as avatars and provides health advice generated by analyzing the user's health data and lifestyle background information.The invention also includes an emotion engine that recognizes the user's emotions and responds appropriately according to the recognized emotions, thereby increasing the user's motivation and supporting sustainable health management.

[1069] User Action

[1070] Users install a dedicated application and create an account. When creating an account, they enter the required information (e.g., name, email address, password) and select a trusted person or character to set as their avatar. Next, users set health goals and enter information about their personality and lifestyle. After that, they enter daily health data (e.g., blood pressure, weight, amount of exercise, dietary details). This data is managed within the application, and users receive specific health advice through conversations with their avatar.

[1071] Furthermore, this invention incorporates an emotion engine that can recognize emotions through the user's voice input and facial expression analysis. Based on the recognized emotions, the avatar provides the user with appropriate health advice and words of encouragement. This motivates the user to take better care of themselves and encourages them to continue managing their health.

[1072] Terminal handling

[1073] The device sends the user's account information and health data to a server. The data entered by the user is uploaded to the server in real time. The device also interacts with the avatar and provides the user with health advice and feedback. At the same time, an emotion engine monitors the user's voice and facial expressions to recognize emotions. This information is also sent to the server and used for analysis.

[1074] Server Processing

[1075] The server stores and analyzes the user's account information and daily health data. Using an AI model, the server generates health advice based on the user's health data and lifestyle background information. It also analyzes emotional data recognized by an emotion engine and generates appropriate feedback based on the user's feelings. The generated advice and feedback are sent to the device and provided in real time via an avatar.

[1076] Specific examples

[1077] User: Ms. Sato (45 years old, female), Trusted person: Nutritionist Ms. Yamada

[1078] 1. Mr. Sato installs the application, creates an account, and sets his trusted nutritionist, Ms. Yamada, as his avatar.

[1079] 2. Mr. Sato sets a goal of "losing 5 kg" and enters his lifestyle information (mainly desk work, mostly eating out).

[1080] 3. Mr. Sato records his daily weight and dietary habits in the application.

[1081] 4. The server analyzes this data and generates advice such as, "We recommend that you change to a balanced diet and take a walk after dinner."

[1082] 5. The avatar Yamada provides advice based on the analysis results and encourages Sato to act.

[1083] 6. If Sato-san shows a depressed expression, the emotion engine will recognize this and Yamada-san's avatar will offer words of encouragement such as, "You're doing well, if you keep going you'll see results."

[1084] In this way, the system of the present invention allows users to receive accurate advice from trusted individuals, enabling them to continuously improve their lifestyle habits. Furthermore, receiving feedback based on their emotions can increase motivation for behavioral change.

[1085] The processing flow will be explained below.

[1086] User Action

[1087] Step 1:

[1088] The user installs the application and launches the app.

[1089] Step 2:

[1090] The user enters their name, email address, and password on the account creation screen.

[1091] Step 3:

[1092] The user selects a person or character they trust and sets their avatar.

[1093] Step 4:

[1094] The user inputs health goals (e.g., blood pressure goal, weight loss goal).

[1095] Step 5:

[1096] The user enters information about their personality and background.

[1097] Step 6:

[1098] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details).

[1099] Step 7:

[1100] The user converses with the avatar and receives specific health advice from the avatar.

[1101] Step 8:

[1102] The device's camera and microphone recognize the user's voice input and facial expressions to collect emotional information.

[1103] Step 9:

[1104] Users can view their progress and set new goals in the app.

[1105] Terminal handling

[1106] Step 1:

[1107] The terminal transmits the user's account information to the server.

[1108] Step 2:

[1109] The terminal stores the avatar information selected by the user.

[1110] Step 3:

[1111] The terminal transmits the health data entered by the user to the server.

[1112] Step 4:

[1113] The advice received by the terminal from the server is displayed as an avatar.

[1114] Step 5:

[1115] The terminal processes the user's voice input and facial expressions with an emotion engine, and transmits the emotion information to the server.

[1116] Step 6:

[1117] The device uploads the conversation log with the avatar to the server.

[1118] Server Processing

[1119] Step 1:

[1120] The server stores the received user account information in a database.

[1121] Step 2:

[1122] The server stores the user's selected avatar information in a database.

[1123] Step 3:

[1124] The server analyzes the health data entered by the user.

[1125] Step 4:

[1126] The server generates health advice based on the analysis results.

[1127] Step 5:

[1128] Based on the emotional information received from the emotion engine, the server modifies the analysis results and generates advice and feedback according to the emotion.

[1129] Step 6:

[1130] The server sends generated advice and feedback to the terminal.

[1131] Step 7:

[1132] The server stores the conversation log with the avatar in a database.

[1133] Specific examples

[1134] User: Ms. Sato (45 years old, female), Trusted person: Nutritionist Ms. Yamada

[1135] User Steps

[1136] Step 1:

[1137] Mr. Sato installs the application and launches it.

[1138] Step 2:

[1139] On the account creation screen, Sato enters his name, email address, and password.

[1140] Step 3:

[1141] Sato sets Yamada, a nutritionist he trusts, as his avatar.

[1142] Step 4:

[1143] Sato enters his health goal of "lose 5 kg."

[1144] Step 5:

[1145] Sato enters background information about his lifestyle, such as the fact that he mainly does desk work and eats out most of the time.

[1146] Step 6:

[1147] Sato enters his daily weight and dietary information into the application.

[1148] Step 7:

[1149] Talk to the avatar Yamada and receive health advice.

[1150] Step 8:

[1151] Sato's voice and facial expressions are recognized by the device's emotion engine, and emotional information is collected.

[1152] Step 9:

[1153] Sato checks the progress of his goals and sets new goals in the app.

[1154] Terminal Steps

[1155] Step 1:

[1156] The device sends Sato's account information to the server.

[1157] Step 2:

[1158] The device saves the avatar information selected by Sato.

[1159] Step 3:

[1160] The device sends the health data entered by Sato to the server.

[1161] Step 4:

[1162] The advice received by the terminal from the server is displayed as an avatar.

[1163] Step 5:

[1164] The device's emotion engine processes Sato's voice input and facial expressions, and sends the emotional information to the server.

[1165] Step 6:

[1166] The device uploads the conversation log with the avatar to the server.

[1167] Server Steps

[1168] Step 1:

[1169] The server stores Mr. Sato's account information in a database.

[1170] Step 2:

[1171] The server saves Sato's selected avatar information in a database.

[1172] Step 3:

[1173] The server analyzes the health data entered by Sato.

[1174] Step 4:

[1175] The server generates appropriate health advice based on the analysis results.

[1176] Step 5:

[1177] The server modifies the analysis results based on the emotional information from the emotion engine and generates advice and feedback according to the emotion.

[1178] Step 6:

[1179] The server sends generated advice and feedback to the terminal.

[1180] Step 7:

[1181] The server stores the conversation log with the avatar in a database.

[1182] Example 2

[1183] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1184] In modern society, lifestyle-related diseases are on the rise, making their prevention and management a critical issue. Patients with lifestyle-related diseases, who are known to have difficulty managing their conditions themselves, are particularly required to receive appropriate health management advice on an ongoing basis while maintaining their motivation. However, there are very few existing systems that provide customized advice tailored to the user's psychological state and individual lifestyle. Furthermore, there is a lack of systems that recognize the user's emotions in real time and provide appropriate feedback based on that information. This poses a challenge, making it difficult for users to motivate themselves to continuously manage their health.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1186] In this invention, the server includes means for analyzing a user's health data and lifestyle background information to generate health advice, means for acquiring the user's voice input and facial expression data to analyze emotions, and means for providing feedback based on the analyzed emotions via the avatar, thereby making it possible to provide health advice and feedback customized to the user's individual lifestyle background and psychological state in real time.

[1187] A "user" is an individual who uses this system to manage their health.

[1188] A "trusted person or character" is a virtual person or character that a user psychologically trusts and that is displayed when receiving health advice.

[1189] An "avatar" is a virtual representation of a trusted person or character, and is a way to provide health advice and feedback to users.

[1190] "Health data" refers to information about a user's physical condition (e.g., weight, blood pressure, amount of exercise, dietary details).

[1191] "Lifestyle background information" is information about the user's lifestyle and environment (e.g., occupation, eating habits, sleeping habits).

[1192] "Analysis" is the process of generating appropriate health advice based on the input health data and lifestyle background information.

[1193] "Health advice" refers to health recommendations or instructions provided to the user based on the analysis results.

[1194] "Voice input" refers to voice information uttered by the user through the microphone of the terminal.

[1195] "Facial expression data" is information about facial expressions that express the user's emotions.

[1196] "Means for analyzing emotions" refers to technology for recognizing and analyzing a user's emotional state based on voice input and facial expression data.

[1197] "Feedback" is a response, such as encouragement or instruction, provided to the user based on the analyzed emotional state.

[1198] The "server" is a central processing unit that receives, stores, and analyzes health data, lifestyle background information, and emotional data, and then generates advice and feedback and sends it to the terminal.

[1199] "Real-time" is a concept that refers to the state in which input information is processed and reflected immediately without delay.

[1200] This invention is a system that analyzes a user's unique health data and lifestyle background information and provides customized health advice through a trusted avatar. It also analyzes the user's voice input and facial expression data and provides feedback according to their emotions, supporting continuous health management.

[1201] System configuration

[1202] The system mainly consists of the following parts:

[1203] 1. User Device

[1204] 2. Server

[1205] 3. Avatar display method

[1206] 4. Voice input and facial expression recognition

[1207] User device functions

[1208] User devices are smart devices (e.g., smartphones, tablets) on which users install and use applications. These devices use the following hardware and software:

[1209] Camera and microphone: Captures your facial expressions and voice.

[1210] Application software (e.g. iOS app, Android app): Enters health data and interacts with the avatar.

[1211] Network communication function: Sends user data to the server and receives data from the server.

[1212] Server Features

[1213] The server acts as a central processing unit and is responsible for:

[1214] Database (e.g. MySQL): Stores user health data and lifestyle information.

[1215] AI module (e.g., Python, TensorFlow): Analyzes health data and lifestyle background information and generates health advice.

[1216] Emotion engine: Analyzes the user's voice input and facial expression data to generate emotion-based feedback.

[1217] Avatar display method

[1218] The avatar display unit displays a trusted person or character selected by the user as a virtual avatar, which provides real-time health advice and emotional feedback to the user.

[1219] Example of operation

[1220] 1. User creates an account:

[1221] Users install the application and create an account by entering basic information such as their name, email address, and password. They then select a trusted person or character to set as their avatar.

[1222] 2. Enter your health goals and lifestyle information:

[1223] The user inputs background information such as their weight loss goal, occupation, diet, etc. For example, they can set a goal such as "lose 5 kg."

[1224] 3. Enter your daily health data:

[1225] Users record their daily health data, such as weight, exercise, and diet, in the application, which is then sent to the server in real time.

[1226] 4. Health advice generation:

[1227] The server analyzes the received health data and generates appropriate health advice, such as "We recommend that you change to a balanced diet and take a walk after dinner."

[1228] 5. Offer advice through an avatar:

[1229] The device receives health advice from the server and provides it to the user through an avatar, which communicates the advice both audibly and visually.

[1230] 6. Sentiment analysis and feedback provision:

[1231] The system analyzes the user's voice input and facial expression data to generate appropriate feedback based on their emotions. For example, if the user is feeling down, the avatar will provide an encouraging message such as, "You're doing well. If you keep going, you'll see results."

[1232] Prompt Sentence Examples

[1233] "I'm a 45-year-old woman who works a desk job. I eat out a lot and would like to lose 5kg. What health advice would you give me?"

[1234] In this way, the present invention is a system that provides health management tailored to the individual needs of the user and supports the formation of sustainable healthy habits.

[1235] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1236] Step 1:

[1237] A user creates an account and enters basic information.

[1238] Enter your name, email address, and password.

[1239] Specific operation: The user installs the application and taps the "New Registration" button. They enter their name, email address, and password in the form displayed on the screen and press the "Register" button.

[1240] Output: The user account information is sent to the server and the account is created.

[1241] Step 2:

[1242] Users select an avatar they trust and set their health goals and lifestyle background information.

[1243] Input: Select a trusted person or character, health goals, and background information (e.g., occupation, diet).

[1244] How it works: After creating an account, users are directed to an avatar selection screen where they can tap to select a trusted person or character. Next, they enter their health goal (e.g., "lose 5kg") and select background information from options.

[1245] Output: Avatar settings, health goals, and lifestyle background information are sent to the server.

[1246] Step 3:

[1247] The user inputs daily health data (weight, amount of exercise, diet, etc.).

[1248] Input: Daily health data such as weight, exercise, and diet.

[1249] Specific actions: The user accesses the health data entry screen and uses sliders and text fields to enter weight, exercise volume, dietary information, etc. Then taps the "Save" button.

[1250] Output: The entered health data is sent to the server and stored.

[1251] Step 4:

[1252] The server stores and analyzes health data and lifestyle background information.

[1253] Input: Health data, lifestyle background information.

[1254] How it works: The server stores the data in a MySQL database and uses Python and TensorFlow to analyze it, using algorithms to generate health advice.

[1255] Output: The generated health advice.

[1256] Step 5:

[1257] The server sends the generated health advice to the terminal.

[1258] Input: Generated health advice.

[1259] Specific operation: The server sends the analysis results, which are health advice, in JSON format to the terminal.

[1260] Output: The device receives the health advice.

[1261] Step 6:

[1262] The device provides advice to the user through an avatar.

[1263] Enter: health advice.

[1264] Specific operation: The device opens the avatar display screen, and the avatar conveys the received advice to the user audibly and visually. An animation is played, and the avatar says, "I recommend you change to a balanced diet and take a walk after dinner."

[1265] Output: The user receives the health advice.

[1266] Step 7:

[1267] The user expresses their emotions through voice and facial expressions.

[1268] Input: Voice input, facial expression data.

[1269] Specific actions: The user looks at the device's camera, makes facial expressions, and speaks into the microphone.

[1270] Output: Voice and facial expression data are input to the device.

[1271] Step 8:

[1272] The device transmits the emotion data to the server.

[1273] Input: speech and facial expression data.

[1274] Specific operation: The device transmits the captured emotion data to the server in real time.

[1275] Output: Emotion data is sent to the server.

[1276] Step 9:

[1277] The server analyzes the emotional data and generates appropriate feedback.

[1278] Input: Emotion data.

[1279] Specific operation: The server analyzes the received voice and facial expression data using an emotion engine to recognize the user's emotional state, and generates appropriate feedback based on the results.

[1280] Output: The generated feedback.

[1281] Step 10:

[1282] The device provides feedback to the user through an avatar.

[1283] Input: Generated feedback.

[1284] Specific operation: The device reflects the feedback on the avatar display screen and plays an animation in which the avatar says, "You're doing well. If you keep at it, you'll definitely see results."

[1285] Output: The user receives feedback.

[1286] (Application example 2)

[1287] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1288] In modern society, the prevention and management of lifestyle-related diseases is an important issue. However, many patients find it difficult to maintain motivation for self-management and do not have easy access to professional health advice. Furthermore, in order to receive regular health checks and lifestyle improvement methods, patients themselves must habitually record their health data and properly analyze it. This process is complex, making sustained management difficult, which presents a challenge.

[1289] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for displaying a person or character trusted by the user as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information to generate health advice, means for providing the user with the health advice generated through the avatar, and means having an emotion engine for analyzing the user's emotions in real time and generating appropriate feedback based on the analysis. This enables the user to continuously manage their health based on advice from the trusted avatar, and to maintain motivation through appropriate feedback according to their emotions.

[1290] A "trusted person or character" is a being that a user psychologically trusts, and is a virtual figure that is set up to provide health advice or increase motivation.

[1291] An "avatar" is a visual or audio representation of a trusted person or character that interacts with the user to provide health advice.

[1292] "Health data" refers to physical information accumulated by a user on a daily basis, specifically data including weight, blood pressure, amount of exercise, dietary details, and the like.

[1293] "Lifestyle background information" refers to information about the user's daily life, and specifically includes data such as occupation, diet, exercise habits, and home environment.

[1294] An "emotion engine" is a software or hardware mechanism that analyzes emotions from a user's facial expressions and voice and generates appropriate feedback based on the results.

[1295] "Analysis" refers to data processing to generate health advice using algorithms and AI technology based on input health data and lifestyle background information.

[1296] "Feedback" refers to appropriate responses and words of encouragement generated based on the user's emotions analyzed by the emotion engine.

[1297] A "server" refers to a computer system that stores and analyzes input data, and this system functions in conjunction with the user's terminal via a network.

[1298] This invention is a system that allows users to prevent and manage lifestyle-related diseases in a physical store. This system displays people or characters that the user trusts as avatars and provides health advice generated by analyzing the user's health data and lifestyle background information. It also has an emotion engine that recognizes the user's emotions in real time and provides appropriate feedback to increase the user's motivation and support continuous health management.

[1299] Hardware and Software

[1300] The user interacts with the device using a smartphone or smart glasses. These devices are equipped with a camera and microphone to capture the user's facial expressions and voice in real time. The camera detects the user's face using OpenCV and analyzes emotions using TensorFlow. Speech recognition is performed using the speech_recognition library.

[1301] Smartphone or smart glasses: A device that has a built-in camera and microphone and is used to run applications.

[1302] OpenCV: An image processing library used to detect faces in camera footage.

[1303] TensorFlow: A machine learning framework that analyzes user emotions using emotion recognition models.

[1304] speech_recognition: A library for speech recognition, converting user speech into text.

[1305] System processing flow

[1306] The server receives, stores, and analyzes the user's health data and lifestyle information. Data entered by the user through a dedicated application is uploaded to the server in real time. The emotion engine also recognizes the user's emotions from facial expressions and voice, which are also sent to the server for analysis.

[1307] Specifically, the following steps are performed:

[1308] 1. Data Collection:

[1309] Users use a smartphone or smart glasses to input health data (weight, blood pressure, exercise, diet, etc.) and lifestyle background information.

[1310] Using a camera and microphone, the user's facial expressions and voice data are also captured in real time.

[1311] 2. Data Analysis:

[1312] The server receives and stores this data.

[1313] The AI ​​model generates health advice based on health data and lifestyle background information.

[1314] The emotion engine analyzes facial and voice data to identify the user's emotional state.

[1315] 3. Providing advice:

[1316] The avatar provides generated health advice to the user.

[1317] At the same time, the avatar provides appropriate feedback and words of encouragement based on the results of emotion analysis.

[1318] Specific examples

[1319] For example, suppose a user says, "I've been finding it difficult to manage my diet lately. Do you have any simple advice?" This voice input is converted into text using the speech_recognition library. The emotion engine analyzes the user's facial expressions and recognizes that the user is in a "happy" emotional state. Based on this data, the server generates advice such as, "Managing your diet is certainly difficult. But start by trying to eat a balanced diet a little at a time every day. If you make small improvements, you'll see big changes." and provides this to the user through an avatar.

[1320] Example prompt sentence:

[1321] plain

[1322] A user is using "Health Concierge" in a physical store. The user speaks to the camera in a relaxed manner, and the microphone picks up the voice input. The emotion recognition model recognizes "happy" from the user's facial expression.

[1323] [Voice Typing]: "I've been having trouble managing my diet lately. Do you have any quick tips?"

[1324] (Expected output)

[1325] Avatar: "Diet management can be difficult, but start by focusing on eating a balanced diet a little at a time every day. Small improvements can make a big difference."

[1326] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1327] Step 1:

[1328] The user activates their smartphone or smart glasses and inputs their health data and lifestyle information. Using a camera and microphone, facial expressions and voice data are also collected in real time. This input data includes health data (e.g., weight, blood pressure, exercise volume, dietary details) and lifestyle information (e.g., occupation, dietary habits, exercise habits, home environment).

[1329] Input: Health data, lifestyle background information, facial expression data, voice data

[1330] Output: Collected health data, lifestyle background information, facial expression data, and voice data are sent to the server.

[1331] Step 2:

[1332] The devices upload the collected data to a server in real time, and the data transfer is done using a secure protocol.

[1333] Input: Collected health data, background information, facial expression data, and voice data

[1334] Output: Data transferred to the server

[1335] Step 3:

[1336] The server stores the received health data and lifestyle background information. At the same time, it analyzes this data and generates optimal health advice for the user. A generative AI model is used for the analysis, and data processing and calculations are performed to generate health advice based on the input data.

[1337] Input: Transferred health data, background information

[1338] Output: Health advice

[1339] Step 4:

[1340] The server inputs facial expression data and voice data into the emotion engine, which analyzes the user's emotions in real time. The emotion engine uses TensorFlow to analyze the facial expression data and converts the voice data into text using the speech_recognition library.

[1341] Input: facial expression data, voice data

[1342] Output: Emotion analysis results

[1343] Step 5:

[1344] The server generates appropriate feedback based on the generated health advice and emotion analysis results.

[1345] Input: Health advice, sentiment analysis results

[1346] Output: Feedback message

[1347] Step 6:

[1348] The device receives health advice and feedback sent from the server and provides it to the user through an avatar, which displays advice and encouragement in real time through conversation with the user.

[1349] Input: Health advice, feedback message

[1350] Output: Advice and feedback displayed through avatars

[1351] As a specific example of how the avatar works, it uses voice recognition to convert what the user says into text, and then uses emotion analysis to provide appropriate advice and feedback based on the user's feelings.

[1352] The above are the specific processing steps for carrying out the invention. This system allows users to receive accurate health advice from a trusted avatar in a physical store, enabling continuous health management.

[1353] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1354] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1355] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1356] [Fourth embodiment]

[1357] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1358] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1359] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1360] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1361] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1362] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1363] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1364] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1365] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1366] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1367] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1368] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1369] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1370] This invention is a system for promoting behavioral change in patients with lifestyle-related diseases. The system allows users to set a person or character they trust as an avatar, and aims to encourage users to improve their lifestyle habits based on health advice provided by the avatar.

[1371] User Action

[1372] First, users install the app and create an account. They choose a person or character they trust and set their avatar. Then, they set their health goals and enter information about their personality and lifestyle.

[1373] Next, users input their daily health data (e.g., blood pressure, weight, exercise, and diet) into the application. Through conversations with the avatar, they receive specific health advice. Based on this advice, users can make changes to their daily behavior.

[1374] Terminal handling

[1375] The device transmits the user's account information and health data to the server, receives the data entered by the user in real time and uploads it to the server, and interacts with the avatar to provide health advice and feedback to the user.

[1376] Server Processing

[1377] The server stores and analyzes the user's account information and daily health data. The server uses an AI model to generate advice based on the user's health data and lifestyle information. This advice is sent to the device in real time and provided to the user through an avatar.

[1378] Specifically, when a user measures their blood pressure and enters the results into the application, the device sends the data to the server. The server analyzes this data and generates advice, such as "Your blood pressure is high. Try some light aerobic exercise." This advice is sent to the device and displayed to the user via an avatar.

[1379] Specific examples

[1380] User: Mr. Tanaka (50 years old, male), Trusted person: Mr. Yamakawa, a sports instructor

[1381] 1. Tanaka installs the application, creates an account, and sets Yamakawa as his avatar.

[1382] 2. Mr. Tanaka set a goal of "keeping his blood pressure below 120 / 80 mmHg" and entered his lifestyle, which mainly involves desk work.

[1383] 3. Tanaka measures his blood pressure every morning and enters the results into the application. He also records his diet and exercise.

[1384] 4. The avatar, Mr. Yamakawa, offers advice: "You should exercise a little more."

[1385] 5. Tanaka follows the advice and starts walking daily, continuing while monitoring fluctuations in his blood pressure.

[1386] As described above, users can voluntarily review their behavior and make sustainable lifestyle improvements. This system is expected to improve the effectiveness of treatments for lifestyle-related diseases and contribute to reducing medical costs.

[1387] The processing flow will be explained below.

[1388] User Action

[1389] Step 1:

[1390] The user installs the application and launches the app.

[1391] Step 2:

[1392] The user enters their name, email address, and password on the account creation screen.

[1393] Step 3:

[1394] The user selects a person or character they trust and sets their avatar.

[1395] Step 4:

[1396] The user inputs health goals (e.g., blood pressure goal, weight loss goal).

[1397] Step 5:

[1398] The user enters information about their personality and background.

[1399] Step 6:

[1400] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details).

[1401] Step 7:

[1402] The user converses with the avatar and receives specific health advice from the avatar.

[1403] Step 8:

[1404] Users can view their progress and set new goals in the app.

[1405] Terminal handling

[1406] Step 1:

[1407] The terminal transmits the user's account information to the server.

[1408] Step 2:

[1409] The terminal stores the avatar information selected by the user.

[1410] Step 3:

[1411] The terminal transmits the health data entered by the user to the server.

[1412] Step 4:

[1413] The advice received by the terminal from the server is displayed as an avatar.

[1414] Step 5:

[1415] The device uploads the conversation log with the avatar to the server.

[1416] Server Processing

[1417] Step 1:

[1418] The server stores the received user account information in a database.

[1419] Step 2:

[1420] The server stores the user's selected avatar information in a database.

[1421] Step 3:

[1422] The server analyzes the health data entered by the user.

[1423] Step 4:

[1424] The server generates health advice based on the analysis results.

[1425] Step 5:

[1426] The server sends the generated advice to the terminal.

[1427] Step 6:

[1428] The server stores the conversation log with the avatar in a database.

[1429] Example 1

[1430] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1431] Conventional lifestyle-related disease management systems simply record users' health data and do not adequately support users in making specific behavioral changes. As a result, users find it difficult to grasp the specific steps they need to take to achieve their health goals, which hinders the improvement of lifestyle-related diseases.

[1432] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1433] In this invention, the server includes means for selecting a character trusted by the user, means for displaying the selected character as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information and generating health advice using an AI model, and means for providing the generated health advice to the user via the avatar, thereby enabling the user to easily understand specific behaviors and voluntarily change their behavior.

[1434] A "user" is an individual who uses this system and inputs health data and lifestyle information and receives advice from an avatar.

[1435] A "character" is a person or fictional entity that a user trusts, displayed as an avatar, and whose role is to provide health advice through interactions with the user.

[1436] An "avatar" is a virtual representation of a character that a user trusts, and serves as a means of providing health advice to the user.

[1437] "Health data" refers to health-related information such as blood pressure, weight, amount of exercise, and dietary details that a user inputs into the application.

[1438] "Lifestyle background information" is data that may affect the achievement of health goals, such as the user's occupation, lifestyle, and home environment.

[1439] The "server" is a central computer system that analyzes users' health data and lifestyle background information and generates health advice using AI models.

[1440] An "AI model" is an artificial intelligence algorithm that analyzes a user's health data and lifestyle background information and generates appropriate health advice based on that data.

[1441] "Health advice" is specific instructions and suggestions for improving health that are generated by the server using an AI model and provided to the user through an avatar.

[1442] "Interaction" refers to a two-way exchange between the user and the avatar in real time, including advice and feedback in response to the user's questions.

[1443] The present invention provides a system in which a user sets a character that the user trusts as an avatar and improves their lifestyle based on health advice provided by the avatar. An embodiment of this system will be described in detail below.

[1444] User Action

[1445] First, users install the dedicated application and create an account. This application is installed on a mobile device such as a smartphone or tablet. The user selects a trusted character and sets that avatar. Next, the user sets health goals and enters information about their personality and lifestyle.

[1446] As a concrete example, consider a case where a user sets a goal of "keeping blood pressure below 120 / 80 mmHg" and leads a desk-based lifestyle. The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, diet) into the application. The user also receives specific health advice through conversations with an avatar. Based on this advice, the user can change their behavior in their daily life.

[1447] Terminal handling

[1448] The device transmits the user's account information and health data to the server in real time, allowing the server to receive the latest health data and analyze it immediately. The device also provides an interface for interacting with the avatar and providing health advice and feedback to the user.

[1449] Server Processing

[1450] The server stores and analyzes the user's account information and daily health data. A generative AI model (e.g., a machine learning algorithm) is used for the analysis. Real-time advice based on the user's health data and lifestyle information is generated and sent to the device.

[1451] Specifically, when a user measures their blood pressure and enters the results into the application, the device sends the data to the server. The server analyzes the data and generates advice such as "Your blood pressure is high. Try some light aerobic exercise." This advice is sent to the device and displayed to the user via an avatar.

[1452] Specific examples

[1453] A middle-aged man installs the application and creates an account. He selects the character of a trusted sports instructor as his avatar. He then sets a goal of "keeping his blood pressure below 120 / 80 mmHg" and enters his desk-bound lifestyle. Every morning, he measures his blood pressure and enters the results into the application. He also records his diet and exercise. The avatar offers specific advice, such as "Try exercising a little more," and he begins walking daily. After a few weeks, he can see that his blood pressure is gradually approaching his target value.

[1454] This system encourages users to make specific behavioral changes and make sustainable lifestyle improvements. By utilizing generative AI models, it is possible to provide personalized health advice and effectively support users' health management.

[1455] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1456] Step 1: Initial User Setup

[1457] The user installs a dedicated application and creates an account, selects a trusted character, and configures its avatar. Input includes a username, password, and the selection of a trusted character. Output is account information and avatar configuration information. Specific actions include character selection and goal setting using drop-down menus and sliders.

[1458] Step 2: Set your health goals

[1459] The user sets health goals and inputs information about their personality and lifestyle. The input includes information such as blood pressure goals, exercise goals, diet, and living environment. The output is the user's goals and lifestyle information. Specific actions involve filling out a form and operating a slider to set the goal value.

[1460] Step 3: Enter your health data

[1461] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details) into the application. The input includes daily blood pressure, dietary details, exercise time, etc. The output is the input health data. Specific operations include measuring blood pressure and inputting the results into the application. The application also records meals and exercise.

[1462] Step 4: Sending data

[1463] The device transmits the health data entered by the user to the server in real time. At this time, the input data is encrypted before transmission. The input includes the health data entered by the user. The output is the data transmitted to the server. Specifically, the data is transmitted via the Internet, and a notification of successful transmission is displayed to the user.

[1464] Step 5: Save and analyze data

[1465] The server stores the received health data and analyzes it using a generative AI model. The input includes the health data sent from the device. The output is a trend analysis based on the analysis results and the user's health status. Specifically, the data is stored in a database and analyzed using machine learning algorithms.

[1466] Step 6: Generating Advice

[1467] The server generates health advice using an AI model based on the analysis results. The input includes the analysis results of the health data. The output is personalized health advice. Specifically, the AI ​​model generates appropriate health advice for each user in text format.

[1468] Step 7: Providing advice

[1469] The terminal provides the user with health advice received from the server. The input includes the health advice sent from the server. The output is the health advice displayed to the user through an avatar. Specifically, the avatar presents the advice to the user in text or voice on the application screen.

[1470] Step 8: Changing user behavior

[1471] The user changes their daily behavior based on advice from the avatar. The input includes health advice from the avatar. The output is a specific change in the user's behavior, which is expected to improve their health. As a specific action, the user performs and continues specific health behaviors such as walking or dietary restrictions.

[1472] (Application example 1)

[1473] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1474] Currently, there are several health management systems on the market for patients with lifestyle-related diseases, but few systems include specific dietary suggestions to comprehensively improve users' lifestyles. As a result, lifestyle-related disease prevention and management are often ineffective. Furthermore, conventional systems have the problem of difficulty in providing real-time dietary suggestions linked to health data. Conventional technologies make it difficult for users to select appropriate diets based on their own health data, which hinders lifestyle improvement.

[1475] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1476] In this invention, the server includes means for selecting a person or character trusted by the user, means for displaying the selected person or character as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information and generating health advice, means for providing the generated health advice to the user via the avatar, and means for suggesting an appropriate diet based on the generated health advice, thereby enabling the user to select an appropriate diet based on their own health condition and enabling more effective prevention and management of lifestyle-related diseases.

[1477] "User" refers to an individual who uses the system.

[1478] A "trusted person or character" refers to a real person or virtual character with whom a user feels a sense of affinity or trust.

[1479] An "avatar" is a digital representation of a person or character that a user trusts, and refers to a virtual presence that is used to interact with the user on the system.

[1480] "Health data" refers to information about a user's health status, such as physiological values ​​such as blood pressure, weight, and blood sugar levels, and diagnostic results.

[1481] "Lifestyle background information" refers to information about the user's living environment and habits that affect health, such as the user's occupation, daily activities, and eating habits.

[1482] "Analysis" refers to the process of evaluating a user's health status and identifying areas for improvement and risks based on the health data and lifestyle background information provided by the user.

[1483] "Health Advice" refers to specific advice or suggestions for improving or maintaining a user's health, generated based on the analysis.

[1484] "Means for suggesting meals" refers to a system that analyzes a user's health data and lifestyle background information and provides appropriate meal content and nutritional balance based on the results.

[1485] To implement the present invention, the following system configuration is required.

[1486] User Action

[1487] The first thing a user does is install a dedicated health management application on their smartphone. The user creates an account and selects a person or character they trust. For example, if the user selects a health and sports advisor as a trusted person, that person will appear as an avatar. The user then sets their own health goals and enters background information about their lifestyle, such as their occupation and eating habits. They enter their daily health data (e.g., blood pressure, weight, amount of exercise, and dietary details) into the app and receive health advice through interactions with the avatar.

[1488] Terminal handling

[1489] The program running on the device sends the user's account information and health data to a server in real time, and provides feedback to the user based on the analysis results returned from the server. An avatar within the application interacts with the user, offering advice such as, "Your blood pressure is high today. We recommend you eat a diet low in carbohydrates."

[1490] Server Processing

[1491] The server is built using programming languages ​​such as Python and stores and analyzes users' account information and daily health data. A generative AI model is used to analyze the health data, and appropriate health advice is generated based on the analysis results. Health advice and dietary suggestions based on the analysis are sent to the device in real time and provided to the user through an avatar.

[1492] Specifically, for example, when a user inputs their blood glucose level and BMI, the following prompt sentence is parsed by the generative AI model:

[1493] text

[1494] My blood sugar level is 150 mg / dL and my BMI is 26. What are some dietary suggestions and advice based on this health condition?

[1495] The generative AI model then generates appropriate advice and dietary suggestions, such as "Your blood sugar level is high, so eat a diet low in carbohydrates."

[1496] In this way, users can receive specific advice based on their own health status and are encouraged to take practical actions to prevent and manage lifestyle-related diseases. This system encourages users to autonomously change their health management behavior and enable them to maintain a healthier lifestyle.

[1497] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1498] Step 1:

[1499] The user installs a dedicated health management application on their smartphone and creates an account. They then select a trusted person or character to set up their avatar. During this initial setup step, the user also enters their health goals and lifestyle background information (e.g., occupation and dietary habits). The input data is collected by the device and sent to the server. The input for this step is the user's profile information and lifestyle background information, and the output is user profile data that stores this information.

[1500] Step 2:

[1501] The user inputs daily health data (e.g., blood pressure, weight, exercise, and dietary information) into the application. The device transmits this data to the server in real time. The input data is health-related information, and the output data is updated health data that is uploaded to the server.

[1502] Step 3:

[1503] The server performs data analysis based on the received health data and user profile. First, it requests an analysis by passing a prompt to the generative AI model. As a specific example, it generates a prompt such as, "Your blood pressure is 150 / 90 mmHg and your BMI is 28. Please give me some dietary suggestions and advice based on these health conditions." The input to this prompt is the health data to be analyzed on the server, and the output is the health advice generated as a result of the analysis.

[1504] Step 4:

[1505] The generative AI model analyzes the prompt text and generates appropriate health advice and dietary suggestions. For example, the generated advice might be, "Your blood pressure is high, so eat a low-salt diet." The input data is the prompt text, and the output data is the generated advice.

[1506] Step 5:

[1507] The server sends the generated health advice to the device. The device then provides the received advice to the user. An avatar within the application provides the advice in the form of voice or text through dialogue with the user. For example, the advice might be displayed in the form of "Your blood pressure is high today. We recommend a low-carbohydrate diet." The input of this step is the generated advice, and the output is feedback advice for the user.

[1508] Step 6:

[1509] The user takes action to improve their lifestyle habits according to the advice provided by the avatar. The user then enters the results into the app again to check the progress of their improvement. This data is also sent from the device to the server and reflected in the next analysis. The input for this step is new health data, and the output data is updated health data and the progress of improvement.

[1510] Through this series of processing steps, users receive specific advice based on their own health condition, enabling them to effectively improve their lifestyle habits.

[1511] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1512] This invention is a system for promoting behavioral change in patients with lifestyle-related diseases, which displays people or characters trusted by the user as avatars and provides health advice generated by analyzing the user's health data and lifestyle background information.The invention also includes an emotion engine that recognizes the user's emotions and responds appropriately according to the recognized emotions, thereby increasing the user's motivation and supporting sustainable health management.

[1513] User Action

[1514] Users install a dedicated application and create an account. When creating an account, they enter the required information (e.g., name, email address, password) and select a trusted person or character to set as their avatar. Next, users set health goals and enter information about their personality and lifestyle. After that, they enter daily health data (e.g., blood pressure, weight, amount of exercise, dietary details). This data is managed within the application, and users receive specific health advice through conversations with their avatar.

[1515] Furthermore, this invention incorporates an emotion engine that can recognize emotions through the user's voice input and facial expression analysis. Based on the recognized emotions, the avatar provides the user with appropriate health advice and words of encouragement. This motivates the user to take better care of themselves and encourages them to continue managing their health.

[1516] Terminal handling

[1517] The device sends the user's account information and health data to a server. The data entered by the user is uploaded to the server in real time. The device also interacts with the avatar and provides the user with health advice and feedback. At the same time, an emotion engine monitors the user's voice and facial expressions to recognize emotions. This information is also sent to the server and used for analysis.

[1518] Server Processing

[1519] The server stores and analyzes the user's account information and daily health data. Using an AI model, the server generates health advice based on the user's health data and lifestyle background information. It also analyzes emotional data recognized by an emotion engine and generates appropriate feedback based on the user's feelings. The generated advice and feedback are sent to the device and provided in real time via an avatar.

[1520] Specific examples

[1521] User: Ms. Sato (45 years old, female), Trusted person: Nutritionist Ms. Yamada

[1522] 1. Mr. Sato installs the application, creates an account, and sets his trusted nutritionist, Ms. Yamada, as his avatar.

[1523] 2. Mr. Sato sets a goal of "losing 5 kg" and enters his lifestyle information (mainly desk work, mostly eating out).

[1524] 3. Mr. Sato records his daily weight and dietary habits in the application.

[1525] 4. The server analyzes this data and generates advice such as, "We recommend that you change to a balanced diet and take a walk after dinner."

[1526] 5. The avatar Yamada provides advice based on the analysis results and encourages Sato to act.

[1527] 6. If Sato-san shows a depressed expression, the emotion engine will recognize this and Yamada-san's avatar will offer words of encouragement such as, "You're doing well, if you keep going you'll see results."

[1528] In this way, the system of the present invention allows users to receive accurate advice from trusted individuals, enabling them to continuously improve their lifestyle habits. Furthermore, receiving feedback based on their emotions can increase motivation for behavioral change.

[1529] The processing flow will be explained below.

[1530] User Action

[1531] Step 1:

[1532] The user installs the application and launches the app.

[1533] Step 2:

[1534] The user enters their name, email address, and password on the account creation screen.

[1535] Step 3:

[1536] The user selects a person or character they trust and sets their avatar.

[1537] Step 4:

[1538] The user inputs health goals (e.g., blood pressure goal, weight loss goal).

[1539] Step 5:

[1540] The user enters information about their personality and background.

[1541] Step 6:

[1542] The user inputs daily health data (e.g., blood pressure, weight, amount of exercise, dietary details).

[1543] Step 7:

[1544] The user converses with the avatar and receives specific health advice from the avatar.

[1545] Step 8:

[1546] The device's camera and microphone recognize the user's voice input and facial expressions to collect emotional information.

[1547] Step 9:

[1548] Users can view their progress and set new goals in the app.

[1549] Terminal handling

[1550] Step 1:

[1551] The terminal transmits the user's account information to the server.

[1552] Step 2:

[1553] The terminal stores the avatar information selected by the user.

[1554] Step 3:

[1555] The terminal transmits the health data entered by the user to the server.

[1556] Step 4:

[1557] The advice received by the terminal from the server is displayed as an avatar.

[1558] Step 5:

[1559] The terminal processes the user's voice input and facial expressions with an emotion engine, and transmits the emotion information to the server.

[1560] Step 6:

[1561] The device uploads the conversation log with the avatar to the server.

[1562] Server Processing

[1563] Step 1:

[1564] The server stores the received user account information in a database.

[1565] Step 2:

[1566] The server stores the user's selected avatar information in a database.

[1567] Step 3:

[1568] The server analyzes the health data entered by the user.

[1569] Step 4:

[1570] The server generates health advice based on the analysis results.

[1571] Step 5:

[1572] Based on the emotional information received from the emotion engine, the server modifies the analysis results and generates advice and feedback according to the emotion.

[1573] Step 6:

[1574] The server sends generated advice and feedback to the terminal.

[1575] Step 7:

[1576] The server stores the conversation log with the avatar in a database.

[1577] Specific examples

[1578] User: Ms. Sato (45 years old, female), Trusted person: Nutritionist Ms. Yamada

[1579] User Steps

[1580] Step 1:

[1581] Mr. Sato installs the application and launches it.

[1582] Step 2:

[1583] On the account creation screen, Sato enters his name, email address, and password.

[1584] Step 3:

[1585] Sato sets Yamada, a nutritionist he trusts, as his avatar.

[1586] Step 4:

[1587] Sato enters his health goal of "lose 5 kg."

[1588] Step 5:

[1589] Sato enters background information about his lifestyle, such as the fact that he mainly does desk work and eats out most of the time.

[1590] Step 6:

[1591] Sato enters his daily weight and dietary information into the application.

[1592] Step 7:

[1593] Talk to the avatar Yamada and receive health advice.

[1594] Step 8:

[1595] Sato's voice and facial expressions are recognized by the device's emotion engine, and emotional information is collected.

[1596] Step 9:

[1597] Sato checks the progress of his goals and sets new goals in the app.

[1598] Terminal Steps

[1599] Step 1:

[1600] The device sends Sato's account information to the server.

[1601] Step 2:

[1602] The device saves the avatar information selected by Sato.

[1603] Step 3:

[1604] The device sends the health data entered by Sato to the server.

[1605] Step 4:

[1606] The advice received by the terminal from the server is displayed as an avatar.

[1607] Step 5:

[1608] The device's emotion engine processes Sato's voice input and facial expressions, and sends the emotional information to the server.

[1609] Step 6:

[1610] The device uploads the conversation log with the avatar to the server.

[1611] Server Steps

[1612] Step 1:

[1613] The server stores Mr. Sato's account information in a database.

[1614] Step 2:

[1615] The server saves Sato's selected avatar information in a database.

[1616] Step 3:

[1617] The server analyzes the health data entered by Sato.

[1618] Step 4:

[1619] The server generates appropriate health advice based on the analysis results.

[1620] Step 5:

[1621] The server modifies the analysis results based on the emotional information from the emotion engine and generates advice and feedback according to the emotion.

[1622] Step 6:

[1623] The server sends generated advice and feedback to the terminal.

[1624] Step 7:

[1625] The server stores the conversation log with the avatar in a database.

[1626] Example 2

[1627] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1628] In modern society, lifestyle-related diseases are on the rise, making their prevention and management a critical issue. Patients with lifestyle-related diseases, who are known to have difficulty managing their conditions themselves, are particularly required to receive appropriate health management advice on an ongoing basis while maintaining their motivation. However, there are very few existing systems that provide customized advice tailored to the user's psychological state and individual lifestyle. Furthermore, there is a lack of systems that recognize the user's emotions in real time and provide appropriate feedback based on that information. This poses a challenge, making it difficult for users to motivate themselves to continuously manage their health.

[1629] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1630] In this invention, the server includes means for analyzing a user's health data and lifestyle background information to generate health advice, means for acquiring the user's voice input and facial expression data to analyze emotions, and means for providing feedback based on the analyzed emotions via the avatar, thereby making it possible to provide health advice and feedback customized to the user's individual lifestyle background and psychological state in real time.

[1631] A "user" is an individual who uses this system to manage their health.

[1632] A "trusted person or character" is a virtual person or character that a user psychologically trusts and that is displayed when receiving health advice.

[1633] An "avatar" is a virtual representation of a trusted person or character, and is a way to provide health advice and feedback to users.

[1634] "Health data" refers to information about a user's physical condition (e.g., weight, blood pressure, amount of exercise, dietary details).

[1635] "Lifestyle background information" is information about the user's lifestyle and environment (e.g., occupation, eating habits, sleeping habits).

[1636] "Analysis" is the process of generating appropriate health advice based on the input health data and lifestyle background information.

[1637] "Health advice" refers to health recommendations or instructions provided to the user based on the analysis results.

[1638] "Voice input" refers to voice information uttered by the user through the microphone of the terminal.

[1639] "Facial expression data" is information about facial expressions that express the user's emotions.

[1640] "Means for analyzing emotions" refers to technology for recognizing and analyzing a user's emotional state based on voice input and facial expression data.

[1641] "Feedback" is a response, such as encouragement or instruction, provided to the user based on the analyzed emotional state.

[1642] The "server" is a central processing unit that receives, stores, and analyzes health data, lifestyle background information, and emotional data, and then generates advice and feedback and sends it to the terminal.

[1643] "Real-time" is a concept that refers to the state in which input information is processed and reflected immediately without delay.

[1644] This invention is a system that analyzes a user's unique health data and lifestyle background information and provides customized health advice through a trusted avatar. It also analyzes the user's voice input and facial expression data and provides feedback according to their emotions, supporting continuous health management.

[1645] System configuration

[1646] The system mainly consists of the following parts:

[1647] 1. User Device

[1648] 2. Server

[1649] 3. Avatar display method

[1650] 4. Voice input and facial expression recognition

[1651] User device functions

[1652] User devices are smart devices (e.g., smartphones, tablets) on which users install and use applications. These devices use the following hardware and software:

[1653] Camera and microphone: Captures your facial expressions and voice.

[1654] Application software (e.g. iOS app, Android app): Enters health data and interacts with the avatar.

[1655] Network communication function: Sends user data to the server and receives data from the server.

[1656] Server Features

[1657] The server acts as a central processing unit and is responsible for:

[1658] Database (e.g. MySQL): Stores user health data and lifestyle information.

[1659] AI module (e.g., Python, TensorFlow): Analyzes health data and lifestyle background information and generates health advice.

[1660] Emotion engine: Analyzes the user's voice input and facial expression data to generate emotion-based feedback.

[1661] Avatar display method

[1662] The avatar display unit displays a trusted person or character selected by the user as a virtual avatar, which provides real-time health advice and emotional feedback to the user.

[1663] Example of operation

[1664] 1. User creates an account:

[1665] Users install the application and create an account by entering basic information such as their name, email address, and password. They then select a trusted person or character to set as their avatar.

[1666] 2. Enter your health goals and lifestyle information:

[1667] The user inputs background information such as their weight loss goal, occupation, diet, etc. For example, they can set a goal such as "lose 5 kg."

[1668] 3. Enter your daily health data:

[1669] Users record their daily health data, such as weight, exercise, and diet, in the application, which is then sent to the server in real time.

[1670] 4. Health advice generation:

[1671] The server analyzes the received health data and generates appropriate health advice, such as "We recommend that you change to a balanced diet and take a walk after dinner."

[1672] 5. Offer advice through an avatar:

[1673] The device receives health advice from the server and provides it to the user through an avatar, which communicates the advice both audibly and visually.

[1674] 6. Sentiment analysis and feedback provision:

[1675] The system analyzes the user's voice input and facial expression data to generate appropriate feedback based on their emotions. For example, if the user is feeling down, the avatar will provide an encouraging message such as, "You're doing well. If you keep going, you'll see results."

[1676] Prompt Sentence Examples

[1677] "I'm a 45-year-old woman who works a desk job. I eat out a lot and would like to lose 5kg. What health advice would you give me?"

[1678] In this way, the present invention is a system that provides health management tailored to the individual needs of the user and supports the formation of sustainable healthy habits.

[1679] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1680] Step 1:

[1681] A user creates an account and enters basic information.

[1682] Enter your name, email address, and password.

[1683] Specific operation: The user installs the application and taps the "New Registration" button. They enter their name, email address, and password in the form displayed on the screen and press the "Register" button.

[1684] Output: The user account information is sent to the server and the account is created.

[1685] Step 2:

[1686] Users select an avatar they trust and set their health goals and lifestyle background information.

[1687] Input: Select a trusted person or character, health goals, and background information (e.g., occupation, diet).

[1688] How it works: After creating an account, users are directed to an avatar selection screen where they can tap to select a trusted person or character. Next, they enter their health goal (e.g., "lose 5kg") and select background information from options.

[1689] Output: Avatar settings, health goals, and lifestyle background information are sent to the server.

[1690] Step 3:

[1691] The user inputs daily health data (weight, amount of exercise, diet, etc.).

[1692] Input: Daily health data such as weight, exercise, and diet.

[1693] Specific actions: The user accesses the health data entry screen and uses sliders and text fields to enter weight, exercise volume, dietary information, etc. Then taps the "Save" button.

[1694] Output: The entered health data is sent to the server and stored.

[1695] Step 4:

[1696] The server stores and analyzes health data and lifestyle background information.

[1697] Input: Health data, lifestyle background information.

[1698] How it works: The server stores the data in a MySQL database and uses Python and TensorFlow to analyze it, using algorithms to generate health advice.

[1699] Output: The generated health advice.

[1700] Step 5:

[1701] The server sends the generated health advice to the terminal.

[1702] Input: Generated health advice.

[1703] Specific operation: The server sends the analysis results, which are health advice, in JSON format to the terminal.

[1704] Output: The device receives the health advice.

[1705] Step 6:

[1706] The device provides advice to the user through an avatar.

[1707] Enter: health advice.

[1708] Specific operation: The device opens the avatar display screen, and the avatar conveys the received advice to the user audibly and visually. An animation is played, and the avatar says, "I recommend you change to a balanced diet and take a walk after dinner."

[1709] Output: The user receives the health advice.

[1710] Step 7:

[1711] The user expresses their emotions through voice and facial expressions.

[1712] Input: Voice input, facial expression data.

[1713] Specific actions: The user looks at the device's camera, makes facial expressions, and speaks into the microphone.

[1714] Output: Voice and facial expression data are input to the device.

[1715] Step 8:

[1716] The device transmits the emotion data to the server.

[1717] Input: speech and facial expression data.

[1718] Specific operation: The device transmits the captured emotion data to the server in real time.

[1719] Output: Emotion data is sent to the server.

[1720] Step 9:

[1721] The server analyzes the emotional data and generates appropriate feedback.

[1722] Input: Emotion data.

[1723] Specific operation: The server analyzes the received voice and facial expression data using an emotion engine to recognize the user's emotional state, and generates appropriate feedback based on the results.

[1724] Output: The generated feedback.

[1725] Step 10:

[1726] The device provides feedback to the user through an avatar.

[1727] Input: Generated feedback.

[1728] Specific operation: The device reflects the feedback on the avatar display screen and plays an animation in which the avatar says, "You're doing well. If you keep at it, you'll definitely see results."

[1729] Output: The user receives feedback.

[1730] (Application example 2)

[1731] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1732] In modern society, the prevention and management of lifestyle-related diseases is an important issue. However, many patients find it difficult to maintain motivation for self-management and do not have easy access to professional health advice. Furthermore, in order to receive regular health checks and lifestyle improvement methods, patients themselves must habitually record their health data and properly analyze it. This process is complex, making sustained management difficult, which presents a challenge.

[1733] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for displaying a person or character trusted by the user as an avatar, means for inputting the user's health data and lifestyle background information, means for analyzing the input health data and lifestyle background information to generate health advice, means for providing the user with the health advice generated through the avatar, and means having an emotion engine for analyzing the user's emotions in real time and generating appropriate feedback based on the analysis. This enables the user to continuously manage their health based on advice from the trusted avatar, and to maintain motivation through appropriate feedback according to their emotions.

[1734] A "trusted person or character" is a being that a user psychologically trusts, and is a virtual figure that is set up to provide health advice or increase motivation.

[1735] An "avatar" is a visual or audio representation of a trusted person or character that interacts with the user to provide health advice.

[1736] "Health data" refers to physical information accumulated by a user on a daily basis, specifically data including weight, blood pressure, amount of exercise, dietary details, and the like.

[1737] "Lifestyle background information" refers to information about the user's daily life, and specifically includes data such as occupation, diet, exercise habits, and home environment.

[1738] An "emotion engine" is a software or hardware mechanism that analyzes emotions from a user's facial expressions and voice and generates appropriate feedback based on the results.

[1739] "Analysis" refers to data processing to generate health advice using algorithms and AI technology based on input health data and lifestyle background information.

[1740] "Feedback" refers to appropriate responses and words of encouragement generated based on the user's emotions analyzed by the emotion engine.

[1741] A "server" refers to a computer system that stores and analyzes input data, and this system functions in conjunction with the user's terminal via a network.

[1742] This invention is a system that allows users to prevent and manage lifestyle-related diseases in a physical store. This system displays people or characters that the user trusts as avatars and provides health advice generated by analyzing the user's health data and lifestyle background information. It also has an emotion engine that recognizes the user's emotions in real time and provides appropriate feedback to increase the user's motivation and support continuous health management.

[1743] Hardware and Software

[1744] The user interacts with the device using a smartphone or smart glasses. These devices are equipped with a camera and microphone to capture the user's facial expressions and voice in real time. The camera detects the user's face using OpenCV and analyzes emotions using TensorFlow. Speech recognition is performed using the speech_recognition library.

[1745] Smartphone or smart glasses: A device that has a built-in camera and microphone and is used to run applications.

[1746] OpenCV: An image processing library used to detect faces in camera footage.

[1747] TensorFlow: A machine learning framework that analyzes user emotions using emotion recognition models.

[1748] speech_recognition: A library for speech recognition, converting user speech into text.

[1749] System processing flow

[1750] The server receives, stores, and analyzes the user's health data and lifestyle information. Data entered by the user through a dedicated application is uploaded to the server in real time. The emotion engine also recognizes the user's emotions from facial expressions and voice, which are also sent to the server for analysis.

[1751] Specifically, the following steps are performed:

[1752] 1. Data Collection:

[1753] Users use a smartphone or smart glasses to input health data (weight, blood pressure, exercise, diet, etc.) and lifestyle background information.

[1754] Using a camera and microphone, the user's facial expressions and voice data are also captured in real time.

[1755] 2. Data Analysis:

[1756] The server receives and stores this data.

[1757] The AI ​​model generates health advice based on health data and lifestyle background information.

[1758] The emotion engine analyzes facial and voice data to identify the user's emotional state.

[1759] 3. Providing advice:

[1760] The avatar provides generated health advice to the user.

[1761] At the same time, the avatar provides appropriate feedback and words of encouragement based on the results of emotion analysis.

[1762] Specific examples

[1763] For example, suppose a user says, "I've been finding it difficult to manage my diet lately. Do you have any simple advice?" This voice input is converted into text using the speech_recognition library. The emotion engine analyzes the user's facial expressions and recognizes that the user is in a "happy" emotional state. Based on this data, the server generates advice such as, "Managing your diet is certainly difficult. But start by trying to eat a balanced diet a little at a time every day. If you make small improvements, you'll see big changes." This advice is provided to the user through an avatar.

[1764] Example prompt sentence:

[1765] plain

[1766] A user is using "Health Concierge" in a physical store. The user speaks to the camera in a relaxed manner, and the microphone picks up the voice input. The emotion recognition model recognizes "happy" from the user's facial expression.

[1767] [Voice Typing]: "I've been having trouble managing my diet lately. Do you have any quick tips?"

[1768] (Expected output)

[1769] Avatar: "Diet management can be difficult, but start by focusing on eating a balanced diet every day. Small improvements can make a big difference."

[1770] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1771] Step 1:

[1772] The user activates their smartphone or smart glasses and inputs their health data and lifestyle information. Using a camera and microphone, facial expressions and voice data are also collected in real time. This input data includes health data (e.g., weight, blood pressure, exercise volume, dietary details) and lifestyle information (e.g., occupation, dietary habits, exercise habits, home environment).

[1773] Input: Health data, lifestyle background information, facial expression data, voice data

[1774] Output: Collected health data, lifestyle background information, facial expression data, and voice data are sent to the server.

[1775] Step 2:

[1776] The devices upload the collected data to a server in real time, and the data transfer is done using a secure protocol.

[1777] Input: Collected health data, background information, facial expression data, and voice data

[1778] Output: Data transferred to the server

[1779] Step 3:

[1780] The server stores the received health data and lifestyle background information. At the same time, it analyzes this data and generates optimal health advice for the user. A generative AI model is used for the analysis, and data processing and calculations are performed to generate health advice based on the input data.

[1781] Input: Transferred health data, background information

[1782] Output: Health advice

[1783] Step 4:

[1784] The server inputs facial expression data and voice data into the emotion engine, which analyzes the user's emotions in real time. The emotion engine uses TensorFlow to analyze the facial expression data and converts the voice data into text using the speech_recognition library.

[1785] Input: facial expression data, voice data

[1786] Output: Emotion analysis results

[1787] Step 5:

[1788] The server generates appropriate feedback based on the generated health advice and emotion analysis results.

[1789] Input: Health advice, sentiment analysis results

[1790] Output: Feedback message

[1791] Step 6:

[1792] The device receives health advice and feedback sent from the server and provides it to the user through an avatar, which displays advice and encouragement in real time through conversation with the user.

[1793] Input: Health advice, feedback message

[1794] Output: Advice and feedback displayed through avatars

[1795] As a specific example of how the avatar works, it uses voice recognition to convert what the user says into text, and then uses emotion analysis to provide appropriate advice and feedback based on the user's feelings.

[1796] The above are the specific processing steps for carrying out the invention. This system allows users to receive accurate health advice from a trusted avatar in a physical store, enabling continuous health management.

[1797] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1798] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1799] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1800] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1801] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1802] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1803] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1804] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1805] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1806] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1807] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1808] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1809] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1810] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1811] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1812] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1813] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1814] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1815] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1816] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1817] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1818] The following is further disclosed regarding the above embodiment.

[1819] (Claim 1)

[1820] a means for the user to select people or characters to trust;

[1821] means for displaying the selected person or character as an avatar;

[1822] a means for inputting user health data and lifestyle background information;

[1823] means for analyzing the input health data and lifestyle background information to generate health advice;

[1824] means for providing the generated health advice to a user through the avatar;

[1825] A system including:

[1826] (Claim 2)

[1827] 2. The system according to claim 1, further comprising means for storing the health data and lifestyle background information in a server and analyzing the stored health data and lifestyle background information.

[1828] (Claim 3)

[1829] 10. The system of claim 1, further comprising means for enabling the avatar to engage in real-time conversation with the user and provide health advice.

[1830] "Example 1"

[1831] (Claim 1)

[1832] a means for a user to select characters to trust;

[1833] means for displaying the selected character as an avatar;

[1834] a means for inputting user health data and lifestyle background information;

[1835] A means for analyzing the input health data and lifestyle background information and generating health advice using an AI model;

[1836] means for providing the generated health advice to a user through the avatar;

[1837] A system including:

[1838] (Claim 2)

[1839] The system according to claim 1, further comprising means for storing the health data and lifestyle background information in a server, and for the server to analyze the health data.

[1840] (Claim 3)

[1841] 10. The system of claim 1, further comprising means for enabling the avatar to engage in real-time conversation with the user and provide health advice.

[1842] "Application Example 1"

[1843] (Claim 1)

[1844] a means for the user to select people or characters to trust;

[1845] means for displaying the selected person or character as an avatar;

[1846] a means for inputting user health data and lifestyle background information;

[1847] means for analyzing the input health data and lifestyle background information to generate health advice;

[1848] means for providing the generated health advice to a user through the avatar;

[1849] means for suggesting an appropriate diet based on the generated health advice;

[1850] A system including:

[1851] (Claim 2)

[1852] 2. The system according to claim 1, further comprising means for storing the health data and lifestyle background information in a server and analyzing the stored health data and lifestyle background information.

[1853] (Claim 3)

[1854] 10. The system of claim 1, further comprising means for enabling the avatar to engage in real-time conversation with the user and provide health advice.

[1855] "Example 2: Combining Emotion Engines"

[1856] (Claim 1)

[1857] a means for the user to select people or characters to trust;

[1858] means for displaying the selected person or character as an avatar;

[1859] a means for inputting user health data and lifestyle background information;

[1860] means for analyzing the input health data and lifestyle background information to generate health advice;

[1861] means for providing the generated health advice to a user through the avatar;

[1862] A means for acquiring user voice input and facial expression data to analyze emotions;

[1863] means for providing feedback based on the analyzed emotion through said avatar;

[1864] A system including:

[1865] (Claim 2)

[1866] 2. The system according to claim 1, further comprising means for storing the health data and lifestyle background information in a server and analyzing the stored health data and lifestyle background information.

[1867] (Claim 3)

[1868] 10. The system of claim 1, further comprising means for enabling the avatar to engage in real-time conversation with the user and provide health advice.

[1869] "Application example 2 when combining emotion engines"

[1870] (Claim 1)

[1871] a means for the user to select people or characters to trust;

[1872] means for displaying the selected person or character as an avatar;

[1873] a means for inputting user health data and lifestyle background information;

[1874] means for analyzing the input health data and lifestyle background information to generate health advice;

[1875] means for providing the generated health advice to a user through the avatar;

[1876] a means for generating appropriate feedback based on an emotion engine that recognizes the user's emotion and that analyzes the user's emotion in real time;

[1877] A system including:

[1878] (Claim 2)

[1879] The system according to claim 1, further comprising means for storing the health data and lifestyle background information in a server and analyzing the stored health data and lifestyle background information.

[1880] (Claim 3)

[1881] 10. The system of claim 1, further comprising means for enabling the avatar to engage in real-time conversation with the user to provide generated health advice and feedback. [Explanation of symbols]

[1882] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for the user to select people or characters to trust; means for displaying the selected person or character as an avatar; a means for inputting user health data and lifestyle background information; means for analyzing the input health data and lifestyle background information to generate health advice; means for providing the generated health advice to a user through the avatar; A system including:

2. 2. The system according to claim 1, further comprising means for storing the health data and lifestyle background information in a server and analyzing the stored health data and lifestyle background information.

3. 10. The system of claim 1, further comprising means for enabling the avatar to converse with the user in real time and provide health advice.

Images