system

Abstract

We provide the system. [Solution] A data collection means consisting of a device for collecting user data, A data management means that anonymizes and manages the data received from the above data collection means, An analysis means for analyzing data managed by the above data management means to evaluate health risks, A plan generation means that generates individual care plans based on the results of the above analysis means, A plan provision means that provides the care plan generated by the above plan generation means to the user, A system that includes this.

Description

Technical Field

[0001] The technology of this disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The decline in the health status of the elderly and middle-aged and senior adults, especially the early detection and management of frailty risk, are social issues. In conventional medical and care services, it is difficult to monitor the health status in real time and identify risks at an early stage, and it has not reached the provision of care optimized individually. Therefore, an effective health management system adapted to different living habits and health statuses for each user is required.

Means for Solving the Problems

[0005] This invention uses data collection means, which consists of a device for collecting user data, to collect diverse health-related data in real time. Through data management means, which anonymizes and manages this data, user privacy is protected while the data is appropriately managed. Subsequently, analysis means, which analyzes the data managed by the data management means to evaluate health risks, are used to identify health conditions such as frailty risk at an early stage. Furthermore, the invention includes plan generation means, which generates individual care plans based on the analysis results, thereby providing optimized plans for each user and enabling specific and effective health management.

[0006] "Data collection means" refers to a device or system that collects various data related to the user's health in real time.

[0007] A "data management system" is a system that anonymizes collected data and manages it appropriately, thereby protecting user privacy while keeping the data available for use.

[0008] An "analysis means" is a system that performs an analysis process to evaluate the user's health risks based on data managed by a data management means.

[0009] The "plan generation means" is a system for generating a health management plan optimized for each user based on the results of the analysis means.

[0010] A "plan delivery method" is a system that notifies users of their generated health management plan and encourages them to implement it. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3]This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] The system of the present invention monitors the user's health status in real time and provides individually optimized care. This system is implemented using data collection means, data management means, analysis means, plan generation means, and plan provision means.

[0033] The data collection method involves collecting user health-related data via the device. This includes measuring heart rate and steps using wearable devices, and recording sleep and meals using smartphone applications. Users input their daily activities and health status into the app, which then allows detailed data to be stored on the device.

[0034] The data management system aggregates data transmitted from terminals onto a server, where it is anonymized and managed. This process is crucial from the perspective of protecting personal information and is designed to allow data to be utilized while protecting privacy.

[0035] The analysis is performed on a server. The collected data is evaluated by an analysis program, and frailty risk and other health risks are calculated. This analysis uses multimodal AI to comprehensively analyze health status from diverse data such as text, voice, images, and video.

[0036] The plan generation system creates individualized care plans for each user based on the analysis results. Specific action plans regarding exercise and diet are formulated according to the user's lifestyle and health goals. For example, users with a high risk of certain frailty may be offered exercises to strengthen muscles and dietary menus to improve balance.

[0037] The plan delivery system sends a plan generated from the server to the terminal and notifies the user. The user can adjust their daily life based on the proposed plan and continuously monitor its effects. The system further optimizes the plan based on the user's feedback.

[0038] As described above, the present invention realizes an advanced health management system that continuously monitors the user's health status and provides appropriate countermeasures. This allows users to engage in health maintenance activities with peace of mind and contributes to improving the operational efficiency of medical institutions and care service providers.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] The device collects the user's health-related data. Wearable devices measure heart rate and steps, while smartphone applications record details about meals and sleep, which the user inputs.

[0042] Step 2:

[0043] The device transmits the collected data to the server in real time. During this process, the data is encrypted to protect privacy and transmitted through a secure channel.

[0044] Step 3:

[0045] The server anonymizes the received data using data management tools and stores it in a database. This process protects users' personal information while preparing the data in a format that can be analyzed.

[0046] Step 4:

[0047] The server evaluates the stored data using analytical tools. Multimodal AI is used to analyze text, audio, images, and videos to comprehensively assess the user's health risks.

[0048] Step 5:

[0049] The server applies a plan generation method based on the analysis results to create an individualized care plan for the user. The plan generates recommendations for exercise and diet tailored to the type and degree of health risks.

[0050] Step 6:

[0051] The server sends the generated care plan to the terminal. A notification, including recommended actions, is delivered to the user using the plan delivery method.

[0052] Step 7:

[0053] Users can view their plans from their devices and incorporate them into their daily activities. The app records user actions and feedback, and sends the results to the server.

[0054] Step 8:

[0055] The server receives feedback from users and updates care plans as needed. It monitors the application and effectiveness of the plans and continuously optimizes them.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] Conventional health management systems have struggled to monitor individual users' health status in real time and provide optimized care plans based on that data. In particular, there is a need for technology that can integrate and analyze diverse information formats to perform more accurate health assessments and provide personalized action plans for each user.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for collecting user information, including biometric information, obtained from an information device worn by the user; information management means for anonymizing and storing the information received from the collection means; and analysis means for evaluating the information managed by the information management means and detecting changes in the user's health status. This makes it possible to monitor the user's health status in detail in real time and provide an individually optimized action plan.

[0061] "Means for collecting user information" refers to a system that acquires various data, including biometric information, from the information devices owned by the user.

[0062] "Information management means" refers to a system that anonymizes collected user information, securely stores it, and keeps it in a state where it can be used for later analysis.

[0063] An "analysis method" is a system that evaluates a user's health status based on managed information and analyzes changes in that status.

[0064] The "plan generation means" is a mechanism that creates individual action plans for users based on the evaluation results obtained from the analysis means.

[0065] A "plan delivery method" is a system that reliably communicates and instructs users on the generated action plan.

[0066] A "response information collection means" is a system that collects user feedback and transmits it to an information management means.

[0067] A "generative algorithm" is a computational method for comprehensively evaluating diverse information formats and deriving valuable insights.

[0068] This invention is an advanced health management system that monitors a user's health status in real time and provides individually optimized care plans. Several hardware and software components are required to implement the invention.

[0069] Users utilize wearable devices and smartphone applications. These devices collect biometric information such as heart rate, steps taken, sleep, and diet. Examples include heart rate monitors, fitness trackers, and smartphone apps for health management. These devices and applications continuously record everyday health-related data.

[0070] The device sends the collected data to the server at regular intervals. This data transmission is typically done via Wi-Fi or a mobile network. The server anonymizes the received data and securely stores it using information management measures.

[0071] The server utilizes multimodal AI technology to comprehensively analyze diverse data formats, including text, audio, images, and video. This allows it to assess the user's health risks and detect changes in their condition in real time. Based on the analysis results, a generative AI model is used to generate an individualized action plan for each user. This plan includes specific exercise and dietary instructions, outlining concrete actions aimed at improving the user's health.

[0072] The server notifies the terminal of the generated plan and provides it to the user. The user can adjust their daily life based on the proposed plan and send feedback from the terminal to the server, enabling continuous optimization.

[0073] As a concrete example, suppose a user regularly records their exercise and manages their calorie expenditure using an app. The server analyzes this data and identifies areas for improvement. As a result, the user receives specific advice such as "jog for 30 minutes every day" or "eat a high-protein meal for dinner."

[0074] An example of a prompt message might be: "A 50-year-old woman with an average daily step count of 5000. Please suggest a specific exercise and diet plan to maintain her health."

[0075] In this way, the present invention supports users in managing their health and enables more effective health maintenance activities.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] Users wear wearable devices to collect health data such as heart rate and steps taken. This allows biometric information to be input into the device in real time. The device aggregates and temporarily stores this data. For example, the average heart rate and total steps taken by the user throughout the day are recorded here.

[0079] Step 2:

[0080] The terminal sends aggregated data to the server at regular intervals. During this process, the data is encrypted during transmission. The server decrypts the received data, anonymizes it, and stores it in an information management system. This input data is transformed into a form that does not identify individual users and is managed securely.

[0081] Step 3:

[0082] The server feeds the data stored in the information management system into the analysis system. The analysis system uses a multimodal AI model to analyze this data and perform data calculations to assess the user's health risks. For example, it detects risks such as abnormal heart rate changes and lack of exercise. As output, it generates a health status assessment report for each user.

[0083] Step 4:

[0084] The server generates individual action plans using a plan generation system based on an assessment of health status obtained through analysis. It utilizes a generation AI model to construct exercise and dietary guidance optimized for the user's lifestyle and health goals. The output is a concrete and actionable plan for improving health, such as "walking three times a week."

[0085] Step 5:

[0086] The server sends the generated action plan to the terminal and provides it to the user. The terminal notifies the user of the received plan and displays it. The user follows this plan in their daily life and records their progress. As output, the user's continuous feedback is stored on the terminal.

[0087] Step 6:

[0088] The user sends feedback to the server via their device. The server analyzes the feedback and evaluates the effectiveness of the action plan that has already been generated. If necessary, it revises and optimizes the plan based on new data and feedback. The output is provided to the user as an improved new action plan.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] There is a need to effectively manage the health status of elderly and care-dependent users and provide care plans optimized for their individual needs, thereby promoting the maintenance of users' health while reducing the burden on caregivers and families. To achieve this kind of health management, it is necessary to efficiently collect users' health information and formulate appropriate care plans based on that information, but conventional systems do not adequately perform real-time analysis and individual optimization.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes an information acquisition means consisting of a device for acquiring user information, an information management means for anonymizing and storing the information obtained from the information acquisition means, and an evaluation means for analyzing the information stored by the information management means and evaluating health risks. This enables the immediate presentation of an individually optimized health management plan to the user, and allows for continuous health monitoring and optimization of the plan through feedback.

[0094] "Information acquisition means" refers to devices or equipment used to acquire user information. This acquired information is related to health status and lifestyle.

[0095] "Information management measures" refer to a system for anonymizing acquired user information and securely storing and managing it. This includes important processes for utilizing data while ensuring data privacy.

[0096] "Evaluation methods" refer to systems and algorithms that analyze managed information to assess health risks for users. A key feature of this analysis is the integrated use of diverse data formats.

[0097] A "health management plan" is a specific action plan aimed at maintaining or improving the health of individual users, based on the results calculated using evaluation methods.

[0098] This system provides advanced health management for users and offers individually optimized health management plans. The system is primarily implemented using information acquisition, information management, and evaluation methods.

[0099] The server uses wearable devices equipped with sensors and smartphone applications to collect user health data. This data includes heart rate, steps taken, sleep patterns, and meal records. Wearable devices transfer data to smartphones via Bluetooth, and smartphones send the data to the server. SSL encryption is used throughout this process to protect the data.

[0100] The server anonymizes received user data and securely stores and manages it as an information management tool. The data is stored using cloud storage and is managed thoroughly to prevent the identification of personal information. Furthermore, the data is regularly backed up to prevent data loss.

[0101] The evaluation method uses an analysis program running on the cloud, and data analysis is performed using generative AI models such as Google Cloud AI. Risk assessments related to the user's health are performed by integrating and analyzing different data formats. Based on the analysis results, the server generates an individualized health management plan and provides it to the user's smartphone as a push notification.

[0102] For example, if data is collected indicating that a woman in her 70s has decreased daily exercise and is at increased risk of frailty, a health management plan can be developed based on this information. The system contributes to improving the user's health by suggesting light walking and a meal plan that considers nutritional balance.

[0103] An example of a prompt sentence applied to a generative AI model is: "A woman in her 70s, recently experiencing a decrease in steps taken, is at high health risk. Please generate suggestions for light exercise and nutritional supplements." This prompt allows the system to concretize the suggestions and develop a personalized plan.

[0104] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0105] Step 1:

[0106] The device collects health data such as the user's heart rate, steps, sleep patterns, and meal records through wearable devices and smartphone applications. Input is data from wearable devices or manual input, and output is a set of this data. Sensor data is acquired directly and transferred to a smartphone using Bluetooth or Wi-Fi.

[0107] Step 2:

[0108] The device sends the collected data to the server. During this process, the data is encrypted via SSL authentication and transmitted securely. The input is the data collected in step 1, and the output is the encrypted data stored on the server. The data transmission protocol is executed, and the information is stored in cloud storage.

[0109] Step 3:

[0110] The server anonymizes the received data and stores it securely using information management measures. The input is the data sent in step 2, and the output is the anonymized data. Personal information is deleted or replaced, and the anonymization process is performed.

[0111] Step 4:

[0112] The server uses generative AI models, such as Google Cloud AI, to analyze anonymized data and assess health risks. The input is anonymized data, and the output is the result of the health risk assessment. The data format is corrected, fed into the AI ​​model, and the assessment results are obtained.

[0113] Step 5:

[0114] The server generates an individualized health management plan based on the analysis results. This plan reflects the user's current health status and lifestyle. The input is the result of a health risk assessment, and the output is an individualized health management plan. A prompt message (e.g., "Female in her 70s, recently decreased step count, high health risk. Please generate suggestions for light exercise and nutritional supplements.") is applied to the generating AI model to formulate a specific plan.

[0115] Step 6:

[0116] The server sends the generated health management plan to the device as a push notification, providing it to the user. The input is a personalized health management plan, and the output is plan information displayed on the device. The plan is delivered in real time using a push notification service.

[0117] This series of processes allows users to constantly receive the latest health management information and effectively optimize their own health status.

[0118] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0119] The present invention provides an advanced health management system that combines an emotion engine that recognizes the user's emotional state. This system includes data collection means, data management means, analysis means, plan generation means, plan provision means, and an emotion engine.

[0120] The data collection method uses a terminal to acquire user health-related and emotion-related data. Health data includes heart rate, steps taken, and sleep duration, while emotion-related data includes the user's spoken content and tone in voice and text.

[0121] The data management system protects user privacy by sending all collected data to a server and anonymizing it. The managed data is securely stored and used for analysis.

[0122] The emotion engine runs on a server and recognizes the user's emotions in real time from voice and text data. This engine uses natural language processing technology and voice analysis to identify emotional patterns.

[0123] The analysis method integrates and analyzes organized data and emotional data generated by an emotion engine. It considers emotional changes along with various data modalities to comprehensively evaluate the user's health risks.

[0124] The plan generation method creates a care plan that reflects the user's emotional state, based on the results of risk analysis and the results of an emotion engine. For example, if high stress levels are detected, a plan emphasizing relaxation will be generated.

[0125] In the plan delivery method, the generated care plan is delivered to the user via a terminal. Suggestions and advice are delivered to the user as notifications to support improvements in their daily life.

[0126] This system also allows users to provide feedback to improve the accuracy of their care plans. User feedback is sent to the server via a feedback collection mechanism and used to adjust the plans. This enables comprehensive and individualized management of users' health status, providing more effective health support that also considers emotional well-being. As a concrete example, if the emotional engine detects that a user is experiencing stress, the server quickly sends a care plan to the terminal, including recommendations for deep breathing exercises and meditation.

[0127] The following describes the processing flow.

[0128] Step 1:

[0129] The device collects the user's health-related and emotional data. Wearable devices measure heart rate and steps, while smartphone applications record emotional data, including the user's voice and text.

[0130] Step 2:

[0131] The device encrypts the collected data and sends it to the server using a secure protocol. For privacy protection, the data is anonymized before transmission.

[0132] Step 3:

[0133] The server organizes the received data using data management tools and stores it in a database. The stored data is then prepared and organized for analysis.

[0134] Step 4:

[0135] The server uses an emotion engine to analyze and recognize the user's emotional state in real time from voice and text data. This analysis utilizes natural language processing and speech pattern recognition technologies.

[0136] Step 5:

[0137] The server integrates accumulated health and emotional data using analytical tools to comprehensively assess the user's health risks. This assessment also reflects the user's emotional state.

[0138] Step 6:

[0139] The server uses a plan generation mechanism to generate individualized care plans based on the results of risk assessments and emotional states. For example, if a person is diagnosed with high stress levels, the plan will include activities that promote relaxation.

[0140] Step 7:

[0141] The server delivers the generated care plan to the terminal. The terminal sends a notification to the user and sets a reminder to encourage them to take action according to the plan at the appropriate time.

[0142] Step 8:

[0143] Users execute the care plan provided through their device and return the results and feedback to the server via the app. The server uses this feedback information to adjust and improve the plan.

[0144] (Example 2)

[0145] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0146] In modern society, personal health management presents diverse and complex challenges, and comprehensive health support systems that take into account users' emotional states are lacking. In particular, there is a need to understand the impact of emotional states on health risks in real time and to provide appropriate care plans quickly based on that understanding. Furthermore, a mechanism is needed to provide personalized health support while protecting user privacy.

[0147] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0148] In this invention, the server includes data collection means consisting of equipment for acquiring biometric and voice information, data management means for anonymizing and managing the information received from the data collection means, and an emotion engine that operates on the server and recognizes emotional states from voice and text information. This makes it possible to comprehensively evaluate health risks and provide care plans tailored to individual users by recognizing the user's emotional state in real time.

[0149] "Biometric information" refers to physiological data that indicates an individual's health status, and specific examples include heart rate, steps taken, and sleep duration.

[0150] "Audio information" refers to data that records the content and tone of a user's speech, and is used in natural language processing and speech analysis.

[0151] "Data collection methods" refer to devices and processes used to acquire users' biometric and voice information, specifically smartphones and smartwatches.

[0152] "Data management means" refers to a system that protects user privacy by organizing collected information and applying anonymization processing.

[0153] An "emotion engine" refers to a program that includes technology to analyze voice and text information and recognize the user's emotional state in real time.

[0154] "Analysis methods" refer to the process of comprehensively analyzing collected and managed data to assess health risks.

[0155] "Plan generation means" refers to a process or technology that automatically creates individual care plans based on the results of analysis and emotional states.

[0156] "Plan delivery method" refers to a system that provides users with generated care plans and offers appropriate guidance and suggestions.

[0157] "Feedback" refers to the information and opinions provided by users, which are used to improve the system and enhance its accuracy.

[0158] This invention is an advanced system that comprehensively manages a user's health and emotional state, and its implementation utilizes multiple hardware and software components. Specifically, smartphones and smartwatches are used as hardware, functioning as data collection tools. This allows for the real-time acquisition of biometric information such as heart rate, steps taken, sleep duration, and voice information.

[0159] The terminal sends this data to the server, where anonymization is applied to ensure the data is securely managed. The server uses natural language processing systems and speech analysis software to process the voice and text information. For example, the natural language processing system may use Python libraries or third-party API services.

[0160] Particularly important is the emotion engine, which operates on a server and detects the user's emotional state in real time from voice and text information. This uses generative AI models to identify emotional patterns. The data analyzed by the emotion engine is integrated with other biometric information on the server to assess overall health risks.

[0161] Based on the evaluation results, the server automatically generates a care plan tailored to the user. This plan generation utilizes an AI model and may include content that promotes stress reduction and relaxation. The created care plan is provided to the user via their device, and a notification function is used to support improvements in their daily life.

[0162] For example, if the emotion engine detects a high stress level in a user, the server immediately generates a care plan recommending deep breathing exercises to help manage stress and notifies the user on their smartphone. The user will then see a prompt message similar to the following:

[0163] Example of a prompt:

[0164] "Based on your current stress level, we'd like to suggest a specially prepared relaxation plan. Let's try a 5-minute meditation."

[0165] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0166] Step 1:

[0167] The device collects biometric and voice information from the user. Inputs include heart rate, steps, sleep duration, and voice data collected by smartwatches and smartphones. This data is recorded in real time and temporarily stored on the device. Specifically, the smartwatch measures heart rate using a pulse sensor, and the smartphone records the user's speech through its microphone.

[0168] Step 2:

[0169] The device sends the collected data to the server. The inputs include biometric and voice information obtained in step 1. The device securely uploads this data to the server and transmits it using a data transfer protocol. Specifically, the device encrypts and transmits the data via Wi-Fi or mobile data communication.

[0170] Step 3:

[0171] The server anonymizes the received data and stores it in a database. The input consists of biometric and voice information sent in step 2. The data is anonymized by removing user identification information and is managed securely. Specifically, the server uses an algorithm to remove user IDs, classify the data, and store it in the database.

[0172] Step 4:

[0173] The server analyzes the voice information using an emotion engine to recognize the user's emotional state. The input is the voice information saved in step 3. Using natural language processing and speech analysis techniques, it identifies emotional patterns and determines the type of emotion. Specifically, the server analyzes keywords and voice tone in the utterances to classify emotions such as joy, sadness, and anger.

[0174] Step 5:

[0175] The server integrates biometric and emotional data to assess health risks. Inputs include biometric data from step 3 and emotional states from step 4. Using a machine learning model, the collected data is analyzed to generate a user health risk profile. Specifically, the server performs data analysis using a Python library and detects anomalies.

[0176] Step 6:

[0177] The server generates a personalized care plan based on the analysis results. The inputs are the health risk profile and emotional state obtained in step 5. The AI ​​model is used to create a specific care plan (e.g., a meditation guide for stress relief). Specifically, the AI ​​model suggests appropriate actions based on the generated risk profile.

[0178] Step 7:

[0179] The device provides the user with the care plan received from the server. The input is the care plan generated in step 6. The device uses push notification functionality to notify the user of the care plan as a suggestion. Specifically, the device displays a notification such as "A relaxation plan is available" to prompt the user to take action.

[0180] (Application Example 2)

[0181] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0182] In modern society, health management and emotional stability are crucial issues. However, many existing systems only consider users' health data, making it difficult to provide comprehensive support that takes emotional states into account. Furthermore, they lack the means to provide flexible support that responds to users' health conditions and emotional changes, and to offer care plans tailored to individual needs.

[0183] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0184] In this invention, the server includes data collection means consisting of equipment for collecting user information, data management means for anonymizing and managing the information received from the data collection means, analysis means equipped with a function for evaluating health risks using analysis means, and emotion recognition means equipped with an analysis engine for recognizing emotional states. This enables the generation of individualized care plans that take into account the user's health condition and emotions, and allows for daily support based on health and emotional states in cooperation with application devices.

[0185] "User information" refers to data necessary to identify the user's health and emotional state, and includes heart rate, steps taken, sleep duration, voice data, and text content.

[0186] "Data collection means" refers to devices or systems used to obtain health data and emotion-related information from users.

[0187] A "data management system" is a system that has the functionality to securely store collected user information, anonymize it to protect privacy, and use it for analysis.

[0188] "Analysis methods" refer to algorithms and computational techniques used to evaluate health and emotional states based on managed data and to identify risks.

[0189] "Plan generation means" refers to a function that automatically creates a care plan optimized for the user based on the analysis results.

[0190] "Plan delivery means" refers to a function that transmits the generated care plan to the user's terminal or application device and provides the user with a proposal in an actionable format.

[0191] "Emotion recognition means" refers to a technology or engine used to analyze a user's voice or text data to recognize and identify their emotional state in real time.

[0192] "Support delivery means" refers to functions that provide users with daily support and suggestions through application devices based on health and emotional data.

[0193] The present invention describes embodiments for carrying out the invention. This invention realizes a system that grasps the user's health and emotional state in real time and provides a personalized care plan based on these. The main components and their functions are described below.

[0194] At the heart of the system lies data used to collect user information. Wearable devices such as smartwatches are used to collect physical information such as heart rate, steps taken, and sleep duration. Emotional information is also collected through user voice and text input.

[0195] This data is securely stored on servers as anonymized information using data management mechanisms. The data engine runs on cloud-based computing platforms (e.g., Amazon Web Services or Microsoft Azure) to analyze the collected data and assess health risks.

[0196] The analyzed data, combined with consideration of health and emotional states, generates an individualized care plan. This plan is then transmitted directly to a smartphone or computer terminal via a plan delivery system, allowing users to receive suggestions and advice through visual and auditory means.

[0197] Furthermore, natural language processing technologies such as the Google Cloud Natural Language API are used as a means of emotion recognition to identify the user's emotions from speech and text, and to provide appropriate actions and suggestions. As part of this process, if an unstable emotional state is detected, relaxation methods such as recommending deep breathing exercises will be suggested via the device.

[0198] For example, if a user uses a voice input system to say, "I'm feeling stressed right now," the AI ​​engine can analyze this data in real time and suggest appropriate relaxation methods, taking into account the user's heart rate.

[0199] An example of a prompt message is, "Generate a prompt that evaluates the user's stress level in real time based on the analysis of the user's voice data." This enables comprehensive care for the user's health and emotions.

[0200] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0201] Step 1:

[0202] The device collects information such as the user's heart rate, steps, sleep duration, and voice data through a smartwatch or voice input device. At this stage, raw data collected from the user is obtained as input. The collected data is temporarily stored on the device.

[0203] Step 2:

[0204] The device sends the collected data to the server. The server receives this data and applies an anonymization process. Here, data reduction and transformation (e.g., hashing) are performed to make it impossible to identify individuals, and the results are output.

[0205] Step 3:

[0206] The server processes anonymized data using analytical tools. It simultaneously analyzes health data (heart rate, steps, etc.) and emotional data (voice tone, keywords), and uses a generative AI model to determine health risks and emotional states. This analysis outputs evaluation results regarding health status and stress levels.

[0207] Step 4:

[0208] Based on the analysis results, the server uses a plan generation mechanism to automatically generate an optimal care plan for the user. At this stage, a plan is output that includes specific actions (e.g., relaxation techniques, exercise suggestions) based on the user's health risks and emotional state.

[0209] Step 5:

[0210] The generated care plan is sent from the server to the terminal. The terminal displays and communicates the plan to the user visually or audibly. In this step, the user can receive specific advice and suggestions based on the analysis results.

[0211] Step 6:

[0212] Users can provide feedback on the proposed care plan. The feedback data is sent from the terminal to the server, which then uses this data to generate future plans. This feedback process results in more refined plans being offered as future proposals.

[0213] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0214] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0215] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0216] [Second Embodiment]

[0217] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0218] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0219] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0220] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0221] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0222] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0223] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0224] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0225] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0226] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0227] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0228] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0229] The system of the present invention monitors the user's health status in real time and provides individually optimized care. This system is implemented using data collection means, data management means, analysis means, plan generation means, and plan provision means.

[0230] The data collection method involves collecting user health-related data via the device. This includes measuring heart rate and steps using wearable devices, and recording sleep and meals using smartphone applications. Users input their daily activities and health status into the app, which then allows detailed data to be stored on the device.

[0231] The data management system aggregates data transmitted from terminals onto a server, where it is anonymized and managed. This process is crucial from the perspective of protecting personal information and is designed to allow data to be utilized while protecting privacy.

[0232] The analysis is performed on a server. The collected data is evaluated by an analysis program, and frailty risk and other health risks are calculated. This analysis uses multimodal AI to comprehensively analyze health status from diverse data such as text, voice, images, and video.

[0233] The plan generation system creates individualized care plans for each user based on the analysis results. Specific action plans regarding exercise and diet are formulated according to the user's lifestyle and health goals. For example, users with a high risk of certain frailty may be offered exercises to strengthen muscles and dietary menus to improve balance.

[0234] The plan delivery system sends a plan generated from the server to the terminal and notifies the user. The user can adjust their daily life based on the proposed plan and continuously monitor its effects. The system further optimizes the plan based on the user's feedback.

[0235] As described above, the present invention realizes an advanced health management system that continuously monitors the user's health status and provides appropriate countermeasures. This allows users to engage in health maintenance activities with peace of mind and contributes to improving the operational efficiency of medical institutions and care service providers.

[0236] The following describes the processing flow.

[0237] Step 1:

[0238] The device collects the user's health-related data. Wearable devices measure heart rate and steps, while smartphone applications record details about meals and sleep, which the user inputs.

[0239] Step 2:

[0240] The device transmits the collected data to the server in real time. During this process, the data is encrypted to protect privacy and transmitted through a secure channel.

[0241] Step 3:

[0242] The server anonymizes the received data using data management tools and stores it in a database. This process protects users' personal information while preparing the data in a format that can be analyzed.

[0243] Step 4:

[0244] The server evaluates the stored data using analytical tools. Multimodal AI is used to analyze text, audio, images, and videos to comprehensively assess the user's health risks.

[0245] Step 5:

[0246] The server applies a plan generation method based on the analysis results to create an individualized care plan for the user. The plan generates recommendations for exercise and diet tailored to the type and degree of health risks.

[0247] Step 6:

[0248] The server sends the generated care plan to the terminal. A notification, including recommended actions, is delivered to the user using the plan delivery method.

[0249] Step 7:

[0250] Users can view their plans from their devices and incorporate them into their daily activities. The app records user actions and feedback, and sends the results to the server.

[0251] Step 8:

[0252] The server receives feedback from users and updates care plans as needed. It monitors the application and effectiveness of the plans and continuously optimizes them.

[0253] (Example 1)

[0254] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0255] Conventional health management systems have struggled to monitor individual users' health status in real time and provide optimized care plans based on that data. In particular, there is a need for technology that can integrate and analyze diverse information formats to perform more accurate health assessments and provide personalized action plans for each user.

[0256] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0257] In this invention, the server includes means for collecting user information, including biometric information, obtained from an information device worn by the user; information management means for anonymizing and storing the information received from the collection means; and analysis means for evaluating the information managed by the information management means and detecting changes in the user's health status. This makes it possible to monitor the user's health status in detail in real time and provide an individually optimized action plan.

[0258] "Means for collecting user information" refers to a system that acquires various data, including biometric information, from the information devices owned by the user.

[0259] "Information management means" refers to a system that anonymizes collected user information, securely stores it, and keeps it in a state where it can be used for later analysis.

[0260] An "analysis method" is a system that evaluates a user's health status based on managed information and analyzes changes in that status.

[0261] The "plan generation means" is a mechanism that creates individual action plans for users based on the evaluation results obtained from the analysis means.

[0262] A "plan delivery method" is a system that reliably communicates and instructs users on the generated action plan.

[0263] A "response information collection means" is a system that collects user feedback and transmits it to an information management means.

[0264] A "generative algorithm" is a computational method for comprehensively evaluating diverse information formats and deriving valuable insights.

[0265] This invention is an advanced health management system that monitors a user's health status in real time and provides individually optimized care plans. Several hardware and software components are required to implement the invention.

[0266] Users utilize wearable devices and smartphone applications. These devices collect biometric information such as heart rate, steps taken, sleep, and diet. Examples include heart rate monitors, fitness trackers, and smartphone apps for health management. These devices and applications continuously record everyday health-related data.

[0267] The device sends the collected data to the server at regular intervals. This data transmission is typically done via Wi-Fi or a mobile network. The server anonymizes the received data and securely stores it using information management measures.

[0268] The server utilizes multimodal AI technology to comprehensively analyze diverse data formats, including text, audio, images, and video. This allows it to assess the user's health risks and detect changes in their condition in real time. Based on the analysis results, a generative AI model is used to generate an individualized action plan for each user. This plan includes specific exercise and dietary instructions, outlining concrete actions aimed at improving the user's health.

[0269] The server notifies the terminal of the generated plan and provides it to the user. The user can adjust their daily life based on the proposed plan and send feedback from the terminal to the server, enabling continuous optimization.

[0270] As a concrete example, suppose a user regularly records their exercise and manages their calorie expenditure using an app. The server analyzes this data and identifies areas for improvement. As a result, the user receives specific advice such as "jog for 30 minutes every day" or "eat a high-protein meal for dinner."

[0271] An example of a prompt message might be: "A 50-year-old woman with an average daily step count of 5000. Please suggest a specific exercise and diet plan to maintain her health."

[0272] In this way, the present invention supports users in managing their health and enables more effective health maintenance activities.

[0273] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0274] Step 1:

[0275] Users wear wearable devices to collect health data such as heart rate and steps taken. This allows biometric information to be input into the device in real time. The device aggregates and temporarily stores this data. For example, the average heart rate and total steps taken by the user throughout the day are recorded here.

[0276] Step 2:

[0277] The terminal sends the data aggregated at regular intervals to the server. At this time, the data is encrypted for communication. The server decrypts the received data and stores it in the information management means through an anonymization process. This input data is converted into a form that cannot identify the user's individual and is managed securely.

[0278] Step 3:

[0279] The server inputs the data stored in the information management means into the analysis means. In the analysis means, using a multimodal AI model, this data is analyzed and data calculations are performed to evaluate the user's health risk. For example, risks such as abnormal heart rate changes and lack of exercise are detected. As an output, a health status evaluation report for each user is generated.

[0280] Step 4:

[0281] The server generates an individual action plan using the plan generation means based on the evaluation of the health status by the analysis means. Utilizing a generation AI model, guidance on exercise and diet optimized for the user's lifestyle and health goals is constructed. What is output is a specific and feasible plan for health improvement. For example, something like "Walking three times a week".

[0282] Step 5:

[0283] The server sends the generated action plan to the terminal and provides it to the user. The terminal notifies and displays the received plan to the user. The user follows this plan in daily life and records the implementation status. As an output, the user's continuous feedback is saved on the terminal.

[0284] Step 6:

[0285] The user sends feedback to the server through the terminal. The server analyzes the feedback and evaluates the effectiveness of the already generated action plan. If necessary, the plan is reviewed and optimized based on new data and feedback. The output is provided to the user as a new improved action plan.

[0286] (Application Example 1)

[0287] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0288] There is a need to effectively manage the health conditions of the elderly and users who require care, and to provide a care plan optimized for individual needs, so as to promote the maintenance of the users' health while reducing the burden on caregivers and family members. In order to perform such health management, it is necessary to efficiently collect the health information of the users and formulate an appropriate care plan based on it. However, in conventional systems, real-time analysis and individual optimization are not sufficiently carried out.

[0289] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0290] In this invention, the server includes an information acquisition means which is a device for acquiring user information, an information management means for anonymizing and storing the information obtained from the above information acquisition means, and an evaluation means for analyzing the information stored by the above information management means and evaluating the risks related to health. Thereby, it becomes possible to present an immediately individually optimized health management plan to the user and to optimize the plan through continuous health monitoring and feedback.

[0291] The "information acquisition means" refers to a device or device for acquiring user information. This acquired information is related to health status and lifestyle.

[0292] "Information management measures" refer to a system for anonymizing acquired user information and securely storing and managing it. This includes important processes for utilizing data while ensuring data privacy.

[0293] "Evaluation methods" refer to systems and algorithms that analyze managed information to assess health risks for users. A key feature of this analysis is the integrated use of diverse data formats.

[0294] A "health management plan" is a specific action plan aimed at maintaining or improving the health of individual users, based on the results calculated using evaluation methods.

[0295] This system provides advanced health management for users and offers individually optimized health management plans. The system is primarily implemented using information acquisition, information management, and evaluation methods.

[0296] The server uses wearable devices equipped with sensors and smartphone applications to collect user health data. This data includes heart rate, steps taken, sleep patterns, and meal records. Wearable devices transfer data to smartphones via Bluetooth, and smartphones send the data to the server. SSL encryption is used throughout this process to protect the data.

[0297] The server anonymizes received user data and securely stores and manages it as an information management tool. The data is stored using cloud storage and is managed thoroughly to prevent the identification of personal information. Furthermore, the data is regularly backed up to prevent data loss.

[0298] The evaluation method involves using an analysis program running on the cloud, with data analysis performed using generative AI models such as Google Cloud AI. Risk assessments related to the user's health are conducted by integrating and analyzing different data formats. Based on the analysis results, the server generates an individualized health management plan and provides it to the user's smartphone as a push notification.

[0299] For example, if data is collected indicating that a woman in her 70s has decreased daily exercise and is at increased risk of frailty, a health management plan can be developed based on this information. The system contributes to improving the user's health by suggesting light walking and a meal plan that considers nutritional balance.

[0300] An example of a prompt sentence applied to a generative AI model is: "A woman in her 70s, recently experiencing a decrease in steps taken, is at high health risk. Please generate suggestions for light exercise and nutritional supplements." This prompt allows the system to concretize the suggestions and develop a personalized plan.

[0301] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0302] Step 1:

[0303] The device collects health data such as the user's heart rate, steps, sleep patterns, and meal records through wearable devices and smartphone applications. Input is data from wearable devices or manual input, and output is a set of this data. Sensor data is acquired directly and transferred to a smartphone using Bluetooth or Wi-Fi.

[0304] Step 2:

[0305] The terminal sends the collected data to the server. In this process, the data is encrypted through SSL authentication and sent securely. The input is the data collected in step 1, and the output is the encrypted data stored on the server. Execute the data transmission protocol and store the information in cloud storage.

[0306] Step 3:

[0307] The server anonymizes the received data and stores it securely by the information management means. The input is the data sent in step 2, and the output is the anonymized data. Delete or replace personal information and implement the anonymization process.

[0308] Step 4:

[0309] The server uses a generative AI model such as Google Cloud AI to analyze the anonymized data and evaluate the health risks. The input is the anonymized data, and the output is the result of the health risk assessment. Format the data format, input it into the AI model, and obtain the evaluation result.

[0310] Step 5:

[0311] The server generates an individual health management plan based on the analysis results. This plan reflects the user's current health status and lifestyle. The input is the result of the health risk assessment, and the output is the individualized health management plan. Apply a prompt sentence (e.g., "A 70-year-old woman, recent decrease in walking distance, high health risk. Please generate suggestions for light exercise and nutritional supplements.") to the generative AI model to formulate a specific plan.

[0312] Step 6:

[0313] The server sends the generated health management plan to the terminal as a push notification and provides it to the user. The input is the individualized health management plan, and the output is the plan information displayed on the terminal. Use the push notification service to deliver the plan in real time.

[0314] This series of processes allows users to constantly receive the latest health management information and effectively optimize their own health status.

[0315] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0316] The present invention provides an advanced health management system that combines an emotion engine that recognizes the user's emotional state. This system includes data collection means, data management means, analysis means, plan generation means, plan provision means, and an emotion engine.

[0317] The data collection method uses a terminal to acquire user health-related and emotion-related data. Health data includes heart rate, steps taken, and sleep duration, while emotion-related data includes the user's spoken content and tone in voice and text.

[0318] The data management system protects user privacy by sending all collected data to a server and anonymizing it. The managed data is securely stored and used for analysis.

[0319] The emotion engine runs on a server and recognizes the user's emotions in real time from voice and text data. This engine uses natural language processing technology and voice analysis to identify emotional patterns.

[0320] The analysis method integrates and analyzes organized data and emotional data generated by an emotion engine. It considers emotional changes along with various data modalities to comprehensively evaluate the user's health risks.

[0321] The plan generation method creates a care plan that reflects the user's emotional state, based on the results of risk analysis and the results of an emotion engine. For example, if high stress levels are detected, a plan emphasizing relaxation will be generated.

[0322] In the plan delivery method, the generated care plan is delivered to the user via a terminal. Suggestions and advice are delivered to the user as notifications to support improvements in their daily life.

[0323] This system also allows users to provide feedback to improve the accuracy of their care plans. User feedback is sent to the server via a feedback collection mechanism and used to adjust the plans. This enables comprehensive and individualized management of users' health status, providing more effective health support that also considers emotional well-being. As a concrete example, if the emotional engine detects that a user is experiencing stress, the server quickly sends a care plan to the terminal, including recommendations for deep breathing exercises and meditation.

[0324] The following describes the processing flow.

[0325] Step 1:

[0326] The device collects the user's health-related and emotional data. Wearable devices measure heart rate and steps, while smartphone applications record emotional data, including the user's voice and text.

[0327] Step 2:

[0328] The device encrypts the collected data and sends it to the server using a secure protocol. For privacy protection, the data is anonymized before transmission.

[0329] Step 3:

[0330] The server organizes the received data using data management tools and stores it in a database. The stored data is then prepared and organized for analysis.

[0331] Step 4:

[0332] The server uses an emotion engine to analyze and recognize the user's emotional state in real time from voice and text data. This analysis utilizes natural language processing and speech pattern recognition technologies.

[0333] Step 5:

[0334] The server integrates accumulated health and emotional data using analytical tools to comprehensively assess the user's health risks. This assessment also reflects the user's emotional state.

[0335] Step 6:

[0336] The server uses a plan generation mechanism to generate individualized care plans based on the results of risk assessments and emotional states. For example, if a person is diagnosed with high stress levels, the plan will include activities that promote relaxation.

[0337] Step 7:

[0338] The server delivers the generated care plan to the terminal. The terminal sends a notification to the user and sets a reminder to encourage them to take action according to the plan at the appropriate time.

[0339] Step 8:

[0340] Users execute the care plan provided through their device and return the results and feedback to the server via the app. The server uses this feedback information to adjust and improve the plan.

[0341] (Example 2)

[0342] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0343] In modern society, personal health management presents diverse and complex challenges, and comprehensive health support systems that take into account users' emotional states are lacking. In particular, there is a need to understand the impact of emotional states on health risks in real time and to provide appropriate care plans quickly based on that understanding. Furthermore, a mechanism is needed to provide personalized health support while protecting user privacy.

[0344] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0345] In this invention, the server includes data collection means consisting of equipment for acquiring biometric and voice information, data management means for anonymizing and managing the information received from the data collection means, and an emotion engine that operates on the server and recognizes emotional states from voice and text information. This makes it possible to comprehensively evaluate health risks and provide care plans tailored to individual users by recognizing the user's emotional state in real time.

[0346] "Biometric information" refers to physiological data that indicates an individual's health status, and specific examples include heart rate, steps taken, and sleep duration.

[0347] "Audio information" refers to data that records the content and tone of a user's speech, and is used in natural language processing and speech analysis.

[0348] "Data collection methods" refer to devices and processes used to acquire users' biometric and voice information, specifically smartphones and smartwatches.

[0349] "Data management means" refers to a system that protects user privacy by organizing collected information and applying anonymization processing.

[0350] An "emotion engine" refers to a program that includes technology to analyze voice and text information and recognize the user's emotional state in real time.

[0351] "Analysis methods" refer to the process of comprehensively analyzing collected and managed data to assess health risks.

[0352] "Plan generation means" refers to a process or technology that automatically creates individual care plans based on the results of analysis and emotional states.

[0353] "Plan delivery method" refers to a system that provides users with generated care plans and offers appropriate guidance and suggestions.

[0354] "Feedback" refers to the information and opinions provided by users, which are used to improve the system and enhance its accuracy.

[0355] This invention is an advanced system that comprehensively manages a user's health and emotional state, and its implementation utilizes multiple hardware and software components. Specifically, smartphones and smartwatches are used as hardware, functioning as data collection tools. This allows for the real-time acquisition of biometric information such as heart rate, steps taken, sleep duration, and voice information.

[0356] The terminal sends this data to the server, where anonymization is applied to ensure the data is securely managed. The server uses natural language processing systems and speech analysis software to process the voice and text information. For example, the natural language processing system may use Python libraries or third-party API services.

[0357] Particularly important is the emotion engine, which operates on a server and detects the user's emotional state in real time from voice and text information. This uses generative AI models to identify emotional patterns. The data analyzed by the emotion engine is integrated with other biometric information on the server to assess overall health risks.

[0358] Based on the evaluation results, the server automatically generates a care plan tailored to the user. This plan generation utilizes an AI model and may include content that promotes stress reduction and relaxation. The created care plan is provided to the user via their device, and a notification function is used to support improvements in their daily life.

[0359] For example, if the emotion engine detects a high stress level in a user, the server immediately generates a care plan recommending deep breathing exercises to help manage stress and notifies the user on their smartphone. The user will then see a prompt message similar to the following:

[0360] Example of a prompt:

[0361] "Based on your current stress level, we'd like to suggest a specially prepared relaxation plan. Let's try a 5-minute meditation."

[0362] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0363] Step 1:

[0364] The device collects biometric and voice information from the user. Inputs include heart rate, steps, sleep duration, and voice data collected by smartwatches and smartphones. This data is recorded in real time and temporarily stored on the device. Specifically, the smartwatch measures heart rate using a pulse sensor, and the smartphone records the user's speech through its microphone.

[0365] Step 2:

[0366] The device sends the collected data to the server. The inputs include biometric and voice information obtained in step 1. The device securely uploads this data to the server and transmits it using a data transfer protocol. Specifically, the device encrypts and transmits the data via Wi-Fi or mobile data communication.

[0367] Step 3:

[0368] The server anonymizes the received data and stores it in a database. The input consists of biometric and voice information sent in step 2. The data is anonymized by removing user identification information and is managed securely. Specifically, the server uses an algorithm to remove user IDs, classify the data, and store it in the database.

[0369] Step 4:

[0370] The server analyzes the voice information using an emotion engine to recognize the user's emotional state. The input is the voice information saved in step 3. Using natural language processing and speech analysis techniques, it identifies emotional patterns and determines the type of emotion. Specifically, the server analyzes keywords and voice tone in the utterances to classify emotions such as joy, sadness, and anger.

[0371] Step 5:

[0372] The server integrates biometric and emotional data to assess health risks. Inputs include biometric data from step 3 and emotional states from step 4. Using a machine learning model, the collected data is analyzed to generate a user health risk profile. Specifically, the server performs data analysis using a Python library and detects anomalies.

[0373] Step 6:

[0374] The server generates a personalized care plan based on the analysis results. The inputs are the health risk profile and emotional state obtained in step 5. The AI ​​model is used to create a specific care plan (e.g., a meditation guide for stress relief). Specifically, the AI ​​model suggests appropriate actions based on the generated risk profile.

[0375] Step 7:

[0376] The device provides the user with the care plan received from the server. The input is the care plan generated in step 6. The device uses push notification functionality to notify the user of the care plan as a suggestion. Specifically, the device displays a notification such as "A relaxation plan is available" to prompt the user to take action.

[0377] (Application Example 2)

[0378] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0379] In modern society, health management and emotional stability are crucial issues. However, many existing systems only consider users' health data, making it difficult to provide comprehensive support that takes emotional states into account. Furthermore, they lack the means to provide flexible support that responds to users' health conditions and emotional changes, and to offer care plans tailored to individual needs.

[0380] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0381] In this invention, the server includes data collection means consisting of equipment for collecting user information, data management means for anonymizing and managing the information received from the data collection means, analysis means equipped with a function for evaluating health risks using analysis means, and emotion recognition means equipped with an analysis engine for recognizing emotional states. This enables the generation of individualized care plans that take into account the user's health condition and emotions, and allows for daily support based on health and emotional states in cooperation with application devices.

[0382] "User information" refers to data necessary to identify the user's health and emotional state, and includes heart rate, steps taken, sleep duration, voice data, and text content.

[0383] "Data collection means" refers to devices or systems used to obtain health data and emotion-related information from users.

[0384] A "data management system" is a system that has the functionality to securely store collected user information, anonymize it to protect privacy, and use it for analysis.

[0385] "Analysis methods" refer to algorithms and computational techniques used to evaluate health and emotional states based on managed data and to identify risks.

[0386] "Plan generation means" refers to a function that automatically creates a care plan optimized for the user based on the analysis results.

[0387] "Plan delivery means" refers to a function that transmits the generated care plan to the user's terminal or application device and provides the user with a proposal in an actionable format.

[0388] "Emotion recognition means" refers to a technology or engine used to analyze a user's voice or text data to recognize and identify their emotional state in real time.

[0389] "Support delivery means" refers to functions that provide users with daily support and suggestions through application devices based on health and emotional data.

[0390] The present invention describes embodiments for carrying out the invention. This invention realizes a system that grasps the user's health and emotional state in real time and provides a personalized care plan based on these. The main components and their functions are described below.

[0391] At the heart of the system lies data used to collect user information. Wearable devices such as smartwatches are used to collect physical information such as heart rate, steps taken, and sleep duration. Emotional information is also collected through user voice and text input.

[0392] This data is securely stored on servers as anonymized information using data management mechanisms. The data engine runs on a cloud-based computing platform (e.g., Amazon Web Services or Microsoft Azure) to analyze the collected data and assess health risks.

[0393] The analyzed data, combined with consideration of health and emotional states, generates an individualized care plan. This plan is then transmitted directly to a smartphone or computer terminal via a plan delivery system, allowing users to receive suggestions and advice through visual and auditory means.

[0394] Furthermore, natural language processing technologies such as the Google Cloud Natural Language API are used as a means of emotion recognition to identify the user's emotions from speech and text, and to provide appropriate actions and suggestions. As part of this process, if an unstable emotional state is detected, relaxation methods such as recommending deep breathing exercises will be suggested via the device.

[0395] For example, if a user uses a voice input system to say, "I'm feeling stressed right now," the AI ​​engine can analyze this data in real time and suggest appropriate relaxation methods, taking into account the user's heart rate.

[0396] An example of a prompt message is, "Generate a prompt that evaluates the user's stress level in real time based on the analysis of the user's voice data." This enables comprehensive care for the user's health and emotions.

[0397] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0398] Step 1:

[0399] The device collects information such as the user's heart rate, steps, sleep duration, and voice data through a smartwatch or voice input device. At this stage, raw data collected from the user is obtained as input. The collected data is temporarily stored on the device.

[0400] Step 2:

[0401] The device sends the collected data to the server. The server receives this data and applies an anonymization process. Here, data reduction and transformation (e.g., hashing) are performed to make it impossible to identify individuals, and the results are output.

[0402] Step 3:

[0403] The server processes anonymized data using analytical tools. It simultaneously analyzes health data (heart rate, steps, etc.) and emotional data (voice tone, keywords), and uses a generative AI model to determine health risks and emotional states. This analysis outputs evaluation results regarding health status and stress levels.

[0404] Step 4:

[0405] Based on the analysis results, the server uses a plan generation mechanism to automatically generate an optimal care plan for the user. At this stage, a plan is output that includes specific actions (e.g., relaxation techniques, exercise suggestions) based on the user's health risks and emotional state.

[0406] Step 5:

[0407] The generated care plan is sent from the server to the terminal. The terminal displays and communicates the plan to the user visually or audibly. In this step, the user can receive specific advice and suggestions based on the analysis results.

[0408] Step 6:

[0409] Users can provide feedback on the proposed care plan. The feedback data is sent from the terminal to the server, which then uses this data to generate future plans. This feedback process results in more refined plans being offered as future proposals.

[0410] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0411] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0412] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0413] [Third Embodiment]

[0414] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0415] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0416] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0417] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0418] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0419] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0420] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0421] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0422] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0423] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0424] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0425] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0426] The system of the present invention monitors the user's health status in real time and provides individually optimized care. This system is implemented using data collection means, data management means, analysis means, plan generation means, and plan provision means.

[0427] The data collection method involves collecting user health-related data via the device. This includes measuring heart rate and steps using wearable devices, and recording sleep and meals using smartphone applications. Users input their daily activities and health status into the app, which then allows detailed data to be stored on the device.

[0428] The data management system aggregates data transmitted from terminals onto a server, where it is anonymized and managed. This process is crucial from the perspective of protecting personal information and is designed to allow data to be utilized while protecting privacy.

[0429] The analysis is performed on a server. The collected data is evaluated by an analysis program, and frailty risk and other health risks are calculated. This analysis uses multimodal AI to comprehensively analyze health status from diverse data such as text, voice, images, and video.

[0430] The plan generation system creates individualized care plans for each user based on the analysis results. Specific action plans regarding exercise and diet are formulated according to the user's lifestyle and health goals. For example, users with a high risk of certain frailty may be offered exercises to strengthen muscles and dietary menus to improve balance.

[0431] The plan delivery system sends a plan generated from the server to the terminal and notifies the user. The user can adjust their daily life based on the proposed plan and continuously monitor its effects. The system further optimizes the plan based on the user's feedback.

[0432] As described above, the present invention realizes an advanced health management system that continuously monitors the user's health status and provides appropriate countermeasures. This allows users to engage in health maintenance activities with peace of mind and contributes to improving the operational efficiency of medical institutions and care service providers.

[0433] The following describes the processing flow.

[0434] Step 1:

[0435] The device collects the user's health-related data. Wearable devices measure heart rate and steps, while smartphone applications record details about meals and sleep, which the user inputs.

[0436] Step 2:

[0437] The device transmits the collected data to the server in real time. During this process, the data is encrypted to protect privacy and transmitted through a secure channel.

[0438] Step 3:

[0439] The server anonymizes the received data using data management tools and stores it in a database. This process protects users' personal information while preparing the data in a format that can be analyzed.

[0440] Step 4:

[0441] The server evaluates the stored data using analytical tools. Multimodal AI is used to analyze text, audio, images, and videos to comprehensively assess the user's health risks.

[0442] Step 5:

[0443] The server applies a plan generation method based on the analysis results to create an individualized care plan for the user. The plan generates recommendations for exercise and diet tailored to the type and degree of health risks.

[0444] Step 6:

[0445] The server sends the generated care plan to the terminal. A notification, including recommended actions, is delivered to the user using the plan delivery method.

[0446] Step 7:

[0447] Users can view their plans from their devices and incorporate them into their daily activities. The app records user actions and feedback, and sends the results to the server.

[0448] Step 8:

[0449] The server receives feedback from users and updates care plans as needed. It monitors the application and effectiveness of the plans and continuously optimizes them.

[0450] (Example 1)

[0451] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0452] Conventional health management systems have struggled to monitor individual users' health status in real time and provide optimized care plans based on that data. In particular, there is a need for technology that can integrate and analyze diverse information formats to perform more accurate health assessments and provide personalized action plans for each user.

[0453] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0454] In this invention, the server includes means for collecting user information, including biometric information, obtained from an information device worn by the user; information management means for anonymizing and storing the information received from the collection means; and analysis means for evaluating the information managed by the information management means and detecting changes in the user's health status. This makes it possible to monitor the user's health status in detail in real time and provide an individually optimized action plan.

[0455] "Means for collecting user information" refers to a system that acquires various data, including biometric information, from the information devices owned by the user.

[0456] "Information management means" refers to a system that anonymizes collected user information, securely stores it, and keeps it in a state where it can be used for later analysis.

[0457] An "analysis method" is a system that evaluates a user's health status based on managed information and analyzes changes in that status.

[0458] The "plan generation means" is a mechanism that creates individual action plans for users based on the evaluation results obtained from the analysis means.

[0459] A "plan delivery method" is a system that reliably communicates and instructs users on the generated action plan.

[0460] A "response information collection means" is a system that collects user feedback and transmits it to an information management means.

[0461] A "generative algorithm" is a computational method for comprehensively evaluating diverse information formats and deriving valuable insights.

[0462] This invention is an advanced health management system that monitors a user's health status in real time and provides individually optimized care plans. Several hardware and software components are required to implement the invention.

[0463] Users utilize wearable devices and smartphone applications. These devices collect biometric information such as heart rate, steps taken, sleep, and diet. Examples include heart rate monitors, fitness trackers, and smartphone apps for health management. These devices and applications continuously record everyday health-related data.

[0464] The device sends the collected data to the server at regular intervals. This data transmission is typically done via Wi-Fi or a mobile network. The server anonymizes the received data and securely stores it using information management measures.

[0465] The server utilizes multimodal AI technology to comprehensively analyze diverse data formats, including text, audio, images, and video. This allows it to assess the user's health risks and detect changes in their condition in real time. Based on the analysis results, a generative AI model is used to generate an individualized action plan for each user. This plan includes specific exercise and dietary instructions, outlining concrete actions aimed at improving the user's health.

[0466] The server notifies the terminal of the generated plan and provides it to the user. The user can adjust their daily life based on the proposed plan and send feedback from the terminal to the server, enabling continuous optimization.

[0467] As a concrete example, suppose a user regularly records their exercise and manages their calorie expenditure using an app. The server analyzes this data and identifies areas for improvement. As a result, the user receives specific advice such as "jog for 30 minutes every day" or "eat a high-protein meal for dinner."

[0468] An example of a prompt message might be: "A 50-year-old woman with an average daily step count of 5000. Please suggest a specific exercise and diet plan to maintain her health."

[0469] In this way, the present invention supports users in managing their health and enables more effective health maintenance activities.

[0470] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0471] Step 1:

[0472] Users wear wearable devices to collect health data such as heart rate and steps taken. This allows biometric information to be input into the device in real time. The device aggregates and temporarily stores this data. For example, the average heart rate and total steps taken by the user throughout the day are recorded here.

[0473] Step 2:

[0474] The terminal sends aggregated data to the server at regular intervals. During this process, the data is encrypted during transmission. The server decrypts the received data, anonymizes it, and stores it in an information management system. This input data is transformed into a form that does not identify individual users and is managed securely.

[0475] Step 3:

[0476] The server feeds the data stored in the information management system into the analysis system. The analysis system uses a multimodal AI model to analyze this data and perform data calculations to assess the user's health risks. For example, it detects risks such as abnormal heart rate changes and lack of exercise. As output, it generates a health status assessment report for each user.

[0477] Step 4:

[0478] The server generates individual action plans using a plan generation system based on an assessment of health status obtained through analysis. It utilizes a generation AI model to construct exercise and dietary guidance optimized for the user's lifestyle and health goals. The output is a concrete and actionable plan for improving health, such as "walking three times a week."

[0479] Step 5:

[0480] The server sends the generated action plan to the terminal and provides it to the user. The terminal notifies the user of the received plan and displays it. The user follows this plan in their daily life and records their progress. As output, the user's continuous feedback is stored on the terminal.

[0481] Step 6:

[0482] The user sends feedback to the server via their device. The server analyzes the feedback and evaluates the effectiveness of the action plan that has already been generated. If necessary, it revises and optimizes the plan based on new data and feedback. The output is provided to the user as an improved new action plan.

[0483] (Application Example 1)

[0484] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0485] There is a need to effectively manage the health status of elderly and care-dependent users and provide care plans optimized for their individual needs, thereby promoting the maintenance of users' health while reducing the burden on caregivers and families. To achieve this kind of health management, it is necessary to efficiently collect users' health information and formulate appropriate care plans based on that information, but conventional systems do not adequately perform real-time analysis and individual optimization.

[0486] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0487] In this invention, the server includes an information acquisition means consisting of a device for acquiring user information, an information management means for anonymizing and storing the information obtained from the information acquisition means, and an evaluation means for analyzing the information stored by the information management means and evaluating health risks. This enables the immediate presentation of an individually optimized health management plan to the user, and allows for continuous health monitoring and optimization of the plan through feedback.

[0488] "Information acquisition means" refers to devices or equipment used to acquire user information. This acquired information is related to health status and lifestyle.

[0489] "Information management measures" refer to a system for anonymizing acquired user information and securely storing and managing it. This includes important processes for utilizing data while ensuring data privacy.

[0490] "Evaluation methods" refer to systems and algorithms that analyze managed information to assess health risks for users. A key feature of this analysis is the integrated use of diverse data formats.

[0491] A "health management plan" is a specific action plan aimed at maintaining or improving the health of individual users, based on the results calculated using evaluation methods.

[0492] This system provides advanced health management for users and offers individually optimized health management plans. The system is primarily implemented using information acquisition, information management, and evaluation methods.

[0493] The server uses wearable devices equipped with sensors and smartphone applications to collect user health data. This data includes heart rate, steps taken, sleep patterns, and meal records. Wearable devices transfer data to smartphones via Bluetooth, and smartphones send the data to the server. SSL encryption is used throughout this process to protect the data.

[0494] The server anonymizes received user data and securely stores and manages it as an information management tool. The data is stored using cloud storage and is managed thoroughly to prevent the identification of personal information. Furthermore, the data is regularly backed up to prevent data loss.

[0495] The evaluation method involves using an analysis program running on the cloud, with data analysis performed using generative AI models such as Google Cloud AI. Risk assessments related to the user's health are conducted by integrating and analyzing different data formats. Based on the analysis results, the server generates an individualized health management plan and provides it to the user's smartphone as a push notification.

[0496] For example, if data is collected indicating that a woman in her 70s has decreased daily exercise and is at increased risk of frailty, a health management plan can be developed based on this information. The system contributes to improving the user's health by suggesting light walking and a meal plan that considers nutritional balance.

[0497] An example of a prompt sentence applied to a generative AI model is: "A woman in her 70s, recently experiencing a decrease in steps taken, is at high health risk. Please generate suggestions for light exercise and nutritional supplements." This prompt allows the system to concretize the suggestions and develop a personalized plan.

[0498] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0499] Step 1:

[0500] The device collects health data such as the user's heart rate, steps, sleep patterns, and meal records through wearable devices and smartphone applications. Input is data from wearable devices or manual input, and output is a set of this data. Sensor data is acquired directly and transferred to a smartphone using Bluetooth or Wi-Fi.

[0501] Step 2:

[0502] The device sends the collected data to the server. During this process, the data is encrypted via SSL authentication and transmitted securely. The input is the data collected in step 1, and the output is the encrypted data stored on the server. The data transmission protocol is executed, and the information is stored in cloud storage.

[0503] Step 3:

[0504] The server anonymizes the received data and stores it securely using information management measures. The input is the data sent in step 2, and the output is the anonymized data. Personal information is deleted or replaced, and the anonymization process is performed.

[0505] Step 4:

[0506] The server uses generative AI models, such as Google Cloud AI, to analyze anonymized data and assess health risks. The input is anonymized data, and the output is the result of the health risk assessment. The data format is corrected, fed into the AI ​​model, and the assessment results are obtained.

[0507] Step 5:

[0508] The server generates an individualized health management plan based on the analysis results. This plan reflects the user's current health status and lifestyle. The input is the result of a health risk assessment, and the output is an individualized health management plan. A prompt message (e.g., "Female in her 70s, recently decreased step count, high health risk. Please generate suggestions for light exercise and nutritional supplements.") is applied to the generating AI model to formulate a specific plan.

[0509] Step 6:

[0510] The server sends the generated health management plan to the device as a push notification, providing it to the user. The input is a personalized health management plan, and the output is plan information displayed on the device. The plan is delivered in real time using a push notification service.

[0511] This series of processes allows users to constantly receive the latest health management information and effectively optimize their own health status.

[0512] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0513] The present invention provides an advanced health management system that combines an emotion engine that recognizes the user's emotional state. This system includes data collection means, data management means, analysis means, plan generation means, plan provision means, and an emotion engine.

[0514] The data collection method uses a terminal to acquire user health-related and emotion-related data. Health data includes heart rate, steps taken, and sleep duration, while emotion-related data includes the user's spoken content and tone in voice and text.

[0515] The data management system protects user privacy by sending all collected data to a server and anonymizing it. The managed data is securely stored and used for analysis.

[0516] The emotion engine runs on a server and recognizes the user's emotions in real time from voice and text data. This engine uses natural language processing technology and voice analysis to identify emotional patterns.

[0517] The analysis method integrates and analyzes organized data and emotional data generated by an emotion engine. It considers emotional changes along with various data modalities to comprehensively evaluate the user's health risks.

[0518] The plan generation method creates a care plan that reflects the user's emotional state, based on the results of risk analysis and the results of an emotion engine. For example, if high stress levels are detected, a plan emphasizing relaxation will be generated.

[0519] In the plan delivery method, the generated care plan is delivered to the user via a terminal. Suggestions and advice are delivered to the user as notifications to support improvements in their daily life.

[0520] This system also allows users to provide feedback to improve the accuracy of their care plans. User feedback is sent to the server via a feedback collection mechanism and used to adjust the plans. This enables comprehensive and individualized management of users' health status, providing more effective health support that also considers emotional well-being. As a concrete example, if the emotional engine detects that a user is experiencing stress, the server quickly sends a care plan to the terminal, including recommendations for deep breathing exercises and meditation.

[0521] The following describes the processing flow.

[0522] Step 1:

[0523] The device collects the user's health-related and emotional data. Wearable devices measure heart rate and steps, while smartphone applications record emotional data, including the user's voice and text.

[0524] Step 2:

[0525] The device encrypts the collected data and sends it to the server using a secure protocol. For privacy protection, the data is anonymized before transmission.

[0526] Step 3:

[0527] The server organizes the received data using data management tools and stores it in a database. The stored data is then prepared and organized for analysis.

[0528] Step 4:

[0529] The server uses an emotion engine to analyze and recognize the user's emotional state in real time from voice and text data. This analysis utilizes natural language processing and speech pattern recognition technologies.

[0530] Step 5:

[0531] The server integrates accumulated health and emotional data using analytical tools to comprehensively assess the user's health risks. This assessment also reflects the user's emotional state.

[0532] Step 6:

[0533] The server uses a plan generation mechanism to generate individualized care plans based on the results of risk assessments and emotional states. For example, if a person is diagnosed with high stress levels, the plan will include activities that promote relaxation.

[0534] Step 7:

[0535] The server delivers the generated care plan to the terminal. The terminal sends a notification to the user and sets a reminder to encourage them to take action according to the plan at the appropriate time.

[0536] Step 8:

[0537] Users execute the care plan provided through their device and return the results and feedback to the server via the app. The server uses this feedback information to adjust and improve the plan.

[0538] (Example 2)

[0539] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0540] In modern society, personal health management presents diverse and complex challenges, and comprehensive health support systems that take into account users' emotional states are lacking. In particular, there is a need to understand the impact of emotional states on health risks in real time and to provide appropriate care plans quickly based on that understanding. Furthermore, a mechanism is needed to provide personalized health support while protecting user privacy.

[0541] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0542] In this invention, the server includes data collection means consisting of equipment for acquiring biometric and voice information, data management means for anonymizing and managing the information received from the data collection means, and an emotion engine that operates on the server and recognizes emotional states from voice and text information. This makes it possible to comprehensively evaluate health risks and provide care plans tailored to individual users by recognizing the user's emotional state in real time.

[0543] "Biometric information" refers to physiological data that indicates an individual's health status, and specific examples include heart rate, steps taken, and sleep duration.

[0544] "Audio information" refers to data that records the content and tone of a user's speech, and is used in natural language processing and speech analysis.

[0545] "Data collection methods" refer to devices and processes used to acquire users' biometric and voice information, specifically smartphones and smartwatches.

[0546] "Data management means" refers to a system that protects user privacy by organizing collected information and applying anonymization processing.

[0547] An "emotion engine" refers to a program that includes technology to analyze voice and text information and recognize the user's emotional state in real time.

[0548] "Analysis methods" refer to the process of comprehensively analyzing collected and managed data to assess health risks.

[0549] "Plan generation means" refers to a process or technology that automatically creates individual care plans based on the results of analysis and emotional states.

[0550] "Plan delivery method" refers to a system that provides users with generated care plans and offers appropriate guidance and suggestions.

[0551] "Feedback" refers to the information and opinions provided by users, which are used to improve the system and enhance its accuracy.

[0552] This invention is an advanced system that comprehensively manages a user's health and emotional state, and its implementation utilizes multiple hardware and software components. Specifically, smartphones and smartwatches are used as hardware, functioning as data collection tools. This allows for the real-time acquisition of biometric information such as heart rate, steps taken, sleep duration, and voice information.

[0553] The terminal sends this data to the server, where anonymization is applied to ensure the data is securely managed. The server uses natural language processing systems and speech analysis software to process the voice and text information. For example, the natural language processing system may use Python libraries or third-party API services.

[0554] Particularly important is the emotion engine, which operates on a server and detects the user's emotional state in real time from voice and text information. This uses generative AI models to identify emotional patterns. The data analyzed by the emotion engine is integrated with other biometric information on the server to assess overall health risks.

[0555] Based on the evaluation results, the server automatically generates a care plan tailored to the user. This plan generation utilizes an AI model and may include content that promotes stress reduction and relaxation. The created care plan is provided to the user via their device, and a notification function is used to support improvements in their daily life.

[0556] For example, if the emotion engine detects a high stress level in a user, the server immediately generates a care plan recommending deep breathing exercises to help manage stress and notifies the user on their smartphone. The user will then see a prompt message similar to the following:

[0557] Example of a prompt:

[0558] "Based on your current stress level, we'd like to suggest a specially prepared relaxation plan. Let's try a 5-minute meditation."

[0559] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0560] Step 1:

[0561] The device collects biometric and voice information from the user. Inputs include heart rate, steps, sleep duration, and voice data collected by smartwatches and smartphones. This data is recorded in real time and temporarily stored on the device. Specifically, the smartwatch measures heart rate using a pulse sensor, and the smartphone records the user's speech through its microphone.

[0562] Step 2:

[0563] The device sends the collected data to the server. The inputs include biometric and voice information obtained in step 1. The device securely uploads this data to the server and transmits it using a data transfer protocol. Specifically, the device encrypts and transmits the data via Wi-Fi or mobile data communication.

[0564] Step 3:

[0565] The server anonymizes the received data and stores it in a database. The input consists of biometric and voice information sent in step 2. The data is anonymized by removing user identification information and is managed securely. Specifically, the server uses an algorithm to remove user IDs, classify the data, and store it in the database.

[0566] Step 4:

[0567] The server analyzes the voice information using an emotion engine to recognize the user's emotional state. The input is the voice information saved in step 3. Using natural language processing and speech analysis techniques, it identifies emotional patterns and determines the type of emotion. Specifically, the server analyzes keywords and voice tone in the utterances to classify emotions such as joy, sadness, and anger.

[0568] Step 5:

[0569] The server integrates biometric and emotional data to assess health risks. Inputs include biometric data from step 3 and emotional states from step 4. Using a machine learning model, the collected data is analyzed to generate a user health risk profile. Specifically, the server performs data analysis using a Python library and detects anomalies.

[0570] Step 6:

[0571] The server generates a personalized care plan based on the analysis results. The inputs are the health risk profile and emotional state obtained in step 5. The AI ​​model is used to create a specific care plan (e.g., a meditation guide for stress relief). Specifically, the AI ​​model suggests appropriate actions based on the generated risk profile.

[0572] Step 7:

[0573] The device provides the user with the care plan received from the server. The input is the care plan generated in step 6. The device uses push notification functionality to notify the user of the care plan as a suggestion. Specifically, the device displays a notification such as "A relaxation plan is available" to prompt the user to take action.

[0574] (Application Example 2)

[0575] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0576] In modern society, health management and emotional stability are crucial issues. However, many existing systems only consider users' health data, making it difficult to provide comprehensive support that takes emotional states into account. Furthermore, they lack the means to provide flexible support that responds to users' health conditions and emotional changes, and to offer care plans tailored to individual needs.

[0577] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0578] In this invention, the server includes data collection means consisting of equipment for collecting user information, data management means for anonymizing and managing the information received from the data collection means, analysis means equipped with a function for evaluating health risks using analysis means, and emotion recognition means equipped with an analysis engine for recognizing emotional states. This enables the generation of individualized care plans that take into account the user's health condition and emotions, and allows for daily support based on health and emotional states in cooperation with application devices.

[0579] "User information" refers to data necessary to identify the user's health and emotional state, and includes heart rate, steps taken, sleep duration, voice data, and text content.

[0580] "Data collection means" refers to devices or systems used to obtain health data and emotion-related information from users.

[0581] A "data management system" is a system that has the functionality to securely store collected user information, anonymize it to protect privacy, and use it for analysis.

[0582] "Analysis methods" refer to algorithms and computational techniques used to evaluate health and emotional states based on managed data and to identify risks.

[0583] "Plan generation means" refers to a function that automatically creates a care plan optimized for the user based on the analysis results.

[0584] "Plan delivery means" refers to a function that transmits the generated care plan to the user's terminal or application device and provides the user with a proposal in an actionable format.

[0585] "Emotion recognition means" refers to a technology or engine used to analyze a user's voice or text data to recognize and identify their emotional state in real time.

[0586] "Support delivery means" refers to functions that provide users with daily support and suggestions through application devices based on health and emotional data.

[0587] The present invention describes embodiments for carrying out the invention. This invention realizes a system that grasps the user's health and emotional state in real time and provides a personalized care plan based on these. The main components and their functions are described below.

[0588] At the heart of the system lies data used to collect user information. Wearable devices such as smartwatches are used to collect physical information such as heart rate, steps taken, and sleep duration. Emotional information is also collected through user voice and text input.

[0589] This data is securely stored on servers as anonymized information using data management mechanisms. The data engine runs on a cloud-based computing platform (e.g., Amazon Web Services or Microsoft Azure) to analyze the collected data and assess health risks.

[0590] The analyzed data, combined with consideration of health and emotional states, generates an individualized care plan. This plan is then transmitted directly to a smartphone or computer terminal via a plan delivery system, allowing users to receive suggestions and advice through visual and auditory means.

[0591] Furthermore, natural language processing technologies such as the Google Cloud Natural Language API are used as a means of emotion recognition to identify the user's emotions from speech and text, and to provide appropriate actions and suggestions. As part of this process, if an unstable emotional state is detected, relaxation methods such as recommending deep breathing exercises will be suggested via the device.

[0592] For example, if a user uses a voice input system to say, "I'm feeling stressed right now," the AI ​​engine can analyze this data in real time and suggest appropriate relaxation methods, taking into account the user's heart rate.

[0593] An example of a prompt message is, "Generate a prompt that evaluates the user's stress level in real time based on the analysis of the user's voice data." This enables comprehensive care for the user's health and emotions.

[0594] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0595] Step 1:

[0596] The device collects information such as the user's heart rate, steps, sleep duration, and voice data through a smartwatch or voice input device. At this stage, raw data collected from the user is obtained as input. The collected data is temporarily stored on the device.

[0597] Step 2:

[0598] The device sends the collected data to the server. The server receives this data and applies an anonymization process. Here, data reduction and transformation (e.g., hashing) are performed to make it impossible to identify individuals, and the results are output.

[0599] Step 3:

[0600] The server processes anonymized data using analytical tools. It simultaneously analyzes health data (heart rate, steps, etc.) and emotional data (voice tone, keywords), and uses a generative AI model to determine health risks and emotional states. This analysis outputs evaluation results regarding health status and stress levels.

[0601] Step 4:

[0602] Based on the analysis results, the server uses a plan generation mechanism to automatically generate an optimal care plan for the user. At this stage, a plan is output that includes specific actions (e.g., relaxation techniques, exercise suggestions) based on the user's health risks and emotional state.

[0603] Step 5:

[0604] The generated care plan is sent from the server to the terminal. The terminal displays and communicates the plan to the user visually or audibly. In this step, the user can receive specific advice and suggestions based on the analysis results.

[0605] Step 6:

[0606] Users can provide feedback on the proposed care plan. The feedback data is sent from the terminal to the server, which then uses this data to generate future plans. This feedback process results in more refined plans being offered as future proposals.

[0607] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0608] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0609] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0610] [Fourth Embodiment]

[0611] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0612] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0613] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0614] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0615] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0616] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0617] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0618] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0619] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0620] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0621] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0622] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0623] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0624] The system of the present invention monitors the user's health status in real time and provides individually optimized care. This system is implemented using data collection means, data management means, analysis means, plan generation means, and plan provision means.

[0625] The data collection method involves collecting user health-related data via the device. This includes measuring heart rate and steps using wearable devices, and recording sleep and meals using smartphone applications. Users input their daily activities and health status into the app, which then allows detailed data to be stored on the device.

[0626] The data management system aggregates data transmitted from terminals onto a server, where it is anonymized and managed. This process is crucial from the perspective of protecting personal information and is designed to allow data to be utilized while protecting privacy.

[0627] The analysis is performed on a server. The collected data is evaluated by an analysis program, and frailty risk and other health risks are calculated. This analysis uses multimodal AI to comprehensively analyze health status from diverse data such as text, voice, images, and video.

[0628] The plan generation system creates individualized care plans for each user based on the analysis results. Specific action plans regarding exercise and diet are formulated according to the user's lifestyle and health goals. For example, users with a high risk of certain frailty may be offered exercises to strengthen muscles and dietary menus to improve balance.

[0629] The plan delivery system sends a plan generated from the server to the terminal and notifies the user. The user can adjust their daily life based on the proposed plan and continuously monitor its effects. The system further optimizes the plan based on the user's feedback.

[0630] As described above, the present invention realizes an advanced health management system that continuously monitors the user's health status and provides appropriate countermeasures. This allows users to engage in health maintenance activities with peace of mind and contributes to improving the operational efficiency of medical institutions and care service providers.

[0631] The following describes the processing flow.

[0632] Step 1:

[0633] The device collects the user's health-related data. Wearable devices measure heart rate and steps, while smartphone applications record details about meals and sleep, which the user inputs.

[0634] Step 2:

[0635] The device transmits the collected data to the server in real time. During this process, the data is encrypted to protect privacy and transmitted through a secure channel.

[0636] Step 3:

[0637] The server anonymizes the received data using data management tools and stores it in a database. This process protects users' personal information while preparing the data in a format that can be analyzed.

[0638] Step 4:

[0639] The server evaluates the stored data using analytical tools. Multimodal AI is used to analyze text, audio, images, and videos to comprehensively assess the user's health risks.

[0640] Step 5:

[0641] The server applies a plan generation method based on the analysis results to create an individualized care plan for the user. The plan generates recommendations for exercise and diet tailored to the type and degree of health risks.

[0642] Step 6:

[0643] The server sends the generated care plan to the terminal. A notification, including recommended actions, is delivered to the user using the plan delivery method.

[0644] Step 7:

[0645] Users can view their plans from their devices and incorporate them into their daily activities. The app records user actions and feedback, and sends the results to the server.

[0646] Step 8:

[0647] The server receives feedback from users and updates care plans as needed. It monitors the application and effectiveness of the plans and continuously optimizes them.

[0648] (Example 1)

[0649] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0650] Conventional health management systems have struggled to monitor individual users' health status in real time and provide optimized care plans based on that data. In particular, there is a need for technology that can integrate and analyze diverse information formats to perform more accurate health assessments and provide personalized action plans for each user.

[0651] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0652] In this invention, the server includes means for collecting user information, including biometric information, obtained from an information device worn by the user; information management means for anonymizing and storing the information received from the collection means; and analysis means for evaluating the information managed by the information management means and detecting changes in the user's health status. This makes it possible to monitor the user's health status in detail in real time and provide an individually optimized action plan.

[0653] "Means for collecting user information" refers to a system that acquires various data, including biometric information, from the information devices owned by the user.

[0654] "Information management means" refers to a system that anonymizes collected user information, securely stores it, and keeps it in a state where it can be used for later analysis.

[0655] An "analysis method" is a system that evaluates a user's health status based on managed information and analyzes changes in that status.

[0656] The "plan generation means" is a mechanism that creates individual action plans for users based on the evaluation results obtained from the analysis means.

[0657] A "plan delivery method" is a system that reliably communicates and instructs users on the generated action plan.

[0658] A "response information collection means" is a system that collects user feedback and transmits it to an information management means.

[0659] A "generative algorithm" is a computational method for comprehensively evaluating diverse information formats and deriving valuable insights.

[0660] This invention is an advanced health management system that monitors a user's health status in real time and provides individually optimized care plans. Several hardware and software components are required to implement the invention.

[0661] Users utilize wearable devices and smartphone applications. These devices collect biometric information such as heart rate, steps taken, sleep, and diet. Examples include heart rate monitors, fitness trackers, and smartphone apps for health management. These devices and applications continuously record everyday health-related data.

[0662] The device sends the collected data to the server at regular intervals. This data transmission is typically done via Wi-Fi or a mobile network. The server anonymizes the received data and securely stores it using information management measures.

[0663] The server utilizes multimodal AI technology to comprehensively analyze diverse data formats, including text, audio, images, and video. This allows it to assess the user's health risks and detect changes in their condition in real time. Based on the analysis results, a generative AI model is used to generate an individualized action plan for each user. This plan includes specific exercise and dietary instructions, outlining concrete actions aimed at improving the user's health.

[0664] The server notifies the terminal of the generated plan and provides it to the user. The user can adjust their daily life based on the proposed plan and send feedback from the terminal to the server, enabling continuous optimization.

[0665] As a concrete example, suppose a user regularly records their exercise and manages their calorie expenditure using an app. The server analyzes this data and identifies areas for improvement. As a result, the user receives specific advice such as "jog for 30 minutes every day" or "eat a high-protein meal for dinner."

[0666] An example of a prompt message might be: "A 50-year-old woman with an average daily step count of 5000. Please suggest a specific exercise and diet plan to maintain her health."

[0667] In this way, the present invention supports users in managing their health and enables more effective health maintenance activities.

[0668] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0669] Step 1:

[0670] Users wear wearable devices to collect health data such as heart rate and steps taken. This allows biometric information to be input into the device in real time. The device aggregates and temporarily stores this data. For example, the average heart rate and total steps taken by the user throughout the day are recorded here.

[0671] Step 2:

[0672] The terminal sends aggregated data to the server at regular intervals. During this process, the data is encrypted during transmission. The server decrypts the received data, anonymizes it, and stores it in an information management system. This input data is transformed into a form that does not identify individual users and is managed securely.

[0673] Step 3:

[0674] The server feeds the data stored in the information management system into the analysis system. The analysis system uses a multimodal AI model to analyze this data and perform data calculations to assess the user's health risks. For example, it detects risks such as abnormal heart rate changes and lack of exercise. As output, it generates a health status assessment report for each user.

[0675] Step 4:

[0676] The server generates individual action plans using a plan generation system based on an assessment of health status obtained through analysis. It utilizes a generation AI model to construct exercise and dietary guidance optimized for the user's lifestyle and health goals. The output is a concrete and actionable plan for improving health, such as "walking three times a week."

[0677] Step 5:

[0678] The server sends the generated action plan to the terminal and provides it to the user. The terminal notifies the user of the received plan and displays it. The user follows this plan in their daily life and records their progress. As output, the user's continuous feedback is stored on the terminal.

[0679] Step 6:

[0680] The user sends feedback to the server via their device. The server analyzes the feedback and evaluates the effectiveness of the action plan that has already been generated. If necessary, it revises and optimizes the plan based on new data and feedback. The output is provided to the user as an improved new action plan.

[0681] (Application Example 1)

[0682] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0683] There is a need to effectively manage the health status of elderly and care-dependent users and provide care plans optimized for their individual needs, thereby promoting the maintenance of users' health while reducing the burden on caregivers and families. To achieve this kind of health management, it is necessary to efficiently collect users' health information and formulate appropriate care plans based on that information, but conventional systems do not adequately perform real-time analysis and individual optimization.

[0684] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0685] In this invention, the server includes an information acquisition means consisting of a device for acquiring user information, an information management means for anonymizing and storing the information obtained from the information acquisition means, and an evaluation means for analyzing the information stored by the information management means and evaluating health risks. This enables the immediate presentation of an individually optimized health management plan to the user, and allows for continuous health monitoring and optimization of the plan through feedback.

[0686] "Information acquisition means" refers to devices or equipment used to acquire user information. This acquired information is related to health status and lifestyle.

[0687] "Information management measures" refer to a system for anonymizing acquired user information and securely storing and managing it. This includes important processes for utilizing data while ensuring data privacy.

[0688] "Evaluation methods" refer to systems and algorithms that analyze managed information to assess health risks for users. A key feature of this analysis is the integrated use of diverse data formats.

[0689] A "health management plan" is a specific action plan aimed at maintaining or improving the health of individual users, based on the results calculated using evaluation methods.

[0690] This system provides advanced health management for users and offers individually optimized health management plans. The system is primarily implemented using information acquisition, information management, and evaluation methods.

[0691] The server uses wearable devices equipped with sensors and smartphone applications to collect user health data. This data includes heart rate, steps taken, sleep patterns, and meal records. Wearable devices transfer data to smartphones via Bluetooth, and smartphones send the data to the server. SSL encryption is used throughout this process to protect the data.

[0692] The server anonymizes received user data and securely stores and manages it as an information management tool. The data is stored using cloud storage and is managed thoroughly to prevent the identification of personal information. Furthermore, the data is regularly backed up to prevent data loss.

[0693] The evaluation method involves using an analysis program running on the cloud, with data analysis performed using generative AI models such as Google Cloud AI. Risk assessments related to the user's health are conducted by integrating and analyzing different data formats. Based on the analysis results, the server generates an individualized health management plan and provides it to the user's smartphone as a push notification.

[0694] For example, if data is collected indicating that a woman in her 70s has decreased daily exercise and is at increased risk of frailty, a health management plan can be developed based on this information. The system contributes to improving the user's health by suggesting light walking and a meal plan that considers nutritional balance.

[0695] An example of a prompt sentence applied to a generative AI model is: "A woman in her 70s, recently experiencing a decrease in steps taken, is at high health risk. Please generate suggestions for light exercise and nutritional supplements." This prompt allows the system to concretize the suggestions and develop a personalized plan.

[0696] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0697] Step 1:

[0698] The device collects health data such as the user's heart rate, steps, sleep patterns, and meal records through wearable devices and smartphone applications. Input is data from wearable devices or manual input, and output is a set of this data. Sensor data is acquired directly and transferred to a smartphone using Bluetooth or Wi-Fi.

[0699] Step 2:

[0700] The device sends the collected data to the server. During this process, the data is encrypted via SSL authentication and transmitted securely. The input is the data collected in step 1, and the output is the encrypted data stored on the server. The data transmission protocol is executed, and the information is stored in cloud storage.

[0701] Step 3:

[0702] The server anonymizes the received data and stores it securely using information management measures. The input is the data sent in step 2, and the output is the anonymized data. Personal information is deleted or replaced, and the anonymization process is performed.

[0703] Step 4:

[0704] The server uses generative AI models, such as Google Cloud AI, to analyze anonymized data and assess health risks. The input is anonymized data, and the output is the result of the health risk assessment. The data format is corrected, fed into the AI ​​model, and the assessment results are obtained.

[0705] Step 5:

[0706] The server generates an individualized health management plan based on the analysis results. This plan reflects the user's current health status and lifestyle. The input is the result of a health risk assessment, and the output is an individualized health management plan. A prompt message (e.g., "Female in her 70s, recently decreased step count, high health risk. Please generate suggestions for light exercise and nutritional supplements.") is applied to the generating AI model to formulate a specific plan.

[0707] Step 6:

[0708] The server sends the generated health management plan to the device as a push notification, providing it to the user. The input is a personalized health management plan, and the output is plan information displayed on the device. The plan is delivered in real time using a push notification service.

[0709] This series of processes allows users to constantly receive the latest health management information and effectively optimize their own health status.

[0710] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0711] The present invention provides an advanced health management system that combines an emotion engine that recognizes the user's emotional state. This system includes data collection means, data management means, analysis means, plan generation means, plan provision means, and an emotion engine.

[0712] The data collection method uses a terminal to acquire user health-related and emotion-related data. Health data includes heart rate, steps taken, and sleep duration, while emotion-related data includes the user's spoken content and tone in voice and text.

[0713] The data management system protects user privacy by sending all collected data to a server and anonymizing it. The managed data is securely stored and used for analysis.

[0714] The emotion engine runs on a server and recognizes the user's emotions in real time from voice and text data. This engine uses natural language processing technology and voice analysis to identify emotional patterns.

[0715] The analysis method integrates and analyzes organized data and emotional data generated by an emotion engine. It considers emotional changes along with various data modalities to comprehensively evaluate the user's health risks.

[0716] The plan generation method creates a care plan that reflects the user's emotional state, based on the results of risk analysis and the results of an emotion engine. For example, if high stress levels are detected, a plan emphasizing relaxation will be generated.

[0717] In the plan delivery method, the generated care plan is delivered to the user via a terminal. Suggestions and advice are delivered to the user as notifications to support improvements in their daily life.

[0718] This system also allows users to provide feedback to improve the accuracy of their care plans. User feedback is sent to the server via a feedback collection mechanism and used to adjust the plans. This enables comprehensive and individualized management of users' health status, providing more effective health support that also considers emotional well-being. As a concrete example, if the emotional engine detects that a user is experiencing stress, the server quickly sends a care plan to the terminal, including recommendations for deep breathing exercises and meditation.

[0719] The following describes the processing flow.

[0720] Step 1:

[0721] The device collects the user's health-related and emotional data. Wearable devices measure heart rate and steps, while smartphone applications record emotional data, including the user's voice and text.

[0722] Step 2:

[0723] The device encrypts the collected data and sends it to the server using a secure protocol. For privacy protection, the data is anonymized before transmission.

[0724] Step 3:

[0725] The server organizes the received data using data management tools and stores it in a database. The stored data is then prepared and organized for analysis.

[0726] Step 4:

[0727] The server uses an emotion engine to analyze and recognize the user's emotional state in real time from voice and text data. This analysis utilizes natural language processing and speech pattern recognition technologies.

[0728] Step 5:

[0729] The server integrates accumulated health and emotional data using analytical tools to comprehensively assess the user's health risks. This assessment also reflects the user's emotional state.

[0730] Step 6:

[0731] The server uses a plan generation mechanism to generate individualized care plans based on the results of risk assessments and emotional states. For example, if a person is diagnosed with high stress levels, the plan will include activities that promote relaxation.

[0732] Step 7:

[0733] The server delivers the generated care plan to the terminal. The terminal sends a notification to the user and sets a reminder to encourage them to take action according to the plan at the appropriate time.

[0734] Step 8:

[0735] Users execute the care plan provided through their device and return the results and feedback to the server via the app. The server uses this feedback information to adjust and improve the plan.

[0736] (Example 2)

[0737] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0738] In modern society, personal health management presents diverse and complex challenges, and comprehensive health support systems that take into account users' emotional states are lacking. In particular, there is a need to understand the impact of emotional states on health risks in real time and to provide appropriate care plans quickly based on that understanding. Furthermore, a mechanism is needed to provide personalized health support while protecting user privacy.

[0739] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0740] In this invention, the server includes data collection means consisting of equipment for acquiring biometric and voice information, data management means for anonymizing and managing the information received from the data collection means, and an emotion engine that operates on the server and recognizes emotional states from voice and text information. This makes it possible to comprehensively evaluate health risks and provide care plans tailored to individual users by recognizing the user's emotional state in real time.

[0741] "Biometric information" refers to physiological data that indicates an individual's health status, and specific examples include heart rate, steps taken, and sleep duration.

[0742] "Audio information" refers to data that records the content and tone of a user's speech, and is used in natural language processing and speech analysis.

[0743] "Data collection methods" refer to devices and processes used to acquire users' biometric and voice information, specifically smartphones and smartwatches.

[0744] "Data management means" refers to a system that protects user privacy by organizing collected information and applying anonymization processing.

[0745] An "emotion engine" refers to a program that includes technology to analyze voice and text information and recognize the user's emotional state in real time.

[0746] "Analysis methods" refer to the process of comprehensively analyzing collected and managed data to assess health risks.

[0747] "Plan generation means" refers to a process or technology that automatically creates individual care plans based on the results of analysis and emotional states.

[0748] "Plan delivery method" refers to a system that provides users with generated care plans and offers appropriate guidance and suggestions.

[0749] "Feedback" refers to the information and opinions provided by users, which are used to improve the system and enhance its accuracy.

[0750] This invention is an advanced system that comprehensively manages a user's health and emotional state, and its implementation utilizes multiple hardware and software components. Specifically, smartphones and smartwatches are used as hardware, functioning as data collection tools. This allows for the real-time acquisition of biometric information such as heart rate, steps taken, sleep duration, and voice information.

[0751] The terminal sends this data to the server, where anonymization is applied to ensure the data is securely managed. The server uses natural language processing systems and speech analysis software to process the voice and text information. For example, the natural language processing system may use Python libraries or third-party API services.

[0752] Particularly important is the emotion engine, which operates on a server and detects the user's emotional state in real time from voice and text information. This uses generative AI models to identify emotional patterns. The data analyzed by the emotion engine is integrated with other biometric information on the server to assess overall health risks.

[0753] Based on the evaluation results, the server automatically generates a care plan tailored to the user. This plan generation utilizes an AI model and may include content that promotes stress reduction and relaxation. The created care plan is provided to the user via their device, and a notification function is used to support improvements in their daily life.

[0754] For example, if the emotion engine detects a high stress level in a user, the server immediately generates a care plan recommending deep breathing exercises to help manage stress and notifies the user on their smartphone. The user will then see a prompt message similar to the following:

[0755] Example of a prompt:

[0756] "Based on your current stress level, we'd like to suggest a specially prepared relaxation plan. Let's try a 5-minute meditation."

[0757] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0758] Step 1:

[0759] The device collects biometric and voice information from the user. Inputs include heart rate, steps, sleep duration, and voice data collected by smartwatches and smartphones. This data is recorded in real time and temporarily stored on the device. Specifically, the smartwatch measures heart rate using a pulse sensor, and the smartphone records the user's speech through its microphone.

[0760] Step 2:

[0761] The device sends the collected data to the server. The inputs include biometric and voice information obtained in step 1. The device securely uploads this data to the server and transmits it using a data transfer protocol. Specifically, the device encrypts and transmits the data via Wi-Fi or mobile data communication.

[0762] Step 3:

[0763] The server anonymizes the received data and stores it in a database. The input consists of biometric and voice information sent in step 2. The data is anonymized by removing user identification information and is managed securely. Specifically, the server uses an algorithm to remove user IDs, classify the data, and store it in the database.

[0764] Step 4:

[0765] The server analyzes the voice information using an emotion engine to recognize the user's emotional state. The input is the voice information saved in step 3. Using natural language processing and speech analysis techniques, it identifies emotional patterns and determines the type of emotion. Specifically, the server analyzes keywords and voice tone in the utterances to classify emotions such as joy, sadness, and anger.

[0766] Step 5:

[0767] The server integrates biometric and emotional data to assess health risks. Inputs include biometric data from step 3 and emotional states from step 4. Using a machine learning model, the collected data is analyzed to generate a user health risk profile. Specifically, the server performs data analysis using a Python library and detects anomalies.

[0768] Step 6:

[0769] The server generates a personalized care plan based on the analysis results. The inputs are the health risk profile and emotional state obtained in step 5. The AI ​​model is used to create a specific care plan (e.g., a meditation guide for stress relief). Specifically, the AI ​​model suggests appropriate actions based on the generated risk profile.

[0770] Step 7:

[0771] The device provides the user with the care plan received from the server. The input is the care plan generated in step 6. The device uses push notification functionality to notify the user of the care plan as a suggestion. Specifically, the device displays a notification such as "A relaxation plan is available" to prompt the user to take action.

[0772] (Application Example 2)

[0773] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0774] In modern society, health management and emotional stability are crucial issues. However, many existing systems only consider users' health data, making it difficult to provide comprehensive support that takes emotional states into account. Furthermore, they lack the means to provide flexible support that responds to users' health conditions and emotional changes, and to offer care plans tailored to individual needs.

[0775] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0776] In this invention, the server includes data collection means consisting of equipment for collecting user information, data management means for anonymizing and managing the information received from the data collection means, analysis means equipped with a function for evaluating health risks using analysis means, and emotion recognition means equipped with an analysis engine for recognizing emotional states. This enables the generation of individualized care plans that take into account the user's health condition and emotions, and allows for daily support based on health and emotional states in cooperation with application devices.

[0777] "User information" refers to data necessary to identify the user's health and emotional state, and includes heart rate, steps taken, sleep duration, voice data, and text content.

[0778] "Data collection means" refers to devices or systems used to obtain health data and emotion-related information from users.

[0779] A "data management system" is a system that has the functionality to securely store collected user information, anonymize it to protect privacy, and use it for analysis.

[0780] "Analysis methods" refer to algorithms and computational techniques used to evaluate health and emotional states based on managed data and to identify risks.

[0781] "Plan generation means" refers to a function that automatically creates a care plan optimized for the user based on the analysis results.

[0782] "Plan delivery means" refers to a function that transmits the generated care plan to the user's terminal or application device and provides the user with a proposal in an actionable format.

[0783] "Emotion recognition means" refers to a technology or engine used to analyze a user's voice or text data to recognize and identify their emotional state in real time.

[0784] "Support delivery means" refers to functions that provide users with daily support and suggestions through application devices based on health and emotional data.

[0785] The present invention describes embodiments for carrying out the invention. This invention realizes a system that grasps the user's health and emotional state in real time and provides a personalized care plan based on these. The main components and their functions are described below.

[0786] At the heart of the system lies data used to collect user information. Wearable devices such as smartwatches are used to collect physical information such as heart rate, steps taken, and sleep duration. Emotional information is also collected through user voice and text input.

[0787] This data is securely stored on servers as anonymized information using data management mechanisms. The data engine runs on a cloud-based computing platform (e.g., Amazon Web Services or Microsoft Azure) to analyze the collected data and assess health risks.

[0788] The analyzed data, combined with consideration of health and emotional states, generates an individualized care plan. This plan is then transmitted directly to a smartphone or computer terminal via a plan delivery system, allowing users to receive suggestions and advice through visual and auditory means.

[0789] Furthermore, natural language processing technologies such as the Google Cloud Natural Language API are used as a means of emotion recognition to identify the user's emotions from speech and text, and to provide appropriate actions and suggestions. As part of this process, if an unstable emotional state is detected, relaxation methods such as recommending deep breathing exercises will be suggested via the device.

[0790] For example, if a user uses a voice input system to say, "I'm feeling stressed right now," the AI ​​engine can analyze this data in real time and suggest appropriate relaxation methods, taking into account the user's heart rate.

[0791] An example of a prompt message is, "Generate a prompt that evaluates the user's stress level in real time based on the analysis of the user's voice data." This enables comprehensive care for the user's health and emotions.

[0792] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0793] Step 1:

[0794] The device collects information such as the user's heart rate, steps, sleep duration, and voice data through a smartwatch or voice input device. At this stage, raw data collected from the user is obtained as input. The collected data is temporarily stored on the device.

[0795] Step 2:

[0796] The device sends the collected data to the server. The server receives this data and applies an anonymization process. Here, data reduction and transformation (e.g., hashing) are performed to make it impossible to identify individuals, and the results are output.

[0797] Step 3:

[0798] The server processes anonymized data using analytical tools. It simultaneously analyzes health data (heart rate, steps, etc.) and emotional data (voice tone, keywords), and uses a generative AI model to determine health risks and emotional states. This analysis outputs evaluation results regarding health status and stress levels.

[0799] Step 4:

[0800] Based on the analysis results, the server uses a plan generation mechanism to automatically generate an optimal care plan for the user. At this stage, a plan is output that includes specific actions (e.g., relaxation techniques, exercise suggestions) based on the user's health risks and emotional state.

[0801] Step 5:

[0802] The generated care plan is sent from the server to the terminal. The terminal displays and communicates the plan to the user visually or audibly. In this step, the user can receive specific advice and suggestions based on the analysis results.

[0803] Step 6:

[0804] Users can provide feedback on the proposed care plan. The feedback data is sent from the terminal to the server, which then uses this data to generate future plans. This feedback process results in more refined plans being offered as future proposals.

[0805] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0806] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0807] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0808] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0809] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0810] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0811] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0812] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0813] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0814] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0815] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0816] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0817] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0818] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0819] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0820] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0821] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0822] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0823] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0824] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0825] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0826] The following is further disclosed regarding the embodiments described above.

[0827] (Claim 1)

[0828] A data collection means consisting of a device for collecting user data,

[0829] A data management means that anonymizes and manages the data received from the above data collection means,

[0830] An analysis means for analyzing data managed by the above data management means to evaluate health risks,

[0831] A plan generation means that generates individual care plans based on the results of the above analysis means,

[0832] A plan provision means that provides the care plan generated by the above plan generation means to the user,

[0833] A system that includes this.

[0834] (Claim 2)

[0835] The system according to claim 1, further comprising a feedback collection means for collecting user feedback and transmitting feedback data to the data management means.

[0836] (Claim 3)

[0837] The system according to claim 1, wherein the analysis means uses a generative model that comprehensively analyzes multiple data modalities.

[0838] "Example 1"

[0839] (Claim 1)

[0840] A means for collecting user information, including biometric information, obtained from an information device worn by the user,

[0841] Information management means for anonymizing and storing information received from the above collection means,

[0842] An analysis means for evaluating the information managed by the above information management means and detecting changes in health status,

[0843] A plan generation means for creating individual action plans based on the evaluation results obtained by the above analysis means,

[0844] A plan provision means for communicating the action plan created by the above plan generation means to the user,

[0845] A system that includes this.

[0846] (Claim 2)

[0847] The system according to claim 1, further comprising a response information collection means for collecting response information from a user and transmitting the response information to the information management means.

[0848] (Claim 3)

[0849] The system according to claim 1, wherein the analysis means uses a generation algorithm that comprehensively evaluates various information formats.

[0850] "Application Example 1"

[0851] (Claim 1)

[0852] Information acquisition means consisting of a device for acquiring user information,

[0853] Information management means for anonymizing and storing the information obtained from the above information acquisition means,

[0854] An evaluation means that analyzes the information stored by the above information management means and evaluates health-related risks,

[0855] A planning tool for formulating individual health management plans based on the results of the above evaluation tool,

[0856] A plan presentation means for presenting the health management plan formulated by the above-mentioned plan formulation means to the user,

[0857] A system that includes this.

[0858] (Claim 2)

[0859] The system according to claim 1, further comprising an opinion aggregation means for collecting opinions from users and transmitting opinion information to the above-mentioned information management means.

[0860] (Claim 3)

[0861] The system according to claim 1, wherein the evaluation means employs a generative model that comprehensively analyzes multiple data formats.

[0862] "Example 2 of combining an emotion engine"

[0863] (Claim 1)

[0864] A data collection means consisting of equipment that acquires biometric information and voice information,

[0865] A data management means that anonymizes and manages the information received from the above data collection means,

[0866] An emotion engine that runs on a server and recognizes emotional states from voice and text information,

[0867] The above data management means and emotion engine analyze the information obtained to assess health risks, and the analysis means considers multiple information modalities.

[0868] A plan generation means that generates an individual care plan based on the results of the above analysis means and the recognized emotional state,

[0869] A plan provision means that provides the care plan generated by the above plan generation means to the user,

[0870] A system that includes this.

[0871] (Claim 2)

[0872] The system according to claim 1, further comprising means for collecting user feedback and transmitting feedback information to the data management means.

[0873] (Claim 3)

[0874] The system according to claim 1, wherein the analysis means includes means for performing an integrated analysis using a generated AI model.

[0875] "Application example 2 when combining with an emotional engine"

[0876] (Claim 1)

[0877] A data collection means consisting of a device that collects user information,

[0878] A data management means that anonymizes and manages the information received from the above data collection means,

[0879] An analysis means for analyzing information managed by the above data management means to evaluate health risks,

[0880] A plan generation means that generates an individual care plan based on the results of the above analysis means,

[0881] A plan provision means that provides the care plan generated by the above plan generation means to the user,

[0882] An emotion recognition means equipped with an analysis engine that recognizes emotional states,

[0883] A support delivery method that provides daily support based on health and emotional state in conjunction with applied devices,

[0884] A system that includes this.

[0885] (Claim 2)

[0886] The system according to claim 1, further comprising an opinion collection means for collecting user opinions and transmitting opinion data to the data management means.

[0887] (Claim 3)

[0888] The system according to claim 1, wherein the analysis means uses a generative model that comprehensively analyzes multiple data modalities, and further provides application suggestions based on the analysis results. [Explanation of symbols]

[0889] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A data collection means consisting of a device for collecting user data, A data management means that anonymizes and manages the data received from the above data collection means, An analysis means for analyzing data managed by the above data management means to evaluate health risks, A plan generation means that generates individual care plans based on the results of the above analysis means, A plan provision means that provides the care plan generated by the above plan generation means to the user, A system that includes this.

2. The system according to claim 1, further comprising a feedback collection means for collecting user feedback and transmitting feedback data to the data management means.

3. The system according to claim 1, wherein the analysis means uses a generative model that comprehensively analyzes multiple data modalities.

Images