system

Abstract

We provide the system. [Solution] Information acquisition methods for obtaining personal physical information, Information management means for recording and managing acquired personal physical information, An analytical means for analyzing recorded and managed information and evaluating physical condition, Based on the analysis results, a predictive means for detecting signs of a physical condition, A visualization method that visualizes detection results and provides them to users, A system including control means for controlling the above means.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, many individuals regularly obtain their physical information, but there is a problem that such information cannot be effectively utilized. As a result, potential health problems may not be discovered early, and preventive measures may not be taken and the situation may progress. Also, there is a lack of effective means for individuals to understand their own physical conditions and make appropriate improvements to their lives.

Means for Solving the Problems

[0005] This invention solves the above problems by providing a system equipped with information acquisition means for acquiring an individual's physical information, information management means for recording and managing information, and analysis means for analyzing information. Furthermore, by detecting signs of physical condition based on the analysis results and providing them to the user in a visualized state, the system makes it possible to intuitively understand one's health status. The system also includes data collection means for acquiring resting states, and by combining this with physical information and analyzing it, it realizes comprehensive physical analysis. Furthermore, by generating suggestions for lifestyle improvements based on the analysis results and notifying the user, the system provides improvement measures tailored to each individual. This supports the health management that individuals perform on a daily basis and improves health prevention.

[0006] "Information acquisition means" refers to the components of devices and software used to measure and collect an individual's physical information.

[0007] "Information management means" refers to technical procedures and devices for storing and managing acquired personal physical information in a database or file system.

[0008] "Analysis methods" refer to techniques for analyzing data based on recorded information, following specific algorithms and rules, and evaluating the physical condition.

[0009] "Predictive detection means" refers to functions and methods that use the results obtained from analysis means to detect potential physical abnormalities or changes at an early stage.

[0010] "Visualization means" refers to interfaces and methods for converting detected information and analysis results into a format that is easy for users to understand and displaying them.

[0011] A "control system" is a system or mechanism that integrates various means to ensure that the overall operation and management are carried out consistently.

[0012] "Data collection means" refers to sensors and devices used to acquire and record information related to the user's rest.

[0013] A "proposal generation method" refers to a process or algorithm for creating personalized lifestyle improvement plans based on analysis results.

[0014] "Notification means" refers to devices or notification systems used to communicate generated proposals and analysis results to users. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the language used in the following description will be explained.

[0018] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] The present invention provides a system for health management purposes that acquires an individual's physical information and performs analysis, visualization, and notification based on that information. This system includes, as an embodiment, information acquisition means, information management means, analysis means, predictive detection means, visualization means, and control means. Specific embodiments are described below.

[0037] Information acquisition means

[0038] Sensors built into the device are used to collect the user's physical information. For example, smartphones and wearable devices acquire data such as heart rate, body temperature, and activity level.

[0039] Information management means

[0040] The device temporarily records the acquired physical information and periodically sends it to the server. The server stores this information in a database and prepares it for subsequent analysis.

[0041] Analysis means

[0042] The server analyzes stored physical information and evaluates the user's health status using a specified algorithm. For example, it can assess stress levels and health risks based on blood pressure data.

[0043] Precursor detection means

[0044] Based on the analysis results, the server detects specific patterns and anomalies, identifying potential signs of illness. For example, it can analyze long-term data to predict the risk of hypertension.

[0045] Visualization means

[0046] The device receives analysis results from the server and displays them graphically. Through the app, users can intuitively understand an overview of their health status and any points to be aware of.

[0047] Control means

[0048] The system is equipped with control mechanisms to manage the operation of the entire system, which integrates and operates multiple mechanisms to ensure consistent service delivery.

[0049] As a concrete example, when a user touches the device in the morning, the system automatically measures their heart rate and body temperature. This data is immediately transmitted to a server and analyzed together with the history from the previous days. If the system determines that the user's physical condition is abnormal that day, the device will send a warning notification to the user and, if necessary, provide advice to encourage them to seek medical attention.

[0050] The following describes the processing flow.

[0051] Step 1:

[0052] The user launches the health management app on their device. The device activates its built-in sensors to measure physical information such as heart rate and body temperature. During this process, the device temporarily stores the measurement data in its internal memory.

[0053] Step 2:

[0054] The terminal sends measurement data to the server based on certain criteria. The server stores the received data in a database and integrates it with the measurement data history.

[0055] Step 3:

[0056] The server uses an analysis algorithm to evaluate the health status based on the received physical information. For example, it analyzes heart rate variability and calculates abnormal patterns and stress indicators.

[0057] Step 4:

[0058] The server uses the analysis results to detect early signs of health problems. If an abnormality or risk is identified, the information is recorded in the database, allowing for follow-up investigations.

[0059] Step 5:

[0060] The device receives analysis results from the server and displays the information graphically in the user interface. Users can view specific figures, risk alerts, and improvement suggestions within the app.

[0061] Step 6:

[0062] The server generates additional lifestyle improvement suggestions and sends notifications to the device, urging it to visit a medical institution if necessary. The device then notifies the user and follows up on the improvement suggestions.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] In modern society, personal health management is a crucial issue, and there is a particular need for real-time monitoring of health status in daily life and early detection of changes. However, conventional systems have not adequately collected and analyzed physiological information, making it difficult to quickly and effectively assess health risks and suggest lifestyle improvements. This has led to the risk of overlooking potential health problems and resulting in insufficient health management for users.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes information gathering means for acquiring personal physiological information, analysis means for analyzing the acquired physiological information, and suggestion generation means for generating lifestyle improvement suggestions based on the analysis results. This makes it possible to effectively monitor the user's health status, detect potential health risks early, and quickly provide appropriate lifestyle improvement suggestions.

[0068] "Personal physiological information" refers to data that represents the user's health status and includes physical data such as heart rate, body temperature, and activity level.

[0069] "Information gathering means" refers to devices or systems used to acquire personal physiological information from users, and includes sensors installed in smartphones and wearable devices.

[0070] "Information management means" refers to means that have the function of temporarily storing acquired physiological information and transferring the data to other devices or systems as needed.

[0071] "Analysis means" refers to a means of analyzing collected data based on a specified calculation method to evaluate the user's health status.

[0072] A "predictive detection method" is a means of detecting specific patterns or anomalies from analyzed data and identifying potential health risks.

[0073] "Visualization means" refers to a means of visually displaying the analyzed and detected results to the user, thereby promoting an understanding of their health status.

[0074] A "control means" is an integrated means for managing the operation of the entire system and ensuring coordination between each means.

[0075] A "proposal generation method" is a method equipped with the function of suggesting improvements to the user's daily life based on the analysis results.

[0076] The system of this invention consists of a process for acquiring, analyzing, and visualizing individual physiological information for the purpose of health management, and notifying the user. This system is mainly implemented using terminals and servers.

[0077] The device uses sensors built into smartphones and wearable devices to collect physiological information such as the user's heart rate, body temperature, and activity level in real time. The acquired data is temporarily stored on the device and sent to a server at regular intervals. Wi-Fi and cellular communication networks are mainly used for this data transmission.

[0078] The server stores the received physiological information in a database and evaluates the user's health status using analysis tools. The analysis utilizes AI-powered algorithms, enabling precise analysis of the user's health status based on heart rate variability and other physiological indicators. These analysis results are then used by predictive detection tools to identify abnormalities and predict potential health risks.

[0079] The analysis results are graphically visualized on the device for easy user understanding. A dedicated application is used, allowing users to grasp an overview of their health status and any points requiring attention through an intuitive interface. Furthermore, if an abnormality is detected, the device will notify the user appropriately and, if necessary, offer suggestions for lifestyle improvements.

[0080] As a concrete example, when a user wakes up in the morning and picks up their smartphone, the device automatically measures their heart rate and body temperature. This information is sent to a server and analyzed by an AI model. For instance, if a slight increase in body temperature persists, the device displays an alert such as, "Your body temperature is higher than normal. We recommend you drink plenty of fluids and get some rest."

[0081] The generating AI model can perform further analysis upon receiving the following prompt: "Explain how the health management system uses heart rate and body temperature data to assess the user's health status."

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] The device uses its built-in sensors to collect the user's physiological information. Specifically, it acquires data such as heart rate, body temperature, and activity level in real time. The input is the user's physiological state, and the output is this data. This provides basic data on the user's daily health status.

[0085] Step 2:

[0086] The terminal temporarily stores the acquired physiological information and sends the data to the server at regular intervals. The input is the physiological information obtained in step 1, and the output is the data sent to the server. Here, communication processing is performed to ensure that the data arrives at the server safely and efficiently.

[0087] Step 3:

[0088] The server stores the received physiological information in a database. The input is physiological information sent from the terminal, and the output is data organized within the database. This step provides a foundation for smooth subsequent analysis.

[0089] Step 4:

[0090] The server analyzes physiological information stored in a database to assess health status. The input is data from the database, and the output is the result of the health status assessment. This analysis utilizes AI models and employs sophisticated pattern recognition on the data.

[0091] Step 5:

[0092] The server detects anomalies and specific patterns based on the analysis results and identifies warning signs. The input is the analysis results obtained in step 4, and the output is risk assessment and necessary alert information. This makes it possible to identify potential health risks in advance.

[0093] Step 6:

[0094] The terminal receives analysis results sent from the server and displays them graphically to the user via an application. The input is the analysis results from the server, and the output is visualized health information for the user. Through this, the user can intuitively understand their own health status.

[0095] Step 7:

[0096] The device notifies the user based on the detection results. Inputs include information on significant risks and suggestions for lifestyle improvements, while outputs are alerts and suggestions sent to the user. For example, if a user's body temperature remains higher than normal, a notification such as "Rest and hydration are recommended" will be sent.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] In health management systems using individual physical information, conventional technologies were limited to merely acquiring and evaluating physical information, lacking a mechanism for users to immediately recognize abnormalities and take rapid emergency action. Furthermore, they did not consider predicting risks due to changes in the surrounding environment and providing users with advance warnings. As a result, there was a problem in that rapid responses to abnormal health conditions or dangers in the external environment could not be made, and the safety of users could not be adequately ensured.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes information acquisition means for acquiring personal physical information, notification operation means for issuing an alarm and notifying emergency contacts when an anomaly is detected, and environmental detection means for monitoring changes in the surrounding environment and predicting dangers. This enables real-time detection and notification of physical abnormalities and environmental risks, allowing for a rapid response.

[0102] "Information acquisition means" refers to devices and sensors that collect physiological data such as heart rate and body temperature from the user's body.

[0103] "Information management means" refers to systems and software for recording and managing acquired physical information.

[0104] "Analysis means" refers to algorithms and software used to evaluate health status using recorded physical information.

[0105] A "predictive detection method" is a method or program for identifying abnormalities or patterns in a person's physical condition based on analyzed health data.

[0106] A "visualization method" is an interface that graphically displays analysis results so that users can intuitively understand them.

[0107] A "notification operation means" is a mechanism that sends an alarm or notification to the user or registered emergency contact when an anomaly is detected.

[0108] "Environmental detection means" refers to a function that monitors changes in the user's surrounding environment, detects predictable risks, and provides early warnings.

[0109] "Control means" refers to hardware or software that integrates and manages each of these means to control the entire system.

[0110] The system for realizing this invention has the function of collecting the user's physical information using sensors installed in smartphones and wearable devices. For example, it acquires data such as heart rate, body temperature, and activity level. The acquired data is temporarily recorded on the terminal and periodically sent to a server. The server stores this data in a database and performs health status analysis.

[0111] The server evaluates the user's health status by applying specific algorithms to the collected physical data. Advanced data processing and analysis are performed using libraries such as Python's NumPy and Pandas. Furthermore, a predictive detection system operates based on the analysis results, detecting specific patterns and anomalies. This allows the system to provide users with information about potential illnesses and health risks.

[0112] Furthermore, the analysis results are sent back to the user's device and visualized in a graphical format for intuitive understanding. The display utilizes the user interface of smart glasses or a mobile app, allowing users to visually check their health status. If an abnormality is detected, a notification is immediately sent to emergency contacts via a notification system, and an alert is also issued to the user. This function utilizes Twilio's API, among others.

[0113] Furthermore, the environmental detection system monitors the user's surroundings and predicts potential hazards such as sudden temperature changes or increases in noise levels. This allows the user to receive advice on how to avoid unnecessary risks.

[0114] For example, if a user's heart rate increases drastically during their commute, the smart glasses will display a visual alert saying, "Your heart rate is high. Please take a short break." Also, if an abnormal temperature fluctuation is predicted, a notification will be sent saying, "The temperature is dropping. Please take precautions against the cold."

[0115] An example of a prompt to a generative AI model is, "Generate health advice for the user based on today's temperature and heart rate data." Using this prompt, the AI ​​model can generate appropriate advice and provide it to the user.

[0116] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0117] Step 1:

[0118] The device uses sensors embedded in smartphones and wearable devices to acquire physiological data such as the user's heart rate, body temperature, and activity level. The input is the sensor signal, and the output is digitized physical data. This allows for the real-time collection of physical information based on the user's daily activities.

[0119] Step 2:

[0120] The terminal temporarily records the acquired physical information in local storage and then sends it to the server. The input is the acquired physical data, and the output is the physical information file sent to the server. This step prepares the data for more advanced analysis.

[0121] Step 3:

[0122] The server stores the received physical information in a database and performs data analysis using Python's NumPy and Pandas libraries. The input is the physical information sent to the server, and the output is the analyzed health status data. This process involves pattern recognition and anomaly detection of the information, and the user's health status is evaluated.

[0123] Step 4:

[0124] The server uses a generative AI model based on the analysis results to predict the user's potential health risks. The input is analyzed health status data, and the output is a risk assessment report. The generative AI model performs an appropriate risk assessment using prompts.

[0125] Step 5:

[0126] The server visualizes the analysis results and risk assessment and transmits this information to the terminal. The input is the risk assessment report, and the output is visualized data that can be displayed on the terminal. The terminal notifies the user through a graphic interface, visually presenting their health status and risks.

[0127] Step 6:

[0128] If an anomaly is detected, the device uses the Twilio API to send a notification to pre-registered emergency contacts. The input is the anomaly detection information, and the output is the notification message sent to the contacts. This procedure allows users and relevant parties to respond to the anomaly quickly.

[0129] Step 7:

[0130] Users can check notifications on their devices and take action directly in response to changes in their health or environment as needed. The input is notification information from the device, and the output is the user's actions based on the acquired health information. This step provides users with useful support for managing their own health and safety.

[0131] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0132] This invention is a system that combines and analyzes physical and emotional information in personal health management to provide personalized health recommendations. This system comprises information acquisition means, information management means, analysis means, predictive detection means, visualization means, control means, and an emotion engine. Specific embodiments are described below.

[0133] Information acquisition means

[0134] The device uses built-in sensors to periodically collect physical information such as the user's heart rate, body temperature, and activity level. In addition, it acquires facial expression data through the device's camera, and an emotion engine analyzes the emotional state from these facial expressions.

[0135] Information management means

[0136] The device temporarily records acquired physical and emotional data and sends it to the server. The server stores this data in a database.

[0137] Analysis means

[0138] The server comprehensively analyzes the received physical and emotional data. The analysis algorithm assesses the user's overall health status by considering not only their heart rate and activity level, but also changes in their emotions.

[0139] Precursor detection means

[0140] The server detects potential health risks and emotional abnormalities based on the analysis results. For example, if a person's heart rate is high when they are under stress, it can predict certain health risks.

[0141] Visualization means

[0142] The device displays simple graphs and summaries through the user interface based on information sent from the server. This allows users to intuitively understand their own health status and emotional tendencies.

[0143] Control means

[0144] The system provides integrated control to ensure that each component works effectively together. This control offers users an effective health management experience.

[0145] Emotional Engine

[0146] The emotion engine analyzes the user's facial expressions and voice data to assess their emotions. This allows for a detailed understanding of how daily emotional changes affect their health.

[0147] As a concrete example, when a user is in a stressful situation, the emotion engine detects the change and sends information to the server. The server analyzes past physical and emotional data and suggests relaxation methods to alleviate mental and physical stress. In this way, incorporating emotional information makes it possible to provide more precise health management and improvement measures.

[0148] The following describes the processing flow.

[0149] Step 1:

[0150] The user launches a health management app on their device. The device activates various sensors to collect physical information such as heart rate, body temperature, and activity level. Simultaneously, the device's camera is used to acquire facial expression data.

[0151] Step 2:

[0152] The device sends collected physical information and facial expression data to the emotion engine. The emotion engine analyzes changes in facial expressions and determines the user's emotional state.

[0153] Step 3:

[0154] The device transmits physical information and emotional state to the server. The server records and manages the received data in a database.

[0155] Step 4:

[0156] The server uses data analysis algorithms to comprehensively analyze physical and emotional data. For example, if a high stress level is detected, it examines heart rate variability to assess whether there are any abnormalities.

[0157] Step 5:

[0158] The server detects potential health risks and emotional abnormalities based on the analysis results. Based on the detection results, it generates health warnings and precautions and adjusts necessary improvement suggestions.

[0159] Step 6:

[0160] The server sends analysis results and improvement suggestions to the terminal. The terminal notifies the user of these and visualizes their health status and emotional tendencies through an interface. The user reviews the suggestions and uses them to improve their life.

[0161] Step 7:

[0162] The terminal receives user feedback on the suggestions and automatically sends it to the server, which is then used to improve future analyses and suggestion generation.

[0163] (Example 2)

[0164] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0165] In modern society, despite the importance of individual health management, physical and emotional information are often treated separately. As a result, it is difficult to grasp the overall picture of one's health, leading to challenges in providing appropriate health management suggestions. Furthermore, there is a lack of information provided in a way that allows users to intuitively understand changes in their health and emotions.

[0166] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0167] In this invention, the server includes data collection means for acquiring an individual's physical and emotional information, information management means for recording and managing the acquired information, and analysis means for comprehensively analyzing the recorded and managed information and evaluating the overall health status. This enables analysis that combines physical and emotional information. Furthermore, it makes it possible to visualize the detection results and provide them in an intuitively understandable format.

[0168] "Data collection means" refers to devices and processes for acquiring an individual's physical and emotional information.

[0169] "Information management means" refers to methods and systems for recording and securely managing acquired physical and emotional information.

[0170] "Analysis means" refers to processes and devices for comprehensively evaluating a user's health status using recorded and managed physical and emotional information.

[0171] "Predictive detection methods" refer to techniques and systems for detecting health risks and emotional abnormalities based on analysis results.

[0172] "Visualization means" refers to methods and technologies for visually representing detection results in an easy-to-understand manner and providing them to the user.

[0173] "Control means" refers to processes or devices that enable each means of a system to work together and function effectively.

[0174] An "emotion analysis engine" is software or a process that analyzes a user's facial expression data and voice data to evaluate their emotional state.

[0175] A "proposal generation method" refers to a process or system that generates suggestions for improving the user's health based on analysis results.

[0176] "Notification means" refers to methods and technologies for communicating generated suggestions and analysis results to the user.

[0177] This invention is designed as a system that enables a detailed assessment of an individual's health status. This system uses sensors and cameras mounted on a device to collect various information, such as the user's heart rate, body temperature, activity level, and facial expression data. This hardware includes vital signs sensors and high-resolution cameras.

[0178] After collecting information, the device temporarily stores this data and sends it to the server. The server securely stores and manages the information using a data management system. Next, the server uses machine learning algorithms and other tools to comprehensively analyze the physical and emotional information. This analysis allows for an assessment of the user's overall health status.

[0179] The analysis results are used by a server-based predictive detection system to detect potential health risks and emotional abnormalities. Health recommendations generated based on this are sent to the terminal and intuitively visualized through a graphical user interface. This makes it easier for users to understand their own health status.

[0180] For example, if a user is in a stressful situation, the server can analyze their increased heart rate and emotional changes and suggest relaxation techniques. This suggestion might appear on the device as a notification advising them to "take deep breaths."

[0181] An example of a prompt to input to a generative AI model is: "Discuss how we should integrate an emotion engine in building a system that understands the user's health in detail and provides personalized suggestions."

[0182] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0183] Step 1:

[0184] The device uses built-in sensors and a camera to collect data on the user's heart rate, body temperature, activity level, and facial expressions. Inputs include this biometric information and facial expression data, while output is an integrated dataset of these. Specific operations include calibrating the sensors and camera and periodically acquiring data.

[0185] Step 2:

[0186] The acquired data is temporarily stored on the device and then sent to the server. The input is the dataset obtained in step 1, and the output is the secure data transfer to the server. Specifically, this involves implementing a secure transmission protocol using data encryption technology.

[0187] Step 3:

[0188] The server records and manages received data in a database. Input is data received from terminals, and output is structured data stored in the database. Specific operations include a process of saving the data while maintaining data integrity and adding necessary metadata.

[0189] Step 4:

[0190] The server analyzes stored data based on machine learning algorithms. The input is data extracted from a database, and the output is the analyzed health status assessment result. Specific operations include data preprocessing, algorithmic pattern recognition, and anomaly detection.

[0191] Step 5:

[0192] Based on the analysis results, the server detects potential health risks and emotional abnormalities. The input is the analyzed evaluation results, and the output is the detected risk information. Specific operations include the application of threshold determination methods based on risk assessment criteria.

[0193] Step 6:

[0194] The server generates appropriate health recommendations based on detected health risks. The input is detected risk information, and the output is health recommendations. Specific operations include comparing the current recommendations with past recommendation history in the database to formulate personalized advice.

[0195] Step 7:

[0196] The terminal visualizes and provides health suggestions received from the server to the user. The input is the generated health suggestions, and the output is the visually represented suggestion information. Specific actions include notifications and graph displays through the user interface.

[0197] (Application Example 2)

[0198] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0199] Conventional health management systems typically evaluate health status based solely on physical information, often lacking analysis that considers the user's emotional state. This can lead to a failure to detect potential health risks caused by stress and emotional fluctuations early, potentially delaying appropriate responses. Furthermore, this can result in inadequate safety management, making it difficult to ensure safety in workplaces and other environments.

[0200] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0201] In this invention, the server includes information acquisition means for acquiring an individual's physical information and emotional state, information management means for recording and managing the acquired data, and analysis means for comprehensively analyzing and evaluating the recorded information. This enables safe and effective health and safety management that takes both physical and emotional aspects into consideration.

[0202] "Information acquisition means" refers to devices and technologies used to collect an individual's physical information and emotional state.

[0203] "Information management means" refers to systems and processes for recording and managing acquired physical information and emotional data.

[0204] "Analysis means" refers to devices or methods for integrating and analyzing recorded physical information and emotional data to evaluate the individual physical and emotional state.

[0205] "Predictive detection means" refers to functions and technologies for recognizing physical and emotional abnormalities based on analysis results.

[0206] "Visualization means" refers to systems and technologies for visually displaying the results of data analysis and providing information to users.

[0207] "Control means" refers to a mechanism or technology that ensures compatibility and coordination among the various means within a system.

[0208] "Data collection means" refers to devices and technologies for effectively collecting information on users' resting states and workers' emotional changes.

[0209] "Proposal generation means" refers to a function or technology that generates guidelines for improving work safety based on analysis results.

[0210] "Notification means" refers to communication technologies in a system for transmitting generated guidelines to relevant individuals or administrators.

[0211] The system according to the present invention provides users with a safe and healthy environment by acquiring and analyzing an individual's physical and emotional information in real time. This system includes information acquisition means, information management means, analysis means, predictive detection means, visualization means, control means, and the like.

[0212] The information acquisition method utilizes sensors and cameras installed in wearable devices such as smart glasses to collect data on the user's heart rate, body temperature, and facial expressions. The data is transmitted to a smartphone via Bluetooth or other means.

[0213] In terms of information management, the smartphone temporarily stores the collected physical and emotional data and uploads it to a cloud server. This cloud server maintains the data in a database, enabling persistent data storage.

[0214] The analysis method involves a server continuously analyzing data. Specifically, it uses Microsoft® Azure® sentiment analysis technology to evaluate changes in heart rate and facial expressions. This allows for a comprehensive assessment of the user's physical condition and emotional changes, and generates necessary countermeasures.

[0215] The server automatically detects stress and anomalies using predictive detection mechanisms. For example, if the heart rate suddenly increases and negative emotions are detected from the user's facial expression, the system will suggest relaxation methods to the user and notify the administrator of an alert.

[0216] The visualization method displays the analysis results as diagrams and charts on the smartphone screen, allowing users to intuitively understand the situation.

[0217] As a concrete example, when a worker in a factory feels fatigued and stressed during work, the system detects these and sends notifications recommending that the worker take a break and that the manager improve the work environment.

[0218] An example of a prompt for a generative AI model is: "Receive heart rate and facial expression data from the user and analyze the current emotional state. If high stress levels or suspicious emotional states are detected, output the details." Through this prompt, sophisticated emotional analysis can be achieved.

[0219] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0220] Step 1:

[0221] The device acquires heart rate, body temperature, and facial expression data in real time from the wearable device's sensors. This data is transmitted to a smartphone via Bluetooth. As input, it receives physical and emotional data from the sensors and temporarily stores it on the smartphone. As output, the data is ready to be sent to a cloud server.

[0222] Step 2:

[0223] The data acquired by the smartphone is transferred to a cloud server. The server stores this data in a database, preparing it for subsequent analysis processes. The input consists of physical and emotional data transmitted from the smartphone, and the output is the data stored in the server's database.

[0224] Step 3:

[0225] The server analyzes the data and evaluates the user's health and emotional state based on heart rate and emotional data. Here, Microsoft Azure's emotion analysis technology is used to send prompts to a generative AI model to analyze the emotional state. The input data consists of physical and emotional data stored on the server, and the output is the analyzed physical and mental state of the user.

[0226] Step 4:

[0227] The server detects potential health risks and emotional abnormalities based on the analysis results. The system monitors changes in heart rate and sudden changes in emotions, and reports on increases in stress levels, etc. The input to this process is analyzed numerical and emotional data, and the output is the detection results of potential risks and emotional abnormalities.

[0228] Step 5:

[0229] The server generates data for visually displaying the analysis and detection results using visualization tools and sends it to the terminal. This allows users to view an overview of their health status and emotional tendencies in graphs and charts on their smartphones. The input is risk information and health status assessments that have been detected as precursors, and the output is displayed data that visualizes these.

[0230] Step 6:

[0231] The user takes necessary actions based on instructions from the system. For example, upon receiving a notification from the system, the user might try relaxation techniques such as taking a break or deep breathing. The input for this step is the visualized data displayed to the user and alerts from the system, while the output is the specific action taken.

[0232] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0233] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0234] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0235] [Second Embodiment]

[0236] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0237] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0238] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0239] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0240] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0241] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0242] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0243] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0244] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0245] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0246] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0247] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0248] The present invention provides a system for health management purposes that acquires an individual's physical information and performs analysis, visualization, and notification based on that information. This system includes, as an embodiment, information acquisition means, information management means, analysis means, predictive detection means, visualization means, and control means. Specific embodiments are described below.

[0249] Information acquisition means

[0250] Sensors built into the device are used to collect the user's physical information. For example, smartphones and wearable devices acquire data such as heart rate, body temperature, and activity level.

[0251] Information management means

[0252] The device temporarily records the acquired physical information and periodically sends it to the server. The server stores this information in a database and prepares it for subsequent analysis.

[0253] Analysis means

[0254] The server analyzes stored physical information and evaluates the user's health status using a specified algorithm. For example, it can assess stress levels and health risks based on blood pressure data.

[0255] Precursor detection means

[0256] Based on the analysis results, the server detects specific patterns and anomalies, identifying potential signs of illness. For example, it can analyze long-term data to predict the risk of hypertension.

[0257] Visualization means

[0258] The device receives analysis results from the server and displays them graphically. Through the app, users can intuitively understand an overview of their health status and any points to be aware of.

[0259] Control means

[0260] The system is equipped with control mechanisms to manage the operation of the entire system, which integrates and operates multiple mechanisms to ensure consistent service delivery.

[0261] As a concrete example, when a user touches the device in the morning, the system automatically measures their heart rate and body temperature. This data is immediately transmitted to a server and analyzed together with the history from the previous days. If the system determines that the user's physical condition is abnormal that day, the device will send a warning notification to the user and, if necessary, provide advice to encourage them to seek medical attention.

[0262] The following describes the processing flow.

[0263] Step 1:

[0264] The user launches the health management app on their device. The device activates its built-in sensors to measure physical information such as heart rate and body temperature. During this process, the device temporarily stores the measurement data in its internal memory.

[0265] Step 2:

[0266] The terminal sends measurement data to the server based on certain criteria. The server stores the received data in a database and integrates it with the measurement data history.

[0267] Step 3:

[0268] The server uses an analysis algorithm to evaluate the health status based on the received physical information. For example, it analyzes heart rate variability and calculates abnormal patterns and stress indicators.

[0269] Step 4:

[0270] The server uses the analysis results to detect early signs of health problems. If an abnormality or risk is identified, the information is recorded in the database, allowing for follow-up investigations.

[0271] Step 5:

[0272] The device receives analysis results from the server and displays the information graphically in the user interface. Users can view specific figures, risk alerts, and improvement suggestions within the app.

[0273] Step 6:

[0274] The server generates additional lifestyle improvement suggestions and sends notifications to the device, urging it to visit a medical institution if necessary. The device then notifies the user and follows up on the improvement suggestions.

[0275] (Example 1)

[0276] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0277] In modern society, personal health management is a crucial issue, and there is a particular need for real-time monitoring of health status in daily life and early detection of changes. However, conventional systems have not adequately collected and analyzed physiological information, making it difficult to quickly and effectively assess health risks and suggest lifestyle improvements. This has led to the risk of overlooking potential health problems and resulting in insufficient health management for users.

[0278] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0279] In this invention, the server includes information gathering means for acquiring personal physiological information, analysis means for analyzing the acquired physiological information, and suggestion generation means for generating lifestyle improvement suggestions based on the analysis results. This makes it possible to effectively monitor the user's health status, detect potential health risks early, and quickly provide appropriate lifestyle improvement suggestions.

[0280] "Personal physiological information" refers to data that represents the user's health status and includes physical data such as heart rate, body temperature, and activity level.

[0281] "Information gathering means" refers to devices or systems used to acquire personal physiological information from users, and includes sensors installed in smartphones and wearable devices.

[0282] "Information management means" refers to means that have the function of temporarily storing acquired physiological information and transferring the data to other devices or systems as needed.

[0283] The "analysis means" is a means for analyzing the collected data based on a specified algorithm and evaluating the user's health status.

[0284] The "symptom detection means" is a means for detecting specific patterns or abnormalities from the analyzed data and discriminating potential health risks.

[0285] The "visualization means" is a means for visually displaying the analyzed and detected results to the user and promoting the understanding of the health status.

[0286] The "control means" is an integrated means for managing the operation of the entire system and ensuring the cooperation between each means.

[0287] The "proposal generation means" is a means equipped with a function of proposing improvement points in the user's daily life based on the analysis results.

[0288] The system of the present invention is configured by a process of acquiring, analyzing, visualizing, and notifying the user of personal physiological information for the purpose of health management. This system is mainly implemented using a terminal and a server.

[0289] The terminal uses sensors installed in smartphones and wearable devices to collect physiological information such as the user's heart rate, body temperature, and activity level in real time. The acquired data is temporarily stored in the terminal and transmitted to the server at regular time intervals. Mainly Wi-Fi and mobile communication networks are used for this data transmission.

[0290] The server stores the received physiological information in a database and evaluates the health status by the analysis means. An algorithm utilizing an AI model is used for the analysis, enabling precise analysis of the user's health status from fluctuations in the heart rate and other physiological indicators. This analysis result is used by the symptom detection means to detect abnormalities and predict potential health risks.

[0291] The analysis results are graphically visualized on the device for easy user understanding. A dedicated application is used, allowing users to grasp an overview of their health status and any points requiring attention through an intuitive interface. Furthermore, if an abnormality is detected, the device will notify the user appropriately and, if necessary, offer suggestions for lifestyle improvements.

[0292] As a concrete example, when a user wakes up in the morning and picks up their smartphone, the device automatically measures their heart rate and body temperature. This information is sent to a server and analyzed by an AI model. For instance, if a slight increase in body temperature persists, the device displays an alert such as, "Your body temperature is higher than normal. We recommend you drink plenty of fluids and get some rest."

[0293] The generating AI model can perform further analysis upon receiving the following prompt: "Explain how the health management system uses heart rate and body temperature data to assess the user's health status."

[0294] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0295] Step 1:

[0296] The device uses its built-in sensors to collect the user's physiological information. Specifically, it acquires data such as heart rate, body temperature, and activity level in real time. The input is the user's physiological state, and the output is this data. This provides basic data on the user's daily health status.

[0297] Step 2:

[0298] The terminal temporarily stores the acquired physiological information and sends the data to the server at regular intervals. The input is the physiological information obtained in step 1, and the output is the data sent to the server. Here, communication processing is performed to ensure that the data arrives at the server safely and efficiently.

[0299] Step 3:

[0300] The server stores the received physiological information in the database. The input is the physiological information sent from the terminal, and the output is the data organized in the database. This step lays the foundation for smooth subsequent analysis.

[0301] Step 4:

[0302] The server analyzes the physiological information stored in the database and evaluates the health status. The input is the data in the database, and the output is the result of evaluating the health status. An algorithm utilizing an AI model is used for this analysis, and advanced pattern recognition is performed on the data.

[0303] Step 5:

[0304] The server detects abnormal values and specific patterns based on the analysis results and discriminates omens. The input is the analysis result obtained in Step 4, and the output is the risk assessment and information on necessary alerts. This enables potential health risks to be grasped in advance.

[0305] Step 6:

[0306] The terminal receives the analysis results sent from the server and graphically displays them to the user via the application. The input is the analysis result from the server, and the output is the health information visualized for the user. Through this, the user can intuitively understand their own health status.

[0307] Step 7:

[0308] The terminal notifies the user according to the detection results. The input is important risk information and suggestions for improving life, and the output is notifications of alerts and suggestions to the user. As a specific example, when the body temperature continues to be higher than normal, a notification such as "It is recommended to rest and replenish fluids" is issued.

[0309] (Application Example 1)

[0310] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0311] In health management systems using individual physical information, conventional technologies were limited to merely acquiring and evaluating physical information, lacking a mechanism for users to immediately recognize abnormalities and take rapid emergency action. Furthermore, they did not consider predicting risks due to changes in the surrounding environment and providing users with advance warnings. As a result, there was a problem in that rapid responses to abnormal health conditions or dangers in the external environment could not be made, and the safety of users could not be adequately ensured.

[0312] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0313] In this invention, the server includes information acquisition means for acquiring personal physical information, notification operation means for issuing an alarm and notifying emergency contacts when an anomaly is detected, and environmental detection means for monitoring changes in the surrounding environment and predicting dangers. This enables real-time detection and notification of physical abnormalities and environmental risks, allowing for a rapid response.

[0314] "Information acquisition means" refers to devices and sensors that collect physiological data such as heart rate and body temperature from the user's body.

[0315] "Information management means" refers to systems and software for recording and managing acquired physical information.

[0316] "Analysis means" refers to algorithms and software used to evaluate health status using recorded physical information.

[0317] A "predictive detection method" is a method or program for identifying abnormalities or patterns in a person's physical condition based on analyzed health data.

[0318] A "visualization method" is an interface that graphically displays analysis results so that users can intuitively understand them.

[0319] A "notification operation means" is a mechanism that sends an alarm or notification to the user or registered emergency contact when an anomaly is detected.

[0320] "Environmental detection means" refers to a function that monitors changes in the user's surrounding environment, detects predictable risks, and provides early warnings.

[0321] "Control means" refers to hardware or software that integrates and manages each of these means to control the entire system.

[0322] The system for realizing this invention has the function of collecting the user's physical information using sensors installed in smartphones and wearable devices. For example, it acquires data such as heart rate, body temperature, and activity level. The acquired data is temporarily recorded on the terminal and periodically sent to a server. The server stores this data in a database and performs health status analysis.

[0323] The server evaluates the user's health status by applying specific algorithms to the collected physical data. Advanced data processing and analysis are performed using libraries such as Python's NumPy and Pandas. Furthermore, a predictive detection system operates based on the analysis results, detecting specific patterns and anomalies. This allows the system to provide users with information about potential illnesses and health risks.

[0324] Furthermore, the analysis results are sent back to the user's device and visualized in a graphical format for intuitive understanding. The display utilizes the user interface of smart glasses or a mobile app, allowing users to visually check their health status. If an abnormality is detected, a notification is immediately sent to emergency contacts via a notification system, and an alert is also issued to the user. This function utilizes Twilio's API, among others.

[0325] Furthermore, the environmental detection system monitors the user's surroundings and predicts potential hazards such as sudden temperature changes or increases in noise levels. This allows the user to receive advice on how to avoid unnecessary risks.

[0326] For example, if a user's heart rate increases drastically during their commute, the smart glasses will display a visual alert saying, "Your heart rate is high. Please take a short break." Also, if an abnormal temperature fluctuation is predicted, a notification will be sent saying, "The temperature is dropping. Please take precautions against the cold."

[0327] An example of a prompt to a generative AI model is, "Generate health advice for the user based on today's temperature and heart rate data." Using this prompt, the AI ​​model can generate appropriate advice and provide it to the user.

[0328] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0329] Step 1:

[0330] The device uses sensors embedded in smartphones and wearable devices to acquire physiological data such as the user's heart rate, body temperature, and activity level. The input is the sensor signal, and the output is digitized physical data. This allows for the real-time collection of physical information based on the user's daily activities.

[0331] Step 2:

[0332] The terminal temporarily records the acquired physical information in local storage and then sends it to the server. The input is the acquired physical data, and the output is the physical information file sent to the server. This step prepares the data for more advanced analysis.

[0333] Step 3:

[0334] The server stores the received physical information in a database and performs data analysis using Python's NumPy and Pandas libraries. The input is the physical information sent to the server, and the output is the analyzed health status data. This process involves pattern recognition and anomaly detection of the information, and the user's health status is evaluated.

[0335] Step 4:

[0336] The server uses a generative AI model based on the analysis results to predict the user's potential health risks. The input is analyzed health status data, and the output is a risk assessment report. The generative AI model performs an appropriate risk assessment using prompts.

[0337] Step 5:

[0338] The server visualizes the analysis results and risk assessment and transmits this information to the terminal. The input is the risk assessment report, and the output is visualized data that can be displayed on the terminal. The terminal notifies the user through a graphic interface, visually presenting their health status and risks.

[0339] Step 6:

[0340] If an anomaly is detected, the device uses the Twilio API to send a notification to pre-registered emergency contacts. The input is the anomaly detection information, and the output is the notification message sent to the contacts. This procedure allows users and relevant parties to respond to the anomaly quickly.

[0341] Step 7:

[0342] Users can check notifications on their devices and take action directly in response to changes in their health or environment as needed. The input is notification information from the device, and the output is the user's actions based on the acquired health information. This step provides users with useful support for managing their own health and safety.

[0343] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0344] This invention is a system that combines and analyzes physical and emotional information in personal health management to provide personalized health recommendations. This system comprises information acquisition means, information management means, analysis means, predictive detection means, visualization means, control means, and an emotion engine. Specific embodiments are described below.

[0345] Information acquisition means

[0346] The device uses built-in sensors to periodically collect physical information such as the user's heart rate, body temperature, and activity level. In addition, it acquires facial expression data through the device's camera, and an emotion engine analyzes the emotional state from these facial expressions.

[0347] Information management means

[0348] The device temporarily records acquired physical and emotional data and sends it to the server. The server stores this data in a database.

[0349] Analysis means

[0350] The server comprehensively analyzes the received physical and emotional data. The analysis algorithm assesses the user's overall health status by considering not only their heart rate and activity level, but also changes in their emotions.

[0351] Precursor detection means

[0352] The server detects potential health risks and emotional abnormalities based on the analysis results. For example, if a person's heart rate is high when they are under stress, it can predict certain health risks.

[0353] Visualization means

[0354] The device displays simple graphs and summaries through the user interface based on information sent from the server. This allows users to intuitively understand their own health status and emotional tendencies.

[0355] Control means

[0356] The system provides integrated control to ensure that each component works effectively together. This control offers users an effective health management experience.

[0357] Emotional Engine

[0358] The emotion engine analyzes the user's facial expressions and voice data to assess their emotions. This allows for a detailed understanding of how daily emotional changes affect their health.

[0359] As a concrete example, when a user is in a stressful situation, the emotion engine detects the change and sends information to the server. The server analyzes past physical and emotional data and suggests relaxation methods to alleviate mental and physical stress. In this way, incorporating emotional information makes it possible to provide more precise health management and improvement measures.

[0360] The following describes the processing flow.

[0361] Step 1:

[0362] The user launches a health management app on their device. The device activates various sensors to collect physical information such as heart rate, body temperature, and activity level. Simultaneously, the device's camera is used to acquire facial expression data.

[0363] Step 2:

[0364] The device sends collected physical information and facial expression data to the emotion engine. The emotion engine analyzes changes in facial expressions and determines the user's emotional state.

[0365] Step 3:

[0366] The device transmits physical information and emotional state to the server. The server records and manages the received data in a database.

[0367] Step 4:

[0368] The server uses data analysis algorithms to comprehensively analyze physical and emotional data. For example, if a high stress level is detected, it examines heart rate variability to assess whether there are any abnormalities.

[0369] Step 5:

[0370] The server detects potential health risks and emotional abnormalities based on the analysis results. Based on the detection results, it generates health warnings and precautions and adjusts necessary improvement suggestions.

[0371] Step 6:

[0372] The server sends analysis results and improvement suggestions to the terminal. The terminal notifies the user of these and visualizes their health status and emotional tendencies through an interface. The user reviews the suggestions and uses them to improve their life.

[0373] Step 7:

[0374] The terminal receives user feedback on the suggestions and automatically sends it to the server, which is then used to improve future analyses and suggestion generation.

[0375] (Example 2)

[0376] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0377] In modern society, despite the importance of individual health management, physical and emotional information are often treated separately. As a result, it is difficult to grasp the overall picture of one's health, leading to challenges in providing appropriate health management suggestions. Furthermore, there is a lack of information provided in a way that allows users to intuitively understand changes in their health and emotions.

[0378] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0379] In this invention, the server includes data collection means for acquiring an individual's physical and emotional information, information management means for recording and managing the acquired information, and analysis means for comprehensively analyzing the recorded and managed information and evaluating the overall health status. This enables analysis that combines physical and emotional information. Furthermore, it makes it possible to visualize the detection results and provide them in an intuitively understandable format.

[0380] "Data collection means" refers to devices and processes for acquiring an individual's physical and emotional information.

[0381] "Information management means" refers to methods and systems for recording and securely managing acquired physical and emotional information.

[0382] "Analysis means" refers to processes and devices for comprehensively evaluating a user's health status using recorded and managed physical and emotional information.

[0383] "Predictive detection methods" refer to techniques and systems for detecting health risks and emotional abnormalities based on analysis results.

[0384] "Visualization means" refers to methods and technologies for visually representing detection results in an easy-to-understand manner and providing them to the user.

[0385] "Control means" refers to processes or devices that enable each means of a system to work together and function effectively.

[0386] An "emotion analysis engine" is software or a process that analyzes a user's facial expression data and voice data to evaluate their emotional state.

[0387] A "proposal generation method" refers to a process or system that generates suggestions for improving the user's health based on analysis results.

[0388] "Notification means" refers to methods and technologies for communicating generated suggestions and analysis results to the user.

[0389] This invention is designed as a system that enables a detailed assessment of an individual's health status. This system uses sensors and cameras mounted on a device to collect various information, such as the user's heart rate, body temperature, activity level, and facial expression data. This hardware includes vital signs sensors and high-resolution cameras.

[0390] After collecting information, the device temporarily stores this data and sends it to the server. The server securely stores and manages the information using a data management system. Next, the server uses machine learning algorithms and other tools to comprehensively analyze the physical and emotional information. This analysis allows for an assessment of the user's overall health status.

[0391] The analysis results are used by a server-based predictive detection system to detect potential health risks and emotional abnormalities. Health recommendations generated based on this are sent to the terminal and intuitively visualized through a graphical user interface. This makes it easier for users to understand their own health status.

[0392] For example, if a user is in a stressful situation, the server can analyze their increased heart rate and emotional changes and suggest relaxation techniques. This suggestion might appear on the device as a notification advising them to "take deep breaths."

[0393] An example of a prompt to input to a generative AI model is: "Discuss how we should integrate an emotion engine in building a system that understands the user's health in detail and provides personalized suggestions."

[0394] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0395] Step 1:

[0396] The device uses built-in sensors and a camera to collect data on the user's heart rate, body temperature, activity level, and facial expressions. Inputs include this biometric information and facial expression data, while output is an integrated dataset of these. Specific operations include calibrating the sensors and camera and periodically acquiring data.

[0397] Step 2:

[0398] The acquired data is temporarily stored on the device and then sent to the server. The input is the dataset obtained in step 1, and the output is the secure data transfer to the server. Specifically, this involves implementing a secure transmission protocol using data encryption technology.

[0399] Step 3:

[0400] The server records and manages received data in a database. Input is data received from terminals, and output is structured data stored in the database. Specific operations include a process of saving the data while maintaining data integrity and adding necessary metadata.

[0401] Step 4:

[0402] The server analyzes stored data based on machine learning algorithms. The input is data extracted from a database, and the output is the analyzed health status assessment result. Specific operations include data preprocessing, algorithmic pattern recognition, and anomaly detection.

[0403] Step 5:

[0404] Based on the analysis results, the server detects potential health risks and emotional abnormalities. The input is the analyzed evaluation results, and the output is the detected risk information. Specific operations include the application of threshold determination methods based on risk assessment criteria.

[0405] Step 6:

[0406] The server generates appropriate health recommendations based on detected health risks. The input is detected risk information, and the output is health recommendations. Specific operations include comparing the current recommendations with past recommendation history in the database to formulate personalized advice.

[0407] Step 7:

[0408] The terminal visualizes and provides health suggestions received from the server to the user. The input is the generated health suggestions, and the output is the visually represented suggestion information. Specific actions include notifications and graph displays through the user interface.

[0409] (Application Example 2)

[0410] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0411] Conventional health management systems typically evaluate health status based solely on physical information, often lacking analysis that considers the user's emotional state. This can lead to a failure to detect potential health risks caused by stress and emotional fluctuations early, potentially delaying appropriate responses. Furthermore, this can result in inadequate safety management, making it difficult to ensure safety in workplaces and other environments.

[0412] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0413] In this invention, the server includes information acquisition means for acquiring an individual's physical information and emotional state, information management means for recording and managing the acquired data, and analysis means for comprehensively analyzing and evaluating the recorded information. This enables safe and effective health and safety management that takes both physical and emotional aspects into consideration.

[0414] "Information acquisition means" refers to devices and technologies used to collect an individual's physical information and emotional state.

[0415] "Information management means" refers to systems and processes for recording and managing acquired physical information and emotional data.

[0416] "Analysis means" refers to devices or methods for integrating and analyzing recorded physical information and emotional data to evaluate the individual physical and emotional state.

[0417] "Predictive detection means" refers to functions and technologies for recognizing physical and emotional abnormalities based on analysis results.

[0418] "Visualization means" refers to systems and technologies for visually displaying the results of data analysis and providing information to users.

[0419] "Control means" refers to a mechanism or technology that ensures compatibility and coordination among the various means within a system.

[0420] "Data collection means" refers to devices and technologies for effectively collecting information on users' resting states and workers' emotional changes.

[0421] "Proposal generation means" refers to a function or technology that generates guidelines for improving work safety based on analysis results.

[0422] "Notification means" refers to communication technologies in a system for transmitting generated guidelines to relevant individuals or administrators.

[0423] The system according to the present invention provides users with a safe and healthy environment by acquiring and analyzing an individual's physical and emotional information in real time. This system includes information acquisition means, information management means, analysis means, predictive detection means, visualization means, control means, and the like.

[0424] The information acquisition method utilizes sensors and cameras installed in wearable devices such as smart glasses to collect data on the user's heart rate, body temperature, and facial expressions. The data is transmitted to a smartphone via Bluetooth or other means.

[0425] In terms of information management, the smartphone temporarily stores the collected physical and emotional data and uploads it to a cloud server. This cloud server maintains the data in a database, enabling persistent data storage.

[0426] The analysis method involves a server continuously analyzing data. Specifically, it uses Microsoft Azure's emotion analysis technology to evaluate changes in heart rate and facial expressions. This allows for a comprehensive assessment of the user's physical condition and emotional changes, and generates necessary countermeasures.

[0427] The server automatically detects stress and anomalies using predictive detection mechanisms. For example, if the heart rate suddenly increases and negative emotions are detected from the user's facial expression, the system will suggest relaxation methods to the user and notify the administrator of an alert.

[0428] The visualization method displays the analysis results as diagrams and charts on the smartphone screen, allowing users to intuitively understand the situation.

[0429] As a concrete example, when a worker in a factory feels fatigued and stressed during work, the system detects these and sends notifications recommending that the worker take a break and that the manager improve the work environment.

[0430] An example of a prompt for a generative AI model is: "Receive heart rate and facial expression data from the user and analyze the current emotional state. If high stress levels or suspicious emotional states are detected, output the details." Through this prompt, sophisticated emotional analysis can be achieved.

[0431] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0432] Step 1:

[0433] The device acquires heart rate, body temperature, and facial expression data in real time from the wearable device's sensors. This data is transmitted to a smartphone via Bluetooth. As input, it receives physical and emotional data from the sensors and temporarily stores it on the smartphone. As output, the data is ready to be sent to a cloud server.

[0434] Step 2:

[0435] The data acquired by the smartphone is transferred to a cloud server. The server stores this data in a database, preparing it for subsequent analysis processes. The input consists of physical and emotional data transmitted from the smartphone, and the output is the data stored in the server's database.

[0436] Step 3:

[0437] The server analyzes the data and evaluates the user's health and emotional state based on heart rate and emotional data. Here, Microsoft Azure's emotion analysis technology is used to send prompts to a generative AI model to analyze the emotional state. The input data consists of physical and emotional data stored on the server, and the output is the analyzed physical and mental state of the user.

[0438] Step 4:

[0439] The server detects potential health risks and emotional abnormalities based on the analysis results. The system monitors changes in heart rate and sudden changes in emotions, and reports on increases in stress levels, etc. The input to this process is analyzed numerical and emotional data, and the output is the detection results of potential risks and emotional abnormalities.

[0440] Step 5:

[0441] The server generates data for visually displaying the analysis and detection results using visualization tools and sends it to the terminal. This allows users to view an overview of their health status and emotional tendencies in graphs and charts on their smartphones. The input is risk information and health status assessments that have been detected as precursors, and the output is displayed data that visualizes these.

[0442] Step 6:

[0443] The user takes necessary actions based on instructions from the system. For example, upon receiving a notification from the system, the user might try relaxation techniques such as taking a break or deep breathing. The input for this step is the visualized data displayed to the user and alerts from the system, while the output is the specific action taken.

[0444] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0445] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0446] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0447] [Third Embodiment]

[0448] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0449] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0450] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0451] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0452] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0453] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0454] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0455] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0456] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0457] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0458] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0459] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0460] The present invention provides a system for health management purposes that acquires an individual's physical information and performs analysis, visualization, and notification based on that information. This system includes, as an embodiment, information acquisition means, information management means, analysis means, predictive detection means, visualization means, and control means. Specific embodiments are described below.

[0461] Information acquisition means

[0462] Sensors built into the device are used to collect the user's physical information. For example, smartphones and wearable devices acquire data such as heart rate, body temperature, and activity level.

[0463] Information management means

[0464] The device temporarily records the acquired physical information and periodically sends it to the server. The server stores this information in a database and prepares it for subsequent analysis.

[0465] Analysis means

[0466] The server analyzes stored physical information and evaluates the user's health status using a specified algorithm. For example, it can assess stress levels and health risks based on blood pressure data.

[0467] Precursor detection means

[0468] Based on the analysis results, the server detects specific patterns and anomalies, identifying potential signs of illness. For example, it can analyze long-term data to predict the risk of hypertension.

[0469] Visualization means

[0470] The device receives analysis results from the server and displays them graphically. Through the app, users can intuitively understand an overview of their health status and any points to be aware of.

[0471] Control means

[0472] The system is equipped with control mechanisms to manage the operation of the entire system, which integrates and operates multiple mechanisms to ensure consistent service delivery.

[0473] As a concrete example, when a user touches the device in the morning, the system automatically measures their heart rate and body temperature. This data is immediately transmitted to a server and analyzed together with the history from the previous days. If the system determines that the user's physical condition is abnormal that day, the device will send a warning notification to the user and, if necessary, provide advice to encourage them to seek medical attention.

[0474] The following describes the processing flow.

[0475] Step 1:

[0476] The user launches the health management app on their device. The device activates its built-in sensors to measure physical information such as heart rate and body temperature. During this process, the device temporarily stores the measurement data in its internal memory.

[0477] Step 2:

[0478] The terminal sends measurement data to the server based on certain criteria. The server stores the received data in a database and integrates it with the measurement data history.

[0479] Step 3:

[0480] The server uses an analysis algorithm to evaluate the health status based on the received physical information. For example, it analyzes heart rate variability and calculates abnormal patterns and stress indicators.

[0481] Step 4:

[0482] The server uses the analysis results to detect early signs of health problems. If an abnormality or risk is identified, the information is recorded in the database, allowing for follow-up investigations.

[0483] Step 5:

[0484] The device receives analysis results from the server and displays the information graphically in the user interface. Users can view specific figures, risk alerts, and improvement suggestions within the app.

[0485] Step 6:

[0486] The server generates additional lifestyle improvement suggestions and sends notifications to the device, urging it to visit a medical institution if necessary. The device then notifies the user and follows up on the improvement suggestions.

[0487] (Example 1)

[0488] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0489] In modern society, personal health management is a crucial issue, and there is a particular need for real-time monitoring of health status in daily life and early detection of changes. However, conventional systems have not adequately collected and analyzed physiological information, making it difficult to quickly and effectively assess health risks and suggest lifestyle improvements. This has led to the risk of overlooking potential health problems and resulting in insufficient health management for users.

[0490] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0491] In this invention, the server includes information gathering means for acquiring personal physiological information, analysis means for analyzing the acquired physiological information, and suggestion generation means for generating lifestyle improvement suggestions based on the analysis results. This makes it possible to effectively monitor the user's health status, detect potential health risks early, and quickly provide appropriate lifestyle improvement suggestions.

[0492] "Personal physiological information" refers to data that represents the user's health status and includes physical data such as heart rate, body temperature, and activity level.

[0493] "Information gathering means" refers to devices or systems used to acquire personal physiological information from users, and includes sensors installed in smartphones and wearable devices.

[0494] "Information management means" refers to means that have the function of temporarily storing acquired physiological information and transferring the data to other devices or systems as needed.

[0495] "Analysis means" refers to a means of analyzing collected data based on a specified calculation method to evaluate the user's health status.

[0496] A "predictive detection method" is a means of detecting specific patterns or anomalies from analyzed data and identifying potential health risks.

[0497] "Visualization means" refers to a means of visually displaying the analyzed and detected results to the user, thereby promoting an understanding of their health status.

[0498] A "control means" is an integrated means for managing the operation of the entire system and ensuring coordination between each means.

[0499] A "proposal generation method" is a method equipped with the function of suggesting improvements to the user's daily life based on the analysis results.

[0500] The system of this invention consists of a process for acquiring, analyzing, and visualizing individual physiological information for the purpose of health management, and notifying the user. This system is mainly implemented using terminals and servers.

[0501] The device uses sensors built into smartphones and wearable devices to collect physiological information such as the user's heart rate, body temperature, and activity level in real time. The acquired data is temporarily stored on the device and sent to a server at regular intervals. Wi-Fi and cellular communication networks are mainly used for this data transmission.

[0502] The server stores the received physiological information in a database and evaluates the user's health status using analysis tools. The analysis utilizes AI-powered algorithms, enabling precise analysis of the user's health status based on heart rate variability and other physiological indicators. These analysis results are then used by predictive detection tools to identify abnormalities and predict potential health risks.

[0503] The analysis results are graphically visualized on the device for easy user understanding. A dedicated application is used, allowing users to grasp an overview of their health status and any points requiring attention through an intuitive interface. Furthermore, if an abnormality is detected, the device will notify the user appropriately and, if necessary, offer suggestions for lifestyle improvements.

[0504] As a concrete example, when a user wakes up in the morning and picks up their smartphone, the device automatically measures their heart rate and body temperature. This information is sent to a server and analyzed by an AI model. For instance, if a slight increase in body temperature persists, the device displays an alert such as, "Your body temperature is higher than normal. We recommend you drink plenty of fluids and get some rest."

[0505] The generating AI model can perform further analysis upon receiving the following prompt: "Explain how the health management system uses heart rate and body temperature data to assess the user's health status."

[0506] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0507] Step 1:

[0508] The device uses its built-in sensors to collect the user's physiological information. Specifically, it acquires data such as heart rate, body temperature, and activity level in real time. The input is the user's physiological state, and the output is this data. This provides basic data on the user's daily health status.

[0509] Step 2:

[0510] The terminal temporarily stores the acquired physiological information and sends the data to the server at regular intervals. The input is the physiological information obtained in step 1, and the output is the data sent to the server. Here, communication processing is performed to ensure that the data arrives at the server safely and efficiently.

[0511] Step 3:

[0512] The server stores the received physiological information in a database. The input is physiological information sent from the terminal, and the output is data organized within the database. This step provides a foundation for smooth subsequent analysis.

[0513] Step 4:

[0514] The server analyzes physiological information stored in a database to assess health status. The input is data from the database, and the output is the result of the health status assessment. This analysis utilizes AI models and employs sophisticated pattern recognition on the data.

[0515] Step 5:

[0516] The server detects anomalies and specific patterns based on the analysis results and identifies warning signs. The input is the analysis results obtained in step 4, and the output is risk assessment and necessary alert information. This makes it possible to identify potential health risks in advance.

[0517] Step 6:

[0518] The terminal receives analysis results sent from the server and displays them graphically to the user via an application. The input is the analysis results from the server, and the output is visualized health information for the user. Through this, the user can intuitively understand their own health status.

[0519] Step 7:

[0520] The device notifies the user based on the detection results. Inputs include information on significant risks and suggestions for lifestyle improvements, while outputs are alerts and suggestions sent to the user. For example, if a user's body temperature remains higher than normal, a notification such as "Rest and hydration are recommended" will be sent.

[0521] (Application Example 1)

[0522] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0523] In health management systems using individual physical information, conventional technologies were limited to merely acquiring and evaluating physical information, lacking a mechanism for users to immediately recognize abnormalities and take rapid emergency action. Furthermore, they did not consider predicting risks due to changes in the surrounding environment and providing users with advance warnings. As a result, there was a problem in that rapid responses to abnormal health conditions or dangers in the external environment could not be made, and the safety of users could not be adequately ensured.

[0524] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0525] In this invention, the server includes information acquisition means for acquiring personal physical information, notification operation means for issuing an alarm and notifying emergency contacts when an anomaly is detected, and environmental detection means for monitoring changes in the surrounding environment and predicting dangers. This enables real-time detection and notification of physical abnormalities and environmental risks, allowing for a rapid response.

[0526] "Information acquisition means" refers to devices and sensors that collect physiological data such as heart rate and body temperature from the user's body.

[0527] "Information management means" refers to systems and software for recording and managing acquired physical information.

[0528] "Analysis means" refers to algorithms and software used to evaluate health status using recorded physical information.

[0529] A "predictive detection method" is a method or program for identifying abnormalities or patterns in a person's physical condition based on analyzed health data.

[0530] A "visualization method" is an interface that graphically displays analysis results so that users can intuitively understand them.

[0531] A "notification operation means" is a mechanism that sends an alarm or notification to the user or registered emergency contact when an anomaly is detected.

[0532] "Environmental detection means" refers to a function that monitors changes in the user's surrounding environment, detects predictable risks, and provides early warnings.

[0533] "Control means" refers to hardware or software that integrates and manages each of these means to control the entire system.

[0534] The system for realizing this invention has the function of collecting the user's physical information using sensors installed in smartphones and wearable devices. For example, it acquires data such as heart rate, body temperature, and activity level. The acquired data is temporarily recorded on the terminal and periodically sent to a server. The server stores this data in a database and performs health status analysis.

[0535] The server evaluates the user's health status by applying specific algorithms to the collected physical data. Advanced data processing and analysis are performed using libraries such as Python's NumPy and Pandas. Furthermore, a predictive detection system operates based on the analysis results, detecting specific patterns and anomalies. This allows the system to provide users with information about potential illnesses and health risks.

[0536] Furthermore, the analysis results are sent back to the user's device and visualized in a graphical format for intuitive understanding. The display utilizes the user interface of smart glasses or a mobile app, allowing users to visually check their health status. If an abnormality is detected, a notification is immediately sent to emergency contacts via a notification system, and an alert is also issued to the user. This function utilizes Twilio's API, among others.

[0537] Furthermore, the environmental detection system monitors the user's surroundings and predicts potential hazards such as sudden temperature changes or increases in noise levels. This allows the user to receive advice on how to avoid unnecessary risks.

[0538] For example, if a user's heart rate increases drastically during their commute, the smart glasses will display a visual alert saying, "Your heart rate is high. Please take a short break." Also, if an abnormal temperature fluctuation is predicted, a notification will be sent saying, "The temperature is dropping. Please take precautions against the cold."

[0539] An example of a prompt to a generative AI model is, "Generate health advice for the user based on today's temperature and heart rate data." Using this prompt, the AI ​​model can generate appropriate advice and provide it to the user.

[0540] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0541] Step 1:

[0542] The device uses sensors embedded in smartphones and wearable devices to acquire physiological data such as the user's heart rate, body temperature, and activity level. The input is the sensor signal, and the output is digitized physical data. This allows for the real-time collection of physical information based on the user's daily activities.

[0543] Step 2:

[0544] The terminal temporarily records the acquired physical information in local storage and then sends it to the server. The input is the acquired physical data, and the output is the physical information file sent to the server. This step prepares the data for more advanced analysis.

[0545] Step 3:

[0546] The server stores the received physical information in a database and performs data analysis using Python's NumPy and Pandas libraries. The input is the physical information sent to the server, and the output is the analyzed health status data. This process involves pattern recognition and anomaly detection of the information, and the user's health status is evaluated.

[0547] Step 4:

[0548] The server uses a generative AI model based on the analysis results to predict the user's potential health risks. The input is analyzed health status data, and the output is a risk assessment report. The generative AI model performs an appropriate risk assessment using prompts.

[0549] Step 5:

[0550] The server visualizes the analysis results and risk assessment and transmits this information to the terminal. The input is the risk assessment report, and the output is visualized data that can be displayed on the terminal. The terminal notifies the user through a graphic interface, visually presenting their health status and risks.

[0551] Step 6:

[0552] If an anomaly is detected, the device uses the Twilio API to send a notification to pre-registered emergency contacts. The input is the anomaly detection information, and the output is the notification message sent to the contacts. This procedure allows users and relevant parties to respond to the anomaly quickly.

[0553] Step 7:

[0554] Users can check notifications on their devices and take action directly in response to changes in their health or environment as needed. The input is notification information from the device, and the output is the user's actions based on the acquired health information. This step provides users with useful support for managing their own health and safety.

[0555] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0556] This invention is a system that combines and analyzes physical and emotional information in personal health management to provide personalized health recommendations. This system comprises information acquisition means, information management means, analysis means, predictive detection means, visualization means, control means, and an emotion engine. Specific embodiments are described below.

[0557] Information acquisition means

[0558] The device uses built-in sensors to periodically collect physical information such as the user's heart rate, body temperature, and activity level. In addition, it acquires facial expression data through the device's camera, and an emotion engine analyzes the emotional state from these facial expressions.

[0559] Information management means

[0560] The device temporarily records acquired physical and emotional data and sends it to the server. The server stores this data in a database.

[0561] Analysis means

[0562] The server comprehensively analyzes the received physical and emotional data. The analysis algorithm assesses the user's overall health status by considering not only their heart rate and activity level, but also changes in their emotions.

[0563] Precursor detection means

[0564] The server detects potential health risks and emotional abnormalities based on the analysis results. For example, if a person's heart rate is high when they are under stress, it can predict certain health risks.

[0565] Visualization means

[0566] The device displays simple graphs and summaries through the user interface based on information sent from the server. This allows users to intuitively understand their own health status and emotional tendencies.

[0567] Control means

[0568] The system provides integrated control to ensure that each component works effectively together. This control offers users an effective health management experience.

[0569] Emotional Engine

[0570] The emotion engine analyzes the user's facial expressions and voice data to assess their emotions. This allows for a detailed understanding of how daily emotional changes affect their health.

[0571] As a concrete example, when a user is in a stressful situation, the emotion engine detects the change and sends information to the server. The server analyzes past physical and emotional data and suggests relaxation methods to alleviate mental and physical stress. In this way, incorporating emotional information makes it possible to provide more precise health management and improvement measures.

[0572] The following describes the processing flow.

[0573] Step 1:

[0574] The user launches a health management app on their device. The device activates various sensors to collect physical information such as heart rate, body temperature, and activity level. Simultaneously, the device's camera is used to acquire facial expression data.

[0575] Step 2:

[0576] The device sends collected physical information and facial expression data to the emotion engine. The emotion engine analyzes changes in facial expressions and determines the user's emotional state.

[0577] Step 3:

[0578] The device transmits physical information and emotional state to the server. The server records and manages the received data in a database.

[0579] Step 4:

[0580] The server uses data analysis algorithms to comprehensively analyze physical and emotional data. For example, if a high stress level is detected, it examines heart rate variability to assess whether there are any abnormalities.

[0581] Step 5:

[0582] The server detects potential health risks and emotional abnormalities based on the analysis results. Based on the detection results, it generates health warnings and precautions and adjusts necessary improvement suggestions.

[0583] Step 6:

[0584] The server sends analysis results and improvement suggestions to the terminal. The terminal notifies the user of these and visualizes their health status and emotional tendencies through an interface. The user reviews the suggestions and uses them to improve their life.

[0585] Step 7:

[0586] The terminal receives user feedback on the suggestions and automatically sends it to the server, which is then used to improve future analyses and suggestion generation.

[0587] (Example 2)

[0588] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0589] In modern society, despite the importance of individual health management, physical and emotional information are often treated separately. As a result, it is difficult to grasp the overall picture of one's health, leading to challenges in providing appropriate health management suggestions. Furthermore, there is a lack of information provided in a way that allows users to intuitively understand changes in their health and emotions.

[0590] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0591] In this invention, the server includes data collection means for acquiring an individual's physical and emotional information, information management means for recording and managing the acquired information, and analysis means for comprehensively analyzing the recorded and managed information and evaluating the overall health status. This enables analysis that combines physical and emotional information. Furthermore, it makes it possible to visualize the detection results and provide them in an intuitively understandable format.

[0592] "Data collection means" refers to devices and processes for acquiring an individual's physical and emotional information.

[0593] "Information management means" refers to methods and systems for recording and securely managing acquired physical and emotional information.

[0594] "Analysis means" refers to processes and devices for comprehensively evaluating a user's health status using recorded and managed physical and emotional information.

[0595] "Predictive detection methods" refer to techniques and systems for detecting health risks and emotional abnormalities based on analysis results.

[0596] "Visualization means" refers to methods and technologies for visually representing detection results in an easy-to-understand manner and providing them to the user.

[0597] "Control means" refers to processes or devices that enable each means of a system to work together and function effectively.

[0598] An "emotion analysis engine" is software or a process that analyzes a user's facial expression data and voice data to evaluate their emotional state.

[0599] A "proposal generation method" refers to a process or system that generates suggestions for improving the user's health based on analysis results.

[0600] "Notification means" refers to methods and technologies for communicating generated suggestions and analysis results to the user.

[0601] This invention is designed as a system that enables a detailed assessment of an individual's health status. This system uses sensors and cameras mounted on a device to collect various information, such as the user's heart rate, body temperature, activity level, and facial expression data. This hardware includes vital signs sensors and high-resolution cameras.

[0602] After collecting information, the device temporarily stores this data and sends it to the server. The server securely stores and manages the information using a data management system. Next, the server uses machine learning algorithms and other tools to comprehensively analyze the physical and emotional information. This analysis allows for an assessment of the user's overall health status.

[0603] The analysis results are used by a server-based predictive detection system to detect potential health risks and emotional abnormalities. Health recommendations generated based on this are sent to the terminal and intuitively visualized through a graphical user interface. This makes it easier for users to understand their own health status.

[0604] For example, if a user is in a stressful situation, the server can analyze their increased heart rate and emotional changes and suggest relaxation techniques. This suggestion might appear on the device as a notification advising them to "take deep breaths."

[0605] An example of a prompt to input to a generative AI model is: "Discuss how an emotion engine should be integrated into building a system that gains a detailed understanding of the user's health and provides personalized suggestions."

[0606] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0607] Step 1:

[0608] The device uses built-in sensors and a camera to collect data on the user's heart rate, body temperature, activity level, and facial expressions. Inputs include this biometric information and facial expression data, while output is an integrated dataset of these. Specific operations include calibrating the sensors and camera and periodically acquiring data.

[0609] Step 2:

[0610] The acquired data is temporarily stored on the device and then sent to the server. The input is the dataset obtained in step 1, and the output is the secure data transfer to the server. Specifically, this involves implementing a secure transmission protocol using data encryption technology.

[0611] Step 3:

[0612] The server records and manages received data in a database. Input is data received from terminals, and output is structured data stored in the database. Specific operations include a process of saving the data while maintaining data integrity and adding necessary metadata.

[0613] Step 4:

[0614] The server analyzes stored data based on machine learning algorithms. The input is data extracted from a database, and the output is the analyzed health status assessment result. Specific operations include data preprocessing, algorithmic pattern recognition, and anomaly detection.

[0615] Step 5:

[0616] Based on the analysis results, the server detects potential health risks and emotional abnormalities. The input is the analyzed evaluation results, and the output is the detected risk information. Specific operations include the application of threshold determination methods based on risk assessment criteria.

[0617] Step 6:

[0618] The server generates appropriate health recommendations based on detected health risks. The input is detected risk information, and the output is health recommendations. Specific operations include comparing the current recommendations with past recommendation history in the database to formulate personalized advice.

[0619] Step 7:

[0620] The terminal visualizes and provides health suggestions received from the server to the user. The input is the generated health suggestions, and the output is the visually represented suggestion information. Specific actions include notifications and graph displays through the user interface.

[0621] (Application Example 2)

[0622] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0623] Conventional health management systems typically evaluate health status based solely on physical information, often lacking analysis that considers the user's emotional state. This can lead to a failure to detect potential health risks caused by stress and emotional fluctuations early, potentially delaying appropriate responses. Furthermore, this can result in inadequate safety management, making it difficult to ensure safety in workplaces and other environments.

[0624] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0625] In this invention, the server includes information acquisition means for acquiring an individual's physical information and emotional state, information management means for recording and managing the acquired data, and analysis means for comprehensively analyzing and evaluating the recorded information. This enables safe and effective health and safety management that takes both physical and emotional aspects into consideration.

[0626] "Information acquisition means" refers to devices and technologies used to collect an individual's physical information and emotional state.

[0627] "Information management means" refers to systems and processes for recording and managing acquired physical information and emotional data.

[0628] "Analysis means" refers to devices or methods for integrating and analyzing recorded physical information and emotional data to evaluate the individual physical and emotional state.

[0629] "Predictive detection means" refers to functions and technologies for recognizing physical and emotional abnormalities based on analysis results.

[0630] "Visualization means" refers to systems and technologies for visually displaying the results of data analysis and providing information to users.

[0631] "Control means" refers to a mechanism or technology that ensures compatibility and coordination among the various means within a system.

[0632] "Data collection means" refers to devices and technologies for effectively collecting information on users' resting states and workers' emotional changes.

[0633] "Proposal generation means" refers to a function or technology that generates guidelines for improving work safety based on analysis results.

[0634] "Notification means" refers to communication technologies in a system for transmitting generated guidelines to relevant individuals or administrators.

[0635] The system according to the present invention provides users with a safe and healthy environment by acquiring and analyzing an individual's physical and emotional information in real time. This system includes information acquisition means, information management means, analysis means, predictive detection means, visualization means, control means, and the like.

[0636] The information acquisition method utilizes sensors and cameras installed in wearable devices such as smart glasses to collect data on the user's heart rate, body temperature, and facial expressions. The data is transmitted to a smartphone via Bluetooth or other means.

[0637] In terms of information management, the smartphone temporarily stores the collected physical and emotional data and uploads it to a cloud server. This cloud server maintains the data in a database, enabling persistent data storage.

[0638] The analysis method involves a server continuously analyzing data. Specifically, it uses Microsoft Azure's emotion analysis technology to evaluate changes in heart rate and facial expressions. This allows for a comprehensive assessment of the user's physical condition and emotional changes, and generates necessary countermeasures.

[0639] The server automatically detects stress and anomalies using predictive detection mechanisms. For example, if the heart rate suddenly increases and negative emotions are detected from the user's facial expression, the system will suggest relaxation methods to the user and notify the administrator of an alert.

[0640] The visualization method displays the analysis results as diagrams and charts on the smartphone screen, allowing users to intuitively understand the situation.

[0641] As a concrete example, when a worker in a factory feels fatigued and stressed during work, the system detects these and sends notifications recommending that the worker take a break and that the manager improve the work environment.

[0642] An example of a prompt for a generative AI model is: "Receive heart rate and facial expression data from the user and analyze the current emotional state. If high stress levels or suspicious emotional states are detected, output the details." Through this prompt, sophisticated emotional analysis can be achieved.

[0643] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0644] Step 1:

[0645] The device acquires heart rate, body temperature, and facial expression data in real time from the wearable device's sensors. This data is transmitted to a smartphone via Bluetooth. As input, it receives physical and emotional data from the sensors and temporarily stores it on the smartphone. As output, the data is ready to be sent to a cloud server.

[0646] Step 2:

[0647] The data acquired by the smartphone is transferred to a cloud server. The server stores this data in a database, preparing it for subsequent analysis processes. The input consists of physical and emotional data transmitted from the smartphone, and the output is the data stored in the server's database.

[0648] Step 3:

[0649] The server analyzes the data and evaluates the user's health and emotional state based on heart rate and emotional data. Here, Microsoft Azure's emotion analysis technology is used to send prompts to a generative AI model to analyze the emotional state. The input data consists of physical and emotional data stored on the server, and the output is the analyzed physical and mental state of the user.

[0650] Step 4:

[0651] The server detects potential health risks and emotional abnormalities based on the analysis results. The system monitors changes in heart rate and sudden changes in emotions, and reports on increases in stress levels, etc. The input to this process is analyzed numerical and emotional data, and the output is the detection results of potential risks and emotional abnormalities.

[0652] Step 5:

[0653] The server generates data for visually displaying the analysis and detection results using visualization tools and sends it to the terminal. This allows users to view an overview of their health status and emotional tendencies in graphs and charts on their smartphones. The input is risk information and health status assessments that have been detected as precursors, and the output is displayed data that visualizes these.

[0654] Step 6:

[0655] The user takes necessary actions based on instructions from the system. For example, upon receiving a notification from the system, the user might try relaxation techniques such as taking a break or deep breathing. The input for this step is the visualized data displayed to the user and alerts from the system, while the output is the specific action taken.

[0656] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0657] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0658] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0659] [Fourth Embodiment]

[0660] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0661] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0662] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0663] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0664] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0665] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0666] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0667] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors in the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0668] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0669] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0670] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0671] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0672] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0673] The present invention provides a system for health management purposes that acquires an individual's physical information and performs analysis, visualization, and notification based on that information. This system includes, as an embodiment, information acquisition means, information management means, analysis means, predictive detection means, visualization means, and control means. Specific embodiments are described below.

[0674] Information acquisition means

[0675] Sensors built into the device are used to collect the user's physical information. For example, smartphones and wearable devices acquire data such as heart rate, body temperature, and activity level.

[0676] Information management means

[0677] The device temporarily records the acquired physical information and periodically sends it to the server. The server stores this information in a database and prepares it for subsequent analysis.

[0678] Analysis means

[0679] The server analyzes stored physical information and evaluates the user's health status using a specified algorithm. For example, it can assess stress levels and health risks based on blood pressure data.

[0680] Precursor detection means

[0681] Based on the analysis results, the server detects specific patterns and anomalies, identifying potential signs of illness. For example, it can analyze long-term data to predict the risk of hypertension.

[0682] Visualization means

[0683] The device receives analysis results from the server and displays them graphically. Through the app, users can intuitively understand an overview of their health status and any points to be aware of.

[0684] Control means

[0685] The system is equipped with control mechanisms to manage the operation of the entire system, which integrates and operates multiple mechanisms to ensure consistent service delivery.

[0686] As a concrete example, when a user touches the device in the morning, the system automatically measures their heart rate and body temperature. This data is immediately transmitted to a server and analyzed together with the history from the previous days. If the system determines that the user's physical condition is abnormal that day, the device will send a warning notification to the user and, if necessary, provide advice to encourage them to seek medical attention.

[0687] The following describes the processing flow.

[0688] Step 1:

[0689] The user launches the health management app on their device. The device activates its built-in sensors to measure physical information such as heart rate and body temperature. During this process, the device temporarily stores the measurement data in its internal memory.

[0690] Step 2:

[0691] The terminal sends measurement data to the server based on certain criteria. The server stores the received data in a database and integrates it with the measurement data history.

[0692] Step 3:

[0693] The server uses an analysis algorithm to evaluate the health status based on the received physical information. For example, it analyzes heart rate variability and calculates abnormal patterns and stress indicators.

[0694] Step 4:

[0695] The server uses the analysis results to detect early signs of health problems. If an abnormality or risk is identified, the information is recorded in the database, allowing for follow-up investigations.

[0696] Step 5:

[0697] The device receives analysis results from the server and displays the information graphically in the user interface. Users can view specific figures, risk alerts, and improvement suggestions within the app.

[0698] Step 6:

[0699] The server generates additional lifestyle improvement suggestions and sends notifications to the device, urging it to visit a medical institution if necessary. The device then notifies the user and follows up on the improvement suggestions.

[0700] (Example 1)

[0701] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0702] In modern society, personal health management is a crucial issue, and there is a particular need for real-time monitoring of health status in daily life and early detection of changes. However, conventional systems have not adequately collected and analyzed physiological information, making it difficult to quickly and effectively assess health risks and suggest lifestyle improvements. This has led to the risk of overlooking potential health problems and resulting in insufficient health management for users.

[0703] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0704] In this invention, the server includes information gathering means for acquiring personal physiological information, analysis means for analyzing the acquired physiological information, and suggestion generation means for generating lifestyle improvement suggestions based on the analysis results. This makes it possible to effectively monitor the user's health status, detect potential health risks early, and quickly provide appropriate lifestyle improvement suggestions.

[0705] "Personal physiological information" refers to data that represents the user's health status and includes physical data such as heart rate, body temperature, and activity level.

[0706] "Information gathering means" refers to devices or systems used to acquire personal physiological information from users, and includes sensors installed in smartphones and wearable devices.

[0707] "Information management means" refers to means that have the function of temporarily storing acquired physiological information and transferring the data to other devices or systems as needed.

[0708] "Analysis means" refers to a means of analyzing collected data based on a specified calculation method to evaluate the user's health status.

[0709] A "predictive detection method" is a means of detecting specific patterns or anomalies from analyzed data and identifying potential health risks.

[0710] "Visualization means" refers to a means of visually displaying the analyzed and detected results to the user, thereby promoting an understanding of their health status.

[0711] A "control means" is an integrated means for managing the operation of the entire system and ensuring coordination between each means.

[0712] A "proposal generation method" is a method equipped with the function of suggesting improvements to the user's daily life based on the analysis results.

[0713] The system of this invention consists of a process for acquiring, analyzing, and visualizing individual physiological information for the purpose of health management, and notifying the user. This system is mainly implemented using terminals and servers.

[0714] The device uses sensors built into smartphones and wearable devices to collect physiological information such as the user's heart rate, body temperature, and activity level in real time. The acquired data is temporarily stored on the device and sent to a server at regular intervals. Wi-Fi and cellular communication networks are mainly used for this data transmission.

[0715] The server stores the received physiological information in a database and evaluates the user's health status using analysis tools. The analysis utilizes AI-powered algorithms, enabling precise analysis of the user's health status based on heart rate variability and other physiological indicators. These analysis results are then used by predictive detection tools to identify abnormalities and predict potential health risks.

[0716] The analysis results are graphically visualized on the device for easy user understanding. A dedicated application is used, allowing users to grasp an overview of their health status and any points requiring attention through an intuitive interface. Furthermore, if an abnormality is detected, the device will notify the user appropriately and, if necessary, offer suggestions for lifestyle improvements.

[0717] As a concrete example, when a user wakes up in the morning and picks up their smartphone, the device automatically measures their heart rate and body temperature. This information is sent to a server and analyzed by an AI model. For instance, if a slight increase in body temperature persists, the device displays an alert such as, "Your body temperature is higher than normal. We recommend you drink plenty of fluids and get some rest."

[0718] The generating AI model can perform further analysis upon receiving the following prompt: "Explain how the health management system uses heart rate and body temperature data to assess the user's health status."

[0719] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0720] Step 1:

[0721] The device uses its built-in sensors to collect the user's physiological information. Specifically, it acquires data such as heart rate, body temperature, and activity level in real time. The input is the user's physiological state, and the output is this data. This provides basic data on the user's daily health status.

[0722] Step 2:

[0723] The terminal temporarily stores the acquired physiological information and sends the data to the server at regular intervals. The input is the physiological information obtained in step 1, and the output is the data sent to the server. Here, communication processing is performed to ensure that the data arrives at the server safely and efficiently.

[0724] Step 3:

[0725] The server stores the received physiological information in a database. The input is physiological information sent from the terminal, and the output is data organized within the database. This step provides a foundation for smooth subsequent analysis.

[0726] Step 4:

[0727] The server analyzes physiological information stored in a database to assess health status. The input is data from the database, and the output is the result of the health status assessment. This analysis utilizes AI models and employs sophisticated pattern recognition on the data.

[0728] Step 5:

[0729] The server detects anomalies and specific patterns based on the analysis results and identifies warning signs. The input is the analysis results obtained in step 4, and the output is risk assessment and necessary alert information. This makes it possible to identify potential health risks in advance.

[0730] Step 6:

[0731] The terminal receives analysis results sent from the server and displays them graphically to the user via an application. The input is the analysis results from the server, and the output is visualized health information for the user. Through this, the user can intuitively understand their own health status.

[0732] Step 7:

[0733] The device notifies the user based on the detection results. Inputs include information on significant risks and suggestions for lifestyle improvements, while outputs are alerts and suggestions sent to the user. For example, if a user's body temperature remains higher than normal, a notification such as "Rest and hydration are recommended" will be sent.

[0734] (Application Example 1)

[0735] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0736] In health management systems using individual physical information, conventional technologies were limited to merely acquiring and evaluating physical information, lacking a mechanism for users to immediately recognize abnormalities and take rapid emergency action. Furthermore, they did not consider predicting risks due to changes in the surrounding environment and providing users with advance warnings. As a result, there was a problem in that rapid responses to abnormal health conditions or dangers in the external environment could not be made, and the safety of users could not be adequately ensured.

[0737] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0738] In this invention, the server includes information acquisition means for acquiring personal physical information, notification operation means for issuing an alarm and notifying emergency contacts when an anomaly is detected, and environmental detection means for monitoring changes in the surrounding environment and predicting dangers. This enables real-time detection and notification of physical abnormalities and environmental risks, allowing for a rapid response.

[0739] "Information acquisition means" refers to devices and sensors that collect physiological data such as heart rate and body temperature from the user's body.

[0740] "Information management means" refers to systems and software for recording and managing acquired physical information.

[0741] "Analysis means" refers to algorithms and software used to evaluate health status using recorded physical information.

[0742] A "predictive detection method" is a method or program for identifying abnormalities or patterns in a person's physical condition based on analyzed health data.

[0743] A "visualization method" is an interface that graphically displays analysis results so that users can intuitively understand them.

[0744] A "notification operation means" is a mechanism that sends an alarm or notification to the user or registered emergency contact when an anomaly is detected.

[0745] "Environmental detection means" refers to a function that monitors changes in the user's surrounding environment, detects predictable risks, and provides early warnings.

[0746] "Control means" refers to hardware or software that integrates and manages each of these means to control the entire system.

[0747] The system for realizing this invention has the function of collecting the user's physical information using sensors installed in smartphones and wearable devices. For example, it acquires data such as heart rate, body temperature, and activity level. The acquired data is temporarily recorded on the terminal and periodically sent to a server. The server stores this data in a database and performs health status analysis.

[0748] The server evaluates the user's health status by applying specific algorithms to the collected physical data. Advanced data processing and analysis are performed using libraries such as Python's NumPy and Pandas. Furthermore, a predictive detection system operates based on the analysis results, detecting specific patterns and anomalies. This allows the system to provide users with information about potential illnesses and health risks.

[0749] Furthermore, the analysis results are sent back to the user's device and visualized in a graphical format for intuitive understanding. The display utilizes the user interface of smart glasses or a mobile app, allowing users to visually check their health status. If an abnormality is detected, a notification is immediately sent to emergency contacts via a notification system, and an alert is also issued to the user. This function utilizes Twilio's API, among others.

[0750] Furthermore, the environmental detection system monitors the user's surroundings and predicts potential hazards such as sudden temperature changes or increases in noise levels. This allows the user to receive advice on how to avoid unnecessary risks.

[0751] For example, if a user's heart rate increases drastically during their commute, the smart glasses will display a visual alert saying, "Your heart rate is high. Please take a short break." Also, if an abnormal temperature fluctuation is predicted, a notification will be sent saying, "The temperature is dropping. Please take precautions against the cold."

[0752] An example of a prompt to a generative AI model is, "Generate health advice for the user based on today's temperature and heart rate data." Using this prompt, the AI ​​model can generate appropriate advice and provide it to the user.

[0753] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0754] Step 1:

[0755] The device uses sensors embedded in smartphones and wearable devices to acquire physiological data such as the user's heart rate, body temperature, and activity level. The input is the sensor signal, and the output is digitized physical data. This allows for the real-time collection of physical information based on the user's daily activities.

[0756] Step 2:

[0757] The terminal temporarily records the acquired physical information in local storage and then sends it to the server. The input is the acquired physical data, and the output is the physical information file sent to the server. This step prepares the data for more advanced analysis.

[0758] Step 3:

[0759] The server stores the received physical information in a database and performs data analysis using Python's NumPy and Pandas libraries. The input is the physical information sent to the server, and the output is the analyzed health status data. This process involves pattern recognition and anomaly detection of the information, and the user's health status is evaluated.

[0760] Step 4:

[0761] The server uses a generative AI model based on the analysis results to predict the user's potential health risks. The input is analyzed health status data, and the output is a risk assessment report. The generative AI model performs an appropriate risk assessment using prompts.

[0762] Step 5:

[0763] The server visualizes the analysis results and risk assessment and transmits this information to the terminal. The input is the risk assessment report, and the output is visualized data that can be displayed on the terminal. The terminal notifies the user through a graphic interface, visually presenting their health status and risks.

[0764] Step 6:

[0765] If an anomaly is detected, the device uses the Twilio API to send a notification to pre-registered emergency contacts. The input is the anomaly detection information, and the output is the notification message sent to the contacts. This procedure allows users and relevant parties to respond to the anomaly quickly.

[0766] Step 7:

[0767] Users can check notifications on their devices and take action directly in response to changes in their health or environment as needed. The input is notification information from the device, and the output is the user's actions based on the acquired health information. This step provides users with useful support for managing their own health and safety.

[0768] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0769] This invention is a system that combines and analyzes physical and emotional information in personal health management to provide personalized health recommendations. This system comprises information acquisition means, information management means, analysis means, predictive detection means, visualization means, control means, and an emotion engine. Specific embodiments are described below.

[0770] Information acquisition means

[0771] The device uses built-in sensors to periodically collect physical information such as the user's heart rate, body temperature, and activity level. In addition, it acquires facial expression data through the device's camera, and an emotion engine analyzes the emotional state from these facial expressions.

[0772] Information management means

[0773] The device temporarily records acquired physical and emotional data and sends it to the server. The server stores this data in a database.

[0774] Analysis means

[0775] The server comprehensively analyzes the received physical and emotional data. The analysis algorithm assesses the user's overall health status by considering not only their heart rate and activity level, but also changes in their emotions.

[0776] Precursor detection means

[0777] The server detects potential health risks and emotional abnormalities based on the analysis results. For example, if a person's heart rate is high when they are under stress, it can predict certain health risks.

[0778] Visualization means

[0779] The device displays simple graphs and summaries through the user interface based on information sent from the server. This allows users to intuitively understand their own health status and emotional tendencies.

[0780] Control means

[0781] The system provides integrated control to ensure that each component works effectively together. This control offers users an effective health management experience.

[0782] Emotional Engine

[0783] The emotion engine analyzes the user's facial expressions and voice data to assess their emotions. This allows for a detailed understanding of how daily emotional changes affect their health.

[0784] As a concrete example, when a user is in a stressful situation, the emotion engine detects the change and sends information to the server. The server analyzes past physical and emotional data and suggests relaxation methods to alleviate mental and physical stress. In this way, incorporating emotional information makes it possible to provide more precise health management and improvement measures.

[0785] The following describes the processing flow.

[0786] Step 1:

[0787] The user launches a health management app on their device. The device activates various sensors to collect physical information such as heart rate, body temperature, and activity level. Simultaneously, the device's camera is used to acquire facial expression data.

[0788] Step 2:

[0789] The device sends collected physical information and facial expression data to the emotion engine. The emotion engine analyzes changes in facial expressions and determines the user's emotional state.

[0790] Step 3:

[0791] The device transmits physical information and emotional state to the server. The server records and manages the received data in a database.

[0792] Step 4:

[0793] The server uses data analysis algorithms to comprehensively analyze physical and emotional data. For example, if a high stress level is detected, it examines heart rate variability to assess whether there are any abnormalities.

[0794] Step 5:

[0795] The server detects potential health risks and emotional abnormalities based on the analysis results. Based on the detection results, it generates health warnings and precautions and adjusts necessary improvement suggestions.

[0796] Step 6:

[0797] The server sends analysis results and improvement suggestions to the terminal. The terminal notifies the user of these and visualizes their health status and emotional tendencies through an interface. The user reviews the suggestions and uses them to improve their life.

[0798] Step 7:

[0799] The terminal receives user feedback on the suggestions and automatically sends it to the server, which is then used to improve future analyses and suggestion generation.

[0800] (Example 2)

[0801] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0802] In modern society, despite the importance of individual health management, physical and emotional information are often treated separately. As a result, it is difficult to grasp the overall picture of one's health, leading to challenges in providing appropriate health management suggestions. Furthermore, there is a lack of information provided in a way that allows users to intuitively understand changes in their health and emotions.

[0803] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0804] In this invention, the server includes data collection means for acquiring an individual's physical and emotional information, information management means for recording and managing the acquired information, and analysis means for comprehensively analyzing the recorded and managed information and evaluating the overall health status. This enables analysis that combines physical and emotional information. Furthermore, it makes it possible to visualize the detection results and provide them in an intuitively understandable format.

[0805] "Data collection means" refers to devices and processes for acquiring an individual's physical and emotional information.

[0806] "Information management means" refers to methods and systems for recording and securely managing acquired physical and emotional information.

[0807] "Analysis means" refers to processes and devices for comprehensively evaluating a user's health status using recorded and managed physical and emotional information.

[0808] "Predictive detection methods" refer to techniques and systems for detecting health risks and emotional abnormalities based on analysis results.

[0809] "Visualization means" refers to methods and technologies for visually representing detection results in an easy-to-understand manner and providing them to the user.

[0810] "Control means" refers to processes or devices that enable each means of a system to work together and function effectively.

[0811] An "emotion analysis engine" is software or a process that analyzes a user's facial expression data and voice data to evaluate their emotional state.

[0812] A "proposal generation method" refers to a process or system that generates suggestions for improving the user's health based on analysis results.

[0813] "Notification means" refers to methods and technologies for communicating generated suggestions and analysis results to the user.

[0814] This invention is designed as a system that enables a detailed assessment of an individual's health status. This system uses sensors and cameras mounted on a device to collect various information, such as the user's heart rate, body temperature, activity level, and facial expression data. This hardware includes vital signs sensors and high-resolution cameras.

[0815] After collecting information, the device temporarily stores this data and sends it to the server. The server securely stores and manages the information using a data management system. Next, the server uses machine learning algorithms and other tools to comprehensively analyze the physical and emotional information. This analysis allows for an assessment of the user's overall health status.

[0816] The analysis results are used by a server-based predictive detection system to detect potential health risks and emotional abnormalities. Health recommendations generated based on this are sent to the terminal and intuitively visualized through a graphical user interface. This makes it easier for users to understand their own health status.

[0817] For example, if a user is in a stressful situation, the server can analyze their increased heart rate and emotional changes and suggest relaxation techniques. This suggestion might appear on the device as a notification advising them to "take deep breaths."

[0818] An example of a prompt to input to a generative AI model is: "Discuss how an emotion engine should be integrated into building a system that gains a detailed understanding of the user's health and provides personalized suggestions."

[0819] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0820] Step 1:

[0821] The device uses built-in sensors and a camera to collect data on the user's heart rate, body temperature, activity level, and facial expressions. Inputs include this biometric information and facial expression data, while output is an integrated dataset of these. Specific operations include calibrating the sensors and camera and periodically acquiring data.

[0822] Step 2:

[0823] The acquired data is temporarily stored on the device and then sent to the server. The input is the dataset obtained in step 1, and the output is the secure data transfer to the server. Specifically, this involves implementing a secure transmission protocol using data encryption technology.

[0824] Step 3:

[0825] The server records and manages received data in a database. Input is data received from terminals, and output is structured data stored in the database. Specific operations include a process of saving the data while maintaining data integrity and adding necessary metadata.

[0826] Step 4:

[0827] The server analyzes stored data based on machine learning algorithms. The input is data extracted from a database, and the output is the analyzed health status assessment result. Specific operations include data preprocessing, algorithmic pattern recognition, and anomaly detection.

[0828] Step 5:

[0829] Based on the analysis results, the server detects potential health risks and emotional abnormalities. The input is the analyzed evaluation results, and the output is the detected risk information. Specific operations include the application of threshold determination methods based on risk assessment criteria.

[0830] Step 6:

[0831] The server generates appropriate health recommendations based on detected health risks. The input is detected risk information, and the output is health recommendations. Specific operations include comparing the current recommendations with past recommendation history in the database to formulate personalized advice.

[0832] Step 7:

[0833] The terminal visualizes and provides health suggestions received from the server to the user. The input is the generated health suggestions, and the output is the visually represented suggestion information. Specific actions include notifications and graph displays through the user interface.

[0834] (Application Example 2)

[0835] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0836] Conventional health management systems typically evaluate health status based solely on physical information, often lacking analysis that considers the user's emotional state. This can lead to a failure to detect potential health risks caused by stress and emotional fluctuations early, potentially delaying appropriate responses. Furthermore, this can result in inadequate safety management, making it difficult to ensure safety in workplaces and other environments.

[0837] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0838] In this invention, the server includes information acquisition means for acquiring an individual's physical information and emotional state, information management means for recording and managing the acquired data, and analysis means for comprehensively analyzing and evaluating the recorded information. This enables safe and effective health and safety management that takes both physical and emotional aspects into consideration.

[0839] "Information acquisition means" refers to devices and technologies used to collect an individual's physical information and emotional state.

[0840] "Information management means" refers to systems and processes for recording and managing acquired physical information and emotional data.

[0841] "Analysis means" refers to devices or methods for integrating and analyzing recorded physical information and emotional data to evaluate the individual physical and emotional state.

[0842] "Predictive detection means" refers to functions and technologies for recognizing physical and emotional abnormalities based on analysis results.

[0843] "Visualization means" refers to systems and technologies for visually displaying the results of data analysis and providing information to users.

[0844] "Control means" refers to a mechanism or technology that ensures compatibility and coordination among the various means within a system.

[0845] "Data collection means" refers to devices and technologies for effectively collecting information on users' resting states and workers' emotional changes.

[0846] "Proposal generation means" refers to a function or technology that generates guidelines for improving work safety based on analysis results.

[0847] "Notification means" refers to communication technologies in a system for transmitting generated guidelines to relevant individuals or administrators.

[0848] The system according to the present invention provides users with a safe and healthy environment by acquiring and analyzing an individual's physical and emotional information in real time. This system includes information acquisition means, information management means, analysis means, predictive detection means, visualization means, control means, and the like.

[0849] The information acquisition method utilizes sensors and cameras installed in wearable devices such as smart glasses to collect data on the user's heart rate, body temperature, and facial expressions. The data is transmitted to a smartphone via Bluetooth or other means.

[0850] In terms of information management, the smartphone temporarily stores the collected physical and emotional data and uploads it to a cloud server. This cloud server maintains the data in a database, enabling persistent data storage.

[0851] The analysis method involves a server continuously analyzing data. Specifically, it uses Microsoft Azure's emotion analysis technology to evaluate changes in heart rate and facial expressions. This allows for a comprehensive assessment of the user's physical condition and emotional changes, and generates necessary countermeasures.

[0852] The server automatically detects stress and anomalies using predictive detection mechanisms. For example, if the heart rate suddenly increases and negative emotions are detected from the user's facial expression, the system will suggest relaxation methods to the user and notify the administrator of an alert.

[0853] The visualization method displays the analysis results as diagrams and charts on the smartphone screen, allowing users to intuitively understand the situation.

[0854] As a concrete example, when a worker in a factory feels fatigued and stressed during work, the system detects these and sends notifications recommending that the worker take a break and that the manager improve the work environment.

[0855] An example of a prompt for a generative AI model is: "Receive heart rate and facial expression data from the user and analyze the current emotional state. If high stress levels or suspicious emotional states are detected, output the details." Through this prompt, sophisticated emotional analysis can be achieved.

[0856] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0857] Step 1:

[0858] The device acquires heart rate, body temperature, and facial expression data in real time from the wearable device's sensors. This data is transmitted to a smartphone via Bluetooth. As input, it receives physical and emotional data from the sensors and temporarily stores it on the smartphone. As output, the data is ready to be sent to a cloud server.

[0859] Step 2:

[0860] The data acquired by the smartphone is transferred to a cloud server. The server stores this data in a database, preparing it for subsequent analysis processes. The input consists of physical and emotional data transmitted from the smartphone, and the output is the data stored in the server's database.

[0861] Step 3:

[0862] The server analyzes the data and evaluates the user's health and emotional state based on heart rate and emotional data. Here, Microsoft Azure's emotion analysis technology is used to send prompts to a generative AI model to analyze the emotional state. The input data consists of physical and emotional data stored on the server, and the output is the analyzed physical and mental state of the user.

[0863] Step 4:

[0864] The server detects potential health risks and emotional abnormalities based on the analysis results. The system monitors changes in heart rate and sudden changes in emotions, and reports on increases in stress levels, etc. The input to this process is analyzed numerical and emotional data, and the output is the detection results of potential risks and emotional abnormalities.

[0865] Step 5:

[0866] The server generates data for visually displaying the analysis and detection results using visualization tools and sends it to the terminal. This allows users to view an overview of their health status and emotional tendencies in graphs and charts on their smartphones. The input is risk information and health status assessments that have been detected as precursors, and the output is displayed data that visualizes these.

[0867] Step 6:

[0868] The user takes necessary actions based on instructions from the system. For example, upon receiving a notification from the system, the user might try relaxation techniques such as taking a break or deep breathing. The input for this step is the visualized data displayed to the user and alerts from the system, while the output is the specific action taken.

[0869] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0870] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0871] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0872] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0873] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0874] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0875] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0876] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0877] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0878] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0879] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0880] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0881] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0882] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0883] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0884] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0885] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0886] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0887] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0888] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0889] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0890] The following is further disclosed regarding the embodiments described above.

[0891] (Claim 1)

[0892] Information acquisition methods for obtaining personal physical information,

[0893] Information management means for recording and managing acquired personal physical information,

[0894] An analytical means for analyzing recorded and managed information and evaluating physical condition,

[0895] Based on the analysis results, a predictive means for detecting signs of a physical condition,

[0896] A visualization method that visualizes detection results and provides them to users,

[0897] A system including control means for controlling the above means.

[0898] (Claim 2)

[0899] Furthermore, it includes means for collecting data to obtain the user's resting state.

[0900] The rest data obtained by the data collection means,

[0901] The system according to claim 1, comprising the analysis means for combining and analyzing an individual's physical information.

[0902] (Claim 3)

[0903] The system further includes a proposal generation mechanism that generates suggestions for improving lifestyle based on the analysis results.

[0904] The system according to claim 1, comprising a notification means for notifying the user of the generated proposal.

[0905] "Example 1"

[0906] (Claim 1)

[0907] Information gathering methods for obtaining personal physiological information,

[0908] Information management means for temporarily storing acquired physiological information and transmitting it to a remote device via a communication line,

[0909] An analysis means for analyzing physiological information stored in a remote device and evaluating health status using a specified calculation method,

[0910] A predictive detection method that detects specific patterns or anomalies based on analysis results and identifies potential health risks,

[0911] A visualization method that visually displays detection results and promotes an intuitive understanding of health status,

[0912] A system that includes control means for comprehensively controlling the overall operation and ensuring coordination between multiple means.

[0913] (Claim 2)

[0914] The system according to claim 1, comprising a function to acquire data on the user's resting state and analyze it in combination with physiological information.

[0915] (Claim 3)

[0916] The system according to claim 1, comprising a function to create suggestions for lifestyle improvements based on analysis results and notify the user.

[0917] "Application Example 1"

[0918] (Claim 1)

[0919] Information acquisition methods for obtaining personal physical information,

[0920] Information management means for recording and managing acquired personal physical information,

[0921] An analytical means for analyzing recorded and managed information and evaluating physical condition,

[0922] Based on the analysis results, a predictive means for detecting signs of a physical condition,

[0923] A visualization method that visualizes detection results and provides them to users,

[0924] A notification system that issues an alarm and notifies emergency contacts when an anomaly is detected,

[0925] A system including control means for controlling the above means.

[0926] (Claim 2)

[0927] Furthermore, it includes means for collecting data to obtain the user's resting state.

[0928] The rest data obtained by the data collection means,

[0929] The system according to claim 1, comprising the analysis means for combining and analyzing individual physical information.

[0930] (Claim 3)

[0931] A proposal generation means that generates suggestions for improving lifestyle based on the analysis results,

[0932] Furthermore, it is equipped with environmental detection means that monitor changes in the surrounding environment and predict hazards.

[0933] The system according to claim 1, further comprising a notification means for notifying the user of the generated proposal.

[0934] "Example 2 of combining an emotion engine"

[0935] (Claim 1)

[0936] A data collection method for acquiring personal physical and emotional information,

[0937] Information management means for recording and managing acquired information,

[0938] An analytical means for comprehensively analyzing recorded and managed information to evaluate overall health status,

[0939] Based on the analysis results, a predictive detection method is used to detect health risks and emotional abnormalities.

[0940] A visualization method that visualizes detection results and provides them in an intuitively understandable format,

[0941] A system including control means for effectively coordinating the above means.

[0942] (Claim 2)

[0943] It also features an emotion analysis engine that analyzes user facial expression and voice data to evaluate their emotional state.

[0944] The system according to claim 1, wherein the analysis means combines and analyzes physical information and emotional information.

[0945] (Claim 3)

[0946] Based on your health condition, we will generate suggestions to improve the balance between mind and body.

[0947] The system according to claim 1, comprising a proposal generation means and a notification means for notifying a user of the generated proposals.

[0948] "Application example 2 when combining with an emotional engine"

[0949] (Claim 1)

[0950] Information acquisition means for acquiring an individual's physical information and emotional state,

[0951] Information management means for recording and managing acquired personal physical information and emotional data,

[0952] An analytical means for comprehensively analyzing recorded information and evaluating the individual physical and emotional state,

[0953] Based on the analysis results, a predictive means for recognizing physical and emotional abnormalities,

[0954] A visualization method that visually displays the results of data analysis and provides them to users,

[0955] A system including control means to ensure compatibility and coordination among the various means.

[0956] (Claim 2)

[0957] The system further includes data collection means for collecting information on the resting state of users and changes in workers' emotions.

[0958] The system according to claim 1, which integrates and analyzes data obtained by the said means with personal physical information and emotional data.

[0959] (Claim 3)

[0960] The system further includes a proposal generation means that generates guidelines for improving work safety based on the analysis results.

[0961] The system according to claim 1, comprising a notification means for notifying a company administrator of the generated guidelines. [Explanation of Symbols]

[0962] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Information acquisition methods for obtaining personal physical information, Information management means for recording and managing acquired personal physical information, An analytical means for analyzing recorded and managed information and evaluating physical condition, Based on the analysis results, a predictive means for detecting signs of a physical condition, A visualization method that visualizes detection results and provides them to users, A system including control means for controlling the above means.

2. Furthermore, it includes means for collecting data to obtain the user's resting state. The rest data obtained by the data collection means, The system according to claim 1, comprising the analysis means for combining and analyzing an individual's physical information.

3. The system further includes a proposal generation mechanism that generates suggestions for improving lifestyle based on the analysis results. The system according to claim 1, comprising a notification means for notifying the user of the generated proposal.

Images