System and method for providing health advice

Abstract

Provided is a system for providing health advice for a user in order to effectively utilize biological data obtained in daily life of the user for maintaining and promoting the health of the user and in order to provide user-friendly health advice that is further customized for the user. The system includes: a sensor that is installed in a facility and continuously collects biological data of the user; an analyzing device that is configured to perform analysis of the biological data collected by the sensor and to determine the health status of the user; a program generation device that is configured to generate a health promotion program tailored to the user by using the determination result of the health status; and an advice generation device that is configured to input the determined health status, the generated health promotion program, and additional information associated with the user into a large-scale language model to obtain health advice for the user.

Description

Systems and methods for providing health advice

[0001] The present invention relates to a system and method for providing health advice according to the health status of users of a facility.

[0002] Smart houses have been gaining attention in recent years. A smart house generally refers to a home that uses information technology (IT) to control electrical and gas-using appliances such as lighting fixtures, cooking appliances, and heating and cooling equipment, thereby optimizing energy consumption.

[0003] In Patent Document 1 (Japanese Patent No. 7285886), the applicant proposed a health promotion program provision system and a health promotion program provision method corresponding to the system, which include a sensor installed in a home and continuously collecting biometric data of the resident of the home, an analysis device that analyzes the collected biometric data and determines the health condition of the resident, and a program generation device that generates a health promotion program tailored to the resident based on the health condition determination results.The system and method have the advantage of being able to effectively utilize biometric data obtained in the user's daily life to maintain and improve the user's health.

[0004] Patent No. 7285886

[0005] The applicant has further researched and discovered a method for providing more customized and user-friendly health advice to users.

[0006] In other words, the present invention aims to provide a system and method that can effectively utilize biometric data obtained in a user's daily life to maintain and improve the user's health, and that can provide the user with more customized and user-friendly health advice.

[0007] In order to solve the above-mentioned problems, one aspect of the present invention provides a system for providing health advice for a user, comprising: a structure installed within a facility and including a sensor that continuously collects biometric data of the user; an analysis device configured to analyze the biometric data collected by the sensor and determine the health condition of the user; a program generation device configured to generate a health promotion program tailored to the user using the health condition determination results; and an advice generation device configured to input the determined health condition, the generated health promotion program, and additional information about the user into a large-scale language model to obtain health advice for the user.

[0008] In the system of the present invention, it is preferable that the additional information includes at least one of the following types of information: basic information including at least one of the user's age, sex, and nationality; biometric information including at least one of the user's height, weight, body composition, basal metabolism, and blood pressure; external information including at least one of the user's financial and economic situation, logistics, seasons, trends, temperature, and humidity; and unique information including at least one of the user's hobbies and preferences, preferences for things and foods, allergies, physical characteristics, personal or household economic situation, and mood of the day.

[0009] In the system of the present invention, it is preferable that the additional information includes the user's reaction to the health advice.

[0010] Furthermore, in the system of the present invention, it is preferable that the biometric data include at least one type of data among data indicating facial expression, heart rate, oxygen saturation, carbon dioxide exhalation, surface body temperature, core body temperature, skin protein analysis, body composition, autonomic nerves, HbA1c, body water content, foot pressure distribution, walking posture, walking speed, changes in joint range of motion, fluctuations in the center of gravity, activity level, electromyography, electrocardiography, electroencephalography, standing and sitting postures, body temperature, blood pressure, blood flow, heart rate, breathing, sweating, eyeballs, sleep time, amount and time of excretion, blood components, urine components, saliva components, images of the oral cavity, and fecal components.

[0011] Furthermore, in the system of the present invention, it is preferable that the analysis device is configured to use a person's biometric data including at least one type of data from data indicating a person's facial expression, heart rate, oxygen saturation, carbon dioxide exhalation, surface body temperature, core body temperature, skin protein analysis, body composition, autonomic nerves, HbA1c, body water content, foot pressure distribution, walking posture, walking speed, changes in joint range of motion, fluctuations in the center of gravity, activity level, electromyography, electrocardiography, electroencephalography, standing and sitting posture, body temperature, blood pressure, blood flow, heart rate, breathing, sweating, eyeballs, sleep time, amount and time of excretion, blood components, urine components, saliva components, images of the oral cavity, and fecal components, as well as the person's health condition including the person's energy expenditure and exercise effect, as training data, and to output the user's health condition from the collected biometric data using a prediction model obtained by machine learning a learning model that takes the person's biometric data as input and outputs the person's health condition.

[0012] Furthermore, in the system of the present invention, it is preferable that the program generation unit is configured to: extract the biometric characteristics of the user by comparing the collected biometric data with predetermined evaluation indices; use the predetermined evaluation indices and evaluation values ​​indicating the effectiveness of the health promotion program as training data; generate a health promotion program for the user from the collected biometric data using an evaluation model obtained by machine learning a learning model in which the input is the biometric data and the output is an evaluation of the health promotion program; and output the extracted biometric characteristics and the generated health promotion program.

[0013] Another aspect of the present invention also provides a method for providing health advice for a user, the method comprising: a computer analyzing biometric data of the user collected by sensors installed in structures within a facility to determine the user's health condition; using the health condition determination result to generate a health promotion program tailored to the user; and inputting the determined health condition, the generated health promotion program, and additional information about the user into a large-scale language model to obtain health advice for the user.

[0014] According to the present invention, a system and method can be provided that can effectively utilize biometric data obtained in a user's daily life to maintain and improve the user's health, and can provide the user with more customized and user-friendly health advice.

[0015] Fig. 1 is a schematic diagram showing an overview of an exemplary embodiment of the present invention. Fig. 2 is a diagram showing an example of a check-up gate including a display device 10. Fig. 3 is a diagram showing another example of the display device 10. Fig. 4 is a block diagram showing the overall configuration of a system 1 for providing health advice. Fig. 5 is a flowchart showing a processing procedure for providing health advice. Fig. 6 is a diagram showing an overview of processing including generation and provision of health advice and response by a user.

[0016] Representative embodiments of the system and method according to the present invention will be described in detail below with reference to the drawings. However, the present invention is not limited to these drawings. Furthermore, since the drawings are intended to conceptually explain the present invention, dimensions, ratios, and numbers may be exaggerated or simplified as necessary to facilitate understanding.

[0017] 1. Overview of the Present Embodiment With reference to FIG. 1, the overview surrounding the system 1 according to the present embodiment will be described.

[0018] The facility 70 in this embodiment (the present invention) is a concept that includes any facility (also referred to as a facility, institution, site, place, activity base, etc.) that can use the system 1. Examples of the facility 70 include homes, workplaces (offices, factories, etc.), educational facilities such as schools, medical facilities such as hospitals and health checkup centers, nursing homes, care facilities, nursing homes, residential facilities for the elderly, exercise facilities, entertainment facilities, assembly halls, accommodation facilities, train stations, and airports. The facility 70 may also include mobile objects such as vehicles, ships, and aircraft.

[0019] When passing through a specific location or staying in a specific area of ​​these facilities 70, various sensors are installed at various locations around these locations or areas, and these sensors sense the user's biometric data. In the present invention, a structure in which various sensors are installed around such locations or areas is referred to as a "check-up gate." Types of sensors that can be used here include, for example, image sensors, temperature sensors, weight sensors, etc., and will be described in detail later. These sensors may be installed independently in the facility 70, or may be incorporated into the display device 10 (described below), which forms the core of the check-up gate.

[0020] Here, biometric data refers to all data obtained directly from the user or indirectly through predetermined processing, including, for example, image data obtained by photographing the user and data obtained by analyzing the image data. Biometric data includes not only dynamic data such as walking posture, walking speed, changes in joint range of motion, center of gravity fluctuation, and activity level, but also time-series data such as electromyography, electrocardiography, and electroencephalography, standing and sitting posture, body temperature, blood pressure and blood flow, heart rate, breathing, sweating, eye movements, sleep time, amount and time of excretion, blood components, urine components such as blood glucose levels, saliva components, images of the oral cavity (tongue, pharynx, etc.), fecal components, facial images, and skin moisture and oil content. The biometric data may be stored in a database (DB) along with other types of data. The database (DB) may be created in a computer belonging to the system 1 or in an external computer 80.

[0021] The system 1 acquires biometric data of the user via the display device 10 and the other device 30, and evaluates the user's health condition by performing computer analysis of the acquired biometric data. All or part of the computer analysis described above may be performed by the display device 10, or may be performed by another computer within the facility 70 or an external computer 80. In this case, the analysis results may be transmitted from the other computer or the external computer 80 to the display device 10.

[0022] Examples of health assessments will be described in detail later, but they include tongue diagnosis, cognitive function and stress checks, blood pressure, infection screening, dementia prediction, posture control prediction, mineral balance checks, grip strength estimation, body composition estimation, lower limb joint disease estimation, and heart rate and pulse wave estimation, and customized functions are provided depending on the location or position where the display device 10 is installed. Moreover, biometric data is acquired naturally in the user's daily life.

[0023] Based on the health status evaluation results, a customized health promotion program may be created for each user. The health promotion program may include an exercise program as well as various programs and health information for maintaining and improving the user's physical and mental health.

[0024] For example, for a user who is assessed as being under excessive tension at work, efforts to alleviate the tension, such as work improvement, taking vacation, engaging in moderate exercise, getting enough sleep, and consulting a doctor, may be suggested. Alternatively, an exercise menu suited to the user's health condition may be presented. Alternatively, for a user who is assessed as being under-exercised at work, moderate exercise may be suggested. Alternatively, for a user suspected of being under- or over-nutritional, a meal menu may be suggested. Alternatively, for a user assessed as having concerns about cognitive function, a doctor consultation may be suggested. Alternatively, for a user suspected of having an infectious disease, a doctor consultation may be suggested. Alternatively, for a user assessed as having an imbalance in minerals, a doctor's consultation may be suggested.

[0025] The health promotion program is provided to the user along with the health status assessment results, and the user implements the program. The user's health functions are improved by repeating the above-mentioned sensing, health status assessment, and creation and implementation of the health promotion program.

[0026] For example, simply by a user standing in front of the display device 10, the display device 10 acquires various health data related to the user. Because the display device 10 can also be used as a normal mirror, it can be naturally incorporated into daily life while maintaining usable mirror image quality. The display device 10 (or external computer 80) uses an artificial intelligence model to integrate and analyze health data stored in a data server and health data sensed daily by the display device 10. As a result, it can detect physical abnormalities that the user, their family, medical professionals, or caregivers may not easily notice, and provide health advice and alerts for each user. This can help prevent illness and promote health.

[0027] In this case, it is preferable to input the determined health condition, the generated health promotion program, and additional information about the user into a large-scale language model to obtain health advice customized for the user. This will result in user-friendly health advice, which is expected to improve the user's understanding of the health advice and make it easier for the user to put the health advice into practice.

[0028] Additionally, the collected biometric data, health status evaluation results, and created health promotion programs may be aggregated in a data server or the like and presented to and used by medical professionals such as doctors with the user's consent. Also, the generated health status assessment results may be presented to and used by medical professionals with the user's consent.

[0029] Furthermore, with the user's consent, the collected biometric data and the generated health status assessment results may be used by government and local communities to create more comfortable and livable communities, to manage the health of residents, and to enable people to continue living independently in their own homes in old age.

[0030] 2. Details of the System According to the Present Embodiment The system 1 according to the present embodiment implements a sensing function that collects biometric data from users within the facility 70, an analysis and assessment function that assesses the health condition of the users by analyzing the collected biometric data, a program generation function that generates a health promotion program based on the health condition assessment results, and an advice generation function that obtains health advice for the users by inputting the assessed health condition, the generated health promotion program, and additional information about the users into a large-scale language model.

[0031] To achieve this function, the system 1 includes a display device 10 and a model generating device 20 (see FIG. 5 ). The system 1 may further include an output device 40, a storage device 50, and an advice generating device 60. These components are communicatively connected via a wired or wireless communication network. The display device 10, the model generating device 20, the output device 40, the storage device 50, and the advice generating device 60 may be separate entities or may be integrated into one body. The storage device 50 may be installed as a component of a data center (not shown), and the data center may store not only collected data but also collected samples (e.g., saliva or fecal samples). In this case, the data center may also be called a biobank. These components will be described in detail below.

[0032] 2-1 Display Device The display device 10 will be described with reference to Figures 2 to 4. Figures 2 to 4 show various aspects of the display device 10.

[0033] The display device 10 can display various characters and images. The display device 10 may be installed in the facility 70 or placed within the facility 70. Examples of locations where the display device 10 can be installed include the entrance, bathroom, living room, and other areas in a home. Other examples include the entrance of an apartment building, the entrance or living room of a lodging facility, the entrance or work area of ​​an office, a fitness or sports gym, a beauty salon, and a beauty counter.

[0034] The display device 10 can acquire various types of biometric data. The biometric data that can be acquired by the display device 10 has been described above. The display device 10 may automatically start acquiring biometric data when it determines, for example, based on the detection result of a human presence sensor, that a user has approached within a predetermined range from the display device 10, or may start acquiring biometric data in response to an operation of the display device 10 by the user. An example of an operation by the user is, but is not limited to, pressing or touching a button on the display device 10.

[0035] For example, the display device 10 may include a mirror surface 11, a display 12, an image sensor 13, a processor, a memory, and a communication device. This type of display device 10 can also be said to be an Internet of Things (IoT) device called a smart mirror.

[0036] That is, the display device 10 has a mirror surface 11. The mirror surface 11 may be configured to reflect the entire body, the upper body, or mainly the face of the user. The mirror surface 11 may be configured as, for example, a laminate of a transparent glass plate and a metallic reflective surface (a back-surface mirror).

[0037] A display 12 is built into the mirror surface 11. The display 12 may be, for example, a liquid crystal display, an organic EL display, or a video projector screen.

[0038] An image sensor 13 is embedded or attached to the mirror surface 11. The image sensor 13 captures an image of a user in front of the display device 10 to obtain image data. The image sensor 13 is, for example, a semiconductor image sensor, and may be a CCD image sensor or a CMOS image sensor. The wavelength range of the image sensor 13 is not limited to visible light. In other words, the image sensor 13 may be an infrared temperature camera, a hyperspectral imaging camera, or an RFID sensor.

[0039] The image data acquired by the image sensor 13 is sent to a processor and memory (not shown) where various processes are performed and the data is stored. The processor may include, for example, a central processing unit (CPU), a graphics processing unit (GPU), or a microprocessor. The memory may include a random access memory (RAM) and a read-only memory (ROM).

[0040] Here, examples of processing performed on the acquired biometric data and the results of the processing are given below: - Estimation of oral or intestinal flora obtained from analysis of tongue images, and evaluation of disease risk based on the estimation results - Evaluation of cognitive function and stress predicted from facial expressions, gaze, and pulse waves obtained from analysis of facial images - Body temperature, heart rate variability, CO2 obtained from analysis of facial images ２ - Detection of breath volume, and health checks and infection screening based on the detection results. Facial images can be used for personal identification and authentication. - Detection of amyloid beta obtained from analysis of palm images, and predicting dementia risk based on the amount of amyloid beta accumulated. Palm images can be used for personal identification and authentication. - Detection of dizziness or unsteadiness obtained from analysis of whole-body images, and predicting nervous system dysfunction based on the detection results. - Detection of toxic metals or minerals obtained from analysis of facial or palm images, and evaluating the body's mineral balance based on the detection results.

[0041] The above-described process can be performed, for example, as follows. That is, pairs of sensing data and corresponding correct labels (correct data, teacher data) are created from data accumulated in a data server (storage device 50), and machine learning (e.g., deep learning) is performed using an artificial intelligence (AI) model. Machine learning is repeated to extract features of the data and predict the user's health condition.

[0042] As described above, the information extracted by the AI ​​model (i.e., the determined health condition and the generated health promotion program) and additional information may be input into a large-scale language model (LLM) to convert it into more user-friendly information. The display device 10 may generate health advice using the large-scale language model itself, or may cooperate with an external computer 80 capable of executing the large-scale language model, for example, via an application programming interface (API). Details of the large-scale language model and the generation of health advice using the large-scale language model will be described later.

[0043] Returning to the explanation of the display device 10, the display device 10 can present the converted information to the user. For example, this can be displayed on the display 12 or by reading out the text. Furthermore, by having the AI ​​model described above learn the user's reaction at that time, more accurate personalized information can be presented, which can lead to a change in the user's behavior.

[0044] The above-described analysis may be realized by executing various analysis programs and various biological data stored in memory on a processor. Alternatively, the analysis results may be obtained by applying various biological data to a trained model stored in memory. Furthermore, appropriate preprocessing may be performed before the analysis is performed. Alternatively, the analysis may be performed by an external computer 80, and the analysis results may be returned to the display device 10.

[0045] The acquired image data, preprocessed data, or analysis results may be stored in memory or transmitted to another mirror device, an external computer 80, or the storage device 50 via a communication device. The communication device may be a wired communication device or a wireless communication device. Examples of wired communication devices include an Ethernet adapter and a modem, while examples of wireless communication devices include Bluetooth (trademark), Wi-Fi, Thread, ZigBee, Matter, and low-power wide-area radio (LPWA). Examples of LPWA include LTE Category M1, NB-IoT, LoRaWAN, Sigfox, Wi-SUN, ZETA, and ELTRES.

[0046] In addition, the display device 10 may include other input devices such as a touch panel (including those equipped with a non-contact interface), a microphone, and other output devices such as a speaker.

[0047] Therefore, the display device 10 can be used not only as a mirror, but also as a device for collecting biometric data, a display device for collected and analyzed biometric data, various information and content, and even as a communication tool.

[0048] For example, the display 12 embedded in the mirror surface 11 can project images onto the surface of the mirror, including images captured by the image sensor 13. The display 12 can also display acquired biometric data. Examples of display methods include displaying text, as well as graphic displays such as graphs and illustrations.

[0049] The display device 10 can perform face recognition and personal authentication using the built-in image sensor 13 or another sensor. In addition, if the display device 10 has a built-in microphone and temperature / humidity sensors, it can perform voice recognition and environmental measurement.

[0050] The display device 10 can communicate with other devices 30 within the facility 70 via a built-in communication device. The other devices 30 include other display devices 10, PCs, and mobile terminals located within the same facility 70. Mobile terminals include, for example, smartphones, tablet terminals, and wearable terminals. The display device 10 can also communicate with an external computer 80 via the Internet. The external computer 80 includes a server, PC, or mobile terminal located outside the facility 70.

[0051] If the display device 10 has a built-in speaker, it can output audio. If the display device 10 has an input device such as a touch panel, it can be easily operated by touching the surface of a mirror, for example.

[0052] In this way, the display device 10 acquires various biological data and evaluates the health condition of the user by performing computer analysis of the acquired biological data. However, all or part of the computer analysis may be performed by another computer (e.g., the external computer 80), and the analysis results may be returned to the display device 10.

[0053] The display device 10 can acquire biometric data of the user at different times (first and second times) to obtain time-series biometric data. Then, analysis of the time-series biometric data enables a time-series evaluation of the user's health status. Alternatively, by comparing the same type of biometric data (first and second biometric data) sensed in different facilities 70 (first and second facilities 70; for example, home and workplace), the user's health status in different environments can be accurately evaluated.

[0054] To give a specific example of the latter, if the measured values ​​of a user's blood pressure, heart rate, respiratory rate, and sweating at the workplace are higher than the corresponding values ​​at the user's home by a preset value or more, the user can be evaluated as being in an excessively tense state at the workplace. Alternatively, if the measured values ​​of a user's blood pressure, heart rate, respiratory rate, blood flow, and sweating at an exercise facility are higher than the corresponding values ​​at the user's home by a preset tolerance, the user can be evaluated as being performing moderate exercise. Alternatively, if the measured or estimated values ​​of a user's travel distance, travel speed, and metabolism at the workplace are lower than the corresponding values ​​at the user's home by a preset value or more, the user can be evaluated as being in a state of insufficient exercise at the workplace. Alternatively, if the measured values ​​of a user's blood pressure, heart rate, and respiratory rate at a user's accommodation and entertainment facilities are lower than the corresponding values ​​at the user's home by a preset value or more, the user can be evaluated as being in an appropriately relaxed state at the accommodation and entertainment facilities. Alternatively, if the brain wave measurements taken at a user's educational facility or workplace differ from the corresponding values ​​at the user's home, the user may be assessed as lacking concentration.

[0055] 2-2. Other devices

[0056] Other devices 30 may be installed within the facility 70. The display device 10 may be combined with or linked to the other devices 30.

[0057] Examples of the other device 30 include, but are not limited to, a body composition scale, a grip force sensor, a pressure sensor, a motion capture device, a temperature sensor, a humidity sensor, a component analyzer, a weight sensor, a flow rate sensor, a position sensor, an infrared temperature camera, a hyperspectral imaging camera, an RFID sensor, and a motion capture device (none of which are shown). The display device 10 may include these devices. In the example of FIG. 2 , the display device 10 has a grip force sensor in the form of a handle as an example of the other device 30 described above.

[0058] The other device 30 also continuously collects biometric data of the user H, and transmits and stores the collected biometric data together with the collection date and time to the storage device 50. The sensing of the biometric data of the user H by the other device 30 may be performed, for example, by constant measurement.

[0059] For example, a body composition scale can be placed at the feet of the display device 10 to measure the user's weight, BMI, body fat percentage, visceral fat level, muscle mass, body water percentage, basal metabolic rate, estimated bone mass, etc. The body composition scale may be embedded in the floor of the facility 70 so that it is flush with the floor. The body composition scale may also be incorporated into a handrail. The body composition scale acquires the user's body composition data, for example, foot floor measurement pressure, and transmits the acquired data to the display device 10. The display device 10 predicts lower limb joint diseases from changes in foot floor measurement pressure.

[0060] The grip sensor is, for example, rod-shaped and is installed beside the display device 10. The grip sensor may also serve as a gripping member or a handrail. The grip sensor detects the pressure applied to the grip sensor when a user grips it and transmits the pressure data to the display device 10. The display device 10 calculates the grip strength using this pressure data and a skeletal analysis based on a full-body image of the user. The display device 10 then predicts the user's frailty using changes in grip strength and the walking speed calculated from the full-body image of the user.

[0061] The infrared temperature camera can be installed in rooms such as the living room, study, bedroom, and office of the facility 70, and can measure the user H's facial expression, heart rate, oxygen saturation, carbon dioxide exhalation, surface body temperature, and core body temperature.

[0062] The hyperspectral imaging camera is installed, for example, at the entrance or nearby, and in addition to the function of personal authentication of user H, can analyze skin proteins of user H, measure body composition (including, for example, the composition ratio of fat, bone, lean soft tissue, etc., weight, body fat, basal metabolic rate, visceral fat, muscle mass, and bone mass), and measure autonomic nerves (including, for example, the degree of autonomic nerve fatigue and the balance between sympathetic and parasympathetic nerves). Alternatively, the hyperspectral imaging camera can be installed at the dining table, in the kitchen, or nearby to measure glycohemoglobin (HbA1c).

[0063] The RFID sensor is installed on a wall surface in a room and can measure the amount of water in the user H's body.

[0064] The pressure sensors are installed, for example, in the hallway and on the floor of the room, and can measure the foot pressure distribution of the user H.

[0065] The motion capture device can be installed, for example, in a hallway in a room, and can measure the user H's walking posture, walking speed, changes in joint range of motion, fluctuations in the center of gravity, and activity level.

[0066] The other device 30 may be installed hidden from view so that the user H cannot immediately find it. This is to prevent the user H from becoming nervous upon noticing the presence of the other device 30 and causing the sensed biometric data to deviate from the everyday values ​​of the user H. In other words, this is to know the user H in a natural state in everyday life.

[0067] Various sensors may also be installed to obtain environmental data, such as a thermometer for measuring room temperature, a hygrometer for measuring room humidity, an illuminance meter for measuring room brightness, and a CO ２ A carbon dioxide meter for measuring the concentration, a noise sensor for measuring environmental sounds, a light sensor for detecting the lighting conditions in the environment, etc. may also be installed. Environmental data collected by these sensors is transmitted to and stored in the storage device 50. Note that the display device 10 may include these sensors.

[0068] Additionally, various sensors built into wearable devices and mobile information terminals such as smartphones may be used to acquire biometric data of the user, which is useful for assisting in determining the user's health condition.

[0069] The collected biometric data, environmental data, and other data are transmitted to and stored in the storage device 50. The storage device 50 may also store the health status assessment results and the generated health promotion program, which will be described later. The various data stored in the storage device 50 may be made available to an external computer 80 used by the doctor and medical institution in charge of user H, with the permission of user H. Such medical collaboration enables various health records necessary for the diagnosis and treatment of user H to be shared among medical professionals in a timely manner. Of course, data collaboration may also be targeted at doctors, dentists, pharmacists, nutritionists, caregivers, etc., and is not limited to these medical professionals.

[0070] 2-3. Model Generation Device The model generation device 20 is a computer that generates various statistical models. The model generation device 20 can also execute various processes using the generated statistical models. Below, as examples of statistical models, a statistical model for analyzing biological data and a statistical model for generating a health promotion program will be described.

[0071] 2-3-1. Biological Data Analysis Model The model generation device 20 can generate a predictive model for analyzing biological data collected by the display device 10 and other devices 30. The model generation device 20 may use AI analysis for data analysis.

[0072] Specifically, the model generation device 20 uses biometric data of a person, including at least one type of data indicating facial expression, heart rate, oxygen saturation, exhaled carbon dioxide, surface body temperature, core body temperature, skin protein analysis, body composition, autonomic nerves, HbA1c, body water content, foot pressure distribution, walking posture, walking speed, changes in joint range of motion, center of gravity fluctuation, activity level, electromyography, electrocardiography, electroencephalography, standing and sitting posture, body temperature, blood pressure, blood flow, heart rate, breathing, sweating, eye movements, sleep time, amount and time of excretion, blood components, urine components, saliva components, images of the oral cavity, and fecal components, as training data, and the person's health condition, including the person's energy expenditure and exercise effect, to generate an analytical model using machine learning, with the biometric data as input and the health condition of the person as output. The model generation device 20 then uses the generated predictive model to output the user's health condition from the collected biometric data. Examples of usable machine learning include, but are not limited to, deep learning.

[0073] For example, the model generation device 20 can use at least one type of biological data, such as a person's facial expression, heart rate, oxygen saturation, carbon dioxide exhalation, surface body temperature, deep body temperature, skin protein analysis, body composition, autonomic nerves, HbA1c, body water content, foot pressure distribution, walking posture, walking speed, changes in joint range of motion, center of gravity fluctuation, and activity level, as existing indicators and teacher variables (teaching data), and perform machine learning using the same type of biological data as the collected data group as explanatory variables to create an analysis program (trained model). The model generation device 20 can also use basic information, such as age, gender, height, and weight, as parameters to perform algorithmic analysis of biological information, such as facial expression, heart rate, oxygen saturation, carbon dioxide exhalation, surface body temperature, deep body temperature, skin protein analysis, body composition, autonomic nerves, HbA1c, body water content, and foot pressure distribution, as well as joint angles during walking and their changes over time, lower limb muscle strength, center of gravity, or motor function information, such as three-dimensional movement distance of observation points correlated thereto, walking posture, walking speed, and activity level. That is, the model generating device 20 can obtain an objective evaluation of the health condition of the user H, including health functions and motor functions, by applying the sensing data to the above model.

[0074] The model generation device 20 can transmit the generated analysis model to the display device 10 or the storage device 50. The model generation device 20 can also transmit the assessment result of user H's health condition to the display device 10, the output device 40, or the storage device 50. The model generation device 20 can further transmit the assessment result of user H's health condition to the device of the user or his / her family. Then, by outputting the assessment result of user H's health condition to the display device 10 or the output device 40, user H and his / her family can detect user H's illness or poor physical condition early and can use the result to manage user H's health. Needless to say, such assessment result can be effectively used by related parties, including medical professionals and caregivers, with user H's permission. As a result, the assessment result can be effectively used to support user H and collaborate with related parties.

[0075] 2-3-2. Health Promotion Program Generation Model The model generation device 20 can also use the health status assessment results to generate a health promotion program customized for each user H. An example of a statistical model for generating a health promotion program is given below.

[0076] That is, the model generation device 20 extracts the user's biometric characteristics by comparing the collected biometric data with predetermined evaluation indices. Then, the model generation device 20 uses the predetermined evaluation indices and evaluation values ​​indicating the effectiveness of the health promotion program as training data, and generates an evaluation model by machine learning, taking the biometric data as input and outputting an evaluation of the health promotion program. The generated evaluation model may be transmitted to the storage device 50 or the display device 10. The model generation device 20 can also use the evaluation model to output a health promotion program to be presented to the user from the collected biometric data, along with the user's biometric characteristics.

[0077] More specifically, health promotion programs are broadly divided into those related to health functions and those related to motor functions. When generating a program related to health functions, the model generation device 20 identifies the user H's biometric characteristics by comparing, for example, facial expression, heart rate, oxygen saturation, carbon dioxide exhalation, surface body temperature, core body temperature, skin protein analysis, body composition, autonomic nerves, HbA1c, body water content, and foot pressure distribution with the normal range for each individual and evaluation indices such as epidemiological and medical guidelines. The model generation device 20 then uses AI analysis of the measured data, basic information such as age, gender, height, and weight, and data on the user H's living environment to construct an algorithm (evaluation model) related to each individual's lifestyle habits. Examples of AI analysis include, but are not limited to, deep learning. The model generation device 20 can also present behavioral changes and the associated effects by applying the constructed evaluation model to the user's biometric data, etc.

[0078] When generating a program related to motor function, the model generation device 20 identifies the dynamic analysis characteristics of the user H, for example, by comparing the position and velocity of each joint, as well as the joint angle and angular velocity. Based on the measured data, the model generation device 20 then estimates the energy consumption during exercise and the exercise effect, and constructs a motion measurement algorithm (evaluation model). Examples of AI analysis include, but are not limited to, deep learning. The model generation device 20 can also present recommended exercises, the energy consumption during exercise, and the exercise effect by applying the constructed evaluation model to the user's biometric data, etc.

[0079] The model generation device 20 can transmit the constructed evaluation model to the display device 10 or the storage device 50. The model generation device 20 can also transmit the various programs that it has generated to the display device 10, the output device 40, or the storage device 50. The model generation device 20 can further transmit the various programs that it has generated to the terminals of the user and his / her family.

[0080] Furthermore, the model generating device 20 may generate its own large-scale language model, which is trained to accept input of a health status assessment, a health promotion program, and additional information (the basic information, biometric information, external information, and unique information described above) for a person and output health advice for that person.

[0081] By outputting the above-mentioned program or health advice to the display device 10 or the output device 40, user H and his / her family can clearly understand what should be done to maintain, restore, and improve user H's health and motor functions, i.e., behavioral changes, and can utilize this information in the health management of user H. It goes without saying that such program or health advice can be shared with related parties, including medical professionals and caregivers, with user H's permission.

[0082] Here is an example of behavioral change. For example, in the kitchen, a user operates a kitchen monitor or their own mobile device to launch a specific application program. The application program evaluates the user's health status by referring to the user's biometric data, and presents menus, ingredients, cooking methods, nutrients, calorie amounts, etc. tailored to the user. The application program can also acquire images of the dishes prepared by the user via the image sensor 13 and calculate and display the amounts of nutrients and energy contained in the dishes.

[0083] In the living room, the user operates a television or monitor to launch an application program. The application program suggests games that use eye tracking to improve or understand cognitive function. When the user selects one of the suggested games, the game starts and cognitive function is measured. During or after the game, the application program references the necessary biometric data and displays the game score and feedback on the improvement of cognitive function.

[0084] In the living room, the user operates a mirror-type monitor (e.g., display device 10) and launches an application program. The application program evaluates the user's health condition by referring to the user's biometric data and presents an exercise menu appropriate for the individual user. A virtual trainer is displayed on the mirror-type monitor, and the user follows the virtual trainer's movements to perform the exercise menu. After exercise, the application program calculates and displays the amount of energy consumed and provides feedback on motor function.

[0085] When the user is out and about, the user operates the mobile terminal and starts the application program, which can display the measurement results of the user's health data along with health advice.

[0086] The model generating device 20 can be configured as one or more computers including an arithmetic circuit (processor) such as a central processing unit (CPU) and memories such as random access memory (RAM) and read-only memory (ROM). The functions of the model generating device 20 described above can be realized by loading an execution program stored in the ROM into the RAM and executing it with the CPU. The model generating device 20 can use continuously collected biological data to determine health conditions, generate health promotion programs, and generate health advice periodically or at times deemed necessary for each individual. Alternatively, the functions of the model generating device 20 described above may be incorporated into the display device 10.

[0087] 2-4. Advice Generation Device The advice generation device 60 is a computer configured to input the determined health condition, the generated health improvement program, and additional information about the user into a large-scale language model and output health advice for user H. That is, the advice generation device 60 stores a trained large-scale language model and can output health advice in response to the input. Alternatively, the advice generation device 60 may access an external computer 80 via, for example, an API and use the trained large-scale language model. Hereinafter, the generation of health advice using a large-scale language model will be described with reference to FIG. 7.

[0088] The large-scale language model is constructed using a large dataset and deep learning technology (e.g., Transformer), and is modeled using the occurrence probability of sentences and words. Because large-scale language models can understand and generate natural language and other content and perform a wide range of tasks, the inventors discovered that by inputting the above information into the large-scale language model, customized, user-friendly health advice for user H can be obtained. Here, known models such as BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), Bard, LLaMA, and Gemini can be used as the large-scale language model. To utilize these known models, the system 1, e.g., the display device 10, may use an API or a computer within the system 1. Alternatively, the system 1, e.g., the model generation device 20, may use machine learning to generate a unique large-scale language model specialized for generating health advice.

[0089] The items input to the large-scale language model can be roughly divided into the following five categories: - Basic information: user's age, gender, nationality, etc. - Biometric information: user's height, weight, body composition, basal metabolism, blood pressure, etc. - External information: financial and economic information, logistics, seasons, trends, temperature, humidity, etc. - Unique information: user's hobbies and preferences, preferences for things and foods, allergies, partial body characteristics (for example, large feet for height), personal or household economic information, mood of the day, etc. - Information extracted as features by the AI ​​model

[0090] Here, information already input into the AI ​​model may be input into the large-scale language model. Basic information, biometric information, external information, and unique information are also input into the large-scale language model as additional information. These types of information are useful for customizing health advice for user H.

[0091] The additional information may include the user's reaction to the provided health advice. For example, consider a scenario in which the large-scale language model outputs health advice to user A, such as, "As a result of measuring user A's nutritional status, we have determined that he or she is deficient in vitamin C. Therefore, we recommend that you eat lemons." In response to this health advice, user A responds by saying that he or she cannot eat lemons due to an allergy or dislike of lemons. This reaction may be acquired, for example, by user A inputting the information via the touch panel or microphone of the display device 10, or by analyzing user A's actions acquired by an image sensor. In other words, as a result of user A's reaction, information about user A's food preference or allergy, i.e., information such as "A cannot eat lemons," is obtained. This information may be registered in a database. When the large-scale language model receives the above-mentioned food preference or allergy information of user A as additional input, it executes processing again and outputs updated health advice. The updated health advice may be, for example, "Then, try eating oranges or kiwifruit. Eating them with yogurt in the morning will improve your intestinal environment. The effects of eating them continuously for one week are as follows: ..." The health advice may be updated at any time depending on the reaction of user A, and there is no limit to the number of updates.

[0092] 2-5. Output Device The output device 40 is a device that outputs health advice including the health status assessment results and a health promotion program. The output device 40 includes the user H's terminal (including a smartphone or personal computer), a display, a printer, a projector, and a speaker. The display device 10 may also function as the output device 40.

[0093] For example, if the health condition assessment result indicates an abnormality in the health condition of user H, i.e., a deviation from the normal range (for example, in the case of data expressed as a numerical value, if the value is above or below a predetermined threshold or outside the predetermined range), the output device 40 may output an alert to the terminals of user H and those related to him / her. The output device 40 may also display other information related to daily life, such as information on energy, housing, and local information.

[0094] 3. System Operation The processing steps for providing a health promotion program by the system 1 will be described with reference to FIG.

[0095] First, in step S1, the display device 10 and the other device 30 sense the biometric data of the user H, and store the data in the storage device 50 in association with the sensing time. The sensing is performed continuously.

[0096] Then, in step S2, the model generation device 20 or the display device 10 analyzes the collected biometric data, determines the health condition of user H, and stores the determination result in the storage device 50. For example, the model generation device 20 or the display device 10 stores a trained model, and obtains the health condition of user H by applying the biometric data of user H to the trained model. Step S2 is executed periodically or at a timing considered necessary for each individual.

[0097] Once the health status assessment results have been generated, in step S3, the model generation device 20 or the display device 10 generates a health promotion program for each user H based on the health status assessment results and stores the program in the storage device 50.

[0098] The health condition assessment results and the health promotion program are displayed on the output device 40 or the display device 10. This allows the user H to understand his / her own health condition (e.g., health function and motor function) and clearly understand what he / she should do to maintain, recover, and improve his / her health function and motor function. The user H can put the presented health promotion program into practice and utilize it in his / her own health management.

[0099] Alternatively, the health status assessment results and the health promotion program may be input into a large-scale language model along with other information about the user to generate health advice customized for the user (step S4). The health advice customized for the user is user-friendly and can be easily implemented by the user.

[0100] Next, in step S5, it is determined whether the user has responded to the presented health advice. If the user's response is confirmed, the process returns to step S4, where various information including the user's response is input into the large-scale language model, and updated health advice is presented to the user. Because the user's response may include user characteristics unknown to the system 1, health advice updated using such a response will be even more tailored to the user. Additionally, each time a user response is obtained, additional information may be input into the large-scale language model, and further updated health advice may be presented to the user. This is expected to further improve the accuracy of the health advice. Alternatively, if the user's response is not confirmed within a predetermined period, the process starts again from step S1.

[0101] An example of the operation is given below. In this example, user A receives various health advice. When user A stands in front of the display device 10, the display device 10 identifies the individual using facial recognition or the like and launches a predetermined application program. This application program displays a virtual person on the display 12 of the display device 10 and gives various instructions to user A. In other words, the display device 10 senses biometric data while a dialogue between the virtual person (e.g., a virtual butler) and the user progresses. Upon completion of the measurement, the display device 10 displays the measurement results and health advice according to user A's selection. At this time, the display device 10 may display a virtual person (e.g., a virtual doctor) on the display 12 and have this person provide explanations. Furthermore, if user A requests the display of an exercise menu, the display device 10 may display a virtual person (e.g., a virtual trainer) on the display 12 and present an exercise menu tailored to the individual.

[0102] Here is an example of the measurement results and health advice: "You have been showing signs of muscle mass decreasing over the past few days. We will present you with a meal and exercise menu that will suit you, so let's work hard together! By achieving these goals, you can expect the following health benefits..."

[0103] Then, the process returns to sensing and repeats the above procedure. This allows user A to effectively utilize biometric data obtained in daily life to improve his / her health functions. User A can also understand the effect of practicing the health promotion program through the health condition assessment results from the next time onwards.

[0104] Another example is a situation in which user B receives various health advice.

[0105] User B sticks out his tongue in front of the display device 10 and an image of his tongue is captured. The image of User B's tongue, along with other information such as User B's age, gender, height, and weight, is input into the tongue diagnosis AI model. Here, the tongue diagnosis AI model is a trained model obtained by machine learning using the tongue image as input and a health status assessment as output. This model is also trained to generate a health promotion program from the health status assessment and the other information described above. The health status assessment may include, for example, prediction of the oral flora and intestinal flora, and assessment of the risk of gastrointestinal disease. The health promotion program may include, for example, suggestions for exercise, diet, and amount of sleep aimed at improving the intestinal flora.

[0106] The information extracted by the AI ​​model is input to a large-scale language model along with other information about user B (e.g., basic information, biometric information, external information, unique information, etc.). The large-scale language model converts the input information into more user-friendly information and presents it to user B.

[0107] For example, when basic information about user B, sample data on the intestinal flora, and a health promotion program such as exercise, diet, and amount of sleep to improve the intestinal flora are input into a large-scale language model (e.g., GPT), the large-scale language model will provide the following answer as an example of daily activities:

[0108] Breakfast examples: Oatmeal (e.g., whole grain oats) with Greek yogurt and a banana Green tea or black coffee Morning activities examples: Walking or biking to work Stretching or a short walk between work Lunch examples: Salad (e.g., containing leaf lettuce, tomato, cucumber, avocado, and carrots). Olive oil and vinegar dressing, whole grain bread with hummus, water or herbal tea. Afternoon activity examples: - Work (including standing), 15-minute walk after lunch. Snack examples: - Nuts (e.g. almonds and walnuts) and fruit (e.g. apple and berries). Dinner examples: - Grilled salmon, stir-fried vegetables (e.g. broccoli, carrots, peppers and zucchini), kimchi, water or herbal tea. Exercise examples: - 30-minute walk or jog after work, strength training twice a week (20-30 minutes; e.g. push-ups, squats, dumbbell exercises). Bedtime activity examples: - Warm herbal tea (e.g. chamomile), reading, meditating, listening to relaxing music, limiting the use of electronic devices. Sleep: - Maintain a consistent bedtime and get 7-8 hours of sleep.

[0109] Assume that user B has reacted in the following ways after receiving the above health advice: - I can't eat bananas because I'm allergic to bananas - I want to eat noodles for lunch - I do desk work and don't like strength training - I want to drink beer with dinner - Instead of reading, I want to think about games or my hobby of fishing before going to bed The above reactions may be made, for example, by placing a check mark in the response box corresponding to each example activity on the display device 10. The above reactions are registered in the DB as information characterizing user B.

[0110] By adding the above reaction information to the input information, the large-scale language model will update the health advice, for example, as follows: Example breakfast: Oatmeal with fruit (strawberries or blueberries) Green tea or iced coffee Example morning activities: Walking to work (e.g., brisk walking, using the stairs) Stretching in between work, simple exercises to improve posture (can be done at your desk) Example lunch: Hiyashi chuka (cold noodles, cucumber, tomato, ham, egg, daikon sprouts, and sesame sauce) Barley tea or water Example afternoon activities: Light stretching or a 15-minute walk after lunch Periodic eye rest away from your desk to prevent eye strain Example snack: Nut and dried fruit mix (e.g., containing almonds, raisins, and cranberries) Example dinner: Grilled chicken or fish (e.g., salmon) Stir-fried vegetables (e.g., containing broccoli, carrots, peppers, and zucchini) Kimchi as a side dish One can of beer However, choose low-alcohol or non-alcoholic drinks. Examples of post-dinner relaxation: ・Preparing for games or fishing ・Light stretching and deep breathing Sleep ・Keep a consistent bedtime and ensure 7-8 hours of sleep

[0111] The updated health advice reflects user B's reaction to the initial health advice and is more in line with the preferences and lifestyle of user B. Alternatively, if there are further reactions from user B, the input to the large-scale language model and the update of the health advice may be repeated to further improve the accuracy of the health advice.

[0112] The sensing data, health condition assessment results, and health promotion program accumulated in this way are stored in the storage device 50 and, with the permission of user H, are shared among medical professionals related to user H. This is expected to enable doctors to make diagnoses and treatments more appropriately and quickly. This effect can be expected not only during face-to-face consultations but also in remote consultations, realizing more comprehensive medical cooperation.

[0113] In addition, by sharing the health information of user H described above among caregivers, it is expected that more appropriate and prompt care support will be provided.

[0114] Of course, the sensing data and the health status assessment results can also be used as a monitoring system or anomaly detection system for user H. In addition, the collected biometric data and health status assessment results can be used by the government and local community with the consent of user H, and can be reflected in policies to create a more comfortable and livable community.

[0115] Because the display device 10 and the other devices 30 are installed in the facility 70, there is no need to bother putting them on or taking them off, as is the case with wearable sensors. Furthermore, because the sensors are hidden from the eyes of the user H, the user H is not aware of the presence of the sensor 10. Therefore, it is possible to obtain daily biological data of the user H, and to grasp the health condition with high reliability.

[0116] Representative embodiments of the present invention have been described above, but the present invention is not limited to these, and various design modifications are possible, which are also included in the present invention.

[0117] According to the present invention, biometric data obtained in the user's daily life can be effectively utilized to maintain and improve the user's health. Utilization of biometric data includes supporting the user and sharing information with medical and nursing care professionals.

[0118] REFERENCE SIGNS LIST 1 System 10 Display device 20 Model generation device 40 Output device 50 Storage device 60 Advice generation device H User

Claims

1. A system for providing health advice for a user, comprising: a structure installed within a facility and including a sensor that continuously collects biometric data of the user; an analysis device configured to analyze the biometric data collected by the sensor and determine the health condition of the user; a program generation device configured to generate a health promotion program tailored to the user using the health condition determination result; and an advice generation device configured to input the determined health condition, the generated health promotion program, and additional information about the user into a large-scale language model to obtain health advice for the user.

2. The system of claim 1, wherein the additional information includes at least one of the following types of information: basic information including at least one of the user's age, sex, and nationality; biometric information including at least one of the user's height, weight, body composition, basal metabolism, and blood pressure; external information including at least one of the user's financial and economic situation, logistics, seasons, trends, temperature, and humidity; and specific information including at least one of the user's hobbies and preferences, preferences for things and foods, allergies, physical characteristics, personal or household economic situation, and mood of the day.

3. The system of claim 1, wherein the additional information includes the user's reaction to the health advice.

4. The system of claim 1, wherein the biometric data includes at least one type of data from the group consisting of facial expression, heart rate, oxygen saturation, carbon dioxide exhalation, surface body temperature, core body temperature, skin protein analysis, body composition, autonomic nerves, HbA1c, body water content, foot pressure distribution, walking posture, walking speed, changes in joint range of motion, fluctuations in the center of gravity, activity level, electromyography, electrocardiography, electroencephalography, standing and sitting posture, body temperature, blood pressure, blood flow, heart rate, breathing, sweating, eye movements, sleep time, amount and time of excretion, blood components, urine components, saliva components, images of the oral cavity, and fecal components.

5. The system according to claim 1, wherein the analysis device is configured to use a person's biometric data including at least one type of data from the group consisting of facial expression, heart rate, oxygen saturation, carbon dioxide exhalation, surface body temperature, core body temperature, skin protein analysis, body composition, autonomic nerves, HbA1c, body water content, foot pressure distribution, walking posture, walking speed, changes in joint range of motion, fluctuations in the center of gravity, activity level, electromyography, electrocardiography, electroencephalography, standing and sitting posture, body temperature, blood pressure, blood flow, heart rate, breathing, sweating, eye movements, sleep time, amount and time of excretion, blood components, urine components, saliva components, images of the oral cavity, and fecal components, as well as the person's health condition including the person's energy expenditure and exercise effects, as training data, and to output the user's health condition from the collected biometric data using a predictive model obtained by machine learning a learning model that takes the person's biometric data as input and outputs the person's health condition.

6. The system described in claim 1, wherein the program generation unit is configured to: extract the biometric characteristics of the user by comparing the collected biometric data with predetermined evaluation indices; use the predetermined evaluation indices and evaluation values ​​indicating the effectiveness of the health promotion program as training data; generate a health promotion program for the user from the collected biometric data using an evaluation model obtained by machine learning a learning model in which the biometric data is used as input and the evaluation of the health promotion program is used as output; and output the extracted biometric characteristics and the generated health promotion program.

7. A method for providing health advice for a user, comprising: a computer performing steps of: analyzing biometric data of the user collected by sensors installed in structures within a facility to determine the health status of the user; generating a health promotion program tailored to the user using the health status determination result; and inputting the determined health status, the generated health promotion program, and additional information about the user into a large-scale language model to obtain health advice for the user.