Personalized health management device and method using non-invasive biometrics

Abstract

The provided personalized health management device and method using non-invasive biometrics using sweat to measure and predict health information of a user on the basis of electrochemical biometrics and lifelog data, and comprise: a communication unit communicatively connected to an external device; a display unit for displaying health information of a user; and a processor for predicting blood glucose levels of the user, wherein the processor acquires body information of the user, and sweat and heartbeat information of the user from an external device, extracts first input data for predicting blood glucose levels of the user from the acquired sweat and heartbeat information, extracts second input data for predicting blood glucose levels of the user from the body information of the user, inputs the first input data and the second input data into a pre-trained neural network model so as to output a blood glucose level prediction result value of the user, generates health information of the user on the basis of the blood glucose level prediction result value of the user, and can control the display unit so as to display the health information of the user.

Description

Personalized health management device and method using non-invasive biometric information

[0001] The present disclosure relates to a personalized health management device and method using non-invasive biometric information that can measure and predict a user's health information based on electrochemical biometric information using sweat and lifelog data.

[0002] Typically, sweat contains dozens of biomarkers.

[0003] By selectively identifying and quantitatively measuring biomarkers in sweat, various body information can be noninvasively confirmed.

[0004] However, since most biomarkers are present at concentrations significantly lower than those in blood or other body fluids, development of highly sensitive biomarker technology with high selectivity is necessary to measure biomarkers in sweat.

[0005] In addition, since human bio-information shows various aspects depending on each individual's various physical indicators, such as age, gender, weight, body mass index, and resting heart rate, there were limitations in accurately predicting each bio-information using only measurement signals from non-invasive biomarkers with high selectivity.

[0006] Therefore, in the future, it is necessary to develop a personalized health management device that can accurately measure and predict a user's health information by simultaneously utilizing electrochemical bioinformation using sweat and individual body indicators.

[0007] The present disclosure aims to solve the above-mentioned problems and other problems.

[0008] The present disclosure provides a personalized health management device and method using non-invasive biometric information, which enables the user to manage his / her health efficiently and conveniently by inputting input data extracted based on the user's sweat and heart rate information and body information into a pre-trained neural network model and outputting the user's blood sugar prediction result value, thereby providing the user with highly accurate and reliable blood sugar prediction information and blood sugar management guide.

[0009] According to one embodiment of the present disclosure, a personalized health management device includes a communication unit that is communicatively connected to an external device, a display unit that displays health information of a user, and a processor that predicts blood sugar of the user, wherein the processor acquires body information of the user and sweat and heart rate information of the user, extracts first input data for predicting blood sugar of the user from the acquired sweat and heart rate information, extracts second input data for predicting blood sugar of the user from the body information of the user, inputs the first input data and the second input data into a pre-learned neural network model to output a blood sugar prediction result value of the user, generates health information of the user based on the blood sugar prediction result value of the user, and controls the display unit to display the health information of the user.

[0010] A health management method according to one embodiment of the present disclosure is a health management method of a personalized health management device that is connected to an external device, the method including: acquiring body information of a user and sweat and heart rate information of the user from the external device; extracting first input data for predicting blood sugar level of the user from the acquired sweat and heart rate information; extracting second input data for predicting blood sugar level of the user from the body information of the user; inputting the first input data and the second input data into a pre-learned neural network model to output a blood sugar level prediction result value of the user; and generating and displaying health information of the user based on the blood sugar level prediction result value of the user.

[0011] According to one embodiment of the present disclosure, a personalized health management device inputs input data extracted based on the user's sweat and heart rate information and body information into a pre-trained neural network model and outputs a user's blood sugar prediction result value, thereby providing the user with highly accurate and reliable blood sugar prediction information and blood sugar management guide, thereby enabling the user to manage his / her health efficiently and conveniently.

[0012] FIG. 1 illustrates an artificial intelligence device according to one embodiment of the present disclosure.

[0013] FIG. 2 illustrates an artificial intelligence server according to one embodiment of the present disclosure.

[0014] FIG. 3 illustrates an artificial intelligence system according to one embodiment of the present disclosure.

[0015] FIG. 4 is a drawing for explaining the operation of a personalized health management device according to one embodiment of the present disclosure.

[0016] FIG. 5 is a drawing for explaining the structure of a personalized health management device according to one embodiment of the present disclosure.

[0017] FIGS. 6(a) and (b) are diagrams illustrating an external device that is communicatively connected to a personalized healthcare device according to one embodiment of the present disclosure.

[0018] FIGS. 7 to 10 are diagrams for explaining a process of acquiring user body information and sweat and heart rate information of a personalized health management device according to an embodiment of the present disclosure.

[0019] FIG. 11 and FIG. 12 are diagrams for explaining a process of generating a blood sugar prediction result value of a personalized health management device according to an embodiment of the present disclosure.

[0020] FIG. 13(a) and (b) are diagrams for explaining the performance of a neural network model according to an embodiment of the present disclosure.

[0021] FIG. 14 and FIG. 15 are drawings for explaining a health information generation process of a personalized health management device according to an embodiment of the present disclosure.

[0022] FIG. 16 is a drawing for explaining a health management method of a personalized health management device according to one embodiment of the present disclosure.

[0023] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present disclosure.

[0024] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0025] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0026] The neural network, artificial neural network, and network function of the present disclosure may often be used interchangeably.

[0027] Additionally, in the present disclosure, the terms "neural network," "neural network," and "network function" may be used interchangeably. A neural network may be comprised of a set of interconnected computational units, generally referred to as "nodes." These "nodes" may also be referred to as "neurons." A neural network comprises at least two or more nodes. The nodes (or neurons) constituting the neural networks may be interconnected by one or more "links."

[0028] Artificial Intelligence (AI)

[0029] Artificial intelligence (AI) is the study of artificial intelligence or the methodologies for creating it, while machine learning (ML) defines various problems in the field of AI and studies the methodologies for solving them. Machine learning is also defined as an algorithm that improves performance on a task through consistent experience.

[0030] An artificial neural network (ANN) is a model used in machine learning. It can refer to a model with problem-solving capabilities, comprised of artificial neurons (nodes) formed by the connection of synapses. An ANN can be defined by the connection patterns between neurons in different layers, the learning process that updates model parameters, and the activation function that generates output values.

[0031] An artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer may contain one or more neurons, and the artificial neural network may include synapses connecting neurons. In an artificial neural network, each neuron can output a function value of an activation function based on input signals, weights, and biases received through the synapses.

[0032] Model parameters are parameters determined through learning, including synaptic connection weights and neuron biases. Hyperparameters are parameters that must be set before learning in machine learning algorithms, including the learning rate, number of iterations, mini-batch size, and initialization function.

[0033] The goal of artificial neural network training can be seen as determining model parameters that minimize a loss function. The loss function can be used as an indicator for determining optimal model parameters during the artificial neural network training process.

[0034] Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning depending on the learning method.

[0035] Supervised learning refers to a method for training an artificial neural network given labels for training data. Labels can refer to the correct answer (or output) that the artificial neural network must infer when the training data is input to the artificial neural network. Unsupervised learning refers to a method for training an artificial neural network without given labels for the training data. Reinforcement learning refers to a learning method in which an agent defined within a given environment is trained to select actions or action sequences that maximize cumulative rewards in each state.

[0036] Among artificial neural networks, machine learning implemented with a deep neural network (DNN) containing multiple hidden layers is sometimes called deep learning, and deep learning is a subset of machine learning. Hereinafter, "machine learning" is used to encompass deep learning.

[0037] Robot

[0038] A robot can be defined as a machine that automatically performs or operates a given task based on its own capabilities. Specifically, a robot capable of perceiving its environment, making independent judgments, and performing actions can be called an intelligent robot.

[0039] Robots can be classified into industrial, medical, household, and military types depending on their purpose or field of use.

[0040] Robots are equipped with actuators or motors, enabling them to perform various physical actions, such as moving robot joints. Furthermore, mobile robots include wheels, brakes, propellers, and other actuators within their actuators, enabling them to move on the ground or fly in the air.

[0041] Self-Driving

[0042] Autonomous driving refers to the technology of driving on its own, and an autonomous vehicle refers to a vehicle that drives without user intervention or with minimal user intervention.

[0043] For example, autonomous driving can include technologies that maintain the driving lane, technologies that automatically adjust speed such as adaptive cruise control, technologies that automatically drive along a set route, and technologies that automatically set a route and drive when a destination is set.

[0044] Vehicles include vehicles equipped only with internal combustion engines, hybrid vehicles equipped with both internal combustion engines and electric motors, and electric vehicles equipped only with electric motors, and may include not only automobiles but also trains, motorcycles, etc.

[0045] At this time, autonomous vehicles can be viewed as robots with autonomous driving functions.

[0046] Extended Reality (XR)

[0047] Extended reality is a general term for virtual reality (VR), augmented reality (AR), and mixed reality (MR). VR technology presents real-world objects and backgrounds as CG images only, AR technology presents virtual CG images over images of real objects, and MR technology is a computer graphics technology that blends and combines virtual objects with the real world.

[0048] MR technology is similar to AR in that it presents both real and virtual objects simultaneously. However, while AR uses virtual objects to complement real objects, MR uses virtual and real objects on an equal footing.

[0049] XR technology can be applied to HMD (Head-Mount Display), HUD (Head-Up Display), mobile phones, tablet PCs, laptops, desktops, TVs, digital signage, etc., and devices to which XR technology is applied can be called XR devices.

[0050] Figure 1 illustrates an AI device (100) according to one embodiment of the present disclosure.

[0051] The AI ​​device (100) can be implemented as a fixed device or a movable device, such as a TV, a projector, a mobile phone, a smart phone, a desktop computer, a laptop, a digital broadcasting terminal, a PDA (personal digital assistant), a PMP (portable multimedia player), a navigation device, a tablet PC, a wearable device, a set-top box (STB), a DMB receiver, a radio, a washing machine, a refrigerator, a desktop computer, digital signage, a robot, a vehicle, etc.

[0052] Referring to FIG. 1, the AI ​​device (100) may include a communication unit (110), an input unit (120), a learning processor (130), a sensing unit (140), an output unit (150), a memory (170), and a processor (180).

[0053] The communication unit (110) can transmit and receive data with external devices such as other AI devices (100a to 100e) or AI servers (200) using wired or wireless communication technology. For example, the communication unit (110) can transmit and receive sensor information, user input, learning models, control signals, etc. with external devices.

[0054] At this time, the communication technologies used by the communication unit (110) include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Bluetooth (Bluetooth), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc.

[0055] The input unit (120) can obtain various types of data.

[0056] At this time, the input unit (120) may include a camera for inputting a video signal, a microphone for receiving an audio signal, a user input unit for receiving information from a user, etc. Here, the camera or microphone may be treated as a sensor, and a signal obtained from the camera or microphone may be referred to as sensing data or sensor information.

[0057] The input unit (120) can obtain input data to be used when obtaining output using learning data and learning models for model learning. The input unit (120) can also obtain unprocessed input data, in which case the processor (180) or learning processor (130) can extract input features as preprocessing for the input data.

[0058] The learning processor (130) can train a model composed of an artificial neural network using learning data. Here, the trained artificial neural network may be referred to as a learning model. The learning model can be used to infer result values ​​for new input data other than the learning data, and the inferred values ​​can be used as a basis for making decisions regarding certain actions.

[0059] At this time, the running processor (130) can perform AI processing together with the running processor (240) of the AI ​​server (200) of FIG. 2.

[0060] At this time, the running processor (130) may include a memory integrated or implemented in the AI ​​device (100). Alternatively, the running processor (130) may be implemented using a memory (170), an external memory directly coupled to the AI ​​device (100), or a memory maintained in an external device.

[0061] The sensing unit (140) can obtain at least one of internal information of the AI ​​device (100), information about the surrounding environment of the AI ​​device (100), and user information using various sensors.

[0062] At this time, the sensors included in the sensing unit (140) include a proximity sensor, a light sensor, an acceleration sensor, a magnetic sensor, a gyro sensor, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, a light sensor, a microphone, a lidar, a radar, etc.

[0063] The output unit (150) can generate output related to vision, hearing, or touch.

[0064] At this time, the output unit (150) may include a display unit that outputs visual information, a speaker that outputs auditory information, a haptic module that outputs tactile information, etc.

[0065] The memory (170) can store data that supports various functions of the AI ​​device (100). For example, the memory (170) can store input data, learning data, learning models, learning history, etc. obtained from the input unit (120).

[0066] The processor (180) may determine at least one executable operation of the AI ​​device (100) based on information determined or generated using a data analysis algorithm or a machine learning algorithm. Then, the processor (180) may control components of the AI ​​device (100) to perform the determined operation.

[0067] To this end, the processor (180) may request, search, receive, or utilize data from the running processor (130) or memory (170), and control components of the AI ​​device (100) to execute at least one of the executable operations, a predicted operation, or an operation determined to be desirable.

[0068] At this time, if connection of an external device is required to perform a determined operation, the processor (180) can generate a control signal for controlling the external device and transmit the generated control signal to the external device.

[0069] The processor (180) can obtain intent information for user input and determine the user's requirement based on the obtained intent information.

[0070] At this time, the processor (180) can obtain intent information corresponding to the user input by using at least one of an STT (Speech To Text) engine for converting voice input into a string or a natural language processing (NLP) engine for obtaining intent information of natural language.

[0071] At this time, at least one of the STT engine or the NLP engine may be configured with an artificial neural network, at least in part, trained according to a machine learning algorithm. Furthermore, at least one of the STT engine or the NLP engine may be trained by a learning processor (130), a learning processor (240) of an AI server (200), or a distributed processing thereof.

[0072] The processor (180) can collect history information including the operation details of the AI ​​device (100) or the user's feedback on the operation, and store the information in the memory (170) or the learning processor (130), or transmit the information to an external device such as an AI server (200). The collected history information can be used to update the learning model.

[0073] The processor (180) can control at least some of the components of the AI ​​device (100) to drive an application program stored in the memory (170). Furthermore, the processor (180) can operate two or more of the components included in the AI ​​device (100) in combination to drive the application program.

[0074] Figure 2 illustrates an AI server (200) according to one embodiment of the present disclosure.

[0075] Referring to FIG. 2, the AI ​​server (200) may refer to a device that trains an artificial neural network using a machine learning algorithm or utilizes a trained artificial neural network. Here, the AI ​​server (200) may be composed of multiple servers to perform distributed processing, and may be defined as a 5G network. In this case, the AI ​​server (200) may be included as a part of the AI ​​device (100) and may perform at least a portion of the AI ​​processing.

[0076] The AI ​​server (200) may include a communication unit (210), memory (230), a learning processor (240), and a processor (260).

[0077] The communication unit (210) can transmit and receive data with an external device such as an AI device (100).

[0078] The memory (230) may include a model storage unit (231). The model storage unit (231) may store a model (or artificial neural network, 231a) that is being learned or has been learned through the learning processor (240).

[0079] The learning processor (240) can train an artificial neural network (231a) using learning data. The learning model can be used while mounted on the AI ​​server (200) of the artificial neural network, or can be mounted on an external device such as an AI device (100).

[0080] The learning model may be implemented in hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory (230).

[0081] The processor (260) can infer a result value for new input data using a learning model and generate a response or control command based on the inferred result value.

[0082] Figure 3 shows an AI system (1) according to one embodiment of the present invention.

[0083] Referring to FIG. 3, an AI system (1) is connected to a cloud network (10) by at least one of an AI server (200), a robot (100a), an autonomous vehicle (100b), an XR device (100c), a smartphone (100d), or an appliance (100e). Here, a robot (100a), an autonomous vehicle (100b), an XR device (100c), a smartphone (100d), or an appliance (100e) to which AI technology is applied may be referred to as an AI device (100a to 100e).

[0084] A cloud network (10) may refer to a network that constitutes part of a cloud computing infrastructure or exists within a cloud computing infrastructure. Here, the cloud network (10) may be configured using a 3G network, a 4G or LTE (Long Term Evolution) network, a 5G network, etc.

[0085] That is, each device (100a to 100e, 200) constituting the AI ​​system (1) can be connected to each other via a cloud network (10). In particular, each device (100a to 100e, 200) can communicate with each other via a base station, but can also communicate with each other directly without going through a base station.

[0086] The AI ​​server (200) may include a server that performs AI processing and a server that performs operations on big data.

[0087] The AI ​​server (200) is connected to at least one of the AI ​​devices constituting the AI ​​system (1), such as a robot (100a), an autonomous vehicle (100b), an XR device (100c), a smartphone (100d), or a home appliance (100e), through a cloud network (10), and can assist at least part of the AI ​​processing of the connected AI devices (100a to 100e).

[0088] At this time, the AI ​​server (200) can train an artificial neural network according to a machine learning algorithm on behalf of the AI ​​devices (100a to 100e), and can directly store the learning model or transmit it to the AI ​​devices (100a to 100e).

[0089] At this time, the AI ​​server (200) can receive input data from the AI ​​devices (100a to 100e), infer a result value for the received input data using a learning model, and generate a response or control command based on the inferred result value and transmit it to the AI ​​devices (100a to 100e).

[0090] Alternatively, the AI ​​device (100a to 100e) may infer a result value for input data using a direct learning model and generate a response or control command based on the inferred result value.

[0091] Below, various embodiments of AI devices (100a to 100e) to which the above-described technology is applied are described. Here, the AI ​​devices (100a to 100e) illustrated in FIG. 3 can be viewed as specific embodiments of the AI ​​device (100) illustrated in FIG. 1.

[0092] <AI+로봇>

[0093] The robot (100a) can be implemented as a guide robot, transport robot, cleaning robot, wearable robot, entertainment robot, pet robot, unmanned flying robot, etc. by applying AI technology.

[0094] The robot (100a) may include a robot control module for controlling movement, and the robot control module may mean a software module or a chip that implements the same in hardware.

[0095] The robot (100a) can obtain status information of the robot (100a), detect (recognize) the surrounding environment and objects, generate map data, determine a movement path and driving plan, determine a response to user interaction, or determine an action using sensor information obtained from various types of sensors.

[0096] Here, the robot (100a) can use sensor information acquired from at least one sensor among lidar, radar, and camera to determine a movement path and driving plan.

[0097] The robot (100a) can perform the above-described operations using a learning model comprised of at least one artificial neural network. For example, the robot (100a) can recognize its surroundings and objects using the learning model, and determine operations using the recognized surrounding environment information or object information. Here, the learning model may be learned directly by the robot (100a) or by an external device such as an AI server (200).

[0098] At this time, the robot (100a) may perform an action by generating a result using a direct learning model, but may also perform an action by transmitting sensor information to an external device such as an AI server (200) and receiving the result generated accordingly.

[0099] The robot (100a) can determine a movement path and a driving plan using at least one of map data, object information detected from sensor information, or object information acquired from an external device, and control a driving unit to drive the robot (100a) according to the determined movement path and driving plan.

[0100] Map data may include object identification information for various objects positioned in the space where the robot (100a) moves. For example, map data may include object identification information for fixed objects such as walls and doors, as well as movable objects such as flower pots and desks. Furthermore, object identification information may include name, type, distance, location, etc.

[0101] Additionally, the robot (100a) can perform actions or drive by controlling the driving unit based on the user's control / interaction. At this time, the robot (100a) can acquire intention information regarding the interaction based on the user's actions or voice utterances, and determine a response based on the acquired intention information to perform the action.

[0102] <AI+자율주행>

[0103] An autonomous vehicle (100b) can be implemented as a mobile robot, vehicle, or unmanned aerial vehicle by applying AI technology.

[0104] The autonomous vehicle (100b) may include an autonomous driving control module for controlling autonomous driving functions. The autonomous driving control module may refer to a software module or a chip implementing the same as hardware. The autonomous driving control module may be included internally as a component of the autonomous vehicle (100b), but may also be configured as separate hardware and connected to the exterior of the autonomous vehicle (100b).

[0105] An autonomous vehicle (100b) can obtain status information of the autonomous vehicle (100b), detect (recognize) the surrounding environment and objects, generate map data, determine a movement path and driving plan, or determine an action using sensor information obtained from various types of sensors.

[0106] Here, the autonomous vehicle (100b) can use sensor information acquired from at least one sensor among lidar, radar, and camera, similar to the robot (100a), to determine a movement path and driving plan.

[0107] In particular, the autonomous vehicle (100b) can recognize the environment or objects in an area where the field of view is obstructed or an area beyond a certain distance by receiving sensor information from external devices, or can receive information recognized directly from external devices.

[0108] The autonomous vehicle (100b) can perform the above-described operations using a learning model comprised of at least one artificial neural network. For example, the autonomous vehicle (100b) can recognize its surroundings and objects using the learning model, and determine a driving route using the recognized surrounding environment information or object information. Here, the learning model may be learned directly by the autonomous vehicle (100b) or by an external device such as an AI server (200).

[0109] At this time, the autonomous vehicle (100b) may perform an action by generating a result using a direct learning model, but may also perform an action by transmitting sensor information to an external device such as an AI server (200) and receiving the result generated accordingly.

[0110] An autonomous vehicle (100b) can determine a movement path and a driving plan using at least one of map data, object information detected from sensor information, or object information acquired from an external device, and control a driving unit to drive the autonomous vehicle (100b) according to the determined movement path and driving plan.

[0111] Map data may include object identification information for various objects located in the space (e.g., a road) where the autonomous vehicle (100b) travels. For example, map data may include object identification information for fixed objects such as streetlights, rocks, and buildings, as well as movable objects such as vehicles and pedestrians. Furthermore, object identification information may include name, type, distance, location, and the like.

[0112] Additionally, the autonomous vehicle (100b) can perform actions or drive by controlling the driving unit based on the user's control / interaction. At this time, the autonomous vehicle (100b) can acquire intention information regarding the interaction based on the user's actions or voice utterances, and determine a response based on the acquired intention information to perform the action.

[0113] <AI+XR>

[0114] The XR device (100c) can be implemented as an HMD (Head-Mount Display), a HUD (Head-Up Display) installed in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a home appliance, digital signage, a vehicle, a fixed robot, or a mobile robot by applying AI technology.

[0115] The XR device (100c) can obtain information about surrounding space or real objects by analyzing 3D point cloud data or image data acquired through various sensors or from an external device to generate location data and attribute data for 3D points, and can render and output an XR object to be output. For example, the XR device (100c) can output an XR object including additional information about a recognized object in correspondence with the recognized object.

[0116] The XR device (100c) can perform the above-described operations using a learning model composed of at least one artificial neural network. For example, the XR device (100c) can recognize a real-world object from 3D point cloud data or image data using the learning model, and provide information corresponding to the recognized real-world object. Here, the learning model may be learned directly in the XR device (100c) or learned from an external device such as an AI server (200).

[0117] At this time, the XR device (100c) may perform an operation by generating a result using a direct learning model, but may also perform an operation by transmitting sensor information to an external device such as an AI server (200) and receiving the result generated accordingly.

[0118] <AI+로봇+자율주행>

[0119] The robot (100a) can be implemented as a guide robot, transport robot, cleaning robot, wearable robot, entertainment robot, pet robot, unmanned flying robot, etc. by applying AI technology and autonomous driving technology.

[0120] A robot (100a) to which AI technology and autonomous driving technology are applied may refer to a robot itself with autonomous driving function, or a robot (100a) that interacts with an autonomous vehicle (100b).

[0121] A robot (100a) with an autonomous driving function can be a general term for devices that move on their own along a given path without user control or move by determining the path on their own.

[0122] A robot (100a) with autonomous driving capabilities and an autonomous vehicle (100b) may use a common sensing method to determine one or more of a movement path or a driving plan. For example, a robot (100a) with autonomous driving capabilities and an autonomous vehicle (100b) may use information sensed via lidar, radar, and cameras to determine one or more of a movement path or a driving plan.

[0123] A robot (100a) interacting with an autonomous vehicle (100b) may exist separately from the autonomous vehicle (100b), and may be linked to autonomous driving functions within the autonomous vehicle (100b) or perform actions linked to a user riding in the autonomous vehicle (100b).

[0124] At this time, the robot (100a) interacting with the autonomous vehicle (100b) can control or assist the autonomous driving function of the autonomous vehicle (100b) by acquiring sensor information on behalf of the autonomous vehicle (100b) and providing it to the autonomous vehicle (100b), or by acquiring sensor information and generating surrounding environment information or object information and providing it to the autonomous vehicle (100b).

[0125] Alternatively, a robot (100a) interacting with an autonomous vehicle (100b) may monitor a user riding in the autonomous vehicle (100b) or control functions of the autonomous vehicle (100b) through interaction with the user. For example, if the robot (100a) determines that the driver is drowsy, it may activate the autonomous driving function of the autonomous vehicle (100b) or assist in controlling the driving unit of the autonomous vehicle (100b). Here, the functions of the autonomous vehicle (100b) controlled by the robot (100a) may include not only the autonomous driving function, but also functions provided by a navigation system or audio system installed inside the autonomous vehicle (100b).

[0126] Alternatively, a robot (100a) interacting with an autonomous vehicle (100b) may provide information to the autonomous vehicle (100b) or assist functions from outside the autonomous vehicle (100b). For example, the robot (100a) may provide traffic information, including signal information, to the autonomous vehicle (100b), such as a smart traffic light, or may interact with the autonomous vehicle (100b) to automatically connect an electric charger to a charging port, such as an automatic electric charger for an electric vehicle.

[0127] <AI+로봇+XR>

[0128] The robot (100a) can be implemented as a guide robot, transport robot, cleaning robot, wearable robot, entertainment robot, pet robot, unmanned flying robot, drone, etc. by applying AI technology and XR technology.

[0129] A robot (100a) to which XR technology is applied may refer to a robot that is the subject of control / interaction within an XR image. In this case, the robot (100a) is distinct from the XR device (100c) and can be linked with each other.

[0130] When a robot (100a) that is the target of control / interaction within an XR image obtains sensor information from sensors including a camera, the robot (100a) or the XR device (100c) can generate an XR image based on the sensor information, and the XR device (100c) can output the generated XR image. In addition, the robot (100a) can operate based on a control signal input through the XR device (100c) or a user's interaction.

[0131] For example, a user can check an XR image corresponding to the viewpoint of a remotely connected robot (100a) through an external device such as an XR device (100c), and through interaction, adjust the autonomous driving path of the robot (100a), control the operation or driving, or check information on surrounding objects.

[0132] <AI+자율주행+XR>

[0133] Autonomous vehicles (100b) can be implemented as mobile robots, vehicles, unmanned aerial vehicles, etc. by applying AI technology and XR technology.

[0134] An autonomous vehicle (100b) to which XR technology is applied may refer to an autonomous vehicle equipped with a means for providing XR images, an autonomous vehicle that is the subject of control / interaction within an XR image, etc. In particular, an autonomous vehicle (100b) that is the subject of control / interaction within an XR image is distinct from an XR device (100c) and can be linked with each other.

[0135] An autonomous vehicle (100b) equipped with a means for providing XR images can acquire sensor information from sensors including cameras and output XR images generated based on the acquired sensor information. For example, the autonomous vehicle (100b) can be equipped with a HUD to output XR images, thereby providing passengers with XR objects corresponding to real objects or objects on the screen.

[0136] At this time, when the XR object is output to the HUD, at least a part of the XR object may be output so as to overlap with an actual object toward which the passenger's gaze is directed. On the other hand, when the XR object is output to a display provided inside the autonomous vehicle (100b), at least a part of the XR object may be output so as to overlap with an object on the screen. For example, the autonomous vehicle (100b) may output XR objects corresponding to objects such as a road, another vehicle, a traffic light, a traffic sign, a two-wheeled vehicle, a pedestrian, a building, etc.

[0137] When an autonomous vehicle (100b) that is the target of control / interaction within an XR image obtains sensor information from sensors including a camera, the autonomous vehicle (100b) or the XR device (100c) can generate an XR image based on the sensor information, and the XR device (100c) can output the generated XR image. In addition, the autonomous vehicle (100b) can operate based on a control signal input through an external device such as the XR device (100c) or a user's interaction.

[0138] FIG. 4 is a drawing for explaining the operation of a personalized health management device according to one embodiment of the present disclosure.

[0139] As illustrated in FIG. 4, the personalized health management device (500) of the present disclosure can be implemented as a fixed device or a movable device, such as a TV, a projector, a mobile phone, a smart phone, a desktop computer, a laptop, a digital broadcasting terminal, a PDA (personal digital assistant), a PMP (portable multimedia player), a navigation device, a tablet PC, a wearable device, a set-top box (STB), a DMB receiver, a radio, a washing machine, a refrigerator, a desktop computer, digital signage, a robot, a vehicle, etc.

[0140] Here, the personalized health management device (500) of the present disclosure can be connected to at least one external device (600) by wire or wireless communication for measuring the biometric information of sweat flowing from the body of the user (700) and the heart rate information of the user (700).

[0141] For example, the external device (600) may include a first device (610) including at least one first sensor for measuring biometric information of sweat flowing from the body of a user (700), and a second device (620) including at least one second sensor for measuring heart rate information of the user (700).

[0142] In addition, the personalized health management device (500) of the present disclosure may be connected to an external server (800) by wire or wireless communication to additionally obtain body information of the user (700) or additionally obtain external environmental information.

[0143] For example, the external server (800) may provide information requested from a personalized health management device (500), store user health information received from the personalized health management device (500) in a database, and provide guide information and recommendation information for personal health management for each user.

[0144] The personalized health management device (500) of the present disclosure acquires the user's (700) body information and the sweat and heart rate information of the user (700) from an external device (600), extracts first input data for predicting the user's (700) blood sugar level from the acquired sweat and heart rate information, extracts second input data for predicting the user's (700) blood sugar level from the user's (700) body information, inputs the first input data and the second input data into a pre-learned neural network model to output the user's (700) blood sugar prediction result value, generates the user's (700) health information based on the user's (700) blood sugar prediction result value, and displays the user's (700) health information.

[0145] Here, the personalized health management device (500) of the present disclosure can request body information of a user (700) when a user input for selecting a health management app is received, and can store body information for each user when body information of the user (700) is input.

[0146] In addition, the personalized health management device (500) of the present disclosure, when obtaining sweat and heart rate information, requests a communication connection to an external device (600), and when the communication connection with the external device (600) is established, the user's sweat and heart rate information can be obtained from the external device (600).

[0147] In addition, the personalized health management device (500) of the present disclosure, when acquiring sweat and heart rate information of a user (700), can acquire sweat and heart rate information of the user (700) at a certain time interval from the time when a certain time has arrived from the time when the user (700) started exercising.

[0148] Next, the personalized health management device (500) of the present disclosure can extract first input data including heart rate-related data and sweat per unit measurement value-related data from the acquired sweat and heart rate information when extracting the first input data.

[0149] For example, heart rate-related data may include at least one of a resting heart rate (RHR), a heart rate at the first hour after the start of the user's exercise, and an average heart rate obtained by subtracting the heart rate at the second hour after the start of the user's exercise and the heart rate at the third hour.

[0150] As another example, the data related to the sweat measurement value may include at least one of the sweat measurement value at the fourth hour after the start of the user's exercise, and the sweat measurement value change amount obtained by subtracting the sweat measurement value at the fifth hour after the start of the user's exercise and the sweat measurement value at the sixth hour.

[0151] As another example, the sweat measurement related data may include at least one of a sweat measurement at the fourth hour after the user started his first exercise, a sweat measurement change amount obtained by subtracting the sweat measurement at the fifth hour after the user started his first exercise from the sweat measurement at the sixth hour, and a second sweat measurement measured after the user started his second exercise.

[0152] Next, the personalized health management device (500) of the present disclosure can extract second input data including the user's gender and user's age from the user's body information when extracting the second input data.

[0153] In some cases, the personalized health management device (500) of the present disclosure may generate an average predicted blood sugar value corresponding to the user's gender and age, and further include the generated average predicted blood sugar value in the second input data.

[0154] For example, the personalized health management device (500) of the present disclosure, when the second input data is extracted, obtains national health and nutrition survey data corresponding to the user's gender and age from an external server (800) based on the second input data, inputs the national health and nutrition survey data corresponding to the user's gender and age into a pre-learned second neural network model, and generates an average predicted blood sugar value corresponding to the user's gender and age, and may further include the generated average predicted blood sugar value in the second input data.

[0155] In addition, the personalized health management device (500) of the present disclosure can obtain information on the surrounding environment where the user is located from at least one of an external server (800) and an external device when the first input data and the second input data are extracted, extract third input data for predicting the user's blood sugar level from the obtained surrounding environment information, and input the extracted third input data into a pre-learned neural network model together with the first and second input data to output a result value of predicting the user's blood sugar level.

[0156] Next, the personalized health management device (500) of the present disclosure, when outputting a blood sugar prediction result value of a user (700), may input first input data including at least one of a resting heart rate (RHR), a heart rate at a first time point after the start of user exercise, an average heart rate obtained by subtracting a heart rate at a second time point after the start of user exercise from a heart rate at a third time point, a sweat glucose measurement value at a fourth time point after the start of user exercise, a sweat glucose measurement value change obtained by subtracting a sweat glucose measurement value at a fifth time point after the start of user exercise from a sweat glucose measurement value at a sixth time point, and an average predicted blood sugar value based on data from the National Health and Nutrition Examination Survey, and second input data including the gender and age of the user into the pre-learned neural network model to output a blood sugar prediction result value of the user (700).

[0157] Here, the personalized health management device (500) of the present disclosure extracts a factor among the factors included in the first input data whose change amount compared to the previous input data is greater than a preset reference change amount, assigns a weight corresponding to the change amount to the extracted factor, and inputs the first input data including the weighted factor into a pre-learned neural network model.

[0158] In addition, the personalized health management device (500) of the present disclosure may, when the user's blood sugar prediction result value is output, feed back the user's blood sugar prediction result value as an input value of a pre-learned neural network model, thereby inputting the user's blood sugar prediction result value into the pre-learned neural network model together with the first and second input data.

[0159] Next, the personalized health management device (500) of the present disclosure, when generating health information of a user (700), can check whether the blood sugar prediction result value of the user (700) belongs to any one of a plurality of preset blood sugar evaluation intervals, and if the blood sugar prediction result value of the user (700) belongs to a specific blood sugar evaluation interval, can display health information including the blood sugar prediction result value of the user (700) and the specific blood sugar evaluation interval to which the blood sugar prediction result value of the user (700) belongs.

[0160] For example, the personalized health management device (500) of the present disclosure can check whether the blood sugar prediction result value of the user (700) belongs to any one of the normal, caution, boundary, and risk sections when the preset multiple blood sugar evaluation sections include a normal section, a caution section, a boundary section, and a risk section.

[0161] In addition, the personalized health management device (500) of the present disclosure can generate blood sugar management guide information corresponding to a specific blood sugar evaluation section, if the blood sugar prediction result value of the user (700) falls outside the normal section, and display the generated blood sugar management guide information together with health information.

[0162] In addition, the personalized health management device (500) of the present disclosure stores the blood sugar prediction result of the user (700) in a database, and when the blood sugar prediction result of the user (700) is accumulated for a certain period of time, the blood sugar change trend of the user (700) can be predicted based on the accumulated data.

[0163] FIG. 5 is a drawing for explaining the structure of a personalized health management device according to one embodiment of the present disclosure.

[0164] As illustrated in FIG. 5, the present disclosure may include a communication unit (510) that is connected to an external device, a display unit (520) that displays health information of the user, and a processor (540) that predicts the blood sugar level of the user.

[0165] In addition, the present disclosure may further include a memory (530) that stores a pre-learned neural network model (590).

[0166] Here, the neural network model (590) may be a deep neural network (DNN) that includes multiple hidden layers in addition to an input layer and an output layer.

[0167] As an example, the neural network model (590) may include a pre-trained linear regression model based on first input data for predicting the user's blood sugar level from the user's sweat and heart rate information and second input data for predicting the user's blood sugar level from the user's body information, but this is only one embodiment and is not limited thereto.

[0168] Next, the communication unit (510) can be connected to at least one external device, either wired or wireless, that measures the user's biometric information of sweat flowing from the user's body and the user's heart rate information.

[0169] Here, the external device may include at least one sensor attached to the user's body to measure biometric information and heart rate information of sweat flowing from the user's body.

[0170] For example, the external device may include a first device including at least one first sensor for measuring biometric information of sweat flowing from the user's body, and a second device including at least one second sensor for measuring heart rate information of the user.

[0171] Next, the processor (540) may obtain the user's body information and the user's sweat and heart rate information from an external device, extract first input data for predicting the user's blood sugar level from the obtained sweat and heart rate information, extract second input data for predicting the user's blood sugar level from the user's body information, input the first input data and the second input data into a pre-learned neural network model to output the user's blood sugar level prediction result value, generate the user's health information based on the user's blood sugar level prediction result value, and control the display unit (520) to display the user's health information.

[0172] Here, the processor (540) can request a communication connection to an external device, and when a communication connection is established with the external device, the processor can obtain the user's sweat and heart rate information from the external device.

[0173] In some cases, when obtaining sweat and heart rate information, the processor (540) may request the user's body information when a user input is received for selecting a health management app displayed on the display unit (520), store the body information for each user when the user's body information is input, request a communication connection to an external device, and obtain the user's sweat and heart rate information from the external device when a communication connection is established with the external device.

[0174] For example, when storing the user's body information, the processor (540) can check whether the body information includes main information including the user's gender and age when the user's body information is input, and if the body information includes the main information, the processor can store the body information for each user.

[0175] Here, the processor (540) can generate and provide a notification requesting main information including gender and age if the main information is not included in the body information.

[0176] For example, the processor (540) may provide a notification in at least one of a first method of generating a notification as a pop-up message in text format and providing it through the display unit (520) and a second method of generating the notification as a voice message in sound format and providing it through a speaker.

[0177] In some cases, the processor (540) may check whether the body information includes sub-information including the user's weight and height when the body information includes main information, and may store the body information for each user when the body information includes sub-information.

[0178] Here, the processor (540) can request the user's sub-information from an external server if the sub-information is not included in the body information, and obtain the user's sub-information from the external server.

[0179] For example, when requesting sub-information of a user, the processor (540) may request additional information including body mass index, skeletal muscle mass, and fat mass in addition to sub-information including the user's weight and height, and may store the body information for each user if the additional information is included in the body information.

[0180] In another case, when requesting additional information from the user, the processor (540) may request additional information from the external server if the additional information is not included in the body information, and may obtain the additional information from the external server.

[0181] Next, when the processor (540) obtains the user's sweat and heart rate information, it controls the display unit (520) to display information of the external device to which communication is connected when a communication connection is established with the external device, checks whether the external device is activated, and when the external device is activated, it can obtain the user's sweat and heart rate information from the external device.

[0182] Here, the processor (540) can generate and provide a notification requesting activation of the external device when the external device is deactivated.

[0183] For example, the processor (540) may provide a notification in at least one of a first method of generating a notification as a pop-up message in text format and providing it through the display unit (520) and a second method of generating the notification as a voice message in sound format and providing it through a speaker.

[0184] In some cases, when the processor (540) obtains the user's sweat and heart rate information, if there are multiple external devices connected to the external device, the processor (540) controls the display unit (520) to display a list of external devices including multiple external devices, and when a user input for selecting at least one external device from the list of external devices is received, the processor (540) can obtain the user's sweat and heart rate information from the external device if the external device is activated.

[0185] Next, when the processor (540) obtains the user's sweat and heart rate information, it may obtain the user's sweat and heart rate information at a certain time interval from the time when the user starts exercising until a certain time has passed.

[0186] For example, the processor (540) can acquire the user's sweat and heart rate information at 5-minute intervals starting 20 minutes after the user starts exercising.

[0187] And, when extracting the first input data, the processor (540) can extract the first input data including heart rate-related data and sweat per unit measurement-related data from the acquired sweat and heart rate information.

[0188] Here, the heart rate-related data may include at least one of a resting heart rate (RHR), a heart rate at the first time point after the start of the user's exercise, and an average heart rate obtained by subtracting the heart rate at the second time point after the start of the user's exercise and the heart rate at the third time point.

[0189] For example, heart rate-related data may include at least one of a resting heart rate (RHR), a heart rate at 20 minutes after the user starts exercising, and an average heart rate obtained by subtracting the heart rate at 5 minutes and the heart rate at 10 minutes after the user starts exercising.

[0190] Additionally, the data related to the sweat measurement value may include at least one of the sweat measurement value at the fourth hour after the start of the user's exercise, and the sweat measurement value change amount obtained by subtracting the sweat measurement value at the fifth hour after the start of the user's exercise and the sweat measurement value at the sixth hour.

[0191] For example, the data related to the sweat measurement value may include at least one of the sweat measurement value at 20 minutes after the start of the user's exercise, and the change in the sweat measurement value obtained by subtracting the sweat measurement value at 20 minutes after the start of the user's exercise from the sweat measurement value at 15 minutes.

[0192] In some cases, the sweat measurement related data may include at least one of a sweat measurement at the fourth hour after the user started their first exercise, a sweat measurement change at the fifth hour after the user started their first exercise minus a sweat measurement at the sixth hour, and a second sweat measurement measured after the user started their second exercise.

[0193] For example, the data related to the sweat measurement may include at least one of a sweat measurement at 20 minutes after the user begins his or her first exercise, a change in the sweat measurement obtained by subtracting the sweat measurement at 20 minutes and the sweat measurement at 15 minutes after the user begins his or her first exercise, and a second sweat measurement measured regardless of the measurement time point after the user begins his or her second exercise.

[0194] Next, when extracting the second input data, the processor (540) can extract the second input data including the user gender and user age from the user's body information.

[0195] In some cases, the processor (540) may generate an average predicted blood sugar value corresponding to the user's gender and age, and further include the generated average predicted blood sugar value in the second input data.

[0196] Here, the processor (540) obtains national health and nutrition survey data corresponding to the user's gender and age from an external server based on the second input data when the second input data is extracted, inputs the national health and nutrition survey data corresponding to the user's gender and age into a pre-learned second neural network model to generate an average predicted blood sugar value corresponding to the user's gender and age, and can further include the generated average predicted blood sugar value in the second input data.

[0197] For example, the second neural network model may include a random forest algorithm, but this is only one embodiment and is not limited thereto.

[0198] Next, the processor (540) may obtain information on the surrounding environment where the user is located from at least one of an external server and an external device when the first input data and the second input data are extracted, extract third input data for predicting the user's blood sugar level from the obtained surrounding environment information, and input the extracted third input data into a pre-learned neural network model together with the first and second input data to output a result value of predicting the user's blood sugar level.

[0199] Here, the processor (540) can extract third input data including temperature, humidity, season, and region from the surrounding environment information.

[0200] And, when outputting the user's blood sugar prediction result value, the processor (540) may input first input data including at least one of a resting heart rate (RHR), a heart rate at the first time point after the start of the user's exercise, an average heart rate obtained by subtracting the heart rate at the second time point after the start of the user's exercise from the heart rate at the third time point, a sweat glucose measurement value at the fourth time point after the start of the user's exercise, a sweat glucose measurement value change obtained by subtracting the sweat glucose measurement value at the fifth time point after the start of the user's exercise from the sweat glucose measurement value at the sixth time point, and an average predicted blood sugar value based on the National Health and Nutrition Examination Survey data, and second input data including the user's gender and age into a pre-learned neural network model to output the user's blood sugar prediction result value.

[0201] Here, the second input data may further include at least one of the user's weight, height, body mass index, skeletal muscle mass, and fat mass.

[0202] Additionally, the neural network model may include a linear regression model trained based on the first input data and the second input data.

[0203] Next, the processor (540) can extract factors among the factors included in the first input data whose change amount compared to the previous input data is greater than a preset reference change amount, assign a weight corresponding to the change amount to the extracted factor, and input the first input data including the weighted factor into a pre-learned neural network model.

[0204] And, when third input data including temperature, humidity, season, and region is extracted, the processor (540) can input the third input data into a pre-learned neural network model together with the first and second input data.

[0205] In addition, when the user's blood sugar prediction result value is output, the processor (540) can feed back the user's blood sugar prediction result value as an input value of a pre-learned neural network model, thereby inputting the user's blood sugar prediction result value into the pre-learned neural network model together with the first and second input data.

[0206] Next, when generating the user's health information, the processor (540) can check whether the user's blood sugar prediction result value belongs to any one of a plurality of preset blood sugar evaluation sections, and if the user's blood sugar prediction result value belongs to a specific blood sugar evaluation section, the processor (540) can control the display unit (520) to display health information including the user's blood sugar prediction result value and the specific blood sugar evaluation section to which the user's blood sugar prediction result value belongs.

[0207] Here, the processor (540) can check whether the user's blood sugar prediction result value belongs to any one of the normal, caution, boundary, and risk sections when the preset plurality of blood sugar evaluation sections include a normal section, a caution section, a boundary section, and a risk section.

[0208] For example, when setting multiple blood sugar evaluation sections, the processor (540) may set a blood sugar evaluation section of 139 mg / dL or less as a normal section, a blood sugar evaluation section of 140 mg / dL to 169 mg / dL as a caution section, a blood sugar evaluation section of 170 mg / dL to 199 mg / dL as a warning section, and a blood sugar evaluation section of more than 200 mg / dL as a risk section.

[0209] In addition, the processor (540) can control the display unit (520) to generate blood sugar management guide information corresponding to the specific blood sugar evaluation section when the specific blood sugar evaluation section to which the user's blood sugar prediction result belongs is outside the normal section, and to display the generated blood sugar management guide information together with health information.

[0210] Here, the blood sugar management guide information may include exercise program information and behavior correction information for the user's blood sugar management.

[0211] In addition, the processor (540) stores the user's blood sugar prediction result value in a database, and when the user's blood sugar prediction result value is accumulated over a certain period of time, the processor can predict the user's blood sugar change trend based on the accumulated data.

[0212] In this way, the present disclosure inputs input data extracted based on the user's sweat and heart rate information and body information into a pre-trained neural network model and outputs the user's blood sugar prediction result value, thereby providing the user with highly accurate and reliable blood sugar prediction information and blood sugar management guide, thereby enabling the user to manage his or her health efficiently and conveniently.

[0213] FIGS. 6(a) and (b) are diagrams illustrating an external device that is communicatively connected to a personalized healthcare device according to one embodiment of the present disclosure.

[0214] As illustrated in FIGS. 6(a) and (b), the external device may include a device (610) including at least one sensor that measures biometric information of sweat flowing from the user's body.

[0215] The device (610) of Fig. 6(a) is a device that measures a single biomarker from sweat flowing out of a user's body, and can measure single biomarkers such as glucose, lactate, electrolytes, etc.

[0216] And, the device (610) of FIG. 6(b) is a device that measures multi-biomarkers from sweat flowing out of the user's body, and can simultaneously measure multi-biomarkers such as glucose, lactate, electrolytes, etc.

[0217] The device (610) illustrated in FIGS. 6(a) and (b) is an electrochemical sensor for detecting various biomarkers in sweat. The electrochemical sensor can be manufactured as a separate disposable strip and detachably attached to a module for use, or can be manufactured as an integrated body by directly forming electrodes on the module itself.

[0218] The present disclosure can provide information on changes in blood sugar concentration, changes in muscle fatigue, and dehydration during exercise of a user based on biomarkers obtained through the device (610) illustrated in FIGS. 6(a) and (b).

[0219] FIGS. 7 to 10 are diagrams for explaining a process of acquiring user body information and sweat and heart rate information of a personalized health management device according to an embodiment of the present disclosure.

[0220] As illustrated in FIG. 7, the health management device (500) of the present disclosure can receive a user input for selecting a health management app (541) displayed on a display screen (520).

[0221] Next, as illustrated in FIG. 8, when a user input for selecting a health management app (541) is received, the present disclosure displays a body information input window (542) for inputting the user's body information on a display screen (522) and requests the user's body information.

[0222] Here, the body information input window (542) may include an input field for entering the user's body information.

[0223] For example, input fields may include a user's gender input field, a user's age input field, etc.

[0224] The present disclosure can store the user's body information for each user when a user input is received in which the user's body information is entered through a body information input window (542) and an input completion button (544) is selected.

[0225] For example, the present disclosure can check whether the body information includes main information including the user's gender and age when the body information is input, and if the body information includes the main information, the body information can be stored for each user.

[0226] In addition, the present disclosure can generate and provide a notification requesting main information including gender and age when the main information is not included in the body information.

[0227] For example, the present disclosure may provide a notification in at least one of a first method of generating a notification as a pop-up message in text format and providing it through a display screen (522) and a second method of generating a notification as a voice message in sound format and providing it through a speaker.

[0228] In some cases, the present disclosure may determine whether the body information includes sub-information including the user's weight and height when the body information includes main information, and may store the body information for each user when the body information includes sub-information.

[0229] Here, the present disclosure can request the user's sub-information from an external server and obtain the user's sub-information from the external server when the sub-information is not included in the body information.

[0230] In addition, the present disclosure requests additional information including body mass index, skeletal muscle mass, and fat mass in addition to sub-information including the user's weight and height, and if the additional information is included in the body information, the body information can be stored for each user.

[0231] Here, the present disclosure can request additional information of the user from an external server and obtain additional information of the user from the external server when additional information is not included in the body information.

[0232] In this way, the reason why various body information of the user is acquired is because it is possible to consider various variables corresponding to the user's personal body characteristics, so the error of the predicted value per sweat can be minimized, and the accuracy and reliability of the user's blood sugar predicted result can be improved.

[0233] Next, as illustrated in FIG. 9, the present disclosure can request a communication connection to an external device, and when a communication connection is established with the external device, the user's sweat and heart rate information can be obtained from the external device.

[0234] The external device may include a first device (610) including at least one first sensor for measuring biometric information of sweat flowing from the body of the user (700), and a second device (620) including at least one second sensor for measuring heart rate information of the user (700).

[0235] And, the present disclosure can display information (546) of the external device connected to the communication connection on the display screen (522) when the external device is connected to the communication connection.

[0236] Next, the present disclosure can check whether an external device is activated, and if the external device is activated, the user's sweat and heart rate information can be obtained from the external device.

[0237] Here, the present disclosure can generate and provide a notification requesting activation of an external device when the external device is deactivated.

[0238] For example, the present disclosure may provide a notification in at least one of a first method of generating a notification as a pop-up message in text form and providing it on a display screen (522) and a second method of generating the notification as a voice message in sound form and providing it through a speaker.

[0239] In some cases, the present disclosure may display a list of external devices including a plurality of external devices on a display screen (522) when the external devices are connected to each other for communication.

[0240] In addition, the present disclosure determines whether the selected external device is activated when a user input for selecting at least one external device from a list of external devices is received, and when the external device is activated, the user's sweat and heart rate information can be obtained from the external device.

[0241] Next, as illustrated in FIG. 10, the present disclosure provides a guide window for requesting exercise start when an external device is activated, and when a user input is received for selecting an exercise start button (548) of the guide window, the user's sweat and heart rate information can be obtained from the external device (600).

[0242] Here, the present disclosure can obtain the user's sweat and heart rate information from an external device (600), and extract input data for predicting the user's blood sugar level from the obtained sweat and heart rate information.

[0243] FIG. 11 and FIG. 12 are diagrams for explaining a process of generating a blood sugar prediction result value of a personalized health management device according to an embodiment of the present disclosure.

[0244] As illustrated in FIG. 11, the present disclosure can obtain a user's sweat and heart rate information from an external device (571), and extract first input data for predicting the user's blood sugar level from the obtained sweat and heart rate information (581).

[0245] Here, the present disclosure can acquire the user's sweat and heart rate information at a certain time interval from the time when the user starts exercising until a certain time has passed.

[0246] In addition, the present disclosure can obtain user body information from user input (572) and extract second input data for predicting the user's blood sugar level from the obtained user body information (582).

[0247] Next, the present disclosure can input first input data and second input data into a pre-learned neural network model (590) to output a user's blood sugar prediction result value.

[0248] Here, the first input data may include heart rate related data and sweat per unit measurement related data.

[0249] For example, heart rate-related data may include at least one of a resting heart rate (RHR), a heart rate at the first hour after the start of the user's exercise, and an average heart rate obtained by subtracting the heart rate at the second hour after the start of the user's exercise and the heart rate at the third hour.

[0250] For example, the data related to the sweat measurement value may include at least one of the sweat measurement value at the fourth hour after the start of the user's exercise, and the sweat measurement value change amount obtained by subtracting the sweat measurement value at the fifth hour after the start of the user's exercise and the sweat measurement value at the sixth hour.

[0251] As another example, the sweat measurement related data may include at least one of a sweat measurement at the fourth hour after the user started his first exercise, a sweat measurement change amount obtained by subtracting the sweat measurement at the fifth hour after the user started his first exercise from the sweat measurement at the sixth hour, and a second sweat measurement measured after the user started his second exercise.

[0252] Additionally, the second input data may include user gender and user age, and may further include an average predicted blood sugar value corresponding to the user gender and age.

[0253] Here, the average predicted blood sugar value can be generated by inputting the National Health and Nutrition Survey data corresponding to the user's gender and age obtained from an external server into another pre-learned neural network model.

[0254] As illustrated in FIG. 12, the present disclosure can obtain a user's sweat and heart rate information from an external device (571), and extract first input data for predicting the user's blood sugar level from the obtained sweat and heart rate information (581).

[0255] In addition, the present disclosure can obtain user body information from user input (572) and extract second input data for predicting the user's blood sugar level from the obtained user body information (582).

[0256] Next, the present disclosure can obtain information about the surrounding environment where the user is located from at least one of an external server and an external device (573), and extract third input data for predicting the user's blood sugar level from the obtained surrounding environment information (583).

[0257] Here, the present disclosure can extract third input data including temperature, humidity, season, and region from the surrounding environment information.

[0258] Next, the present disclosure can input first input data, second input data, and third input data into a pre-learned neural network model to output a user's blood sugar prediction result value.

[0259] Here, the present disclosure may input first input data including at least one of a resting heart rate (RHR), a heart rate at a first time point after the start of a user's exercise, an average heart rate obtained by subtracting the heart rate at a second time point after the start of the user's exercise from the heart rate at a third time point, a sweat glucose measurement value at a fourth time point after the start of the user's exercise, a sweat glucose measurement value change obtained by subtracting the sweat glucose measurement value at a fifth time point after the start of the user's exercise from the sweat glucose measurement value at a sixth time point, and an average predicted blood glucose value based on data from the National Health and Nutrition Examination Survey, second input data including the user's gender and age, and third input data including temperature, humidity, season, and region into a pre-learned neural network model to output a user's blood glucose prediction result value.

[0260] FIG. 13(a) and (b) are diagrams for explaining the performance of a neural network model according to an embodiment of the present disclosure.

[0261] FIG. 13(a) is a diagram showing evaluation indices of a modeling set for a neural network model of the present disclosure, and FIG. 13(b) is a diagram showing evaluation indices of a validation set for a neural network model of the present disclosure.

[0262] As shown in Fig. 13(a), the neural network model of the present disclosure has a coefficient of determination R as a performance evaluation index applied to about 150 adults. 2 is about 0.364, the coefficient of determination R 2 We can see that adj is approximately 0.319, MAPE (Mean Absolute Percentage Error) is approximately 5.88%, and the error rate of 10% is approximately 81.3%, or 122 out of 150 people.

[0263] As shown in Fig. 13(b), the neural network model of the present disclosure is applied to about 30 adults selected from about 150 people, and the coefficient of determination R is used as a performance evaluation index. 2 is about 0.417, the coefficient of determination R 2 We can see that adj is approximately 0.19, MAPE (Mean Absolute Percentage Error) is approximately 3.93%, and the error rate of 10% is approximately 96.6%, which is 29 out of 30 people.

[0264] In this way, the neural network model of the present disclosure can improve the rate of error rate within 10% of sweat sugar prediction values ​​for pre-exercise blood sugar to approximately 96.6%.

[0265] Here, the list of predictive variables applied to the present disclosure may include at least one of the user's sex, age, resting heart rate (RHR), heart rate at 20 minutes after the user starts exercising (HR20min), average heart rate obtained by subtracting the heart rate at 5 minutes after the user starts exercising and the heart rate at 10 minutes after the user starts exercising (HRpost 5-10avg), sweat glucose measurement value at 20 minutes after the user starts exercising (SW20min), sweat glucose measurement change amount obtained by subtracting the sweat glucose measurement value at 20 minutes after the user starts exercising and the sweat glucose measurement value at 15 minutes after the user starts exercising (SW20-15delta), a second sweat glucose measurement value measured regardless of the measurement time after the user starts exercising for the second time (SW2), and an average predicted blood glucose value (BG RF) based on data from the Korea National Health and Nutrition Examination Survey.

[0266] FIG. 14 and FIG. 15 are drawings for explaining a health information generation process of a personalized health management device according to an embodiment of the present disclosure.

[0267] As illustrated in FIGS. 14 and 15, the present disclosure can determine whether a user's blood sugar prediction result value belongs to any one of a plurality of preset blood sugar evaluation intervals, and if the user's blood sugar prediction result value belongs to a specific blood sugar evaluation interval, display health information including the user's blood sugar prediction result value and the specific blood sugar evaluation interval to which the user's blood sugar prediction result value belongs.

[0268] As illustrated in FIG. 14, the present disclosure can determine whether a user's blood sugar prediction result value belongs to any one of the normal, caution, boundary, and risk sections when a plurality of preset blood sugar evaluation sections include a normal section, a caution section, a boundary section, and a risk section.

[0269] For example, the present disclosure can set a blood sugar evaluation range of about 139 mg / dL or less as a normal range and output an alarm message corresponding to maintaining lifestyle habits.

[0270] In addition, the present disclosure sets a blood sugar evaluation range of about 140 mg / dL to about 169 mg / dL as a caution range, and can output an alarm message and diet / exercise guidance information corresponding to lifestyle management.

[0271] In addition, the present disclosure sets a blood sugar evaluation range of about 170 mg / dL to about 199 mg / dL as a boundary range, and can output an alarm message and diet / exercise guidance information corresponding to active management.

[0272] In addition, the present disclosure can set a blood sugar evaluation range exceeding about 200 mg / dL as a risk range and output an alarm message corresponding to a blood sugar measurement recommendation.

[0273] In this way, the present disclosure can generate blood sugar management guide information corresponding to a specific blood sugar evaluation range when a specific blood sugar evaluation range to which a user's blood sugar prediction result falls deviates from a normal range, and display the generated blood sugar management guide information together with health information.

[0274] Here, the blood sugar management guide information may include exercise program information and behavior correction information for the user's blood sugar management.

[0275] In addition, as illustrated in FIG. 15, the personalized health management device (500) of the present disclosure can display health information (810) including the user's blood sugar prediction result value and the specific blood sugar evaluation section to which the user's blood sugar prediction result value belongs on the display screen.

[0276] In addition, the present disclosure can display blood sugar management guide information (820) corresponding to a specific blood sugar evaluation section on a display screen together with health information (810).

[0277] Here, the present disclosure stores the user's blood sugar prediction result value in a database, and when the user's blood sugar prediction result value is accumulated for a certain period of time, the user's blood sugar change trend can be predicted and displayed on a display screen based on the accumulated data.

[0278] FIG. 16 is a drawing for explaining a health management method of a personalized health management device according to one embodiment of the present disclosure.

[0279] As illustrated in FIG. 16, the present disclosure can obtain the user's body information and the user's sweat and heart rate information from an external device (S20).

[0280] Here, the present disclosure can request the user's body information when a user input for selecting a health management app is received, and when the user's body information is input, the body information can be stored for each user.

[0281] Additionally, the present disclosure can request a communication connection to an external device, and when a communication connection is established with the external device, the user's sweat and heart rate information can be obtained from the external device.

[0282] That is, the present disclosure can display information about the external device when a communication connection is made with the external device, check whether the external device is activated, and obtain the user's sweat and heart rate information from the external device when the external device is activated.

[0283] At this time, the present disclosure can acquire the user's sweat and heart rate information at a certain time interval from the time when the specific time has arrived when the user starts exercising.

[0284] Next, the present disclosure can extract first input data for predicting the user's blood sugar level from the acquired sweat and heart rate information, and can extract second input data for predicting the user's blood sugar level from the user's body information (S30).

[0285] Here, the present disclosure can extract first input data including heart rate-related data and sweat per unit measurement-related data from the acquired sweat and heart rate information.

[0286] For example, heart rate-related data may include at least one of a resting heart rate (RHR), a heart rate at the first hour after the start of the user's exercise, and an average heart rate obtained by subtracting the heart rate at the second hour after the start of the user's exercise and the heart rate at the third hour.

[0287] As another example, the data related to the sweat measurement may include at least one of the sweat measurement at the fourth hour after the start of the user's exercise, and the sweat measurement change amount obtained by subtracting the sweat measurement at the fifth hour after the start of the user's exercise from the sweat measurement at the sixth hour.

[0288] In some cases, the sweat measurement related data may include at least one of a sweat measurement at the fourth hour after the user started their first exercise, a sweat measurement change at the fifth hour after the user started their first exercise minus a sweat measurement at the sixth hour, and a second sweat measurement measured after the user started their second exercise.

[0289] Additionally, the present disclosure can extract second input data including user gender and user age from the user's body information.

[0290] Here, the second input data may further include an average predicted blood sugar value corresponding to the user's gender and age.

[0291] And, the present disclosure can output a user's blood sugar prediction result value by inputting first input data and second input data into a pre-learned neural network model (S40).

[0292] Here, the present disclosure may obtain information on the surrounding environment where the user is located from at least one of an external server and an external device when first input data and second input data are extracted, extract third input data for predicting the user's blood sugar level from the obtained surrounding environment information, and input the extracted third input data into a pre-learned neural network model together with the first and second input data to output a result value of predicting the user's blood sugar level.

[0293] In addition, the present disclosure extracts factors among factors included in first input data, the amount of change of which is greater than a preset reference amount of change compared to previous input data, assigns a weight corresponding to the amount of change to the extracted factors, and inputs the first input data including the weighted factors into a pre-learned neural network model.

[0294] In addition, the present disclosure may feed back the user's blood sugar prediction result value as an input value of a pre-trained neural network model when the user's blood sugar prediction result value is output, thereby inputting the user's blood sugar prediction result value into the pre-trained neural network model together with the first and second input data.

[0295] Next, the present disclosure can generate and display the user's health information based on the user's blood sugar prediction result value (S50).

[0296] Here, the present disclosure can determine whether the user's blood sugar prediction result value belongs to any one of a plurality of preset blood sugar evaluation ranges, and if the user's blood sugar prediction result value belongs to a specific blood sugar evaluation range, display health information including the user's blood sugar prediction result value and the specific blood sugar evaluation range to which the user's blood sugar prediction result value belongs.

[0297] In some cases, the present disclosure may generate blood sugar management guide information corresponding to a specific blood sugar evaluation range to which the user's blood sugar prediction result falls outside the normal range, and display the generated blood sugar management guide information together with the health information.

[0298] In this way, the present disclosure inputs input data extracted based on the user's sweat and heart rate information and body information into a pre-trained neural network model and outputs the user's blood sugar prediction result value, thereby providing the user with highly accurate and reliable blood sugar prediction information and blood sugar management guide, thereby enabling the user to manage his or her health efficiently and conveniently.

[0299] The above-described present disclosure can be implemented as computer-readable code on a program-recorded medium. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disk drives (SSDs), silicon disk drives (SDDs), read-only memory (ROM), random access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. In addition, the computer may include a processor (180) of an artificial intelligence device.

[0300] The personalized health management device according to the present disclosure has remarkable industrial applicability because it can provide users with highly accurate and reliable blood sugar prediction information and blood sugar management guide.

Claims

1. A communication unit that connects to an external device; A display unit that displays the user's health information; and A processor for predicting the blood sugar level of the user is included, The above processor, A personalized health management device characterized in that it acquires the user's body information and the user's sweat and heart rate information, extracts first input data for predicting the user's blood sugar level from the acquired sweat and heart rate information, extracts second input data for predicting the user's blood sugar level from the user's body information, inputs the first input data and the second input data into a pre-learned neural network model to output the user's blood sugar level prediction result value, generates the user's health information based on the user's blood sugar level prediction result value, and controls the display unit to display the user's health information.

2. In paragraph 1, The above communication department, A personalized health management device characterized in that it is connected to at least one external device, either wired or wirelessly, for measuring biometric information of sweat flowing from the user's body and heart rate information of the user.

3. In paragraph 1, The above processor, A personalized health management device characterized in that, when obtaining the sweat and heart rate information, a communication connection is requested to the external device, and when the communication connection with the external device is established, the sweat and heart rate information of the user is obtained from the external device.

4. In paragraph 3, The above processor, A personalized health management device characterized in that, when acquiring the user's sweat and heart rate information, when a specific time has arrived from the time when the user started exercising, the user's sweat and heart rate information is acquired at a specific time interval from the time when the specific time has arrived.

5. In paragraph 1, The above processor, A personalized health management device characterized in that, when extracting the first input data, the first input data including heart rate-related data and sweat per unit measurement value-related data is extracted from the acquired sweat and heart rate information.

6. In paragraph 5, The above heart rate related data is, A personalized health management device characterized by including at least one of a resting heart rate (RHR), a heart rate at a first time point after the start of a user's exercise, and an average heart rate obtained by subtracting a heart rate at a second time point after the start of a user's exercise and a heart rate at a third time point.

7. In paragraph 5, The data related to the above sweat measurement value is, A personalized health management device characterized by including at least one of a sweat measurement value at the fourth hour after the start of a user's exercise, and a sweat measurement value change amount obtained by subtracting a sweat measurement value at the fifth hour after the start of a user's exercise and a sweat measurement value at the sixth hour.

8. In paragraph 5, The data related to the above sweat measurement value is, A personalized health management device characterized in that it includes at least one of a sweat measurement value measured at the fourth hour after the user starts his first exercise, a sweat measurement change amount obtained by subtracting the sweat measurement value measured at the fifth hour after the user starts his first exercise from the sweat measurement value measured at the sixth hour, and a second sweat measurement value measured after the user starts his second exercise.

9. In paragraph 1, The above processor, A personalized health management device characterized in that, when extracting the second input data, the second input data including the user's gender and user's age are extracted from the user's body information.

10. In paragraph 9, The above processor, A personalized health management device characterized in that, when the second input data is extracted, national health and nutrition survey data corresponding to the user's gender and age is acquired from an external server based on the second input data, the national health and nutrition survey data corresponding to the user's gender and age is input into a pre-trained second neural network model to generate an average predicted blood sugar value corresponding to the user's gender and age, and the generated average predicted blood sugar value is further included in the second input data.

11. In paragraph 1, The above processor, A personalized health management device characterized in that, when the first input data and the second input data are extracted, information on the surrounding environment where the user is located is obtained from at least one of an external server and an external device, third input data for predicting the user's blood sugar level is extracted from the obtained surrounding environment information, and the extracted third input data is input into the pre-learned neural network model together with the first and second input data to output a blood sugar level prediction result value of the user.

12. In paragraph 1, The above processor, A personalized health management device characterized in that, when outputting the user's blood sugar prediction result value, the first input data including at least one of a resting heart rate (RHR), a heart rate at the first hour after the user starts exercising, an average heart rate obtained by subtracting the heart rate at the second hour after the user starts exercising from the heart rate at the third hour, a sweat glucose measurement value at the fourth hour after the user starts exercising, a sweat glucose measurement value change obtained by subtracting the sweat glucose measurement value at the fifth hour after the user starts exercising from the sweat glucose measurement value at the sixth hour, and an average predicted blood sugar value based on data from the National Health and Nutrition Examination Survey, and second input data including the user's gender and age are input into the pre-learned neural network model, thereby outputting the user's blood sugar prediction result value.

13. In paragraph 1, The above processor, A personalized health management device characterized in that, when generating health information of the user, it is determined whether the user's blood sugar prediction result value belongs to any one of a plurality of preset blood sugar evaluation intervals, and if the user's blood sugar prediction result value belongs to a specific blood sugar evaluation interval, the display unit is controlled to display health information including the user's blood sugar prediction result value and the specific blood sugar evaluation interval to which the user's blood sugar prediction result value belongs.

14. In paragraph 13, The above processor, A personalized health management device characterized in that, when a specific blood sugar evaluation section to which the user's blood sugar prediction result belongs falls outside the normal section, blood sugar management guide information corresponding to the specific blood sugar evaluation section is generated, and the display section is controlled to display the generated blood sugar management guide information together with the health information.

15. In a health management method of a personalized health management device that is connected to an external device, A step of obtaining the user's body information and the user's sweat and heart rate information from the external device; A step of extracting first input data for predicting the user's blood sugar level from the acquired sweat and heart rate information; A step of extracting second input data for predicting blood sugar level of the user from the user's body information; A step of inputting the first input data and the second input data into a pre-trained neural network model to output the user's blood sugar prediction result value; and A personalized health management method, characterized by including a step of generating and displaying health information of the user based on the blood sugar prediction result of the user.

Images