Electronic device for providing health information, operating method thereof, and computer-readable non-transitory storage medium

Abstract

Disclosed are an electronic device for providing health information related to hemoglobin measurement and an operating method thereof. The electronic device may acquire a skin temperature of a user and acquire a nail image of the user by means of a camera. The electronic device may determine a hemoglobin index on the basis of the skin temperature and the nail image, and display a user interface related to the hemoglobin index or health information by means of a display. The electronic device may determine a hemoglobin index on the basis of the nail image, and correct the hemoglobin index on the basis of the difference between a designated skin temperature and the obtained skin temperature. The electronic device may generate a corrected nail image from the nail image on the basis of the obtained skin temperature, and determine a hemoglobin index on the basis of the corrected nail image. Various other embodiments identified by the present document are possible.

Description

Electronic device providing health information, method of operation thereof, and computer-readable non-transient storage medium

[0001] The present disclosure relates to an electronic device for providing health information, a method of operating the same, and a computer-readable non-transient storage medium, and more specifically, to an electronic device for providing health information related to hemoglobin measurement, a method of operating the same, and a computer-readable non-transient storage medium.

[0002] With the recent advancement of mobile communication technology, the use of portable or mobile electronic devices (e.g., smartphones, wearable electronic devices, mobile terminals, or tablet PCs (personal computers)) has become widespread, and the services that can be provided through these electronic devices are becoming increasingly diverse. Since these electronic devices are implemented in a form that users can carry or wear and are closely integrated into their daily lives, they can be effectively utilized for various services.

[0003] For example, an electronic device may provide healthcare services that continuously monitor a user's biometric data, or data related to exercise, sleep, and / or diet, and manage health. The electronic device may acquire a user's biometric data or motion data through one or more sensors, analyze the acquired data, and store the analysis results in conjunction with an application (e.g., health application, exercise application), or provide guidance or coaching based on the analysis results.

[0004] In addition, electronic devices can provide various functions utilizing cameras or images (e.g., shooting functions, filter functions, photo editing functions, album functions, transmission and reception functions, video calling functions, or messenger functions). Various attempts are being made to improve camera performance or enhance image quality (e.g., photographs) in electronic devices.

[0005] The information described above may be provided as related art to aid in understanding the present disclosure. None of the foregoing is to be claimed as prior art related to the present disclosure, nor is it to be used to determine prior art related to the present disclosure.

[0006] An electronic device according to one embodiment of the present disclosure may include at least one processor comprising processing circuitry, a memory for storing instructions, a camera, a communication circuit, and a display. The instructions may be executed individually or collectively by the at least one processor to enable the electronic device to acquire the skin temperature of a user of the electronic device, acquire an image of the user's fingernails through the camera, determine the user's hemoglobin index based on the skin temperature and the fingernail image, and display a user interface through the display that indicates the hemoglobin index or health information related to the hemoglobin index.

[0007] A method of operation of an electronic device according to one embodiment of the present disclosure may include the operation of acquiring the skin temperature of a user of the electronic device, the operation of acquiring an image of the user's fingernail through a camera of the electronic device, the operation of determining the user's hemoglobin index based on the skin temperature and the fingernail image, and the operation of displaying a user interface that indicates the hemoglobin index or health information related to the hemoglobin index through a display of the electronic device.

[0008] A storage medium according to one embodiment of the present disclosure may be a computer-readable, non-transient storage medium. The storage medium may have at least one program recorded thereon, comprising instructions for executing a method of operating an electronic device. The storage medium may have at least one program recorded thereon for executing a method comprising: acquiring the skin temperature of a user of the electronic device; acquiring an image of the user's fingernails through a camera of the electronic device; determining the user's hemoglobin index based on the skin temperature and the fingernail image; and displaying a user interface that indicates the hemoglobin index or health information related to the hemoglobin index through a display of the electronic device.

[0009] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0010] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0011] FIG. 2 is a block diagram of an electronic device according to one embodiment.

[0012] FIG. 3a is a flowchart illustrating the operation method of an electronic device according to one embodiment.

[0013] FIGS. 3b and FIGS. 3c are flowcharts illustrating a process in which an electronic device according to one embodiment determines a hemoglobin index.

[0014] FIG. 4 is a diagram illustrating a process for determining a hemoglobin index using a corrected image of an electronic device according to one embodiment.

[0015] FIG. 5a is an example of a user interface related to a hemoglobin indicator displayed on an electronic device according to one embodiment.

[0016] FIG. 5b is a diagram illustrating the relationship between finger skin temperature and nail color according to one embodiment.

[0017] FIG. 6 is a flowchart illustrating a process in which an electronic device according to one embodiment determines a hemoglobin index using an artificial intelligence model.

[0018] FIG. 7 is a flowchart illustrating the operation method of an electronic device according to one embodiment.

[0019] FIG. 8a is a flowchart illustrating a process in which an electronic device according to one embodiment triggers a hemoglobin measurement function.

[0020] FIG. 8b is a flowchart illustrating the process of an electronic device measuring hemoglobin according to one embodiment.

[0021] FIG. 8c is a flowchart illustrating a process for adjusting personalized parameters for hemoglobin measurement using blood strip test data according to one embodiment.

[0022] FIG. 9a is an example of a user interface related to a process in which an electronic device according to one embodiment triggers a hemoglobin measurement function.

[0023] FIG. 9b is an example of a user interface related to a process in which an electronic device according to one embodiment registers a reference image for hemoglobin measurement.

[0024] FIG. 10 is a drawing illustrating a wearable electronic device according to one embodiment.

[0025] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0026] Invasive hemoglobin measurement technology measures actual hemoglobin levels through the collection and testing of blood samples, so while it offers high accuracy, it may have limitations such as poor user-friendliness due to pain and low accessibility because it requires specialized personnel and equipment. Accordingly, there may be a need for non-invasive hemoglobin measurement technology that offers relatively high user-friendliness, accessibility, and convenience.

[0027] The hemoglobin component in the blood that carries oxygen is supplied to the area under the fingernails, and since this hemoglobin is red, it may show through the nails and appear pink. Fingernails not only serve to protect the skin at the fingertips but can also reflect one's health condition.

[0028] For example, if blood circulation is good, normal fingernails may appear light pink. If hemoglobin levels in the blood drop, the nails may turn pale or white due to insufficient oxygen supply. In such cases, anemia may be suspected. Additionally, cyanosis, where nails turn blue due to inadequate oxygen supply, can be a sign of respiratory or heart disease. If nails turn yellow, liver or lung disease may be suspected.

[0029] As such, nail color can be a factor related to hemoglobin levels or health status, and methods to non-invasively monitor hemoglobin levels or health status by utilizing this relationship may be considered.

[0030] However, nail color can vary from person to person due to physical, genetic, and personal factors, and can be influenced not only by actual hemoglobin levels but also by the surrounding environment (or external environment). Therefore, estimating hemoglobin levels based on nail color may result in relatively lower measurement accuracy. Low measurement accuracy can lead to poor usability resulting from user dissatisfaction.

[0031] An electronic device and a method of operation thereof according to various embodiments of the present disclosure may be intended to improve the accuracy of non-invasive hemoglobin measurement and to ensure the reliability and consistency of said measurement.

[0032] An electronic device and its method of operation according to various embodiments of the present disclosure may be intended to provide a hemoglobin measurement result suitable for an individual user (personalized hemoglobin measurement result) by taking into account the individual characteristics of the user.

[0033] The electronic device and the method of operation thereof according to various embodiments of the present disclosure may be intended to improve the accuracy of hemoglobin measurement by excluding temporary influences caused by the surrounding environment (or external environment) or physiological state when analyzing nail color for hemoglobin measurement.

[0034] An electronic device and its method of operation according to various embodiments of the present disclosure may be intended to guide a user in real time to a suitable environment or conditions required for hemoglobin measurement and to improve the user experience.

[0035] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0036] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0037] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence is performed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0038] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, software (e.g., program (140)) and input data or output data for related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0039] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0040] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0041] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0042] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0043] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0044] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0045] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0046] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0047] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0048] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0049] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0050] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0051] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0052] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0053] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0054] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0055] At least some of the above components can be connected to each other via a communication method between peripheral area devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0056] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0057] FIG. 2 is a block diagram of an electronic device (200) according to one embodiment.

[0058] An electronic device (200) according to one embodiment may be implemented in various types. For example, the electronic device (200) may be implemented as a type that can be carried and touched (or held) (e.g., a smartphone (201) shown in FIG. 5a). For another example, the electronic device (200) may be implemented as a wearable electronic device of a type that can be worn (or attached) (e.g., a smart watch (401) shown in FIG. 5a, a wearable electronic device (1010) of the smart glass type shown in FIG. 10).

[0059] Referring to FIG. 2, an electronic device (200) according to one embodiment may include a processor (210), memory (220), camera (230), and display (240). The electronic device (200) may further include a communication module (250) and / or a sensor module (260).

[0060] According to one embodiment, the electronic device (200) of FIG. 2 may correspond to the electronic device (101) shown in FIG. 1. The processor (210), memory (220), camera (230), display (240), communication module (250), and sensor module (260) included in the electronic device (200) of FIG. 2 may correspond to the processor (120), memory (130), camera module (180), display module (160), and sensor module (176) of FIG. 1, respectively.

[0061] In some embodiments, at least one of the illustrated components of the electronic device (200) may be omitted, integrated with one another, or additionally provided. The processor (210), memory (220), camera (230), display (240), communication module (250), and sensor module (260) included in the electronic device (200) may be electrically and / or operationally connected to one another to exchange signals (e.g., commands or data) with one another.

[0062] According to one embodiment, the number of processors (210) (e.g., processor (120) of FIG. 1) may be one or more. For example, the processor (210) may have the structure of a multi-core processor such as a dual core, a quad core, or a hexa core. The processor (210) can control the operations of the electronic device (200) by executing instructions stored in memory (220). For example, the processor (210) may correspond to a plurality of processors that collectively perform a plurality of operations by dividing them among the processors.

[0063] According to one embodiment, the processor (210) (e.g., the processor (120) of FIG. 1) may include at least one processor comprising processing circuitry. The processor (210) may execute and / or control various functions supported by the electronic device (200). The processor (210) may control at least some of the memory (220), camera (230), display (240), communication module (250), and sensor module (260). The processor (210) may execute applications and control various hardware by executing code written in a programming language stored in the memory (220) of the electronic device (200). For example, the processor (210) may perform a hemoglobin measurement function and / or a function of providing health information related to hemoglobin indicators by executing instructions stored in the memory (220). For example, the processor (210) can execute an application (e.g., a health application) and use the application to perform hemoglobin measurement operations and / or health information provision operations. The application executed on the electronic device (200) can operate independently or in conjunction with an external electronic device (e.g., the electronic device (102, 104) of FIG. 1, the server (108)).

[0064] According to one embodiment, memory (220) (e.g., memory (130) of FIG. 1) may store instructions. As instructions stored in memory (220) are executed, operations of the processor (210) may be performed. For example, the processor (210) may perform hemoglobin measurement operations and / or health information provision operations by executing instructions stored in memory (220). The processor (210) may perform operations or control components of the electronic device (200) by executing instructions stored in memory (220) individually or collectively. In the present disclosure, operations of the electronic device (200) may be understood as being performed when the processor (210) executes instructions.

[0065] According to one embodiment, a camera (230) (e.g., camera module (180) of FIG. 1) can acquire an image (still image, e.g., a photograph) or a video (e.g., a video) through a shooting (or capture) operation. The capture result (e.g., an image and / or video) can be stored in memory (220). A processor (210) can control the camera (230). The processor (210) can support various functions using the camera (230). The processor (210) can store the image acquired through the camera (230) in memory (220) and display the stored image through a display (240). For example, the display (240) may display the execution screen of an application executed by the processor (210) or the image stored in memory (220).

[0066] According to one embodiment, a display (240) (e.g., the display module (160) of FIG. 1) can display content such as an execution screen of an application executed by a processor (210) or an image or video stored in memory (130) on the display (240). The processor (210) can display an image or video acquired through a camera (230) on the display (240) in real time. The processor (210) can display (or output) a visual type user interface through the display (240).

[0067] According to one embodiment, the display (240) may include at least one of an LCD (liquid crystal display), a TFT-LCD (thin film transistor LCD), an OLED (organic light emitting diodes), an LED, an AMOLED (active matrix organic LED), a flexible display, and a three-dimensional display. In some embodiments, some of these displays may be configured to be transparent or light-transmitting so that the outside can be seen through them. For example, a transparent or light-transmitting display may be configured in the form of a transparent display including a TOLED (transparent OLED).

[0068] According to one embodiment, the display (240) may be implemented as an integral part of the touch panel. The display (240) may support touch functions and may detect user input (e.g., touch) and transmit it to the processor (210). A display (240) that supports touch functions may be referred to as a touch screen. The display (240) may further include a structure capable of detecting input using a stylus pen, such as EMR (electro-magnetic resonance) or AES (active electrostatic solution).

[0069] According to one embodiment, a communication module (250) (e.g., a communication module (190) of FIG. 1) may include communication circuitry. The communication module (250) may establish a communication connection with at least one external electronic device (e.g., an electronic device (102, 104) of FIG. 1, or a server (108) of FIG. 1) and transmit and receive various data.

[0070] According to one embodiment, the communication module (250) may support a short-range wireless communication connection of the electronic device (200). For example, the communication module (250) may support a short-range wireless communication (e.g., Bluetooth, Bluetooth LE (Bluetooth Low Energy), WiFi (Wireless Fidelity), NFC (Near Field Communication), RFID (Radio-Frequency Identification), UWB (Ultra Wide Band), or IrDA (Infrared Data Association)) connection between the electronic device (200) (e.g., electronic device (101) of FIG. 1, user's smartphone) and an external electronic device (e.g., electronic device (102) of FIG. 1, wearable electronic device worn by the user).

[0071] According to one embodiment, the communication module (250) can support a long-distance wireless communication connection of the electronic device (200). For example, the communication module (250) can support a long-distance wireless communication (e.g., cellular communication, or the Internet) connection between an external server (e.g., server (108) of FIG. 1) and the electronic device (200) to provide health information related to hemoglobin indicators to the external server. The health information can be stored in conjunction with a user account managed by the external server.

[0072] According to one embodiment, the sensor module (260) (e.g., the sensor module (176) of FIG. 1) may include at least one sensor and / or sensor circuitry.

[0073] According to one embodiment, the sensor module (260) may include at least one temperature sensor. The at least one temperature sensor may be for measuring temperature. For example, one of the at least one temperature sensor may be for measuring the skin temperature of a user from a body part (e.g., finger, palm, wrist) in contact with one surface of the electronic device (200). For example, another of the at least one temperature sensor may be for measuring the external temperature of the electronic device (200).

[0074] According to one embodiment, the sensor module (260) may further include a biosensor. The biosensor may be for measuring biosignals and / or biodata (e.g., data on at least one of heart rate, heart rate variability, blood pressure, electrocardiogram, blood glucose, blood volume, oxygen saturation, body fat, electromyogram, or electroencephalogram). For example, the biosensor may include an electrode sensor having an electrode that detects characteristics of the body through contact with the body. For example, the biosensor may include at least one of a photoplethysmography (PPG) sensor, a near-infrared spectroscopy (NIRS) sensor, an electrocardiography (ECG) sensor, a galvanic skin response (GSR) sensor, a bioelectrical impedance analysis (BIA) sensor, an electrodermal response (EDR) sensor, an electromyogram (EMG) sensor, an electroencephalogram (EEG) sensor, or a biomarker sensor that detects specific substances or components within the body.

[0075] According to one embodiment, the sensor module (260) may further include a motion sensor. The motion sensor may be for measuring the movement of the electronic device (200) (or user). For example, the motion sensor may include at least one of an accelerometer, a gyroscope, a barometer (or altitude sensor), a gesture sensor, or a grip sensor.

[0076] According to one embodiment, the sensor module (260) may further include an illuminance sensor. The illuminance sensor may be for measuring illuminance (e.g., brightness of external light).

[0077] According to one embodiment, the processor (210) can acquire (e.g., take, capture) an image of the user's fingernail through the camera (230).

[0078] According to one embodiment, the processor (210) can obtain the skin temperature of the user of the electronic device (200). The processor (210) can obtain the skin temperature of the user through a temperature sensor (or internal sensor) within a communication module (250) (or communication circuit) and / or a sensor module (260).

[0079] For example, the processor (210) of an electronic device (200) (e.g., electronic device (101) of FIG. 1, smartphone (201) of FIG. 5a) can receive the skin temperature measured by the wearable electronic device from an external wearable electronic device (e.g., electronic device (102) of FIG. 1, smart ring (402) of FIG. 5a) through a communication module (250).

[0080] For example, the processor (210) of an electronic device (200) (e.g., the smartphone (201) of FIG. 5a) can establish (or open) a short-range wireless communication connection with an external wearable electronic device (e.g., the smart watch (401), smart ring (402) of FIG. 5a) through a communication module (250). The processor (210) can request a skin temperature from a temperature sensor of the wearable electronic device or receive the skin temperature from the temperature sensor of the wearable electronic device through the short-range wireless communication connection.

[0081] For example, the processor (210) of the electronic device (200) (e.g., a smart watch) can measure the skin temperature of the user wearing the electronic device (200) through a temperature sensor in the sensor module (260).

[0082] According to one embodiment, the processor (210) can perform non-invasive hemoglobin measurement (or monitoring). The processor (210) can provide health information related to a hemoglobin index (e.g., hemoglobin level, hemoglobin concentration, hemoglobin level).

[0083] According to one embodiment, the processor (210) can non-invasively measure (e.g. estimate) hemoglobin using skin temperature and nail images. The processor (210) can determine the user's hemoglobin index based on the skin temperature and nail images.

[0084] According to one embodiment, the processor (210) can correct a nail image based on skin temperature and then measure (e.g., estimate) hemoglobin using the corrected nail image. The processor (210) can generate a corrected nail image from the nail image based on the skin temperature. The processor (210) can determine the hemoglobin index based on the corrected nail image.

[0085] According to one embodiment, the processor (210) can identify whether the user's skin temperature (e.g., current skin temperature) falls within a specified range. If the skin temperature falls within the specified range, the processor (210) can determine a hemoglobin index based on a nail image (e.g., a captured image). If the skin temperature falls outside the specified range (or if the skin temperature does not fall within the specified range), the processor (210) can determine a hemoglobin index based on a corrected nail image.

[0086] According to one embodiment, the processor (210) can extract color information from a nail image. For example, the color information may correspond to histogram information representing the color distribution of a region of interest (e.g., nail area) within the image, or color profile information of the region of interest. The processor (210) may correct the color information based on the difference between a specified skin temperature and the current skin temperature, and personalization parameters. The personalization parameters may be parameters for correcting the color information of the nail image. The processor (210) may generate the corrected nail image by applying the corrected color information to the nail image. For example, the personalization parameters may include correction coefficients for correcting the color information. For example, the personalization parameters may be personal characteristics of a user representing the relationship between skin temperature and nail color. For example, the personalization parameters may be a baseline representing the user's basic nail color, or parameters for quantifying or numerically measuring the sensitivity or adaptability of nail color change with respect to temperature.

[0087] According to one embodiment, the processor (210) of an electronic device (200) (e.g., electronic device (101) of FIG. 1, smartphone (201) of FIG. 5a) can receive the skin temperature of a body part (e.g., finger, wrist) on which the wearable electronic device is worn from an external wearable electronic device (e.g., electronic device (102) of FIG. 1, smart ring (402) of FIG. 5a, smart band) through a communication module (250).

[0088] For example, the wearable electronic device may be a ring-type wearable electronic device (e.g., the smart ring (402) of FIG. 5a). The processor (210) may receive the finger skin temperature from the ring-type wearable electronic device via a communication module (250). The processor (210) may generate a corrected nail image from a nail image (e.g., a captured image) based on the finger skin temperature.

[0089] For example, the wearable electronic device may be a watch-type wearable electronic device (e.g., the smart watch (401) of FIG. 5a) or a band-type wearable electronic device (e.g., a smart band (not shown)). The processor (210) may receive the skin temperature of the wearing area (e.g., wrist, ankle) from the watch-type wearable electronic device or the band-type wearable electronic device. The processor (210) may convert the received skin temperature into a finger skin temperature. The processor (210) may generate a corrected fingernail image from a fingernail image (e.g., a captured image) based on the finger skin temperature.

[0090] According to one embodiment, the processor (210) may collect skin temperatures for different body parts from two or more wearable electronic devices (e.g., a smart watch worn on the wrist, a smart band worn on the ankle). The processor (210) may determine the skin temperature of a finger based on the collected skin temperatures for different body parts. For example, the processor (210) may estimate the first finger skin temperature from the wrist skin temperature measured by the first wearable electronic device (e.g., a smart watch worn on the wrist). The processor (210) may estimate the second finger skin temperature from the ankle skin temperature measured by the second wearable electronic device (e.g., a smart watch worn on the ankle). The processor (210) may set the average value of the first finger skin temperature and the second finger skin temperature as the finger skin temperature. The processor (210) may generate a corrected nail image from a nail image (e.g., a captured image) based on the finger skin temperature.

[0091] According to one embodiment, the processor (210) may measure (e.g., estimate) hemoglobin based on a nail image and then correct the hemoglobin measurement result using skin temperature. The processor (210) may determine a user's hemoglobin indicator (e.g., hemoglobin value, hemoglobin concentration, hemoglobin level) based on the nail image. The processor (210) may correct (or compensate) the determined hemoglobin indicator based on the difference between a specified skin temperature and the skin temperature (e.g., current skin temperature).

[0092] According to one embodiment, the processor (210) can correct the hemoglobin index based on the difference between the specified skin temperature and the current skin temperature, and personalization parameters. For example, the personalization parameters may include a correction coefficient for correcting the hemoglobin index. For example, the personalization parameters may be a characteristic element of the individual user indicating the relationship between nail color and the hemoglobin index. For example, the personalization parameters may be parameters for quantifying or numerically measuring at least one of the user's baseline, sensitivity, or adaptability related to the hemoglobin index.

[0093] According to one embodiment, the processor (210) may display a user interface (e.g., a second screen (530) shown in FIG. 5a) that indicates the hemoglobin indicator or health information related to the hemoglobin indicator through a display (240). For example, the processor (210) may display a user interface containing information indicating the hemoglobin indicator (e.g., hemoglobin measurement results such as hemoglobin value, hemoglobin concentration, or hemoglobin level). For example, the processor (210) may convert the hemoglobin indicator into a hemoglobin-associated indicator (e.g., change in hemoglobin value, anemia index, female health indicator) and display a user interface containing information indicating the hemoglobin-associated indicator. For example, the processor (210) may determine a health condition (e.g., whether hemoglobin levels are abnormal, type of anemia, iron deficiency, suspected chronic or lung disease, dehydration or overhydration, cause or symptom of abnormality) based on the hemoglobin indicator and / or the hemoglobin-associated indicator, and display a user interface containing information about the health condition.

[0094] According to one embodiment, the electronic device (200) may further include an output interface other than the display (240) (e.g., at least one of the acoustic output module (155), audio module (170), haptic module (179), or lamp (not shown) of FIG. 1).

[0095] According to one embodiment, a processor (210) of an electronic device (200) may output a user interface that displays a hemoglobin index or health information related to the hemoglobin index using a display (240) and / or the output interface. The user interface may be implemented as a visual type (e.g., screen, text), an auditory type (e.g., audio, sound, sound), a tactile type (e.g., vibration), or a hybrid type combining at least some of these.

[0096] According to one embodiment, the processor (210) can detect an external temperature through another temperature sensor within the sensor module (260). The processor (210) can identify whether the external temperature is outside a specified threshold range. If the external temperature is outside the threshold range, the processor (210) may skip the hemoglobin measurement. If the external temperature is outside the threshold range, the processor (210) may display a user interface (e.g., a guide message such as "The current external temperature is too low to measure hemoglobin. Please try again later.") through the display (240) indicating that the hemoglobin measurement is impossible.

[0097] According to one embodiment, the processor (210) can display an execution screen of an application (e.g., a health application) (e.g., the third screen (910) of FIG. 9a) through a display (240). The processor (210) can activate a camera (230) while the execution screen of the application is displayed. The processor (210) can acquire a plurality of nail images (or videos) through the camera (230). The processor (210) can determine at least one nail image among the plurality of nail images to be used for determining a hemoglobin index based on at least one of the meta information of the plurality of nail images (e.g., skin temperature per image, shooting angle per image) or attribute information per image extracted from the plurality of images (e.g., illumination per image, size of the nail area per image).

[0098] For example, the processor (210) may store the image-specific skin temperature and / or shooting angle in the metadata of each image while multiple nail images are being captured. The processor (210) may use the metadata to identify the nail image among the multiple nail images that is closest to a specified skin temperature or that was captured at a specified shooting angle. The processor (210) may select the identified nail image as the image to be used for determining the hemoglobin index.

[0099] For example, the processor (210) can identify image-specific attribute information (e.g., illumination per image, size of the nail area per image) from multiple nail images included in a video captured for a specified time (e.g., 5 seconds). The illumination per image may vary depending on the shooting conditions of each image (e.g., lighting, shooting angle, flash on / off). If the reflectance increases during shooting due to lighting or shooting angle, or if the nail area is not captured normally, the accuracy of hemoglobin measurement may be relatively reduced. Using the image-specific attribute information, the processor (210) can identify among the multiple images a nail image in which the reflectance is below a specified threshold or the size of the nail area is greater than a certain ratio. The processor (210) can select the identified nail image as the image to be used for determining the hemoglobin index.

[0100] According to one embodiment, the processor (210) may display an execution screen of an application (e.g., a health application) (e.g., the fourth screen (920) of FIG. 9b) through a display (240). The execution screen may include a user interface that guides shooting conditions for registering a specified number (e.g., N) of reference nail images. The shooting conditions may include conditions related to temperature. The shooting conditions may further include conditions related to at least one of lighting, shooting angle, or time. For example, the user interface that guides the shooting conditions may include guide information to guide the user's skin temperature to a specified skin temperature during shooting (e.g., guide messages such as "Please shoot in a warm indoor environment," or "Please rub your hands together vigorously before shooting!"). For example, the user interface that guides the shooting conditions may include guide information regarding at least one of a shooting posture that makes the nails visible (e.g., hand posture), a shooting location (e.g., indoor), lighting, or a shooting angle (e.g., high angle). The processor (210) can activate the camera (230) while the execution screen of the application (e.g., health application) is displayed. The processor (210) can acquire the reference nail images through the camera (230). The processor (210) can store the reference nail images and / or parameter information regarding the reference nail images in memory (220). For example, the processor (210) can adjust the value of a personalized parameter from a default value (e.g., 1) to a personalized value (e.g., 1.2) based on the reference nail images. The processor (210) can store the personalized value as the parameter information. The processor (210) can perform hemoglobin measurement (e.g., estimation) using the parameter information.

[0101] At least some of the operations of the method of operation of the electronic device (200) according to the various embodiments described below may be performed in correspondence with each other or in combination with each other.

[0102] FIG. 3a is a flowchart illustrating the operation method of an electronic device (200) according to one embodiment.

[0103] According to one embodiment, the method of operation of the electronic device (200) may be for hemoglobin measurement (e.g., estimation, monitoring). The method may be for non-invasive hemoglobin measurement. The method may be for providing health information related to hemoglobin indicators.

[0104] According to one embodiment, the operations illustrated in FIG. 3a may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. In some embodiments, some of the illustrated operations may be omitted, some operations may be integrated, some operations may be changed, or other operations may be added.

[0105] According to one embodiment, operations 310 to 340 may be understood to be performed in a processor (e.g., processor (210) of FIG. 2) of an electronic device (e.g., electronic device (200) of FIG. 2).

[0106] Referring to FIG. 3a, the operation method of the electronic device (200) may include operation 310, operation 320, operation 330 and operation 340.

[0107] According to one embodiment, in operation 310, the electronic device (200) can obtain the skin temperature of the user of the electronic device (200). The electronic device (200) can obtain the skin temperature of the user using a temperature sensor (or internal temperature sensor) within a communication module (250) (or communication circuit) and / or a sensor module (260).

[0108] In one embodiment, the electronic device (200) can obtain skin temperature by interacting with an external electronic device connected via short-range wireless communication. The external electronic device may be a wearable electronic device worn by a user, such as a smart watch (401) or a smart ring (402). The electronic device (200) can receive the user's skin temperature from the wearable electronic device through a communication module (250). For example, the electronic device (200) (e.g., a smartphone (201)) can receive skin temperature from a wearable electronic device worn by a user (e.g., a smart watch (401), a smart ring (402)) through a communication module (250). For example, the electronic device (200) (e.g., a smart watch (401)) can receive skin temperature from a second wearable electronic device worn by a user (e.g., a smart ring (402)) through a communication module (250) as a first wearable electronic device.

[0109] In one embodiment, an electronic device (200) (e.g., smartphone (201), smart watch (401), smart ring (402)) can detect skin temperature through a temperature sensor (or internal temperature sensor) within a sensor module (260).

[0110] In one embodiment, the electronic device (200) can acquire (e.g., receive, detect) skin temperature through at least one electronic device selected by user input among the electronic device (200) and an external electronic device (e.g., a wearable electronic device such as a smart watch (401), a smart ring (402)). For example, the electronic device (200) may display a user interface on a display (240) that includes a list of devices capable of measuring skin temperature (e.g., a smartphone (401) held by the user, a smart watch (401) or a smart ring (402) worn by the user) and an indicator for selecting at least one electronic device from the list of devices (e.g., a guide message saying, "Accurate hemoglobin measurement is currently possible using 'xxx's smartphone, smart watch, and smart ring. Would you like to start hemoglobin measurement using these devices?" and a selection menu for "Yes / No"). The electronic device (200) can select at least one electronic device from the list of devices based on user input to the user interface and obtain skin temperature through the selected at least one electronic device.

[0111] According to one embodiment, an electronic device (200) (e.g., electronic device (101) of FIG. 1, smartphone (201) of FIG. 5a, smart watch (401)) can measure skin temperature through an external wearable electronic device (e.g., electronic device (102) of FIG. 1, smart ring (402) of FIG. 5a). For example, the electronic device (200) (e.g., electronic device (101) of FIG. 1, smartphone (201) of FIG. 5a, smart watch (401)) can receive the skin temperature measured by the wearable electronic device from the wearable electronic device (e.g., electronic device (102) of FIG. 1, smart ring (402) of FIG. 5a) worn by the user through a communication module (250). For example, an electronic device (200) (e.g., smartphone (201) of FIG. 5a, smart watch (401)) can establish (or open) a short-range wireless communication connection with an external wearable electronic device (e.g., smart ring (402) of FIG. 5a) through a communication module (250). The electronic device (200) can request a skin temperature from a temperature sensor included in the wearable electronic device or receive the skin temperature from the temperature sensor of the wearable electronic device through the short-range wireless communication connection.

[0112] According to one embodiment, an electronic device (200) (e.g., the electronic device (101) of FIG. 1, the smart watch (401) of FIG. 5a) can measure skin temperature on its own. For example, the electronic device (200) (e.g., the smart watch (401) of FIG. 5a) can measure the skin temperature of a user wearing the electronic device (200) through an internal temperature sensor.

[0113] According to one embodiment, in operation 320, the electronic device (200) can acquire (e.g., take, capture) an image of the user's fingernail through a camera (230).

[0114] According to one embodiment, an electronic device (200) (e.g., a smartphone (201)) can induce a user to photograph a fingernail corresponding to the wearing position of a wearable electronic device (e.g., a smart watch (401), a smart ring (402) worn by the user) used for measuring skin temperature. For example, the electronic device (200) can identify the wearing position of the wearable electronic device (e.g., one of the left hand or the right hand). The electronic device (200) can identify the wearing position entered by user settings from memory (220) or identify the wearing position based on a user gesture detected through a motion sensor within a sensor module (260). The electronic device (200) can output a user interface (e.g., a guide message such as "Please position the fingers of the left hand, which is wearing the watch for hemoglobin measurement, in the direction of the shot and then take a shot") that induces (or requests) the user to photograph a fingernail or hand that matches the wearing position.

[0115] According to one embodiment, the electronic device (200) may set or change the shooting mode of the camera (230) to be suitable for acquiring a nail image. For example, the electronic device (200) may start capturing a nail image after switching from a first shooting mode (e.g., a general photo shooting mode) to a second shooting mode (e.g., a shooting mode for a health application, a shooting mode for measuring hemoglobin or health information) in response to a triggering event. In the first shooting mode and the second shooting mode, the hardware attributes and / or post-processing attributes of the camera (230) used for shooting may differ from one another. For example, in the second shooting mode, compared to the first shooting mode, the camera hardware attributes (e.g., automatic exposure, focus adjustment, use of multiple lenses or special lenses, shutter speed, continuous scanning) and / or camera post-processing attributes (e.g., noise reduction, peripheral distortion correction, background blur, color correction, skin correction, tone adjustment, sharpness enhancement, image stabilization) may be set to be suitable for acquiring a nail image.

[0116] According to one embodiment, in operation 330, the electronic device (200) can measure (e.g., estimate) hemoglobin in a non-invasive manner using the skin temperature obtained through operation 310 and the nail image obtained through operation 320. The electronic device (200) can determine the user's hemoglobin index (e.g., hemoglobin value, hemoglobin concentration, hemoglobin level) based on the skin temperature and the nail image.

[0117] According to one embodiment, in operation 330, the electronic device (200) can determine a hemoglobin index based on a nail image obtained through operation 320. The electronic device (200) can correct the hemoglobin index based on the difference between a specified skin temperature and a skin temperature obtained through operation 310.

[0118] According to one embodiment, in operation 330, the electronic device (200) can generate a corrected nail image from a nail image obtained through operation 320 based on the skin temperature obtained through operation 310. The electronic device (200) can determine a hemoglobin index based on the corrected nail image.

[0119] According to one embodiment, in operation 330, the electronic device (200) can identify whether the skin temperature obtained through operation 310 falls within a specified range. If the skin temperature falls within the specified range, the electronic device (200) can determine a hemoglobin index based on color information of a nail image. If the skin temperature falls outside the specified range, the electronic device (200) can determine a hemoglobin index based on color information of the nail image, the difference between the specified skin temperature and the skin temperature, and personalization parameters.

[0120] According to one embodiment, in operation 330, the electronic device (200) may receive the skin temperature of the body part on which the wearable electronic device is worn through a communication module (250) from a wearable electronic device worn by a user (e.g., a smart watch (401) or a smart ring (402) of FIG. 5a). For example, the wearable electronic device may be a ring-type wearable electronic device (e.g., a smart ring (402) of FIG. 5a). The electronic device (200) may receive the finger skin temperature from the ring-type wearable electronic device through a communication module (250). The electronic device (200) may determine a hemoglobin index based on the finger skin temperature. For example, the wearable electronic device may be a watch-type or band-type wearable electronic device (e.g., a smart watch (401) of FIG. 5a). The electronic device (200) can receive the skin temperature of the wearing area from the watch-type or band-type wearable electronic device. The electronic device (200) can convert the received skin temperature into a finger skin temperature. The electronic device (200) can determine a hemoglobin index based on the finger skin temperature. For example, the electronic device (200) can collect skin temperatures by body part from two or more wearable electronic devices worn by a user (e.g., a smart watch (401) and a smart ring (402) of FIG. 5a) via a communication module (250). The electronic device (200) can determine the finger skin temperature based on the collected skin temperatures by body part. The electronic device (200) can determine a hemoglobin index based on the finger skin temperature.

[0121] According to one embodiment, the electronic device (200) can display an execution screen of an application (e.g., a health application) (e.g., the third screen (910) of FIG. 9a) through a display (240). The electronic device (200) can activate a camera (230) (e.g., a rear camera) while the execution screen of the application is displayed. The electronic device (200) can acquire a plurality of nail images (or videos) through the camera (230). The electronic device (200) can determine at least one nail image among the plurality of nail images to be used for determining a hemoglobin index based on at least one of the meta information of the plurality of nail images (e.g., skin temperature per image, shooting angle per image) or attribute information per image extracted from the plurality of images (e.g., illumination per image, size of the nail area per image).

[0122] For example, while multiple nail images are being captured, the electronic device (200) may store the image-specific skin temperature and / or shooting angle in the metadata of each image. The electronic device (200) may use the metadata to identify the nail image among the multiple nail images that is closest to a specified skin temperature or that was captured at a specified shooting angle. The electronic device (200) may select the identified nail image as the image to be used for determining the hemoglobin index.

[0123] For example, an electronic device (200) can identify attribute information per image (e.g., illumination per image, size of the nail area per image) from multiple nail images included in a video captured for a specified time (e.g., 5 seconds). The illumination per image may vary depending on the shooting conditions of each image (e.g., lighting, shooting angle, flash on / off). If the reflectance increases during shooting due to lighting or shooting angle, or if the nail area is not captured normally, the accuracy of hemoglobin measurement may be relatively reduced. Using the attribute information per image, the electronic device (200) can identify among the multiple images a nail image in which the reflectance is below a specified threshold or the size of the nail area is greater than a certain ratio. The electronic device (200) can select the identified nail image as the image to be used for determining the hemoglobin index.

[0124] According to one embodiment, the electronic device (200) can relatively reduce color distortion or reflection distortion caused by lighting through the synthesis of a flash image and a non-flash image. The flash image may be captured based on natural light or indoor light in a flash-off state. The non-flash image may be captured in a flash-on state with substantially the same composition as the flash image. The electronic device (200) can align the flash image and the non-flash image and, by analyzing the difference in lighting between the two images, estimate information regarding at least one of actual color, texture, or reflectance. The electronic device (200) can correct the captured fingernail image based on the estimated information and select the corrected fingernail image as the image to be used for determining the hemoglobin index.

[0125] According to one embodiment, in operation 340, the electronic device (200) may display a user interface (e.g., a second screen (530) illustrated in FIG. 5a) that indicates the hemoglobin indicator or health information related to the hemoglobin indicator through a display (240). For example, the user interface may include information indicating the hemoglobin measurement result and / or health information processed from the hemoglobin measurement result. For example, the user interface may include information indicating the hemoglobin indicator (e.g., hemoglobin value, hemoglobin concentration, or hemoglobin level). For example, the user interface may include information indicating the hemoglobin-related indicator converted from the hemoglobin indicator (e.g., change in hemoglobin value, anemia index, female health indicator). For example, the user interface may include information on a health condition estimated based on the hemoglobin indicator and / or the hemoglobin-associated indicator (e.g., whether hemoglobin levels are abnormal, type of anemia, iron deficiency, suspected chronic or lung disease, dehydration or overhydration, cause or symptom of abnormality).

[0126] According to one embodiment, an electronic device (200) can acquire (e.g., receive, detect) an external temperature. The electronic device (200) can identify whether the external temperature is outside a specified threshold range. If the external temperature is outside the threshold range, the electronic device (200) may omit the hemoglobin measurement. If the external temperature is outside the threshold range, the electronic device (200) may display a user interface indicating that the hemoglobin measurement is impossible (e.g., a guide message such as "The current external temperature is too low to measure hemoglobin. Please try again later.").

[0127] Although not illustrated, according to one embodiment, the method of operation of the electronic device (200) may further include the operation of registering at least one reference nail image. For example, the electronic device (200) may register at least one reference nail image before performing a shot for acquiring a nail image of operation 320.

[0128] According to one embodiment, the electronic device (200) may display an execution screen of an application (e.g., a health application) (e.g., the fourth screen (920) of FIG. 9b) through a display (240). The execution screen may include a user interface that guides shooting conditions for registering a specified number (e.g., N) of reference nail images. The shooting conditions may include conditions related to temperature. The shooting conditions may further include conditions related to at least one of lighting, shooting angle, or time. For example, the user interface that guides the shooting conditions may include guide information to guide the user's skin temperature to a specified skin temperature during shooting (e.g., guide messages such as "Please shoot in a warm indoor environment," or "Please rub your hands together vigorously before shooting!"). For example, the user interface that guides the shooting conditions may include guide information regarding at least one of a shooting posture that makes the nails visible (e.g., hand posture), a shooting location (e.g., indoor), lighting, or a shooting angle (e.g., high angle). The electronic device (200) can activate the camera (230) while the execution screen of the application (e.g., health application) is displayed. The electronic device (200) can acquire the reference nail images through the camera (230). The electronic device (200) can store the reference nail images and / or information about the reference nail images in memory (220). For example, the electronic device (200) can adjust the value of a personalization parameter from a default value (e.g., 1) to a personalized value (e.g., 1.2) based on the reference nail images. The electronic device (200) can store the personalized value as the parameter information. The electronic device (200) can perform hemoglobin measurement (e.g., estimation) using the parameter information.

[0129] FIG. 3b is a flowchart illustrating a process (e.g., operation 330 of FIG. 3a) in which an electronic device (200) according to one embodiment determines a hemoglobin indicator.

[0130] According to one embodiment, the electronic device (200) can determine a hemoglobin index using a corrected image. The electronic device (200) can correct a nail image based on skin temperature and then measure (e.g. estimate) hemoglobin using the corrected nail image.

[0131] Referring to FIG. 3b, a process for determining a hemoglobin index based on skin temperature and a nail image (e.g., operation 330) may include operation 331 of generating a corrected nail image from the nail image and operation 332 of determining a hemoglobin index based on the corrected nail image.

[0132] According to one embodiment, in operation 331, the electronic device (200) can generate a corrected nail image from a nail image (e.g., a captured image) based on the user's skin temperature.

[0133] According to one embodiment, an electronic device (200) can extract color information from a nail image (e.g., a captured image). For example, the color information may correspond to histogram information representing the color distribution of a region of interest (e.g., a nail area) within the image, or color profile information of the region of interest. The electronic device (200) can correct the color information based on the difference between a specified skin temperature and a current skin temperature, and personalization parameters. The personalization parameters may be parameters for correcting the color information of the nail image. The electronic device (200) can generate the corrected nail image by applying the corrected color information to the nail image. For example, the personalization parameters may include a correction coefficient for correcting the color information. For example, the personalization parameters may be a characteristic element of a user individual representing the relationship between skin temperature and nail color. For example, the personalization parameter may be a baseline representing the user's basic nail color, or a parameter for quantifying or numerically measuring the sensitivity or adaptability of nail color change with respect to temperature.

[0134] According to one embodiment, in operation 332, the electronic device (200) can determine the user's hemoglobin indicator (e.g., hemoglobin level, hemoglobin concentration, hemoglobin level) based on the nail image (or corrected nail image) generated through operation 331.

[0135] According to one embodiment, the electronic device (200) can identify whether the user's skin temperature (e.g., current skin temperature) falls within a specified range. If the skin temperature falls within the specified range, the electronic device (200) can determine a hemoglobin index based on a nail image (e.g., captured image). If the skin temperature falls outside the specified range (or if the skin temperature does not fall within the specified range), the electronic device (200) can determine a hemoglobin index based on a nail image (or a corrected nail image) generated through operation 331.

[0136] FIG. 3c is a flowchart illustrating a process (e.g., operation 330 of FIG. 3a) in which an electronic device (200) according to one embodiment determines a hemoglobin indicator.

[0137] According to one embodiment, the electronic device (200) can measure (e.g., estimate) hemoglobin based on a nail image (e.g., a captured image) and then correct the hemoglobin measurement result using skin temperature.

[0138] Referring to FIG. 3c, a process for determining a hemoglobin index based on skin temperature and a nail image (e.g., operation 330) may include operation 335 for determining a hemoglobin index using the nail image and operation 336 for correcting the hemoglobin index based on the skin temperature.

[0139] According to one embodiment, in operation 335, the electronic device (200) can determine the user's hemoglobin indicator (e.g., hemoglobin level, hemoglobin concentration, hemoglobin level) based on the user's fingernail image (e.g., captured image).

[0140] According to one embodiment, in operation 336, the electronic device (200) can correct the hemoglobin index determined through operation 335 based on the difference between a specified skin temperature and the skin temperature (e.g., current skin temperature).

[0141] According to one embodiment, the electronic device (200) can correct the hemoglobin index based on the difference between the specified skin temperature and the current skin temperature, and personalization parameters. For example, the personalization parameters may include a correction coefficient for correcting the hemoglobin index. For example, the personalization parameters may be a characteristic element of the individual user indicating the relationship between nail color and the hemoglobin index. For example, the personalization parameters may be parameters for quantifying or numerically measuring at least one of the user's baseline, sensitivity, or adaptability related to the hemoglobin index.

[0142] FIG. 4 is a diagram illustrating a process (e.g., operation 331 and operation 332 of FIG. 3b) in which an electronic device (200) according to one embodiment determines a hemoglobin index using a corrected image.

[0143] According to one embodiment, in operation 410 (e.g., operation 310 of FIG. 3a), the electronic device (200) (e.g., the smartphone (201) of FIG. 5a) can acquire (e.g., receive, detect) the user's skin temperature.

[0144] According to one embodiment, an electronic device (200) (e.g., a smartphone (201) of FIG. 5a) can receive the skin temperature of a body part (e.g., finger, wrist, ankle) on which the wearable electronic device is worn through a communication module (250) from an external wearable electronic device (e.g., at least one of a smart watch (401), smart ring (402), or smart band (not shown) of FIG. 4 and FIG. 5a).

[0145] In one embodiment, the skin temperature may be the finger skin temperature.

[0146] In one embodiment, the wearable electronic device may be a ring-type wearable electronic device (e.g., a smart ring (402)). The electronic device (200) may receive finger skin temperature from the ring-type wearable electronic device through a communication module (250).

[0147] In one embodiment, the wearable electronic device may be a watch-type wearable electronic device (e.g., a smart watch (401)) or a band-type wearable electronic device (e.g., a smart band (not shown)). The electronic device (200) may receive the skin temperature of a wearing area (e.g., wrist, ankle) from the watch-type wearable electronic device or the band-type wearable electronic device. The electronic device (200) may convert the received skin temperature into a finger skin temperature.

[0148] In one embodiment, the electronic device (200) may collect skin temperatures for different body parts from two or more wearable electronic devices (e.g., a smart watch (401) worn on the wrist, a smart band worn on the ankle). The electronic device (200) may determine the skin temperature of a finger based on the collected skin temperatures for different body parts. For example, the electronic device (200) may estimate the first finger skin temperature from the wrist skin temperature measured by the first wearable electronic device (e.g., a smart watch worn on the wrist). The electronic device (200) may estimate the second finger skin temperature from the ankle skin temperature measured by the second wearable electronic device (e.g., a smart watch worn on the ankle). The electronic device (200) may set the average value of the first finger skin temperature and the second finger skin temperature as the finger skin temperature.

[0149] According to one embodiment, in operation 420 (e.g., operation 320 of FIG. 3a), an electronic device (200) (e.g., a smartphone (201) of FIG. 5a) can acquire (e.g., capture) an image of the user's fingernail through a camera (230).

[0150] According to one embodiment, in operation 430 (e.g., operation 331 of FIG. 3b), an electronic device (200) (e.g., a smartphone (201) of FIG. 5a) can correct a nail image (e.g., a captured image) obtained through operation 420 based on a finger skin temperature obtained through operation 410. The electronic device (200) can generate a corrected nail image from the nail image (e.g., a captured image) based on the finger skin temperature. For example, the corrected nail image may be obtained by processing the nail image (e.g., a captured image) (e.g., noise reduction, image normalization, magnification by a specified magnification) and correcting the color information of the processed nail image.

[0151] In operation 440 (e.g., operation 332 in FIG. 3b), an electronic device (200) (e.g., a smartphone (201) in FIG. 5a) can determine a hemoglobin indicator based on a fingernail image (or a corrected fingernail image) generated through operation 430.

[0152] FIG. 5a is an example of a user interface related to a hemoglobin indicator displayed on an electronic device (200) according to one embodiment. FIG. 5b is a diagram illustrating the relationship between finger skin temperature and nail color according to one embodiment.

[0153] According to one embodiment, the electronic device (200) can perform an operation (e.g., operation 340 of FIG. 3a) that displays a user interface related to a hemoglobin indicator (e.g., the second screen (530) of FIG. 5a).

[0154] According to one embodiment, the electronic device (200) may correspond to a smartphone (201). The smartphone (201) may output a user interface such as a second screen (530). For example, the user interface may include information regarding a hemoglobin index determined based on the user's skin temperature and an image of the user's fingernails. For example, the user interface may include information indicating a hemoglobin measurement result and / or health information processed from the hemoglobin measurement result.

[0155] According to one embodiment, the electronic device (200) may receive the skin temperature measured by the wearable electronic device from an external wearable electronic device. The wearable electronic device may correspond to a smart watch (401) or a smart ring (402).

[0156] According to one embodiment, a smartphone (201) can trigger a hemoglobin measurement function. For example, the smartphone (201) can trigger the hemoglobin measurement function when the execution of an application (e.g., a health application) is detected, or when user input selecting a specific button within the application (e.g., a hemoglobin measurement button, a camera button) is detected. In response to the triggering of the hemoglobin measurement function, the smartphone (201) can display a first screen (510) for capturing (or taking) a nail image necessary for hemoglobin measurement. In response to the triggering of the hemoglobin measurement function, the smartphone (201) can activate a camera (e.g., a rear camera). The smartphone (201) can capture (or take) a nail image using the activated camera.

[0157] According to one embodiment, when a user wears both a smart watch (401) and a smart ring (402), the smartphone (201) can acquire (e.g., receive, detect) skin temperature using the wearable electronic device selected by the user among the smart watch (401) and the smart ring (402).

[0158] According to one embodiment, the smartphone (201) can acquire (e.g., receive, detect) the finger skin temperature through a smart watch (401) or a smart ring (402).

[0159] For example, a smartphone (201) can receive the finger skin temperature measured by a smart ring (402).

[0160] For example, a smartphone (201) may receive a wrist skin temperature measured by a smartwatch (401) and convert the wrist skin temperature into a finger skin temperature. The smartphone (201) may estimate the finger skin temperature from the skin temperature of the wearing area based on an external temperature and / or a specified offset. The offset may be a value corresponding to the general temperature difference (or distance) between the wearing area and the finger. The offset may be adjusted according to the external temperature. For example, if the external temperature is below a specified threshold, the smartphone (201) may estimate the finger skin temperature by subtracting the offset from the wrist skin temperature measured by the smartwatch (401).

[0161] According to one embodiment, a smartphone (201) can obtain skin temperature from a wearable electronic device worn on the body part closest to the fingertip among a plurality of wearable electronic devices worn by a user. If the user wears both a smart watch (401) and a smart ring (402), the smartphone (201) can obtain (e.g., receive, detect) skin temperature using the wearable electronic device with the higher priority among the smart watch (401) and the smart ring (402). For example, among the user's body parts, the fingertip may have the highest correlation with hemoglobin indicators and the highest priority. The priority of each wearable electronic device may be set according to the distance from the fingertip. For example, the closer the wearing area is to the fingertip, the higher the priority of the wearable electronic device may be. Using the skin temperature of the body part closest to the fingertip may improve the accuracy of hemoglobin measurement.

[0162] According to one embodiment, a smartphone (201) can measure (e.g., estimate) hemoglobin using fingernail images (521, 522) captured through a camera and finger skin temperature obtained (e.g., received, detected) through a smart watch (401) or a smart ring (402). The smartphone (201) can display a second screen (530) showing the hemoglobin measurement result.

[0163] FIG. 5b is a diagram illustrating the relationship between finger skin temperature and nail color according to one embodiment.

[0164] Referring to Fig. 5b, among the user's body parts, the fingertips may have the highest relative correlation with hemoglobin levels. When hemoglobin is measured (e.g., estimated) using skin temperature and nail images of body parts close to the fingertips, measurement accuracy can be improved.

[0165] According to one embodiment, vasodilation (551), in which blood vessels widen, or vasoconstriction (552), in which blood vessels narrow, may occur in the nail bed (550) at the fingertip depending on the skin temperature of the finger. Due to the vasodilation or vasoconstriction, the nail color corresponding to the nail bed (550) may appear differently.

[0166] For example, if the skin temperature of the finger exceeds a first temperature (e.g., 37.5°C), vasodilation (551) may occur, in which the diameter of the blood vessels (e.g., arteries) increases. In this case, the hand is warm, and due to the vasodilation (551), the color of the nail bed (550) within the nail image (521) (or nail color) may exhibit excessive redness. The red color may appear more intense in the area of ​​the nail bed (550) that is close to the eponychium (560) located at the root of the nail.

[0167] For example, if the skin temperature of the finger is below a second temperature (e.g., 36.5°C), vasoconstriction (552) may occur, in which the diameter of the blood vessels (e.g., arteries) decreases. In this case, the hand is cold, and due to the vasoconstriction (552), the color of the nail bed (550) within the nail image (522) (or nail color) may appear bluish or purple tones. The bluish or purple tones may appear more strongly in the area of ​​the nail bed (550) that is close to the eponychium (560) located at the root of the nail.

[0168] According to one embodiment, a smartphone (201) can perform hemoglobin measurement based on color information of a nail area within a nail image (521, 522).

[0169] According to one embodiment, if the finger skin temperature falls outside a specified range, the measurement accuracy may be relatively reduced when measuring hemoglobin based on color information of the nail area. A smartphone (201) can identify whether the finger skin temperature falls within a specified range to improve measurement accuracy. If the finger skin temperature falls within a specified range (e.g., if the finger skin temperature is 36.5°C or higher and less than 37.5°C), the smartphone (201) can determine a hemoglobin index based on color information of the nail area within the nail image (521). If the finger skin temperature falls outside a specified range (e.g., if the finger skin temperature is less than 36.5°C or exceeds 37.5°C), the smartphone (201) can correct the color information of the nail area within the nail image (522). The smartphone (201) can determine a hemoglobin index based on the corrected color information.

[0170] FIG. 6 is a flowchart illustrating a process in which an electronic device (200) according to one embodiment determines a hemoglobin index using an artificial intelligence (AI) model.

[0171] According to one embodiment, the electronic device (200) can determine a hemoglobin indicator (e.g., hemoglobin value, concentration, or level) using at least one artificial intelligence model (e.g., the calibration model of reference numeral 630, the machine learning model of reference numeral 650). The operation of determining the hemoglobin indicator may correspond to a hemoglobin measurement or estimation operation.

[0172] According to one embodiment, the at least one AI model (the correction model of reference numeral 630, the machine learning model of reference numeral 650) may each be implemented in hardware and / or software to perform a predetermined function.

[0173] In the embodiment of FIG. 6, the number of AI models used for hemoglobin estimation is exemplified as two, but the number of AI models is not limited to two. For example, additional AI models may be used, or at least some of the illustrated AI models may be integrated. For example, some of the illustrated AI models may be omitted.

[0174] According to one embodiment, the electronic device (200) of FIG. 2 may be configured to include at least some of the AI ​​models shown in FIG. 6 (the correction model of reference numeral 630, the machine learning model of reference numeral 650). At least some of the AI ​​models may be included in the electronic device (200) (e.g., memory (220)) in the form of on-device AI models, but are not limited thereto. For example, at least some of the AI ​​models may be included in an external server (e.g., the server (108) of FIG. 1). In some embodiments, an AI model integrating at least some of the AI ​​models may be used, or only some of the AI ​​models may be optionally used.

[0175] Referring to FIG. 6, a hemoglobin measurement operation of an electronic device (200) according to one embodiment may include an operation of receiving temperature data and image data (blocks 601, 602), a preprocessing operation for temperature data and image data (block 610), an operation of determining whether the skin temperature from the temperature data is within a specified range (block 620), an operation of correcting a nail image from the image data based on the skin temperature (blocks 630, 640), an operation of learning or training the temperature data and / or the image data (block 650), an operation of estimating hemoglobin based on the corrected nail image (block 660), and an operation of outputting data according to the hemoglobin estimation result (block 670).

[0176] According to one embodiment, the correction model (or compensation model) of block 630 may be an AI model trained to generate an image in which color information (e.g., color intensity of a fingernail image) of an input image of a living organism is adjusted (or corrected, compensated) based on the input temperature.

[0177] According to one embodiment, the machine learning model of block 650 may be an AI model trained to enhance the performance of the correction model. The machine learning model may be an AI model trained to update parameters to be used in the correction model. The machine learning model may be an AI model trained for the purpose of learning (or training) patterns regarding the relationship between an individual's skin temperature and nail color (or nail bed color) based on temperature data and / or image data under various shooting conditions, and dynamically setting and / or adjusting parameters based on the learning results.

[0178] According to one embodiment, in block 601, the electronic device (200) may receive temperature data. The temperature data may include the user's skin temperature. The skin temperature may be used to determine whether correction is required to ensure accurate hemoglobin estimation. For example, the electronic device (200) may receive the temperature data from an external wearable electronic device (e.g., the smart watch (401) and smart ring (402) of FIG. 5a).

[0179] According to one embodiment, in block 602, the electronic device (200) may receive image data necessary for hemoglobin estimation. The image data may include at least one nail image. The image data may include an image (e.g., a single image capturing a nail bed) or a video (e.g., a short video of a nail bed) captured (or filmed) through a camera (230).

[0180] According to one embodiment, when a video (e.g., video) is used for hemoglobin estimation, an electronic device (200) processes the frames of the video and can select a frame (or image) suitable for hemoglobin estimation among the frames based on frame-by-frame illumination interference (e.g., reflectance) and / or frame-by-frame image quality. For example, the electronic device (200) can select an optimal frame among the frames of the input video that has relatively minimal illumination interference and relatively the highest image quality as the frame for hemoglobin estimation, thereby allowing preprocessing to be performed on the frame.

[0181] According to one embodiment, in block 610, the electronic device (200) can perform pre-processing on temperature data input through block 601 and image data input through block 602.

[0182] According to one embodiment, at least one operation among noise reduction, image normalization, or temperature analysis may be performed in the preprocessing block of block 610. For example, the noise reduction operation may be an operation that filters out defects such as glare, uneven lighting, or motion blurs from the input image and converts the input image into a relatively sharp image suitable for further analysis. The image normalization operation may be an operation that adjusts the brightness, contrast, or color level of the input image to standardize the input image. In the image normalization operation, a region of interest (e.g., nail area, nail bed area) may be isolated for precise analysis. The temperature analysis operation may be an operation that evaluates whether the input skin temperature (e.g., current skin temperature) is suitable for hemoglobin estimation or determines whether the input skin temperature falls within a specified range.

[0183] According to one embodiment, the user's current skin temperature and the user's fingernail image to be used for hemoglobin estimation can be output through the preprocessing process of block 610.

[0184] According to one embodiment, block 620 may be for evaluating whether an input skin temperature (e.g., current skin temperature) is suitable for hemoglobin measurement. In block 620, the electronic device (200) can identify whether the skin temperature input via block 610 falls within a specified range (e.g., 36.5°C to 37.5°C).

[0185] According to one embodiment, if the identification result of block 620 indicates that the skin temperature (e.g., current skin temperature) falls within a specified range, the process may proceed to block 660.

[0186] According to one embodiment, block 660 may be for estimating hemoglobin by directly using an input image (e.g., a captured original image, a captured nail image) without correction. In block 660, the electronic device (200) may perform hemoglobin estimation based on the skin temperature and the input image. The hemoglobin estimation operation may correspond to an operation for determining a hemoglobin index.

[0187] According to one embodiment, in block 670, the electronic device (200) may display a user interface indicating the hemoglobin estimation result of operation 660 (e.g., a hemoglobin indicator corresponding to any one of hemoglobin values, concentrations, or levels).

[0188] According to one embodiment, if the identification result of block 620 shows that the skin temperature (e.g., current skin temperature) is outside a specified range (or if the skin temperature is not included in the specified range), the process may proceed to block 630.

[0189] According to one embodiment, block 630 may be for adjusting skin temperature and / or images to be used for hemoglobin estimation using a correction model. Block 630 may be for temperature correction and / or image correction.

[0190] According to one embodiment, in block 630, the electronic device (200) can correct the input image (e.g., captured original image, captured nail image) based on the skin temperature (e.g., current skin temperature). The electronic device (200) can perform temperature correction and / or image correction using the correction model.

[0191] According to one embodiment, the correction model of block 630 may be intended to adjust color information (e.g., color intensity) of an image according to temperature. For example, the correction model may be for color intensity (C) adjusted by the formula of Equation 1 below. adjusted ) can be calculated.

[0192]

[0193] The above mathematical formula 1 is merely an example to aid understanding and is not limited thereto; it can be modified, applied, or extended in various ways.

[0194] In the above mathematical formula 1, C adjusted may be an adjusted (or corrected or compensated) color intensity. For example, the adjusted color intensity (C adjusted ) can correspond to the adjusted metric values ​​of the RGB channels (red, green, blue channels). The adjusted color intensity (C adjusted ) can correspond to a histogram adjustment value or a color profile adjustment value.

[0195] C inputmay be the initial color intensity from an image (e.g., captured image, input image). For example, the initial color intensity (C input ) can correspond to the initial metric values ​​of the RGB channels. The initial color intensity (C input ) can correspond to the histogram initial value or the color profile initial value.

[0196] k can be a correction coefficient. For example, k can be a personalization parameter.

[0197] T optimal can be a specified skin temperature (optimal skin temperature). For example, T optimal is the appropriate upper limit temperature (T optimal, upper ) and / or appropriate lower limit temperature (T optimal, lower It may include ).

[0198] T input It can be the current skin temperature (or measured skin temperature, input skin temperature).

[0199] "T optimal -T input It can correspond to the difference between the specified skin temperature and the current skin temperature.

[0200] According to one embodiment, the electronic device (200) has a current skin temperature (T input The color intensity of the input image (e.g., captured original image, captured nail image) can be adjusted according to ). The above correction model uses the current skin temperature (T) using Equation 1 above. input The color intensity of the input image can be dynamically increased or decreased based on whether ) is higher or lower than the specified skin temperature (whether it is warmer or colder than the specified range).

[0201] For example, a specified skin temperature (T optimal)(e.g., values ​​between 36.5℃ and 37.5℃) can be defined by boundary values ​​prior to vasoconstriction and vasodilation (e.g., 36.5℃, 37.5℃). Current skin temperature (T input If ) is higher than the designated first temperature (e.g., 37.5 °C), vasodilation occurs, and the nail bed may appear redder than usual. On the other hand, the current skin temperature (T input If the temperature is lower than the specified second temperature (e.g., 36.5℃), vasoconstriction occurs, and the nail bed may appear paler or even bluish than usual.

[0202] As such, nail color (or nail bed color) can change depending on skin temperature regardless of the actual hemoglobin concentration. Therefore, if the skin temperature falls outside the specified range, temperature correction and / or image correction may be required to improve the accuracy of hemoglobin measurement.

[0203] According to one embodiment, the electronic device (200) can improve the accuracy of hemoglobin measurement by performing temperature correction and / or image correction using the correction model.

[0204] The correction model of Equation 1 or Block 630 above is merely an example to explain the relationship between skin temperature and image color (e.g., nail color, nail bed color), and the scope of the embodiments is not limited thereto. For example, the relationship between skin temperature and image color (e.g., nail color, nail bed color) may be defined by a non-linear regression model rather than a linear model such as Equation 1 above. In this case, the electronic device (200) [represents] a color intensity (C) adjusted from the color intensity of the input image through non-linear regression analysis by the non-linear regression model. adjusted ) can be calculated. The electronic device (200) can generate a corrected image from the input image using the adjusted color intensity.

[0205] According to one embodiment, in block 630, the electronic device (200) can correct an input image (e.g., a captured original image, a captured nail image). The electronic device (200) can adjust the color intensity of the input image. For color intensity adjustment, the electronic device (200) can perform at least one operation of high-temperature adjustment, low-temperature adjustment, or skin tone adaptation. The high-temperature adjustment operation may be to reduce the red channel intensity due to a skin temperature higher than the first temperature (e.g., 37.5 °C). The low-temperature adjustment operation may be to reduce the blue channel intensity due to a skin temperature lower than the second temperature (e.g., 36.5 °C). The skin tone adaptation may be to adjust the degree of increase or decrease in color intensity by taking into account the individual skin tone of the individual.

[0206] For example, if the current skin temperature is relatively high, which may make the skin appear redder due to vasodilation, the electronic device (200) can normalize the nail color (or the color of the nail bed) by reducing the effect of the red tone. On the other hand, if the current skin temperature is relatively low, which may cause a bluish or purple tone in the skin due to vasoconstriction, the electronic device (200) can increase the base color component while reducing the effect of the blue tone.

[0207] According to one embodiment, in block 640, an image corrected through block 630 (e.g., a corrected fingernail image) may be output. The corrected image may be the input image corrected.

[0208] According to one embodiment, in block 660, the electronic device (200) can perform hemoglobin estimation based on the corrected image (e.g., corrected nail image) and the skin temperature (e.g., current skin temperature) of block 640. The hemoglobin estimation operation may correspond to an operation to determine a hemoglobin index.

[0209] According to one embodiment, in block 670, the electronic device (200) may display a user interface indicating a hemoglobin estimation result (e.g., a hemoglobin indicator corresponding to any one of hemoglobin values, concentrations, or levels). For example, the user interface may further include information regarding the corrected image of operation 640 and / or shooting conditions (e.g., current skin temperature, corrected skin temperature, lighting, shooting angle).

[0210] According to one embodiment, temperature data and image data preprocessed through block 610 can be transferred to a machine learning model of block 650. The machine learning model of operation 650 may be for setting and / or adjusting parameters to be used in the calibration model of operation 630.

[0211] According to one embodiment, the machine learning model may perform at least one of the following operations: generating training data regarding the relationship between an individual's skin temperature and nail images, dynamically adjusting parameters for hemoglobin estimation, or updating the training data or parameters through a feedback loop.

[0212] For example, the machine learning model receives a plurality of input images of the user's fingernails and can identify the shooting conditions for each of the input images (e.g., skin tone condition, skin temperature condition, lighting condition). The machine learning model can learn the change in fingernail color according to the user's skin temperature using the input images and the shooting conditions for each image. The machine learning model can quantify how nail color (e.g., nail bed color) shifts with temperature changes regardless of actual hemoglobin concentration by utilizing color information (e.g., color intensity of RGB channels) from input images (e.g., nail images) captured under various shooting conditions, and can dynamically set and / or adjust personalized parameters (e.g., correction coefficient (k) of Equation 1) to be used for hemoglobin estimation by reflecting the quantified results. The machine learning model can generate training data through training on various datasets of shooting conditions (e.g., skin tone condition, skin temperature condition, or lighting condition). Based on the training data, the machine learning model can improve the performance of the correction model and / or the accuracy of hemoglobin measurement by dynamically adjusting the values ​​of the parameters to be used in the correction model of Block 630.

[0213] FIG. 7 is a flowchart illustrating the operation method of an electronic device (200) according to one embodiment.

[0214] According to one embodiment, the method of operation of the electronic device (200) may be for hemoglobin measurement (e.g., estimation, monitoring). The method may be for non-invasive hemoglobin measurement. The method may be for providing health information related to hemoglobin indicators.

[0215] According to one embodiment, the operations illustrated in FIG. 7 may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. In some embodiments, some of the illustrated operations may be omitted, some operations may be integrated, some operations may be changed, or other operations may be added.

[0216] According to one embodiment, at least some of the operations shown in FIG. 7 and the operations shown in FIG. 3a may be performed in correspondence with each other or in combination with each other.

[0217] Referring to FIG. 7, the operation method of the electronic device (200) may include operation 710, operation 720, operation 730, operation 740, operation 750, operation 760, operation 770, and operation 780.

[0218] According to one embodiment, in operation 710, an electronic device (200) (e.g., a user's smartphone (201)) may receive temperature data through an external wearable electronic device (e.g., a smart watch (401), a smart ring (402) worn by the user) connected via short-range wireless communication. The temperature data may correspond to the user's skin temperature.

[0219] According to one embodiment, the electronic device (200) can continuously monitor the skin temperature of a body part (e.g., wrist, finger) on which the wearable electronic device is worn by using a temperature sensor within the wearable electronic device. The electronic device (200) can collect (or measure) temperature data corresponding to the skin temperature at regular intervals and store it in memory (220).

[0220] According to one embodiment, in operation 720, the electronic device (200) may receive image data through a camera (230). The image data may correspond to an image of the user's fingernail (or an image of the nail bed). The electronic device (200) may capture the fingernail image through the camera (230). For example, the electronic device (200) may capture an image containing a fingernail area or shoot a short video containing fingernail areas through the camera (230).

[0221] According to one embodiment, the electronic device (200) may perform image capture based on the satisfaction of specified shooting conditions. For example, the electronic device (200) may determine that the specified shooting conditions are satisfied if the skin temperature is within a specified range. In this case, the electronic device (200) may trigger a hemoglobin measurement function to display the execution screen of an application (e.g., a health application) or activate the camera (230). The electronic device (200) may capture a nail image using the activated camera (230). The electronic device (200) may ensure (or provide) appropriate shooting conditions (e.g., lighting conditions, flash on / off, shooting direction) to increase image clarity and hemoglobin measurement accuracy when capturing the nail image.

[0222] According to one embodiment, in operation 730, the electronic device (200) can link (e.g., pair) temperature data (e.g., skin temperature) input through operation 710 and image data (e.g., nail image) input through operation 720.

[0223] According to one embodiment, the electronic device (200) can improve the quality of the captured image (e.g., fingernail image) by applying a pre-processing technique (e.g., noise reduction, contrast enhancement) to the captured image.

[0224] According to one embodiment, an electronic device (200) can perform color analysis on an image. The electronic device (200) can analyze the color of a region of interest (e.g., fingernail area) within a pre-processed captured image and determine RGB color metric values ​​and / or other color metric values ​​based on the analysis results. The electronic device (200) can isolate a region of interest (e.g., fingernail area) from the captured image based on the RGB metric values ​​that have a relatively high correlation with blood hemoglobin concentration compared to other colors, and can analyze the color profile (or histogram) of the region of interest.

[0225] According to one embodiment, in operation 740, the electronic device (200) can perform temperature correction and / or image correction on the image. The electronic device (200) can generate a corrected image from a pre-processed captured image.

[0226] According to one embodiment, the electronic device (200) can adjust the color metric value of the pre-processed captured image according to the relationship between the skin temperature and the change in nail color from the temperature data. For example, the electronic device (200) can adjust the color metric value by considering the effect of a relatively high skin temperature on the color of the nail bed (reddish color) or the effect of a relatively low skin temperature on the color of the nail bed (bluish color).

[0227] According to one embodiment, in operations 750 and 760, the electronic device (200) can perform a hemoglobin measurement operation using the corrected image.

[0228] According to one embodiment, the electronic device (200) can improve the accuracy or reliability of the hemoglobin measurement through a validation and feedback process for the corrected image.

[0229] According to one embodiment, the electronic device (200) can determine a hemoglobin concentration value (g / dL) based on a color metric value adjusted through operation 740. The electronic device (200) can compare the hemoglobin concentration value with a known hemoglobin concentration range in a hemoglobin database. For example, the hemoglobin database may be stored in memory (220). The hemoglobin database may include information on at least some of a plurality of images of the user's fingernails, color metric values ​​of the images, shooting conditions per image, or hemoglobin concentration ranges per shooting condition. Based on the comparison result, the electronic device (200) can determine whether the hemoglobin concentration value falls within a known hemoglobin concentration range under shooting conditions that are the same or similar to the current shooting conditions (e.g., factors such as skin tone, ambient lighting conditions, or individual skin temperature changes). The electronic device (200) can improve the reliability of temperature correction and / or image correction by adjusting the hemoglobin concentration value according to the above judgment result. The electronic device (200) can assign a confidence score to the hemoglobin measurement result based on the quality, validation, and feedback of the input data (temperature data and / or image data) applied to the adjusted hemoglobin concentration value. If the confidence score to the hemoglobin measurement result exceeds a certain value, the electronic device (200) can determine the adjusted hemoglobin concentration value as the final hemoglobin concentration value.

[0230] According to one embodiment, in operation 770, the electronic device (200) may display a user interface for the hemoglobin measurement result. The user interface may include the final hemoglobin concentration value. The user interface may further include information on at least one of shooting conditions (e.g., current skin temperature) or hemoglobin measurement quality (e.g., confidence score).

[0231] According to one embodiment, the electronic device (200) can perform necessary interlocking operations based on the quality of the hemoglobin measurement (e.g., confidence score). For example, the electronic device (200) can provide feedback that a re-measurement is required if the quality of the hemoglobin measurement is relatively low.

[0232] According to one embodiment, in operation 780, the electronic device (200) may perform data logging and sharing operations regarding the hemoglobin measurement results. For example, the electronic device (200) may include information on at least some of the hemoglobin measurement results (e.g., hemoglobin concentration values), the nail image used for the hemoglobin measurement, the color metric value of the nail image, shooting conditions (e.g., current skin temperature, lighting conditions, confidence score), or hemoglobin measurement history (e.g., notification history).

[0233] FIG. 8a is a flowchart illustrating a process in which an electronic device (200) according to one embodiment triggers a hemoglobin measurement function. For example, the operations of FIG. 8a may be performed prior to operation 310 of FIG. 3a.

[0234] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0235] According to one embodiment, operations 811 to 813 may be understood to be performed in a processor (e.g., processor (210) of FIG. 2) of an electronic device (e.g., electronic device (200) of FIG. 2).

[0236] According to one embodiment, in operation 811, an electronic device (200) (e.g., a user's smartphone (201)) can collect biometric data (e.g., body temperature, heart rate) through an external wearable electronic device (e.g., a smart watch (401), a smart ring (402)). The electronic device (200) can measure the biometric data in real time or continuously track it.

[0237] According to one embodiment, in operation 812, the electronic device (200) can evaluate whether the biometric data collected through operation 811 falls within a specified range for hemoglobin measurement. The electronic device (200) can compare the biometric data with the specified range required for hemoglobin measurement. For example, the electronic device (200) can determine whether a first condition is satisfied in which the body temperature falls within a specified range (e.g., a normal body temperature range between 36.5°C and 37.5°C) and a second condition is satisfied in which the heart rate falls within a specified range (e.g., a normal heart rate range between about 60 and about 100 beats per minute). If the first condition and the second condition are satisfied, the electronic device (200) can proceed to operation 813 and display a user interface (e.g., the third screen (910) of FIG. 9a) asking whether to proceed with hemoglobin measurement via image capture.

[0238] According to one embodiment, the electronic device (200) may allow hemoglobin measurement to proceed even if the first condition and the second condition are not satisfied, for example, even if the body temperature and heart rate deviate from a specified range for a certain period of time or more than a certain number of times due to frequent temperature changes or heart rate abnormalities. In such cases, the electronic device (200) may perform temperature correction and / or image correction using a correction model (e.g., the correction model shown in block 630 of FIG. 6). The electronic device (200) may correct distortion caused by temperature or correct the captured nail image. Accordingly, even if temporarily abnormal biological data is acquired, a more accurate and reliable hemoglobin estimate can be obtained through temperature correction and / or image correction.

[0239] FIG. 8b is a flowchart illustrating the process of an electronic device (200) according to one embodiment measuring hemoglobin.

[0240] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0241] According to one embodiment, operations 821 to 823 may be understood to be performed in a processor (e.g., processor (210) of FIG. 2) of an electronic device (e.g., electronic device (200) of FIG. 2).

[0242] According to one embodiment, in operation 821 (e.g., operations 310, 320 of FIG. 3A), an electronic device (200) (e.g., a user's smartphone (201)) can acquire (e.g., receive, detect) the user's skin temperature (e.g., finger skin temperature) through an external wearable electronic device (e.g., a smart watch (401), a smart ring (402)). The electronic device (200) can capture (or photograph) an image of the user's fingernail through a camera (230).

[0243] According to one embodiment, in operation 822 (e.g., operation 330 of FIG. 3a), the electronic device (200) can generate a corrected nail image by adjusting the color information (e.g., color intensity) of the nail image. The electronic device (200) can perform a hemoglobin measurement based on the adjusted color information (e.g., adjusted color intensity) of the corrected nail image.

[0244] According to one embodiment, in operation 823 (e.g., operation 340 of FIG. 3a), the electronic device (200) may display a user interface indicating the hemoglobin measurement result of operation 822.

[0245] FIG. 8c is a flowchart illustrating a process in which an electronic device (200) according to one embodiment adjusts personalized parameters for hemoglobin measurement using blood strip test data.

[0246] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0247] According to one embodiment, operations 831 to 834 may be understood to be performed in a processor (e.g., processor (210) of FIG. 2) of an electronic device (e.g., electronic device (200) of FIG. 2).

[0248] According to one embodiment, the personalization parameters may be used in the process of performing temperature correction and / or image correction using the correction model shown in block 630 of FIG. 6. For example, the blood strip test data may be stored as the user's personal information (security information) in a secure memory area (e.g., a storage linked to a user account, a storage requiring a user login for access) of a memory (220) or an external server (e.g., the server (108) of FIG. 1, the management server of the health application).

[0249] According to one embodiment, the electronic device (200) may perform hemoglobin measurement based further on the user's blood strip test data. The blood strip test data corresponds to the user's blood test results (e.g., actual hemoglobin concentration) and may be used to correct non-invasive hemoglobin measurement results. The electronic device (200) may adjust personalization parameters required for hemoglobin measurement based on the blood strip test data.

[0250] According to one embodiment, in operation 831, the electronic device (200) may collect the user's blood strip test result data, image data, and temperature data. The image data may include a plurality of nail images taken of the user's nails. The temperature data may include the skin temperature for each of the nail images. For example, the electronic device (200) (e.g., the user's smartphone (201)) may capture (or photograph) the plurality of nail images through a camera (230), and while capturing each nail image, may acquire (e.g., receive, detect) the user's skin temperature (e.g., finger skin temperature) through an external wearable electronic device (e.g., smart watch (401), smart ring (402)). The electronic device (200) may read the blood strip test data from memory (220) or an external server (e.g., server (108) of FIG. 1, management server of a health application) when user consent is set.

[0251] According to one embodiment, in operation 832, the electronic device (200) can link (e.g., pair) the blood strip test result data, the image data, and the temperature data obtained through operation 831. For example, when capturing a nail image, the electronic device (200) can obtain (e.g., receive, detect) the current skin temperature and extract the actual hemoglobin level from the blood strip test result data. The electronic device (200) can match and store the nail image, the current skin temperature, and the actual hemoglobin level. The actual hemoglobin level may additionally be paired with other physiological parameters (e.g., heart rate) other than the current skin temperature.

[0252] According to one embodiment, in operation 833, the electronic device (200) can adjust a personalization parameter (e.g., a correction factor (k)) required for hemoglobin measurement based on the result of data linkage in operation 832. For example, the personalization parameter may be used for correction of skin temperature and / or correction of nail images to be used for hemoglobin level estimation.

[0253] According to one embodiment, in operation 834, the electronic device (200) may store the value of the personalization parameter adjusted through operation 833. The electronic device (200) may perform hemoglobin measurements using the value of the personalization parameter for a certain period. The electronic device (200) may assign a confidence score to the hemoglobin measurement result. The electronic device (200) may maintain, readjust, or reset the value of the personalization parameter by reflecting feedback based on the confidence score.

[0254] According to one embodiment, the electronic device (200) can update the values ​​of a calibration model used for hemoglobin measurement and / or a personalized parameter used in the calibration model by repeating operations 831, 832, 833, and 834 at regular intervals (e.g., every 1 to 2 months). Accordingly, the calibration model can adapt to physiological changes of the user and improve the performance of the calibration model, the accuracy or reliability of the hemoglobin measurement.

[0255] FIG. 9a is an example of a user interface related to a process in which an electronic device (200) (e.g., a smartphone (201)) according to one embodiment triggers a hemoglobin measurement function.

[0256] According to one embodiment, an electronic device (200) (e.g., a smartphone (201)) may display a user interface such as the third screen (910) shown in FIG. 9a. For example, the user interface may be a user interface that asks whether to proceed with hemoglobin measurement via image capture. For example, the user interface may be displayed in response to the triggering condition of the hemoglobin measurement function being satisfied. For example, the electronic device (200) may determine that the triggering condition is satisfied when the execution of an application (e.g., a health application) is detected, when user input selecting a specific button within the application (e.g., a hemoglobin measurement button, a camera button) is detected, or when the user's body temperature falls within a specified range (e.g., a normal body temperature range between 36.5°C and 37.5°C).

[0257] Referring to FIG. 9a, the third screen (910) may include a notification message (911) asking whether to proceed with hemoglobin measurement via camera shooting (or image capture) and / or a shooting area (912) for capturing a nail image. For example, a guide image for a shooting posture (e.g., hand posture) may be displayed in the shooting area (912).

[0258] According to one embodiment, an electronic device (200) (e.g., a smartphone (201)) may display a user interface for guiding shooting conditions suitable for hemoglobin measurement and a specified skin temperature in real time.

[0259] FIG. 9b is an example of a user interface related to the process of an electronic device (200) (e.g., a smartphone (201)) according to one embodiment registering a reference image for hemoglobin measurement.

[0260] According to one embodiment, an electronic device (200) (e.g., a smartphone (201)) may display a user interface such as the fourth screen (920) shown in FIG. 9b. The user interface may be for registering a specified number (e.g., N) of reference nail images (or videos of a specified length) for hemoglobin measurement.

[0261] Referring to FIG. 9b, the fourth screen (920) may include a notification message (921) asking whether to register a reference nail image for hemoglobin measurement and / or a shooting area (922) for capturing the reference nail image. For example, a guide image for a shooting posture (e.g., hand posture) may be displayed in the shooting area (922). For example, information about current shooting conditions (e.g., current skin temperature, lighting, or shooting angle) may be displayed in the shooting area (922).

[0262] FIG. 10 is a drawing illustrating a wearable electronic device (1010) according to one embodiment.

[0263] According to one embodiment, the electronic device (200) may be a smart glass type wearable electronic device (1010).

[0264] Referring to FIG. 10, a wearable electronic device (1010) (e.g., smart glasses) may include a display (1020), a temperature sensor (1030), and a camera (1040). The temperature sensor (1030) may be in contact with a part of the user's body (e.g., around the ear) to measure body temperature.

[0265] According to one embodiment, the wearable electronic device (1010) can capture (or photograph) images of the user's fingernails (1051, 1052) through a camera (1040) while the wearable electronic device (1010) is worn by the user. The wearable electronic device (1010) can receive the user's skin temperature (e.g., finger skin temperature) from another wearable electronic device (e.g., smart watch (401), smart ring (402)) connected via short-range wireless communication. The wearable electronic device (1010) can measure hemoglobin using the fingernail images (1051, 1052) and the skin temperature (e.g., finger skin temperature) and display a user interface (e.g., augmented reality, AR object) for the hemoglobin measurement result through a display (1020).

[0266] According to one embodiment, the wearable electronic device (1010) can capture an image of the user's fingernails through a camera (1040) while the wearable electronic device (1010) is worn by the user. The wearable electronic device (1010) can detect the user's first skin temperature (e.g., body temperature) itself through a temperature sensor (1030). The wearable electronic device (1010) can receive the user's second skin temperature (e.g., hand temperature) from another wearable electronic device (e.g., smart watch (401), smart ring (402)) connected via short-range wireless communication. The first skin temperature and the second skin temperature may differ. For example, if the user has cold hands and feet, the difference between the hand temperature and the body temperature may be greater than that of a normal person. The wearable electronic device (1010) may select the first skin temperature for hemoglobin measurement when the difference between the first skin temperature (e.g., body temperature) and the second skin temperature (e.g., hand temperature) is greater than or equal to a specified offset, or when the second skin temperature deviates from a specified threshold. The wearable electronic device (1010) may measure hemoglobin using the first skin temperature and the nail images (1051, 1052). The wearable electronic device (1010) may display a user interface (e.g., AR object) for the hemoglobin measurement result through a display (1020).

[0267] An electronic device according to one embodiment of the present disclosure (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2) may include at least one processor (e.g., processor (210) of FIG. 2) having processing circuitry, a memory (e.g., memory (220) of FIG. 2) storing instructions, a camera (e.g., camera (230) of FIG. 2), a communication circuit (e.g., communication module (250) of FIG. 2), and a display (e.g., display (240) of FIG. 2). The above commands may be executed individually or collectively by the at least one processor to enable the electronic device to acquire the skin temperature of the user of the electronic device, acquire an image of the user's fingernail through the camera, determine the user's hemoglobin index based on the skin temperature and the fingernail image, and display a user interface that indicates the hemoglobin index or health information related to the hemoglobin index through the display.

[0268] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor to enable the electronic device to determine the hemoglobin index based on the nail image and to correct the hemoglobin index based on the difference between a specified skin temperature and the skin temperature.

[0269] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor to enable the electronic device to generate a corrected nail image from the nail image based on the skin temperature and to determine the hemoglobin index based on the corrected nail image.

[0270] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor to enable the electronic device to identify whether the skin temperature is included in a specified range, and if the skin temperature is included in the specified range, to determine the hemoglobin index based on color information of the nail image, and if the skin temperature is outside the specified range, to determine the hemoglobin index based on color information of the nail image, the difference between the specified skin temperature and the skin temperature, and personalization parameters.

[0271] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor so that the electronic device receives the skin temperature of the body part on which the wearable electronic device is worn through the communication circuit from the wearable electronic device worn by the user.

[0272] According to one embodiment of the present disclosure, the wearable electronic device may be a ring-type wearable electronic device. The instructions may be executed individually or collectively by the at least one processor so that the electronic device receives a finger skin temperature from the ring-type wearable electronic device through the communication circuit and determines the hemoglobin index based on the finger skin temperature.

[0273] According to one embodiment of the present disclosure, the wearable electronic device may be a watch-type or band-type wearable electronic device. The instructions may be executed individually or collectively by the at least one processor so that the electronic device receives the skin temperature of the wearing area from the watch-type or band-type wearable electronic device, converts the received skin temperature into a finger skin temperature, and determines the hemoglobin index based on the finger skin temperature.

[0274] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor so that the electronic device collects skin temperatures by body part from two or more wearable electronic devices worn by the user through the communication circuit, determines a finger skin temperature based on the collected skin temperatures by body part, and determines a hemoglobin index based on the finger skin temperature.

[0275] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor so that the electronic device displays an execution screen of an application through the display, activates the camera while the execution screen is displayed, acquires a plurality of fingernail images through the camera, and determines at least one fingernail image among the plurality of fingernail images to be used for determining the hemoglobin index based on at least one of the meta information of the plurality of fingernail images or image-specific attribute information extracted from the plurality of images.

[0276] According to one embodiment of the present disclosure, the instructions are executed individually or collectively by the at least one processor, so that the electronic device displays an execution screen of an application through the display, wherein the execution screen includes a user interface that guides shooting conditions for registering a specified number of reference nail images, and while the execution screen is displayed, the camera is activated, the reference nail images are acquired through the camera, and information regarding the reference nail images is stored in the memory.

[0277] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor so that the electronic device adjusts the value of a personalization parameter from a default value to a personalized value based on the reference nail images and stores the personalized value as information.

[0278] According to one embodiment of the present disclosure, the shooting conditions may include conditions related to temperature.

[0279] According to one embodiment of the present disclosure, the shooting conditions may further include conditions related to at least one of lighting, shooting angle, or time.

[0280] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor so that the electronic device obtains an external temperature through another temperature sensor, identifies whether the external temperature is outside a threshold range, and displays a user interface through the display indicating that hemoglobin measurement is impossible if the skin temperature is outside the threshold range.

[0281] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor to enable the electronic device to identify whether the skin temperature is included in a specified range, determine the hemoglobin index based on the nail image if the skin temperature is included in the specified range, and determine the hemoglobin index based on the corrected nail image if the skin temperature is outside the specified range.

[0282] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor to enable the electronic device to extract color information from the nail image, correct the color information based on a specified skin temperature and the difference between the skin temperature and a personalization parameter, and to generate the corrected nail image by applying the corrected color information to the nail image.

[0283] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor so that the electronic device receives the skin temperature through the communication circuit from a wearable electronic device worn by the user, detects the skin temperature through a temperature sensor of the electronic device, or obtains the skin temperature through at least one electronic device selected by user input among the wearable electronic device and the electronic device.

[0284] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor so that the electronic device receives a finger skin temperature from the ring-type wearable electronic device through the communication circuit and generates the corrected nail image based on the finger skin temperature.

[0285] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor so that the electronic device receives a skin temperature of a wearing area from the watch-type or band-type wearable electronic device, converts the received skin temperature into a finger skin temperature, and generates the corrected nail image based on the finger skin temperature.

[0286] According to one embodiment of the present disclosure, the instructions may be executed individually or collectively by the at least one processor so that the electronic device collects skin temperatures by body part from two or more wearable electronic devices through the communication circuit, determines a finger skin temperature based on the collected skin temperatures by body part, and generates a corrected nail image based on the finger skin temperature.

[0287] A method of operation of an electronic device according to one embodiment of the present disclosure (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2) may include the operation of acquiring the skin temperature of a user of the electronic device, the operation of acquiring an image of the user's fingernail through a camera of the electronic device, the operation of determining the user's hemoglobin index based on the skin temperature and the fingernail image, and the operation of displaying a user interface that indicates the hemoglobin index or health information related to the hemoglobin index through a display of the electronic device.

[0288] According to one embodiment of the present disclosure, the operation of determining the hemoglobin index may include the operation of determining the hemoglobin index based on the nail image, and the operation of correcting the hemoglobin index based on the difference between a specified skin temperature and the skin temperature.

[0289] According to one embodiment of the present disclosure, the operation of determining the hemoglobin index may include the operation of generating a corrected nail image from the nail image based on the skin temperature, and the operation of determining the hemoglobin index based on the corrected nail image.

[0290] According to one embodiment of the present disclosure, the operation of determining the hemoglobin index may include the operation of identifying whether the skin temperature is included in a specified range, the operation of determining the hemoglobin index based on color information of the nail image when the skin temperature is included in the specified range, and the operation of determining the hemoglobin index based on color information of the nail image, the difference between the specified skin temperature and the skin temperature, and personalization parameters when the skin temperature is outside the specified range.

[0291] According to one embodiment of the present disclosure, the operation of obtaining the skin temperature may include at least one of the operation of receiving the skin temperature through the communication circuit from a wearable electronic device worn by the user, the operation of detecting the skin temperature through a temperature sensor of the electronic device, or the operation of obtaining the skin temperature through at least one electronic device selected by user input among the wearable electronic device and the electronic device.

[0292] According to one embodiment of the present disclosure, the operation of obtaining the skin temperature may include receiving the skin temperature of the wearing area from a watch-type or band-type wearable electronic device worn by the user, and converting the received skin temperature into a finger skin temperature.

[0293] A storage medium according to one embodiment of the present disclosure may be a computer-readable, non-transient storage medium. The storage medium may have at least one program recorded thereon, comprising instructions for executing a method of operation of an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2). The storage medium may have at least one program recorded thereon for executing a method including the operation of acquiring the skin temperature of a user of the electronic device, the operation of acquiring an image of the user's fingernails through a camera of the electronic device, the operation of determining the user's hemoglobin index based on the skin temperature and the fingernail image, and the operation of displaying a user interface that indicates the hemoglobin index or health information related to the hemoglobin index through a display of the electronic device.

[0294]

[0295] An electronic device and its method of operation according to various embodiments of the present disclosure can improve the accuracy of non-invasive hemoglobin measurement and ensure the reliability and consistency of said measurement.

[0296] An electronic device and its method of operation according to various embodiments of the present disclosure can provide a hemoglobin measurement result suitable for an individual user (personalized hemoglobin measurement result) by taking into account the individual characteristics of the user.

[0297] An electronic device and its method of operation according to various embodiments of the present disclosure can improve the accuracy of hemoglobin measurement by excluding temporary influences caused by the surrounding environment (or external environment) or physiological state when analyzing nail color for hemoglobin measurement.

[0298] An electronic device and its method of operation according to various embodiments of the present disclosure can guide the user in real time to suitable environments or conditions required for hemoglobin measurement and improve the user experience.

[0299] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description of the present disclosure.

[0300] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0301] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0302] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0303] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0304] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0305] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device, At least one processor including processing circuitry; Memory for storing instructions; camera; Communication circuit; and Includes a display, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Acquire the skin temperature of the user of the above electronic device, and Acquire an image of the user's fingernail through the camera above, and Determining the user's hemoglobin index based on the skin temperature and nail image, An electronic device that displays a user interface indicating the hemoglobin indicator or health information related to the hemoglobin indicator through the above display.

2. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Determining the hemoglobin indicator based on the above nail image, An electronic device for correcting the hemoglobin index based on the difference between a specified skin temperature and the skin temperature.

3. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Based on the above skin temperature, a corrected nail image is generated from the nail image, and An electronic device for determining the hemoglobin indicator based on the above-mentioned corrected nail image.

4. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Identify whether the above skin temperature falls within a specified range, and If the above skin temperature falls within the above specified range, the hemoglobin index is determined based on the color information of the above nail image, and An electronic device that determines the hemoglobin index based on color information of the nail image, the difference between the specified skin temperature and the skin temperature, and personalization parameters when the skin temperature falls outside the specified range.

5. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Receiving the skin temperature through the communication circuit from a wearable electronic device worn by the above user, Detecting the skin temperature through the temperature sensor of the electronic device, An electronic device that obtains the skin temperature through the above-mentioned wearable electronic device and at least one electronic device selected by user input among the above-mentioned electronic devices.

6. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Receiving finger skin temperature through the communication circuit from a ring-type wearable electronic device worn by the above user, and An electronic device for determining the hemoglobin indicator based on the finger skin temperature.

7. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Receiving the skin temperature of the wearing area from a watch-type or band-type wearable electronic device worn by the above user, and Converting the received skin temperature above into finger skin temperature, and An electronic device for determining the hemoglobin indicator based on the finger skin temperature.

8. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Collecting skin temperatures by body part from two or more wearable electronic devices worn by the above user through the communication circuit, and Determine the finger skin temperature based on the skin temperatures of each body part collected above, and An electronic device for determining the hemoglobin indicator based on the finger skin temperature.

9. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: The application execution screen is displayed through the above display, and Activate the camera while the above execution screen is displayed, and Acquire multiple fingernail images through the above camera, and An electronic device for determining at least one nail image among the plurality of nail images to be used for determining the hemoglobin index based on at least one of the meta information of the plurality of nail images or image-specific attribute information extracted from the plurality of images.

10. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: The application execution screen is displayed through the above display, and the execution screen includes a user interface that guides shooting conditions for registering a specified number of standard fingernail images. Activate the camera while the above execution screen is displayed, and Acquire the reference fingernail images through the camera above, and An electronic device that stores information about the reference fingernail images in the memory.

11. In Claim 10, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Based on the above reference nail images, the value of the personalization parameter is adjusted from the default value to a personalized value, and An electronic device that stores the above personalized value as the above information.

12. In Claim 10, The above shooting conditions are an electronic device including temperature-related conditions.

13. In Claim 11, The above shooting conditions include an electronic device further comprising conditions related to at least one of lighting, shooting angle, or time.

14. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, and the electronic device: Obtain the external temperature through another temperature sensor, and Identify whether the above external temperature exceeds the critical range, and An electronic device that displays a user interface indicating that hemoglobin measurement is impossible through the display when the skin temperature exceeds the threshold range.

15. In a method of operating an electronic device, The operation of obtaining the user's skin temperature of the above electronic device; The operation of acquiring an image of the user's fingernail through the camera of the electronic device; The operation of determining the user's hemoglobin index based on the skin temperature and the nail image; and A method comprising the operation of displaying a user interface indicating the hemoglobin indicator or health information related to the hemoglobin indicator through a display of the electronic device.

Images