Electronic device and food recommendation method in electronic device

Abstract

An electronic device according to an embodiment comprises a camera, a communication circuit, a display, a memory storing instructions, and at least one processor, wherein the instructions, when individually and / or collectively executed by the at least one processor, cause the electronic device to: acquire images through the camera; receive, through the communication circuit, sensor values detected by at least one external sensor; identify food image areas included in the images; identify a user's gesture for food intake on the basis of the food image areas and the sensor values; identify, on the basis of the identification of the user's gesture for food intake, foods corresponding to the food image areas and sugar content values corresponding to the foods; identify priorities of the foods by using the sugar content values corresponding to the foods; and display information indicating an intake order of the foods through the display on the basis of the priorities of the foods. Other embodiments may be included.

Description

Electronic devices and food recommendation methods in electronic devices

[0001] Various embodiments of the present disclosure relate to an electronic device and a method for recommending food in an electronic device.

[0002] Various services and additional functions provided through electronic devices, such as smartphones and wearable devices, are gradually increasing. To enhance the utility value of these electronic devices and satisfy the needs of diverse users, telecommunications service providers and electronic device manufacturers are competitively developing devices to offer various functions and differentiate themselves from competitors. Consequently, the functions provided through electronic devices are also becoming increasingly sophisticated.

[0003] Recently, there has been an increasing number of people who are interested in health and want information on what foods to eat for their health, and electronic devices may provide a function that offers information about foods that are good for the user's body.

[0004] There are various ways for electronic devices to provide information about food. Conventional electronic devices may not be able to recommend specifically which foods to consume and how to consume them (e.g., the order of consumption) while the user is eating (e.g., during a meal). Furthermore, conventional electronic devices may not be able to notify the user in real-time during a meal if it is anticipated that the food consumed will cause a blood sugar spike.

[0005] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs.

[0006] An electronic device according to one embodiment of the present disclosure may include a camera, a communication circuit, a display, a memory for storing commands, and at least one processor. According to one embodiment, the commands, when executed individually and / or collectively by the at least one processor, may cause the electronic device to acquire images through the camera. The commands, when executed individually and / or collectively by the at least one processor, may cause the electronic device to receive sensor values ​​detected by at least one external sensor through the communication circuit. The commands, when executed individually and / or collectively by the at least one processor, may cause the electronic device to identify food image regions included in the images. The commands, when executed individually and / or collectively by the at least one processor, may cause the electronic device to identify a user's gesture for eating food based on the food image regions and the sensor values. When the above commands are executed individually and / or collectively by the at least one processor, the electronic device may identify foods corresponding to the food image regions and sugar content values ​​corresponding to the foods based on the identification of the user's gesture for food consumption. When the above commands are executed individually and / or collectively by the at least one processor, the electronic device may identify priorities of the foods using the sugar content values ​​corresponding to the foods. When the above commands are executed individually and / or collectively by the at least one processor, the electronic device may display information indicating the order of consumption of the foods through the display based on the priorities of the foods.

[0007] A food recommendation method in an electronic device according to one embodiment of the present disclosure may include an operation of acquiring images through a camera of the electronic device. The method may include an operation of receiving sensor values ​​detected by at least one external sensor through a communication circuit of the electronic device. The method may include an operation of identifying food image regions included in the images. The method may include an operation of identifying a user's gesture for food consumption based on the food image regions and the sensor values. The method may include an operation of identifying foods corresponding to the food image regions and sugar content values ​​corresponding to the foods based on the identification of the user's gesture for food consumption. The method may include an operation of identifying priorities of the foods using the sugar content values ​​corresponding to the foods. The method may include an operation of displaying information indicating the order of consumption of the foods through a display of the electronic device based on the priorities of the foods.

[0008] According to one embodiment of the present disclosure, in a non-transitory storage medium storing commands, the commands are configured to cause the electronic device to perform at least one operation when executed by the electronic device, wherein the at least one operation may include the operation of acquiring images through a camera of the electronic device. The at least one operation may include the operation of receiving sensor values ​​detected by at least one external sensor through a communication circuit of the electronic device. The at least one operation may include the operation of identifying food image regions included in the images. The at least one operation may include the operation of identifying a user's gesture for eating food based on the food image regions and the sensor values. The at least one operation may include the operation of identifying foods corresponding to the food image regions and sugar content values ​​corresponding to the foods based on the identification of the user's gesture for eating food. The at least one operation may include the operation of identifying priorities of the foods using the sugar content values ​​corresponding to the foods. The above at least one operation may include an operation of displaying information indicating the order of consuming the foods through the display of the electronic device based on the priorities of the foods.

[0009] FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment.

[0010] FIG. 2a is a drawing showing an electronic device according to one embodiment.

[0011] FIG. 2b is a diagram showing the structure of a display and an eye-tracking camera of an electronic device according to one embodiment.

[0012] FIG. 3 is a diagram showing an example of a case where a user is eating while an electronic device according to one embodiment is being worn by the user.

[0013] FIG. 4 is a block diagram of an electronic device according to one embodiment.

[0014] FIG. 5 is a flowchart illustrating the operation of providing a user with the order of consumption of foods in an electronic device according to one embodiment.

[0015] FIG. 6 is a flowchart illustrating an operation to provide information to lower a user's blood glucose rise rate value in an electronic device according to one embodiment.

[0016] FIG. 7 is a flowchart illustrating the operation of obtaining a sugar content value and a blood glucose rise rate value corresponding to a new food in an electronic device according to one embodiment and updating a database.

[0017] FIG. 8 is a diagram showing an example of displaying the order of consumption of foods while the foods are displayed to the user through an electronic device according to one embodiment.

[0018] FIG. 9 is a diagram showing an example indicating that the amount of sugar consumed by a user through an electronic device according to one embodiment exceeds a specified amount of sugar intake.

[0019] FIG. 10 is a diagram showing an example of displaying information recommending food to be consumed by a user through an electronic device according to one embodiment.

[0020] FIG. 11 is a diagram showing an example of displaying information about food that is not recommended for consumption by a user through an electronic device according to one embodiment.

[0021] FIG. 12 is a diagram showing an example of displaying information that guides a user to reduce the speed of consuming food through an electronic device according to one embodiment.

[0022] FIG. 13a is a diagram showing an example of a first exercise path recommended based on the user's location in an electronic device according to one embodiment.

[0023] FIG. 13b is a diagram showing an example of a second exercise path recommended based on the user's location in an electronic device according to one embodiment.

[0024] FIG. 14 is a diagram showing an example of eating while an electronic device according to one embodiment is worn by a user.

[0025] FIG. 15 is a diagram showing an example of a screen displaying the user's intake of nutrients and recommendation information according to one embodiment.

[0026] FIG. 16 is a diagram showing an example of a screen displaying the user's intake of nutrients, whether insufficient, appropriate, or excessive, according to one embodiment.

[0027] FIG. 17a is a diagram illustrating an example of recommending food when a user's carbohydrate intake is identified through an electronic device as exceeding a specified amount of carbohydrate intake while a menu is being displayed to the user according to one embodiment.

[0028] FIG. 17b is a diagram illustrating an example of recommending food through an electronic device when the intake of nutritional components of a user satisfies the intake range of specified image components while a menu is displayed to the user according to one embodiment.

[0029] Hereinafter, electronic devices according to various embodiments will be examined with reference to the attached drawings. In the various embodiments, the term "user" may refer to a person using the electronic device or a device using the electronic device (e.g., an artificial intelligence electronic device).

[0030] The terms used in this document are used merely to describe specific embodiments and are not intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly indicates otherwise. All terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art of the present invention. Terms defined in commonly used dictionaries may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this document. In some cases, even terms defined in this document shall not be interpreted to exclude embodiments of the present invention.

[0031] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to one embodiment.

[0032] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0033] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0034] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0035] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, software (e.g., program (140)) and input data or output data for related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0036] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0037] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0038] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0039] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0040] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0041] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0042] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0043] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0044] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0045] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0046] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0047] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0048] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0049] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0050] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0051] According to one embodiment, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0052] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0053] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology. The electronic device according to one embodiment disclosed in this document may be a device of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, a home appliance device, or an XR (extended reality) electronic device. The electronic device according to the embodiment of this document is not limited to the devices described above.

[0054] FIG. 2a is a drawing showing an electronic device according to one embodiment.

[0055] Referring to FIG. 2a, an electronic device (200) according to one embodiment (e.g., the electronic device (101) of FIG. 1) (or an XR (extended reality) electronic device or AR (augment reality) glasses) can perform XR functions or provide XR content.

[0056] An electronic device (200) according to one embodiment may include a frame (205), a first support member (201), a second support member (202), a first hinge member (203) connecting the frame (205) and the first support member (201), and / or a second hinge member (204) connecting the frame (205) and the second support member (202). According to one embodiment, the frame (205) may include at least one camera (e.g., a first camera (211-1, 211-2), a second camera (212-1, 212-2) and / or a third camera (213)), one or more light-emitting elements (214-1, 214-2), at least one display (e.g., a first display (251), a second display (252)), one or more sound input devices (262-1, 262-2, 262-3), or one or more transparent members (290-1, 290-2). An electronic device (200) according to one embodiment may include one or more first cameras (211-1, 211-2), one or more second cameras (212-1, 212-2), and / or one or more third cameras (213). Images acquired through one or more first cameras (211-1, 211-2) according to one embodiment may be used for detecting hand gestures by a user, tracking the user's head, and / or spatial recognition. One or more first cameras (211-1, 211-2) according to one embodiment may be GS (global shutter) cameras. One or more first cameras (211-1, 211-2) according to one embodiment may perform SLAM (simultaneous localization and mapping) operations through depth imaging. One or more first cameras (211-1, 211-2) according to one embodiment may perform spatial recognition for 6 DoF (degrees of freedom). Images acquired through one or more second cameras (212-1, 212-2) according to one embodiment may be used to detect and track the user's pupils.For example, images acquired through one or more second cameras (212-1, 212-2) may be used to track the user's gaze direction. One or more second cameras (212-1, 212-2) according to one embodiment may be GS cameras. One or more second cameras (212-1, 212-2) according to one embodiment may correspond to the left eye and the right eye, respectively, and the performance of one or more second cameras (212-1, 212-2) may be substantially identical. One or more third cameras (213) according to one embodiment may be high-resolution cameras. One or more third cameras (213) according to one embodiment may perform an auto-focusing (AF) function or a shake correction function. One or more third cameras (213) according to one embodiment may be GS cameras or RS (rolling shutter) cameras. An electronic device (200) according to one embodiment may include one or more light-emitting elements (214-1, 214-2). The light-emitting elements (214-1, 214-2) are different from the light source described below, which irradiates light onto a screen output area of ​​a display. The light-emitting elements (214-1, 214-2) according to one embodiment may irradiate light to facilitate pupil detection in detecting and tracking a user's pupil through one or more second cameras (212-1, 212-2). The light-emitting elements (214-1, 214-2) according to one embodiment may include LEDs. The light-emitting elements (214-1, 214-2) according to one embodiment may irradiate light in the infrared region. The light-emitting elements (214-1, 214-2) according to one embodiment may be attached around the frame (205) of the electronic device (200).A light-emitting element (214-1, 214-2) according to one embodiment is located around one or more first cameras (211-1, 211-2) and can assist gesture detection, head tracking, and / or spatial recognition by one or more first cameras (211-1, 211-2) when the electronic device (200) is used in a dark environment. A light-emitting element (214-1, 214-2) according to one embodiment is located around one or more third cameras (413) and can assist image acquisition by one or more third cameras (213) when the electronic device (200) is used in a dark environment. An electronic device (200) according to one embodiment may include a first display (251), a second display (252), one or more input optical members (253-1, 253-2), one or more transparent members (290-1, 290-2), and / or one or more screen display portions (254-1, 254-2) located in a frame (205). The first display (251) and the second display (252) according to one embodiment may include, for example, a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED). In the case where the first display (251) and the second display (252) according to one embodiment are composed of a liquid crystal display device, a digital mirror display device, or a silicon liquid crystal display device, the electronic device (200) may include a light source that irradiates light to the screen output area of ​​the display.In one embodiment, if the first display (251) and / or the second display (252) can generate light on their own, for example, by being composed of one of an organic light-emitting diode or a micro LED, the electronic device (200) can provide a good quality virtual image to the user without including a separate light source. In one embodiment, one or more transparent members (290-1, 290-2) may be positioned facing the user's eyes when the user wears the electronic device (200). In one embodiment, one or more transparent members (290-1, 290-2) may include at least one of a glass plate, a plastic plate, or a polymer. In one embodiment, the user can see the outside world through one or more transparent members (290-1, 290-2) when wearing the electronic device (200). According to one embodiment, one or more input optical members (253-1, 253-2) can guide light generated from the first display (251) and / or the second display (252) to the user's eye. According to one embodiment, an image based on the light generated from the first display (251) and the second display (252) is formed on one or more screen display portions (254-1, 254-2) on one or more transparent members (290-1, 290-2), and the user can view the image formed on the one or more screen display portions (254-1, 254-2). According to one embodiment, the electronic device (200) may include one or more optical waveguides (not shown). The optical waveguides can transmit light generated from the first display (251) and the second display (252) to the user's eye. The electronic device (200) may include one optical waveguide each corresponding to the left eye and the right eye. An optical waveguide according to one embodiment may include at least one of glass, plastic, or polymer.An optical waveguide according to one embodiment may include a nano-pattern formed on an internal or external surface, for example, a grating structure having a polygonal or curved shape. An optical waveguide according to one embodiment may include a free-form prism, in which case the optical waveguide may provide incident light to a user through a reflective mirror. An optical waveguide according to one embodiment may include at least one diffractive element (e.g., DOE (diffractive optical element), HOE (holographic optical element)) or a reflective element (e.g., a reflective mirror), and may guide display light emitted from a light source to the user's eye using at least one diffractive element or reflective element included in the optical waveguide. An diffractive element according to one embodiment may include an input / output optical member. An reflective element according to one embodiment may include a member that causes total internal reflection. An electronic device (200) according to one embodiment may include one or more sound input devices (262-1, 262-2, 262-3) (e.g., a microphone), and the one or more sound input devices (262-1, 262-2, 262-3) may receive sound from a user or sound generated in the vicinity of the electronic device (200). For example, the one or more sound input devices (262-1, 262-2, 262-3) may receive sound generated in the vicinity and transmit it to a processor (e.g., the processor (420) of FIG. 4). One or more support members (e.g., first support member (201), second support member (202)) according to one embodiment may include a PCB (e.g., first PCB (270-1) and second PCB (270-2)), one or more sound output devices (263-1, 263-2), and / or one or more batteries (235-1, 235-2).The first PCB (270-1) and the second PCB (270-2) can transmit electrical signals to components included in the electronic device (400), such as the first camera (411), the second camera (412), the third camera (413), the display (450), the audio device (461), and / or the sensor (480), which will be described later with reference to FIG. 4. In one embodiment, the first PCB (270-1) and the second PCB (270-2) may be FPCBs. In one embodiment, the first PCB (270-1) and the second PCB (270-2) may each include a first substrate, a second substrate, and an interposer disposed between the first substrate and the second substrate. In one embodiment, the electronic device (200) may include a battery (235-1, 235-2). A battery (235-1, 235-2) can store power to operate the remaining components of the electronic device (200). One or more sound output devices (263-1, 263-2) (e.g., speakers) according to one embodiment can output audio data to a user. For example, they can provide feedback on a user's command (or input) or provide information about a virtual object to the user through audio data. An electronic device (200) according to one embodiment may include one or more hinge portions (e.g., a first hinge portion (203), a second hinge portion (204)). For example, the first hinge portion (203) can be configured so that the first support portion (201) is coupled to the frame (205) and can rotate relative to the frame (205), and the second hinge portion (204) can be configured so that the second support portion (202) is coupled to the frame (205) and can rotate relative to the frame (205).

[0057] FIG. 2b is a diagram showing the structure of a display and an eye-tracking camera of an electronic device according to one embodiment.

[0058] Referring to FIG. 2b, an electronic device (200) according to one embodiment may include a display (221), an input optical member (222), a display optical waveguide (223), an output optical member (224), an eye-tracking camera (210), a first splitter (241), an eye-tracking optical waveguide (242), and / or a second splitter (243).

[0059] In an electronic device (200) according to one embodiment, the display (221) may be the first display (251) or the second display (252) shown in FIG. 2a. Light output from the display (221) passes through an input optical member (222) (e.g., input optical member (253-1, 253-2) of FIG. 2a) and is incident on a display optical waveguide (223), and may pass through the display optical waveguide (223) and be output through an output optical member (224). The light output from the output optical member (224) can be seen by the user's eye (230). According to one embodiment, light output from the display (221) is output through the output optical member (224), and the shape of an object is seen by the user's eye (230) by the light output through the output optical member (224), which may mean displaying an object on the display. Light output from a display (221) according to one embodiment is output through an output optical member (224), and controlling the display (221) so that the shape of an object is visible to the user's eye (230) by the light output through the output optical member (224) may mean controlling to display the object. Light (232) reflected from the user's eye (230) passes through a first splitter (241) and is incident on an eye-tracking optical waveguide (242), and may pass through the eye-tracking optical waveguide (242) and be output to an eye-tracking camera (210) through a second splitter (243). According to various embodiments, the light (232) reflected from the user's eye (230) may be light output from the light-emitting elements (214-1, 214-2) of FIG. 2A and reflected from the user's eye (230). According to various embodiments, the eye-tracking camera (210) may be one or more second cameras (212-1, 212-2) as shown in FIG. 2a.

[0060] FIG. 3 is a diagram showing an example of a case where a user is eating while an electronic device according to one embodiment is being worn by the user.

[0061] Referring to FIG. 3, an electronic device (200) according to one embodiment (e.g., the electronic device (101) of FIG. 1) can acquire information related to the user's meal in real time while the user is wearing (or carrying) the device. For example, information related to the user's meal may include information about foods around the user (e.g., types of food), gestures for the user to consume food (e.g., gestures of bringing food to the mouth), personal health information of the user (e.g., health information requiring dietary therapy such as obesity, diabetes, hypertension, malnutrition, information about medications the user is taking, and / or other information related to the user's personal health), and / or location information of the user. An electronic device (200) according to one embodiment can photograph the surroundings of a user through at least one camera (e.g., camera module (180) of FIG. 1 or the first camera (211-1, 211-2), second camera (212-1, 212-2) and / or third camera (213)) while the user is eating, and can obtain information about food around the user using the captured images. An electronic device (200) according to one embodiment receives sensor data (or sensor value) sensed by at least one external sensor (e.g., at least one sensor of an external electronic device (302) (e.g., an electronic device (102) of FIG. 1 (e.g., a wearable electronic device or a smart watch or other electronic device capable of sensing a user’s gesture)) and can identify a user’s gesture for eating food using the received sensor value. An electronic device (200) according to one embodiment can obtain user’s personal health information stored in advance through a memory (e.g., memory (130) of FIG. 1) or obtain user’s personal health information from the outside through a communication circuit (e.g., communication module (190) of FIG. 1). An electronic device (200) according to one embodiment can obtain user’s location information through an external or internal location sensor (e.g., sensor module (176) of FIG. 1) (e.g., GPS sensor).An electronic device (200) according to one embodiment may provide the user with at least one food-related recommendation information using information related to the user's meal obtained in real time during the user's meal. For example, the electronic device (200) may provide an order of intake for at least some or all of the foods using information related to the user's meal obtained in real time during the user's meal. For example, the electronic device (200) may provide information to lower the user's blood sugar rise rate using information related to the user's meal obtained in real time during the user's meal. For example, the electronic device (200) may provide information indicating that the amount of sugar consumed by the user during the meal exceeds a specified intake threshold value, information recommending food for the user to consume, information not recommended for the user, information guiding the user to reduce the speed of food intake, information on foods that are good to consume together, and / or information recommending exercise (e.g., displayed via a display or sound output via a speaker) using information related to the user's meal obtained in real time during the user's meal. For example, the electronic device (200) can identify information about the medication the user is taking based on the user's personal health information and provide a notification (e.g., display via a display or output sound via a speaker) to prevent blood sugar spikes or hypoglycemic shocks from occurring when blood sugar management is required based on the information about the medication the user is taking.

[0062] FIG. 4 is a block diagram of an electronic device according to one embodiment.

[0063] Referring to FIG. 4, an electronic device (400) according to one embodiment (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2) may include a first camera (411), a second camera (412), a third camera (413), a processor (420), a power management integrated chip (PMIC) (430), a battery (435), a memory (440), a display (450), an audio device (461), a sound input device (462), a sound output device (463), a communication circuit (470), and / or a sensor (480). The electronic device (400) according to one embodiment is not limited thereto and may be configured to include various additional components or to exclude some of the above components. The electronic device (400) according to one embodiment may further include all or part of the electronic device (101) shown in FIG. 1 (and / or the electronic device (200) shown in FIG. 2).

[0064] A processor (420) according to one embodiment may be composed of one or more processors. In this case, the one or more processors may be a general-purpose processor such as a CPU, AP, DSP (Digital Signal Processor), a graphics-dedicated processor such as a GPU, VPU (Vision Processing Unit), or an artificial intelligence-dedicated processor such as an NPU. The one or more processors may be controlled to process input data according to a predefined operation rule or artificial intelligence model stored in memory (440). Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0065] Predefined behavioral rules or artificial intelligence models can be created through learning. Here, being created through learning means that a basic artificial intelligence model is trained using multiple learning data by a learning algorithm, thereby creating predefined behavioral rules or artificial intelligence models configured to perform desired characteristics (or objectives). Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0066] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0067] A processor (420) according to one embodiment can acquire images through a first camera (411), a second camera (412), and / or a third camera (413).

[0068] A processor (420) according to one embodiment can acquire images of the user's surroundings (e.g., the user's field of vision while eating) through a first camera (411), a second camera (412), and / or a third camera (413) while the electronic device (400) is worn (or carried) by the user. A processor (420) according to one embodiment can identify food image regions included in the images. A processor (420) according to one embodiment can identify food image regions included in the images. According to one embodiment, when a first image is acquired, the processor (420) can detect at least one or more food objects (or food image regions) included in the first image using an engine (or software or program) (or artificial intelligence model) for object detection. For example, the engine for object detection may include an engine based on R-CNN (regions with convolutional neutral network) or YOLO (you only look once) technology.

[0069] A processor (420) according to one embodiment may receive sensor values ​​detected by at least one external sensor (e.g., an external electronic device (302) (e.g., at least one sensor of the electronic device (102) of FIG. 1 (e.g., a wearable electronic device or a smart watch)) through a communication circuit (470). According to one embodiment, the at least one sensor may include an accelerometer, a gyroscope, and / or a gravity sensor. According to one embodiment, the processor (420) may identify a user's gesture using the sensor values ​​detected by the at least one sensor.

[0070] A processor (420) according to one embodiment can identify a user's gesture for eating food based on food image regions and sensor values. For example, the processor (420) can detect a user's gesture for eating food if at least one food image region is identified and the user gesture based on the sensor values ​​is a user gesture associated with at least one food image region.

[0071] A processor (420) according to one embodiment can identify at least one food (e.g., at least one type of food) corresponding to at least one food image area. A processor (420) according to one embodiment can identify at least one food corresponding to (or matching with) the image characteristics of at least one food image area by using an artificial intelligence model that has learned the image characteristics of each of a plurality of foods (e.g., braised short ribs, rice, pork belly, lettuce, seasoned spinach, braised fish, broccoli, spicy stir-fried chicken, and / or other foods). A processor (420) according to one embodiment can identify each food in at least one food image area by using a large-scale pre-trained artificial intelligence model learned based on a large number of food images. A processor (420) according to one embodiment can identify each food in at least one food image area by using an artificial intelligence model learned from food images taken by a user. For example, the processor (420) can identify a general braised short rib included in the taken images by using an artificial intelligence model learned from a large number of various braised short rib images. For example, the processor (420) can identify the specified galbi-jjim included in the captured images using an artificial intelligence model that has learned galbi-jjim images (of a specific place or a specific store or a specific chef) associated with the user.

[0072] A processor (420) according to one embodiment can identify a sugar content value corresponding to each of at least one food by using a database containing predetermined (or defined, detected, or investigated) sugar content values ​​corresponding to each of a plurality of foods (or a plurality of food types). A database according to one embodiment may include predetermined sugar content values ​​corresponding to each of a plurality of foods. A database according to one embodiment may further include blood glucose rise rate values ​​measured after a designated amount of each of a plurality of foods is consumed by a user, along with the predetermined sugar content values ​​corresponding to each of a plurality of foods.

[0073] Table 1 below is a table showing examples of sugar content values ​​and blood glucose elevation rate values ​​corresponding to each of multiple foods (e.g., multiple types of food).

[0074] Food (e.g., Food Type) Per Serving Value (%) User Blood Sugar Increase Rate Value (%) (Value measured by user) Braised Short Ribs 3320 White Rice 7540 Pork Belly 00 Lettuce 0.941 Seasoned Spinach 0.591 Braised Fish 3.34 Broccoli 1.382 Spicy Stir-fried Chicken 2.513 … … …

[0075] Referring to Table 1 above, the sugar content value (%) according to one embodiment may be the amount of sugar contained per 100g of food. The user blood sugar rise rate value according to one embodiment may refer to the ratio of the increase in blood sugar measured after a specified time following the user's consumption of 100g of food from the blood sugar value measured before consuming the food. A processor (420) according to one embodiment can identify, using Table 1, that the sugar content value corresponding to galbi-jjim is 33% and the user blood sugar rise rate value is 20% when the food in the food image area is galbi-jjim. A processor (420) according to one embodiment can identify, using Table 1, that the sugar content corresponding to rice is 75% and the user blood sugar rise rate is 40% when the food corresponding to the food image area is white rice. The foods, sugar content values, and user blood glucose rise rate values ​​shown in Table 1 above are merely examples, and other foods, other sugar content values, and other user blood glucose rise rate values ​​may be used. A processor (420) according to one embodiment can identify the priority of at least one food using at least one sugar content value (and / or blood glucose rise rate value) corresponding to at least one food in at least one food area. A processor (420) according to one embodiment can identify the priorities of multiple foods by assigning higher priority to multiple sugar content values ​​(and / or user blood glucose rise rate values) corresponding to multiple foods in multiple food areas in order of decreasing value. For example, the processor (420) can identify the food having the smallest sugar content value (and / or user blood glucose rise rate value) as the food of the highest priority. A processor (420) according to one embodiment can display information indicating the order of consumption of the foods based on the priority of the foods through a display (450).A processor (420) according to one embodiment can determine the order of consumption of foods in order of highest priority and display the determined order of consumption of foods through a display (450). For example, the processor (420) can display a virtual image indicating the order of consumption of foods while the actual foods are visible to the user. For example, the processor (420) can identify the order of consumption of foods including spinach, braised fish, and rice as Vegetable (Spinach) -> Meat (Braised Fish) -> Rice and display a virtual image indicating the identified order of consumption. This can lower the user's blood sugar rise rate by causing the user to consume foods in order of lowest sugar content (and / or user blood sugar rise rate value).

[0076] A processor (420) according to one embodiment can obtain the value of at least one food consumed by the user and the amount of at least one food consumed based on captured images and acquired sensor values. For example, the processor (420) can obtain the value of at least one food consumed by the user and the amount of at least one food consumed based on the amount of food spread (or tossed or picked up) by the user through the captured images. For example, the processor (420) can obtain the value of the amount of food that can be consumed by the user based on the ratio between the size of an object containing food (e.g., cutlery, plate, bowl, cup, and / or other objects capable of containing food) and the size of the food object within the object through the captured images, and obtain the value of the amount of food consumed by the user based on the reduction in the size of the food object.

[0077] A processor (420) according to one embodiment can obtain at least one sugar content value corresponding to each of at least one food consumed by a user.

[0078] A processor (420) according to one embodiment can identify whether a first sugar content value corresponding to the first food exists in a database when the value of the amount of the first food consumed by the user is obtained. For example, if the first food is a general braised short rib, the first sugar content value may exist in the database. If the first food is not a general braised short rib but a braised short rib designated in association with the user (of a specific place, a specific store, or a specific chef), the first sugar content value may not exist in the database.

[0079] A processor (420) according to one embodiment can obtain a first sugar content value consumed by a user based on the amount of the first food consumed by the user by using the first sugar content value corresponding to the first food when a first sugar content value corresponding to the first food exists in the database.

[0080] A processor (420) according to one embodiment may, if a first sugar content value corresponding to a first food does not exist in the database, measure a first blood sugar rise rate value measured for the user after the first food (e.g., a designated amount of the first food) is consumed by the user, and obtain a second sugar content value and a second blood sugar rise rate value corresponding to a second food (e.g., a general braised short rib) associated with the first food (e.g., braised short ribs designated in association with the user) from the database. A processor (420) according to one embodiment may calculate (or predict) and obtain a first sugar content value corresponding to the first food using the first blood sugar rise rate value, the second sugar content value, and the second blood sugar rise rate value. A processor (420) according to one embodiment may update the database to include a first sugar content value corresponding to the first food (e.g., braised short ribs designated in association with the user). According to one embodiment, the processor (420) may request user input for an unidentifiable food image area when it is impossible to identify the food in the food image areas while identifying the food in the food image areas using an artificial intelligence model. For example, if the first food does not exist in the database, the processor (420) may display an indicator (e.g., color, area, shape, and / or indicator object) requesting information about the first food through the artificial intelligence model, output a question about the first food (e.g., "What is the food visible in the red square?"), receive a voice corresponding to the type of food (e.g., "Minari beef belly pancake") as an answer to the question, obtain the received type of food and the sugar content value (or nutritional value) of the type of food, and register it in the database.

[0081] A processor (420) according to one embodiment can obtain a value of the amount of sugar consumed by a user (e.g., a value of the total amount of sugar consumed by the user from the food consumed by the user) by using the value of the amount consumed of each of at least one food consumed by the user and the value of the sugar content corresponding to each of at least one food.

[0082] A processor (420) according to one embodiment may obtain a sugar threshold value associated with a user's blood sugar rise rate. According to one embodiment, the sugar threshold value may be set based on the user's intake of sugar at which the user's blood sugar rise rate exceeds a specified blood sugar rise rate (e.g., a blood sugar rise rate corresponding to a blood sugar spike), and may have a different value for each user.

[0083] A processor (420) according to one embodiment may display information on a display (450) to lower the user's blood sugar rise rate value when the sugar amount value consumed by the user exceeds a sugar amount threshold. A processor (420) according to one embodiment may display information on a display (450) to lower the user's blood sugar rise rate value when the blood sugar change amount (or blood sugar rise rate by the blood sugar rise curve) based on the sugar amount value consumed by the user exceeds a specified blood sugar change amount threshold. A processor (420) according to one embodiment may display information on a display (450) to lower the user's blood sugar rise rate value when a blood sugar rise (e.g., a sudden blood sugar rise or a blood sugar peak) exceeding a threshold within a specified period occurs based on the sugar amount value consumed by the user. A processor (420) according to one embodiment may display information on a display (450) to lower the user's blood sugar rise rate value when the sugar intake rate based on the sugar amount value consumed by the user exceeds a specified sugar intake rate.

[0084] A processor (420) according to one embodiment may display on a display (450) information indicating that the amount of sugar consumed by the user exceeds the intake sugar threshold, information indicating that the amount of blood sugar change exceeds the specified blood sugar change threshold, information indicating that a rise in blood sugar exceeding the threshold occurs within a specified period, or information indicating that the rate of sugar intake exceeds the specified rate of sugar intake. In the case where the amount of sugar consumed by the user exceeds the intake sugar threshold, the amount of blood sugar change exceeds the specified blood sugar change threshold, a rise in blood sugar exceeding the threshold occurs within a specified period, or the rate of sugar intake exceeds the specified rate of sugar intake. A processor (420) according to one embodiment may display on a display (450) information recommending food for the user to consume among foods when the amount of sugar consumed by the user exceeds the intake sugar threshold, the amount of blood sugar change exceeds the specified blood sugar change threshold, a rise in blood sugar exceeding the threshold occurs within a specified period, or the rate of sugar intake exceeds the specified rate of sugar intake. A processor (420) according to one embodiment may display information on a display (450) indicating foods that are not recommended for consumption among foods where the user's intake amount exceeds a sugar threshold, the blood sugar change amount exceeds a specified blood sugar change amount threshold, a blood sugar rise exceeding a threshold occurs within a specified period, or the sugar intake rate exceeds a specified sugar intake rate. A processor (420) according to one embodiment may display information on a display (450) instructing the user to reduce the intake rate of foods when the user's intake amount exceeds a sugar threshold, the blood sugar change amount exceeds a specified blood sugar change amount threshold, a blood sugar rise exceeding a threshold occurs within a specified period, or the sugar intake rate exceeds a specified sugar intake rate.A processor (420) according to one embodiment may display information on a display (450) guiding a combination of at least two foods to be consumed together among foods when the user's sugar intake value exceeds a sugar threshold, the blood sugar change amount exceeds a specified blood sugar change amount threshold, a blood sugar rise exceeding a threshold occurs within a specified period, or the sugar intake rate exceeds a specified sugar intake rate. For example, if foods including pork belly and lettuce are identified, the processor (420) may display information on the display (450) guiding the consumption of pork belly and lettuce together (e.g., "Wrap the pork belly in lettuce and eat"). A processor (420) according to one embodiment may display information on the display (450) recommending exercise when the user's sugar intake value exceeds a sugar threshold, the blood sugar change amount exceeds a specified blood sugar change amount threshold, a blood sugar rise exceeding a threshold occurs within a specified period, or the sugar intake rate exceeds a specified sugar intake rate. Information recommending exercise according to one embodiment may include the types of recommended exercises, recommended exercise intensity, and / or recommended exercise time. If the recommended exercise is a running exercise or a walking exercise, the processor (420) according to one embodiment may display a recommended running or walking path on the display (450) based on the user's location (or location information).

[0085] According to one embodiment, a PMIC (430) (e.g., power management module (188) of FIG. 1) can provide power to each component of an electronic device (400) using the capacity of a battery (435) (e.g., battery (189) of FIG. 1).

[0086] A memory (440) according to one embodiment (e.g., memory (130) of FIG. 1) can store various data used by at least one component of an electronic device (400) (e.g., first camera (411), second camera (412), third camera (413), processor (420), PMIC (430), display (450), audio device (461), sound input device (462), sound output device (463), communication circuit (470), and / or sensor (480)). A memory (440) according to one embodiment can store instructions (or commands or programs) that cause the processor (420) to operate when executed.

[0087] A display (450) according to one embodiment may display a screen to be provided to a user. A display (450) according to one embodiment may include a first display (251), a second display (252), one or more input optical members (253-1, 253-2), one or more transparent members (290-1, 290-2), and / or one or more screen display portions (254-1, 254-2), as shown in FIG. 2a.

[0088] An audio device (461) according to one embodiment is connected to a sound input device (462) (e.g., a microphone) and a sound output device (463) to convert data input through the sound input device (462) and convert data to be output to the sound output device (463). A sound output device (463) according to one embodiment may include a speaker and an amplifier. The sound output device (463) may refer to one or more sound output devices (263-1, 263-2) of FIG. 2A. The sound input device (462) may refer to one or more sound input devices (262-1, 262-2, 262-3) of FIG. 2A.

[0089] A communication circuit (470) according to one embodiment may support the establishment of a wireless communication channel with an electronic device outside the electronic device (400) (e.g., an external electronic device (302) (e.g., the electronic device (102) of FIG. 1) (e.g., a wearable electronic device or a smart watch)) and the performance of communication through the established communication channel. Wireless communication according to one embodiment may include short-range wireless communication.

[0090] A sensor (480) according to one embodiment may include a 6-axis sensor (481), a magnetic sensor (482), a proximity sensor (483), and / or a light sensor (484). A sensor (480) according to one embodiment may further include a sensor for acquiring a biosignal to detect whether the electronic device (400) is being worn by a user. For example, the sensor (480) may further include at least one of a heart rate sensor, a skin sensor, or a temperature sensor. A processor (420) according to one embodiment may confirm (or identify) the movement of a user wearing the electronic device (400) based on sensing information acquired through the 6-axis sensor (481). For example, the 6-axis sensor (481) may generate a sensor value by detecting a change in the direction the user is facing (or the direction the user looks through the electronic device (400)) while the electronic device (400) is being worn, and transmit the generated sensor value or the amount of change in the sensor value to the processor (420). A processor (420) according to one embodiment can perform AR (augmented reality), VR (virtual reality), or XR (extended reality) functions based on a sensor value or a change in sensor value by a 6-axis sensor (481).

[0091] An electronic device (101, 200, or 400) according to one embodiment of the present disclosure may include a camera (180 or, 411, 412, and / or 413), a communication circuit (190 or 470), a display (160 or 450), a memory (130 or 440) for storing commands, and at least one processor (120 or 420). The commands, when executed individually and / or collectively by the at least one processor (420), may cause the electronic device (400) to acquire images through the camera (411, 412, and / or 413). The commands, when executed individually and / or collectively by the at least one processor (420), may cause the electronic device (400) to receive sensor values ​​detected by at least one external sensor through the communication circuit (470). When the above commands are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may identify food image regions included in the images. When the above commands are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may identify a user's gesture for eating food based on the food image regions and the sensor values. When the above commands are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may identify foods corresponding to the food image regions and sugar content values ​​corresponding to the foods based on the identification of the user's gesture for eating food.When the above commands are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may identify the priorities of the foods using the sugar content values ​​corresponding to the foods. When the above commands are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may display information indicating the order of consumption of the foods through the display (450) based on the priorities of the foods.

[0092] The commands according to one embodiment, when executed individually and / or collectively by the at least one processor (420), may cause the electronic device (400) to identify at least one food among the foods consumed by the user based on the images and the user's gesture. The commands, when executed individually and / or collectively by the at least one processor (420), may cause the electronic device (400) to obtain the value of the amount of each of the at least one food consumed by the user. The commands, when executed individually and / or collectively by the at least one processor (420), may cause the electronic device (400) to obtain the value of the amount of sugar consumed by the user using the value of the sugar content corresponding to each of the at least one food and the value of the amount of each of the at least one food consumed by the user. When the above commands are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may obtain a sugar amount threshold value associated with the user's blood sugar rise rate. When the above commands are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may display information through the display (450) to lower the user's blood sugar rise rate value when the sugar amount value consumed by the user exceeds the sugar amount threshold value.

[0093] When the commands according to one embodiment of the present disclosure are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may identify whether there is a first sugar content value corresponding to a first food consumed by the user from a database containing a plurality of sugar content values ​​corresponding to each of a plurality of foods. When the commands are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may obtain a first sugar content value corresponding to the first food from the database if a first sugar content value corresponding to the first food exists in the database. When the above commands are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may be configured to obtain a first blood sugar increase rate value measured for the user after a designated amount of the first food is consumed by the user, if a first sugar content value corresponding to the first food does not exist in the database, obtain a second blood sugar increase rate value corresponding to the second food associated with the first food from the database, obtain a second blood sugar increase rate value measured for the user after a designated amount of the second food is consumed by the user from the database, calculate a first sugar content value corresponding to the first food using the first blood sugar increase rate value, the second sugar content value, and the second blood sugar increase rate value, and update the database to include the first sugar content value corresponding to the first food.

[0094] When the commands according to one embodiment are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may display information through the display (450) indicating that the user's consumed sugar value exceeds the consumed sugar threshold value when the consumed sugar value by the user exceeds the sugar threshold value.

[0095] When the commands according to one embodiment are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may display information recommending food to be consumed by the user through the display (450) when the user's intake amount value exceeds the intake amount threshold value.

[0096] When the commands according to one embodiment are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may display information indicating which of the foods are not recommended for consumption by the user through the display (450) when the user's intake amount value exceeds the intake threshold value.

[0097] When the commands according to one embodiment are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may display information through the display (450) guiding the user to reduce the speed of consumption of the foods when the user's intake amount value exceeds the intake amount threshold value.

[0098] When the commands according to one embodiment are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may display information through the display (450) guiding the user to consume a combination of at least two of the foods when the user's intake amount value exceeds the intake amount threshold value.

[0099] When the commands according to one embodiment are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may display information recommending exercise through the display when the user's intake amount value exceeds the intake threshold value, and the information recommending exercise may include the type of exercise, exercise intensity, and / or exercise time.

[0100] When the commands according to one embodiment are executed individually and / or collectively by the at least one processor (420), the electronic device (400) may identify a route to be recommended for running or walking based on the user's location, and display the route to be recommended for running or walking through the display (450).

[0101] FIG. 5 is a flowchart illustrating the operation of providing a user with the order of consumption of foods in an electronic device according to one embodiment.

[0102] Referring to FIG. 5, the processor (e.g., the processor (120) of FIG. 1 or the processor (420) of FIG. 4) of the electronic device (400) according to one embodiment (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2a) can perform at least one of 510 operations to 570 operations.

[0103] In operation 510, a processor (420) according to one embodiment may acquire images through a first camera (411), a second camera (412), and / or a third camera (413). A processor (420) according to one embodiment may acquire images of the user's surroundings (e.g., a field of view or scene viewed by the user while eating) through the first camera (411), the second camera (412), and / or the third camera (413) while the electronic device (400) is worn (or carried) by the user.

[0104] In operation 520, a processor (420) according to one embodiment can identify food image regions included in images. According to one embodiment, when a first image is acquired, the processor (420) can detect at least one or more food objects (or food image regions) included in the first image using an engine (or software or program) (or artificial intelligence model) for object detection. For example, the engine for object detection may include R-CNN (regions with convolutional neutral network) or YOLO (you only look once).

[0105] In operation 530, a processor (420) according to one embodiment may receive sensor values ​​detected by at least one external sensor (e.g., an external electronic device (302) (e.g., at least one sensor of the electronic device (102) of FIG. 1 (e.g., a wearable electronic device or a smart watch)) through a communication circuit (470). The at least one external sensor according to one embodiment may include an accelerometer, a gyroscope, and / or a gravity sensor. The processor (420) according to one embodiment may identify a user's gesture using the sensor values ​​detected by the at least one external sensor.

[0106] In operation 540, a processor (420) according to one embodiment can identify a user's gesture for eating food based on food image regions and sensor values. For example, the processor (420) can detect a user's gesture for eating food if at least one food image region is identified and the user gesture based on the sensor values ​​is a user gesture associated with at least one food image region.

[0107] In operation 550, a processor (420) according to one embodiment can identify foods corresponding to food image regions and sugar content values ​​corresponding to each of the foods based on the identification of a user's gesture for food consumption. A processor (420) according to one embodiment can identify at least one food (e.g., at least one type of food) corresponding to at least one food image region. A processor (420) according to one embodiment can identify at least one food corresponding to (or matched to) at least one food image region by using the image characteristics of at least one food image region and the image characteristics of a plurality of pre-learned foods (e.g., braised short ribs, rice, pork belly, lettuce, seasoned spinach, braised fish, broccoli, spicy stir-fried chicken, and / or other foods). A processor (420) according to one embodiment can identify each food in at least one food image region using a large-scale pre-trained artificial intelligence model learned based on a large amount of food images. A processor (420) according to one embodiment can identify each food in at least one food image area using an artificial intelligence model that has learned food images taken by a user. For example, the processor (420) can identify a general galbi-jjim included in the taken images using an artificial intelligence model that has learned a large number of various galbi-jjim images. For example, the processor (420) can identify a specific galbi-jjim included in the taken images using an artificial intelligence model that has learned galbi-jjim images designated in association with the user (specific place or specific store or specific chef). A processor (420) according to one embodiment can identify a sugar content value corresponding to each of at least one food using a database (e.g., the database of Table 1) containing predetermined (or defined or detected or investigated) sugar content values ​​corresponding to each of a plurality of foods (or a plurality of food types).A database according to one embodiment may include predetermined sugar content values ​​corresponding to each of a plurality of foods. A database according to one embodiment may further include blood glucose elevation rate values ​​measured after a specified amount of each of the plurality of foods is consumed by a user, along with the predetermined sugar content values ​​corresponding to each of the plurality of foods.

[0108] In operation 560, a processor (420) according to one embodiment can identify the priority of foods using sugar content values ​​(and / or blood glucose rise rate values) corresponding to the foods. A processor (420) according to one embodiment can identify the priority of at least one food using at least one sugar content value (and / or blood glucose rise rate value) corresponding to at least one food in at least one food area. A processor (420) according to one embodiment can identify the priority of multiple foods by assigning higher priority to multiple sugar content values ​​(and / or user blood glucose rise rate values) corresponding to multiple foods in multiple food areas in order of decreasing value. For example, the processor (420) can identify the food having the smallest sugar content value (and / or user blood glucose rise rate value) as the food of the highest priority.

[0109] In operation 570, a processor (420) according to one embodiment may display information indicating the order of consumption of foods based on the priority of the foods through a display (450). A processor (420) according to one embodiment may determine the order of consumption of foods in order of highest priority and display the determined order of consumption of foods through a display (450). For example, the processor (420) may display a virtual image indicating the order of consumption of foods while the actual foods are visible to the user. This may allow the user to consume foods in order of lowest sugar content (and / or user blood sugar rise rate value), thereby lowering the user's blood sugar rise rate.

[0110] FIG. 6 is a flowchart illustrating an operation to provide information to lower a user's blood glucose rise rate value in an electronic device according to one embodiment.

[0111] Referring to FIG. 6, the processor (e.g., the processor (120) of FIG. 1 or the processor (420) of FIG. 4) of the electronic device (400) according to one embodiment (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2a) can perform at least one of 610 operations to 650 operations.

[0112] In operation 610, a processor (420) according to one embodiment can obtain a value of the amount of at least one food consumed by the user among the foods based on captured images and acquired sensor values. For example, the processor (420) can obtain a value of the amount of at least one food consumed by the user and the amount of at least one food consumed by the user by identifying the amount of food spread (or scooped or picked up) by the user through the captured images. For example, the processor (420) can obtain a value of the amount of food that can be consumed by the user based on the ratio between the size of the object containing the food (e.g., spoon, plate, bowl, cup, and / or other object capable of containing food) and the size of the food object within the object based on the size of the food object, and obtain a value of the amount of food consumed by the user based on the reduction in the size of the food object.

[0113] In operation 620, the processor (420) according to one embodiment can obtain at least one sugar content value corresponding to each of at least one food consumed by the user.

[0114] In operation 630, a processor (420) according to one embodiment can obtain a value of the amount of sugar consumed by the user (e.g., a total amount of sugar consumed by the user from the food consumed by the user during a specified period) by using the value of the amount consumed of each of at least one food consumed by the user and the value of the sugar content corresponding to each of at least one food.

[0115] In operation 640, the processor (420) according to one embodiment may obtain a sugar threshold value associated with the user's blood sugar rise rate. The sugar threshold value according to one embodiment may be set based on the user's intake of sugar at which the user's blood sugar rise rate exceeds a specified blood sugar rise rate (e.g., a blood sugar rise rate corresponding to a blood sugar spike), and may have a different value for each user.

[0116] In operation 650, a processor (420) according to one embodiment may display information on the display (450) to lower the user's blood sugar rise rate value based on the fact that the sugar amount value consumed by the user exceeds the sugar amount threshold. A processor (420) according to one embodiment may display information on the display (450) indicating that the sugar amount value consumed by the user exceeds the intake sugar threshold when the sugar amount value consumed by the user exceeds the intake sugar threshold. A processor (420) according to one embodiment may display information on the display (450) recommending food for the user to consume among foods when the user's intake sugar amount value exceeds the sugar amount threshold. A processor (420) according to one embodiment may display information on the display (450) indicating foods that are not recommended for consumption among foods where the user's intake sugar amount value exceeds the sugar amount threshold. A processor (420) according to one embodiment may display information on a display (450) that guides the user to reduce the speed of consuming food when the user's intake value exceeds a sugar threshold. A processor (420) according to one embodiment may display information on a display (450) that guides the user to consume at least two foods together when the user's intake value exceeds a sugar threshold. A processor (420) according to one embodiment may display information on a display (450) that recommends exercise when the user's intake value exceeds a sugar threshold. The information recommending exercise according to one embodiment may include the type of recommended exercise, the recommended exercise intensity, and / or the recommended exercise time. A processor (420) according to one embodiment may display a recommended running or walking path on a display (450) based on the user's location (or location information) when the recommended exercise is running or walking.

[0117] FIG. 7 is a flowchart illustrating the operation of obtaining a sugar content value and a blood glucose rise rate value corresponding to food in an electronic device according to one embodiment and updating a database.

[0118] Referring to FIG. 7, the processor (e.g., the processor (120) of FIG. 1 or the processor (420) of FIG. 4) of the electronic device (400) according to one embodiment (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2a) can perform at least one of 710 operations to 770 operations.

[0119] In operation 710, the processor (420) according to one embodiment can identify the first food consumed by the user (or the value of the amount of the first food consumed).

[0120] In operation 720, the processor (420) according to one embodiment can identify whether there is a first sugar content value corresponding to the first food in the database.

[0121] In operation 730, the processor (420) according to one embodiment can obtain the first sugar content value corresponding to the first food if the first sugar content value corresponding to the first food exists in the database.

[0122] In operation 740, the processor (420) according to one embodiment may measure a first blood sugar increase rate value measured for the user after the first food (e.g., a specified amount of the first food) is consumed by the user, if there is no first sugar content value corresponding to the first food in the database.

[0123] In operation 750, a processor (420) according to one embodiment can obtain a second sugar content value and a second blood sugar increase rate value corresponding to a second food (e.g., regular braised short ribs) associated with a first food (e.g., braised short ribs different from regular braised short ribs) from a database.

[0124] In operation 760, a processor (420) according to one embodiment can calculate (or predict) and obtain a first sugar content value corresponding to a first food using a first blood sugar increase rate value, a second sugar content value, and a second blood sugar increase rate value.

[0125] In operation 770, the processor (420) according to one embodiment may update the database to include a first sugar content value corresponding to the first food.

[0126] A food recommendation method in an electronic device (101, 200, or 400) according to one embodiment of the present disclosure may include the operation of acquiring images through a camera (180 or 411, 412, and / or 413) of the electronic device (400). The method may include the operation of receiving sensor values ​​detected by at least one external sensor through a communication circuit (190 or 470) of the electronic device (400). The method may include the operation of identifying food image regions included in the images. The method may include the operation of identifying a user's gesture for food consumption based on the food image regions and the sensor values. The method may include the operation of identifying foods corresponding to the food image regions and sugar content values ​​corresponding to the foods based on the identification of the user's gesture for food consumption. The method may include the operation of identifying priorities of the foods using the sugar content values ​​corresponding to the foods. The above method may include an operation of displaying information indicating the order of consuming the foods through the display (160 or 450) of the electronic device based on the priorities of the foods.

[0127] The method according to one embodiment may include an operation of identifying at least one food among the foods consumed by the user based on the images and the user's gesture. The method may include an operation of obtaining a value of the amount of each of the at least one food consumed by the user. The method may include an operation of obtaining a value of the amount of sugar consumed by the user using a sugar content value corresponding to each of the at least one food and the value of the amount of each of the at least one food consumed by the user. The method may include an operation of obtaining a sugar threshold value associated with the user's blood sugar rise rate. The method may include an operation of displaying information to lower the user's blood sugar rise rate value through the display when the value of the sugar consumed by the user exceeds the sugar threshold value.

[0128] According to one embodiment, the method may include an operation of identifying whether a first sugar content value corresponding to a first food consumed by the user exists from a database containing a plurality of sugar content values ​​corresponding to each of a plurality of foods. If the first sugar content value corresponding to the first food exists in the database, the method may include an operation of obtaining the first sugar content value corresponding to the first food from the database. If the first sugar content value corresponding to the first food does not exist in the database, the method may include an operation of obtaining a first blood glucose rise rate value measured for the user after a designated amount of the first food is consumed by the user. The method may include an operation of obtaining a second sugar content value corresponding to a second food associated with the first food from the database. The method may obtain a second blood glucose rise rate value measured for the user after a designated amount of the second food is consumed by the user from the database. The above method may include an operation of calculating a first sugar content value corresponding to the first food using the first blood sugar increase rate value, the second sugar content value, and the second blood sugar increase rate value. The above method may include an operation of updating the database to include the first sugar content value corresponding to the first food.

[0129] The method according to one embodiment may include an operation of displaying information through the display (450) indicating that the user's consumed sugar value exceeds the consumed sugar threshold value when the consumed sugar value exceeds the consumed sugar threshold value.

[0130] The method according to one embodiment may include an operation of displaying information recommending food to be consumed by the user through the display (450) when the user's intake amount value exceeds the intake amount threshold value.

[0131] The method according to one embodiment may include an operation of displaying information through the display (450) indicating foods among the foods that are not recommended for consumption by the user when the user's intake amount value exceeds the intake amount threshold value.

[0132] The method according to one embodiment may include an operation of displaying information through the display (450) to guide the user to reduce the speed of consumption of the foods when the user's intake amount value exceeds the intake amount threshold value.

[0133] The method according to one embodiment may include an operation of displaying information through the display (450) to guide the user to consume at least two of the foods in combination when the user's intake amount value exceeds the intake amount threshold value.

[0134] The method according to one embodiment may include an action of displaying information recommending exercise through the display (450) when the user's intake amount value exceeds the intake threshold value. The information recommending exercise in the method may include the type of exercise, exercise intensity, and / or exercise time.

[0135] FIG. 8 is a diagram showing an example of displaying the order of consumption of foods while the foods are displayed to the user through an electronic device according to one embodiment.

[0136] Referring to FIG. 8, the processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of the electronic device (400) according to one embodiment (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2a) can display the order of consumption of the foods (e.g., 810, 820, 830, 840, 850, 860, 870, 880, 890) in the field of view (800) through one or more transparent members (e.g., 290-1 and 290-2 of FIG. 2a) through a screen on one or more transparent members (290-1, 290-2) while the user can see the foods (e.g., 810, 820, 830, 840, 850, 860, 870, 880, 890) in the field of view (800) through one or more transparent members (e.g., 290-1 and 290-2 of FIG. 2a). An electronic device (400) according to one embodiment can identify the priority of at least one food in order of smallest sugar content value (and / or user blood sugar rise rate value) using at least one sugar content value (and / or user blood sugar rise rate value) corresponding to each of the foods (e.g., 810, 820, 830, 840, 850, 860, 870, 880, 890). For example, the processor (420) can identify the food (810) having the smallest sugar content value (and / or user blood sugar rise rate value) as the food of the highest priority. The processor (420) according to one embodiment can display information indicating the order of consumption of the foods based on the priority of the foods through a display (450). A processor (420) according to one embodiment may display virtual images (e.g., at least some or all of ①, ②, ③, ④, ⑤, ⑥, ⑦, ⑧, and ⑨) indicating the order of consumption of the food in each food image area in food image areas while actual foods (e.g., 810, 820, 830, 840, 850, 860, 870, 880, 890) are visible to the user's eyes. According to one embodiment, if the user consumes the food according to the information indicating the order of consumption of the foods, the rate of blood sugar increase may be lowered.

[0137] FIG. 9 is a diagram showing an example indicating that the amount of sugar consumed by a user through an electronic device according to one embodiment exceeds a specified amount of sugar intake.

[0138] Referring to FIG. 9, a processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of an electronic device (400) according to one embodiment (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2a) may display information (900) (e.g., "Recommended sugar intake has been exceeded") on a screen above one or more transparent members (e.g., 290-1 and 290-2 of FIG. 2a) when the sugar intake value consumed by the user during a meal exceeds a sugar intake threshold value while the user is viewing foods (e.g., 810, 820, 830, 840, 850, 860, 870, 880, 890) in the field of view (800) through one or more transparent members (e.g., 290-1 and 290-2 of FIG. 2a) while the user is viewing the foods (e.g., 810, 820, 830, 840, 850, 860, 870, 880, 890) through one or more transparent members (290-1, 290-2) and the sugar intake value consumed by the user exceeds the intake sugar intake threshold value.

[0139] FIG. 10 is a diagram showing an example of displaying information recommending food to be consumed by a user through an electronic device according to one embodiment.

[0140] Referring to FIG. 10, a processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of an electronic device (400) according to one embodiment (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2a) may, through one or more transparent members (e.g., 290-1 and 290-2 of FIG. 2a), allow the user to view foods (e.g., 810, 820, 830, 840, 850, 860, 870, 880, 890) in the user's field of vision (800) through one or more transparent members, and if the amount of sugar consumed by the user during a meal exceeds a sugar threshold, if the amount of blood sugar change exceeds a specified blood sugar change threshold, if a blood sugar rise exceeding a threshold occurs within a specified period, or if the rate of sugar intake exceeds a specified rate of sugar intake, the consumption by the user via a screen on one or more transparent members (290-1, 290-2) Recommended foods (e.g., low-sugar foods) (810) and foods not recommended for consumption by the user (e.g., high-sugar foods) (820, 830, 840, 850, 860, 870, 880, 890) can be displayed with different visual effects. For example, information recommending food for consumption by the user can be displayed by highlighting the foods recommended for consumption by the user (810) (e.g., in bold or an appetite-stimulating color to indicate recommendation) and by not highlighting the remaining foods (820, 830, 840, 850, 860, 870, 880, 890) (e.g., in dim or an appetite-stimulating color to indicate not recommending). For example, the food (810) to be consumed by the user may be the food with the lowest sugar content among the foods (810, 820, 830, 840, 850, 860, 870, 880, 890).

[0141] FIG. 11 is a diagram showing an example of displaying information about food that is not recommended for consumption by a user through an electronic device according to one embodiment.

[0142] Referring to FIG. 11, a processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of an electronic device (400) according to one embodiment (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2a) can, through one or more transparent members (e.g., 290-1 and 290-2 of FIG. 2a), [review] the user's intake via a screen on one or more transparent members (290-1, 290-2) when the user sees foods (e.g., 810, 820, 830, 840, 850, 860, 870, 880, 890) that have come into the user's field of vision (800) through one or more transparent members, if the value of the amount of sugar consumed by the user during a meal exceeds a sugar amount threshold, if the amount of blood sugar change exceeds a specified blood sugar change threshold, if a blood sugar rise exceeding a threshold occurs within a specified period, or if the rate of sugar intake exceeds a specified rate of sugar intake, [review] the user's intake via a screen on one or more transparent members (290-1, 290-2). Information about foods that are not recommended for consumption by the user can be displayed by covering or changing the color (1100) of at least one food (860, 880, 890) that is not recommended. For example, at least one food (860, 880, 890) that is not recommended for consumption by the user may be a food among the foods (810, 820, 830, 840, 850, 860, 870, 880, 890) that has a sugar content higher than a specified standard.

[0143] FIG. 12 is a diagram showing an example of displaying information that guides a user to reduce the speed of consuming food through an electronic device according to one embodiment.

[0144] Referring to FIG. 12, the processor (e.g., the processor (120) of FIG. 1 or the processor (420) of FIG. 4) of the electronic device (400) according to one embodiment (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2a) may display information (1200) (e.g., "Shall we put down the spoon and focus on chewing?") on the display (450) to guide the user to reduce the speed of eating the food through a screen on one or more transparent members (290-1, 290-2) when the sugar amount consumed by the user exceeds a sugar amount threshold, the blood sugar change amount exceeds a specified blood sugar change amount threshold, a blood sugar rise exceeding a threshold occurs within a specified period, or the sugar intake rate exceeds a specified sugar intake rate.

[0145] FIG. 13a is a diagram showing an example of a first exercise path recommended based on the user's location in an electronic device according to one embodiment.

[0146] Referring to FIG. 13a, a processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of an electronic device (400) according to one embodiment (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2a) may display on a display (450) a screen (1310) containing a first exercise path (e.g., running or walking path) (1311) recommended based on the user's location (or location information) when the amount of sugar consumed by the user exceeds a sugar amount threshold, the amount of blood sugar change exceeds a specified blood sugar change threshold, a rise in blood sugar exceeding a threshold occurs within a specified period, or the rate of sugar intake exceeds a specified rate of sugar intake.

[0147] FIG. 13b is a diagram showing an example of a second exercise path recommended based on the user's location in an electronic device according to one embodiment.

[0148] Referring to FIG. 13b, a processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of an electronic device (400) according to one embodiment (e.g., electronic device (101) of FIG. 1 or electronic device (200) of FIG. 2a) may display on a display (450) a screen (1320) containing a second exercise path (e.g., running or walking path) (1321) recommended based on the user's location (or location information) when the amount of sugar consumed by the user exceeds a sugar amount threshold, the amount of blood sugar change exceeds a specified blood sugar change threshold, a rise in blood sugar exceeding a threshold occurs within a specified period, or the rate of sugar intake exceeds a specified rate of sugar intake.

[0149] A processor (420) according to one embodiment may recommend different exercise paths based on the magnitude of the excess sugar value, the magnitude of the excess blood sugar change, the magnitude of the excess blood sugar rise, or the magnitude of the excess sugar intake rate value when the sugar value consumed by the user exceeds a sugar value threshold, when the blood sugar change exceeds a specified blood sugar change threshold, when a blood sugar rise exceeding a threshold occurs within a specified period, or when the sugar intake rate exceeds a specified sugar intake rate. A processor (420) according to one embodiment may recommend a first exercise path (1311) when the sugar value consumed by the user exceeds the sugar value threshold below a specified standard, and may recommend a second exercise path (1321) which is longer than the first exercise path (1311) when the sugar value consumed by the user exceeds the sugar value threshold above a specified standard. Examples of exercise paths may include any other exercise paths in addition to the examples shown in FIG. 13a and FIG. 13b.

[0150] FIG. 14 is a diagram showing an example of eating while an electronic device according to one embodiment is worn by a user.

[0151] Referring to FIG. 14, a processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of an electronic device (1400) according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2a, or electronic device (400) of FIG. 4) can obtain the value of at least one food consumed by the user and the value of the amount of at least one food consumed among the foods (1410, 1420, 1430, 1440, 1450, 1460) based on images taken while being worn by the user and acquired sensor values. For example, the processor (420) can obtain the value of at least one food consumed by the user and the amount of at least one food consumed by the user by identifying at least one food spread (or lifted or picked up) by the user through the captured images. For example, the processor (420) can identify the amount of food that can be consumed by the user based on the ratio between the size of the object containing the food (e.g., spoon, plate, bowl, cup, and / or other object capable of containing food) and the size of the food object within the object through the captured images, and obtain the value of the amount of food consumed by the user based on the reduction in the size of the food object.

[0152] A processor (420) according to one embodiment may store at least one food consumed by the user, the value of the amount of at least one food consumed, and the value of the amount of at least one nutrient consumed. A processor (420) according to one embodiment may use the value of the amount of at least one nutrient consumed by the user to identify the nutrient that the user is consuming insufficiently compared to the recommended intake amount for each nutrient. A processor (420) according to one embodiment may display information on a display (450) recommending the food containing the nutrient that the user is consuming insufficiently (e.g., 1440) among the foods (1410, 1420, 1430, 1440, 1450, 1460) (e.g., “Try the seasoned broccoli and tofu. Broccoli is rich in Vitamin C.” or “To meet your daily protein intake, you need a protein-first diet. However, I recommend spicy chicken stew rather than pork belly.”).

[0153] FIG. 15 is a diagram showing an example of a screen displaying the user's intake of nutrients and recommendation information according to one embodiment.

[0154] Referring to FIG. 15, a processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of an electronic device (1400) according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2a, or electronic device (400) of FIG. 4) may display information (or a screen) (1500) on a display (450) indicating the current intake amount (e.g., carbohydrates, fat, protein) and recommended intake amount (e.g., carbohydrates 55%, fat 25%, protein 20%) of at least one nutritional component (e.g., carbohydrates, fat, protein) by the user (e.g., carbohydrates 64%, fat 20%, protein 16%).

[0155] FIG. 16 is a diagram showing an example of a screen displaying the user's intake of nutrients, whether insufficient, appropriate, or excessive, according to one embodiment.

[0156] Referring to FIG. 16, a processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of an electronic device (1400) according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2a, or electronic device (400) of FIG. 4) may display information (or a screen) (1600) indicating insufficient, adequate, or excessive intake of each nutrient using the amount of at least one nutrient stored and consumed each time food is consumed by a user on a display (450). For example, at least one nutrient may include protein, dietary fiber, vitamin A, vitamin C, potassium, calcium, iron, saturated fat, sodium, and / or sugar, and any other nutrient may also be possible.

[0157] FIG. 17a is a diagram showing an example of recommending food when a user's carbohydrate intake is identified as exceeding a specified amount of carbohydrate intake while a menu is being displayed to the user through an electronic device according to one embodiment.

[0158] Referring to FIG. 17a, a processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of an electronic device (1400) according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2a, or electronic device (400) of FIG. 4) can identify the menu (1700) through images taken while the menu (1700) is being viewed by the user through one or more transparent members (290-1, 290-2) worn by the user. A processor (420) according to one embodiment can identify at least one food image area of ​​the menu (1700) and identify the nutritional components of at least one food corresponding to at least one food image area. A processor (420) according to one embodiment can identify a food image area (1711) (e.g., food menu) corresponding to a food with less carbohydrates (e.g., the least) among at least one food image area when it is identified that the user's carbohydrate intake has exceeded a specified daily carbohydrate intake based on the previously stored intake amounts of the user's nutritional components. A processor (420) according to one embodiment can display information (1712) recommending a food with less carbohydrates on a menu board (1700) (e.g., "Today's carbohydrate intake has exceeded the limit. We recommend a chicken tender salad that is rich in protein and vitamins") on a display (450).

[0159] FIG. 17b is a diagram illustrating an example of recommending food when the user's intake of nutritional components meets the specified range of intake of nutritional components while a menu is displayed to the user through an electronic device according to one embodiment.

[0160] Referring to FIG. 17b, a processor (e.g., processor (120) of FIG. 1 or processor (420) of FIG. 4) of an electronic device (1400) according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2a, or electronic device (400) of FIG. 4) can identify the menu (1700) through images taken while the menu (1700) is being viewed by the user through one or more transparent members (290-1, 290-2) worn by the user. A processor (420) according to one embodiment can identify at least one food image area of ​​the menu (1700) and identify the nutritional components of each of at least one food corresponding to at least one food image area. A processor (420) according to one embodiment can compare the intake amounts of the user's nutritional components with the intake amounts of designated image components and identify at least one food image area (1721) (e.g., at least one food menu) containing the image component required by the user. A processor (420) according to one embodiment can display information (1722) recommending at least one food from the menu board (1700) (e.g., "You are eating a balanced diet. If you eat the recommended menu, you will meet 40% of today's protein intake.") on a display (450).

[0161] A processor (420) according to one embodiment records whether the user has consumed the recommended food after recommending food on the menu (1700) and can identify the user's food consumption tendency (e.g., a tendency to strictly manage diet or a tendency to loosely manage diet) based on whether the recommended food has been consumed. A processor (420) according to one embodiment can recommend different foods based on the user's food consumption tendency even if the user's intake of nutritional components is the same when recommending food on the menu (1700).

[0162] One embodiment of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, each of phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another component and do not limit the components in any other aspect (e.g., importance or order). Where any (e.g., first) component is referred to as “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicationally,” it means that said component may be connected to said other component directly (e.g., wired), wirelessly, or through a third component.

[0163] The term "module" as used in an embodiment of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to an embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0164] One embodiment of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101) or electronic device (200)). For example, a processor (e.g., processor (420)) of the machine (e.g., electronic device (200)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0165] In a non-transient storage medium storing commands according to one embodiment of the present disclosure, the commands are configured to cause the electronic device (101, 200, or 400) to perform at least one operation when executed by the electronic device, wherein the at least one operation may include the operation of acquiring images through a camera (180, or 411, 412, and / or 413) of the electronic device (400). The at least one operation may include the operation of receiving sensor values ​​detected by at least one external sensor through a communication circuit (190 or 470) of the electronic device. The at least one operation may include the operation of identifying food image regions included in the images. The at least one operation may include the operation of identifying a user's gesture for food consumption based on the food image regions and the sensor values. The at least one operation may include identifying foods corresponding to the food image regions and sugar content values ​​corresponding to the foods based on the identification of the user's gesture for consuming the food. The at least one operation may include identifying priorities of the foods using the sugar content values ​​corresponding to the foods. The at least one operation may include displaying information indicating the order of consumption of the foods through the display (160 or 450) of the electronic device based on the priorities of the foods.

[0166] According to one embodiment, the method according to one embodiment disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0167] According to one embodiment, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to one embodiment, one or more of the components or operations among the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to one embodiment, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101, 200, 400), Camera(180, 411, 412, 413); Communication circuit (190, 470); Display (160, 450); Memory for storing instructions (130, 440); and It includes at least one processor (120, 420), When the above commands are executed individually and / or collectively by the at least one processor, the electronic device, Images are acquired through the above camera, and Receive sensor values ​​detected by at least one external sensor through the above communication circuit, and Identify food image regions included in the above images, and Identifying a user's gesture for food intake based on the above food image regions and the above sensor values, and Based on the identification of the user's gesture for consuming the food, the foods corresponding to the food image regions and the sugar content values ​​corresponding to the foods are identified, and Identifying the priorities of the foods using the sugar content values ​​corresponding to the foods, and An electronic device configured to display information indicating the order of consumption of the foods through the display based on the priorities of the foods.

2. In Paragraph 1, When the above commands are executed individually and / or collectively by the at least one processor, the electronic device, Identifying at least one food among the foods consumed by the user based on the above images and the user's gesture, and Obtaining the value of the amount of each of the at least one food consumed by the above user, A value of the amount of sugar consumed by the user is obtained by using the value of the sugar content corresponding to each of the at least one food and the value of the amount of each of the at least one food consumed by the user, and Obtaining a sugar threshold value associated with the blood glucose elevation rate of the above user, and An electronic device that displays information to lower the user's blood sugar rise rate value when the sugar amount value consumed by the user through the above display exceeds the sugar amount threshold value.

3. In Paragraph 1 or 2, When the above commands are executed individually and / or collectively by the at least one processor, the electronic device, Identifying whether a first sugar content value corresponding to a first food consumed by the user exists from a database containing multiple sugar content values ​​corresponding to each of multiple foods, and If a first sugar content value corresponding to the first food exists in the above database: Obtaining a first sugar content value corresponding to the first food from the above database, and If a first sugar content value corresponding to the first food does not exist in the above database: After a designated amount of the first food is consumed by the user, a first blood glucose elevation rate value is obtained for the user, and Obtain a second salt content value corresponding to the second food associated with the first food from the above database, and Obtaining a second blood sugar increase rate value measured for the user after a specified amount of the second food is consumed by the user from the above database, and Calculate a first sugar content value corresponding to the first food using the first blood sugar increase rate value, the second sugar content value, and the second blood sugar increase rate value, and An electronic device that updates the database to include the first sugar content value corresponding to the first food.

4. In any one of paragraphs 1 through 3, When the above commands are executed individually and / or collectively by the at least one processor, the electronic device, An electronic device that displays information indicating that the user's consumed sugar value exceeds the consumed sugar threshold value through the display when the sugar value consumed by the user exceeds the sugar threshold value.

5. In any one of paragraphs 1 through 4, When the above commands are executed individually and / or collectively by the at least one processor, the electronic device, An electronic device that displays information recommending food to be consumed by the user among the foods through the display when the user's intake amount value exceeds the intake threshold value.

6. In any one of paragraphs 1 through 5, When the above commands are executed individually and / or collectively by the at least one processor, the electronic device, An electronic device that displays information indicating foods among the foods that are not recommended for consumption by the user through the display when the user's intake amount value exceeds the intake threshold value.

7. In any one of paragraphs 1 through 6, When the above commands are executed individually and / or collectively by the at least one processor, the electronic device, An electronic device that displays information through the display to guide the user to reduce the speed of consumption of the foods when the user's intake amount exceeds the intake threshold value.

8. In any one of paragraphs 1 through 7, When the above commands are executed individually and / or collectively by the at least one processor, the electronic device, An electronic device that displays information through the display instructing the user to consume a combination of at least two of the foods when the user's intake amount exceeds the intake threshold value.

9. In any one of paragraphs 1 through 8, When the above commands are executed individually and / or collectively by the at least one processor, the electronic device, If the user's sugar intake value exceeds the sugar threshold, information recommending exercise is displayed through the display, and The information recommending the above exercise is an electronic device including the type of exercise, exercise intensity, and / or exercise time.

10. In any one of paragraphs 1 through 9, When the above commands are executed individually and / or collectively by the at least one processor, the electronic device, An electronic device that, when the above types of exercise include running or walking, identifies a route to be recommended for running or walking based on the user's location and displays the recommended route to be recommended for running or walking through the display.

11. In a food recommendation method in an electronic device (101, 200, 400), The operation of acquiring images through the cameras (180, 411, 412, 413) of the electronic device; The operation of receiving sensor values ​​detected by at least one external sensor through the communication circuit (190, 470) of the electronic device; An operation to identify food image regions included in the above images; An action of identifying a user's gesture for food intake based on the above food image regions and the above sensor values; An operation of identifying foods corresponding to the food image regions and sugar content values ​​corresponding to the foods based on the identification of the user's gesture for consuming the food; The operation of identifying priorities of the foods using the sugar content values ​​corresponding to the foods; and A method including the operation of displaying information indicating the order of consuming the foods through the display (160, 450) of the electronic device based on the priorities of the foods.

12. In Paragraph 11, An action of identifying at least one food among the foods consumed by the user based on the above images and the user's gesture; The operation of obtaining the value of the amount of each of the at least one food consumed by the above user; The operation of obtaining a sugar amount value consumed by the user by using a sugar content value corresponding to each of the at least one food and a value of the amount of each of the at least one food consumed by the user; The operation of obtaining a sugar threshold value associated with the blood glucose rise rate of the above user; and A method including an operation to display information to lower the user's blood sugar rise rate value when the sugar amount value consumed by the user through the above display exceeds the sugar amount threshold value.

13. In Paragraph 11 or 12, An operation of identifying whether a first sugar content value corresponding to a first food consumed by the user exists from a database containing multiple sugar content values ​​corresponding to each of multiple foods; If a first sugar content value corresponding to the first food exists in the database, the operation of obtaining the first sugar content value corresponding to the first food from the database; and A method comprising the operation of, if a first sugar content value corresponding to the first food does not exist in the database, obtaining a first blood sugar increase rate value measured for the user after a designated amount of the first food is consumed by the user, obtaining a second sugar content value corresponding to the second food associated with the first food from the database, obtaining a second blood sugar increase rate value measured for the user after a designated amount of the second food is consumed by the user from the database, calculating a first sugar content value corresponding to the first food using the first blood sugar increase rate value, the second sugar content value, and the second blood sugar increase rate value, and updating the database to include the first sugar content value corresponding to the first food.

14. In any one of paragraphs 11 through 13, A method comprising the operation of displaying information through the display indicating that the user's consumed sugar value exceeds the consumed sugar threshold value when the sugar value consumed by the user exceeds the sugar threshold value.

15. In a non-transient storage medium storing instructions, said instructions are configured such that when executed by an electronic device (101, 200, 400), said electronic device performs at least one operation, said at least one operation being, The operation of acquiring images through the cameras (180, 411, 412, 413) of the electronic device; The operation of receiving sensor values ​​detected by at least one external sensor through the communication circuit (190, 470) of the electronic device; An operation to identify food image regions included in the above images; An action of identifying a user's gesture for food intake based on the above food image regions and the above sensor values; An operation of identifying foods corresponding to the food image regions and sugar content values ​​corresponding to the foods based on the identification of the user's gesture for consuming the food; The operation of identifying priorities of the foods using the sugar content values ​​corresponding to the foods; and A storage medium comprising an operation of displaying information indicating the order of consumption of the foods through the display (160, 450) of the electronic device based on the priorities of the foods.

Images