Electronic device and method for measuring heart rate

Abstract

An electronic device and method for measuring heart rate are disclosed. The heart rate measuring method of the electronic device according to the present invention comprises the following steps: capturing an image including a user's face; grouping the user's face included in the image into a plurality of regions including a plurality of pixels of similar colors; inputting information about the grouped plurality of regions into an artificial intelligence learning model to obtain information about the user's heart rate; and outputting the obtained information about the heart rate. Therefore, the electronic device can more accurately measure the user's heart rate by capturing an image.

Description

Technical Field

[0001] The present disclosure relates to an electronic device for measuring a heart rate and a measuring method thereof, and more particularly, to an electronic device for measuring a user's heart rate using a captured facial image and a measuring method thereof. Background Art

[0002] In a general heart rate measurement method, a sensor is attached to a body part such as a user's finger, and the user's heart rate is measured by using sensing information sensed by the attached sensor.

[0003] With the development of electronic technology, a camera-based non-contact heart rate measurement method for measuring a user's heart rate through an image captured by a camera without attaching a separate sensor to a body part of the user has been developed.

[0004] The camera-based non-contact heart rate measurement method is a method for capturing an image including a user's face and measuring the user's heart rate through a color change of the user's facial skin included in the captured image.

[0005] The above-mentioned heart rate measurement method may have a problem of measuring an incorrect heart rate when measuring the user's heart rate based on an image of the face captured under event circumstances (such as when the user's face is captured in a state where the facial color is dark or light due to the surrounding environment (e.g., indoor lighting), or when the user's face is captured when the user's skin temporarily changes according to the user's movement). Summary of the invention

[0006] Technical issues

[0007] An object of the present disclosure is to accurately measure a user's heart rate through an image captured by an electronic device.

[0008] Technical Solution

[0009] According to an embodiment, a method for measuring heart rate of an electronic device includes: capturing an image including a user's face; grouping the user's face included in the image into multiple regions including multiple pixels of similar colors; inputting information about the grouped multiple regions into an artificial intelligence learning model to obtain information about the user's heart rate; and outputting the obtained information about the heart rate.

[0010] The grouping step may include: grouping the user's face into multiple areas based on color information and position information of the multiple pixels constituting the user's face, acquiring a color value corresponding to each of the multiple areas after grouping, grouping multiple areas within a predetermined color range into the same group based on the color value corresponding to each of the multiple areas acquired, and acquiring pulse signals of the multiple areas grouped into the same group using the color value of each area in the multiple areas grouped into the same group.

[0011] The acquiring step may include acquiring information about the user's heart rate by inputting pulse signals for the plurality of regions grouped into the same group into an artificial intelligence learning model.

[0012] The artificial intelligence learning model may include a frequency decomposition layer and a complex layer, wherein the frequency decomposition layer is configured to obtain periodic attribute information of periodic iteration from the input pulse signal, and the complex layer is configured to convert the periodic attribute information obtained by the frequency decomposition layer into a value that can be recognized by the artificial intelligence learning model.

[0013] The capturing step may include: obtaining the facial area of ​​the user in the captured image, and the obtaining step may include: obtaining the facial area of ​​the user in the captured image using a support vector machine (SVM) algorithm, and removing the eyes, mouth and neck from the obtained facial area of ​​the user.

[0014] The grouping may include grouping the image of a remaining area from which the eye, mouth, and neck areas are removed into a plurality of areas including a plurality of pixels of similar colors.

[0015] The removing step may also include removing a forehead area from the user's facial area, and the grouping step may include grouping the image of the remaining area from which the eye, mouth, neck and forehead areas are removed into a plurality of areas including a plurality of pixels of similar colors.

[0016] The removing step may also include: removing an area of ​​the forehead part from the user's facial area, and the grouping step may include grouping images of some areas of the remaining areas from which the eyes, mouth, neck, and forehead are removed into multiple areas including multiple pixels of similar colors, and the some areas may include the area from which the mouth area is removed.

[0017] According to another embodiment, an electronic device includes: a capturer; an outputter configured to output information about a heart rate; and a processor configured to group a face of a user included in an image captured by the capturer into a plurality of regions including a plurality of pixels of similar colors, input information about the grouped plurality of regions into an artificial intelligence learning model to obtain information about the user's heart rate, and control the outputter to output the obtained information about the heart rate.

[0018] The processor can group the user's face into multiple areas based on the color information and position information of the multiple pixels constituting the user's face and obtain a color value corresponding to each of the multiple areas after grouping, and group multiple areas within a predetermined color range into the same group based on the color value corresponding to each of the multiple areas obtained, and then use the color value of each area in the multiple areas grouped into the same group to obtain the pulse signals of the multiple areas grouped into the same group.

[0019] The processor may acquire information about the user's heart rate by inputting pulse signals for the plurality of regions grouped into the same group into an artificial intelligence learning model.

[0020] The artificial intelligence learning model may include a frequency decomposition layer and a complex layer, wherein the frequency decomposition layer is configured to obtain periodic attribute information of periodic iteration from the input pulse signal, and the complex layer is configured to convert the periodic attribute information obtained by the frequency decomposition layer into a value that can be recognized by the artificial intelligence learning model.

[0021] The processor may acquire a facial region of the user in the captured image using a support vector machine (SVM) algorithm, and remove eyes, a mouth, and a neck from the acquired facial region of the user.

[0022] The processor may group the image of the remaining area from which the eye, mouth, and neck areas are removed into a plurality of areas including a plurality of pixels of similar colors.

[0023] The processor may also remove a forehead region from the user's facial region, and group the image of the remaining region from which the eye, mouth, neck, and forehead regions are removed into a plurality of regions including a plurality of pixels of similar colors.

[0024] The processor may group images of some of the remaining areas from which the eye, mouth, and forehead areas are removed into a plurality of areas including a plurality of pixels of similar colors, and the some of the areas may include an area from which the mouth area is removed.

[0025] Beneficial Effects

[0026] According to an embodiment, an electronic device can more accurately measure a user's heart rate through a captured image by grouping the user's face included in the captured image into regions by color, and using data based on color values ​​of the grouped regions as input values ​​of an artificial intelligence (AI) model. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is an example diagram showing measurement of a user's heart rate by an electronic device according to an embodiment;

[0028] Figure 2 is a block diagram illustrating an electronic device providing information about a user's heart rate according to an embodiment;

[0029] Figure 3 is a detailed block diagram of an electronic device providing information about a user's heart rate according to an embodiment;

[0030] Figure 4 is an example diagram showing an artificial intelligence learning model according to an embodiment;

[0031] Figure 5 is a first example diagram of acquiring a user's facial region from a captured image by a processor according to an embodiment;

[0032] Figure 6 is a second exemplary diagram illustrating obtaining a user's facial region from a captured image by a processor according to another embodiment;

[0033] Figure 7 is a detailed block diagram of a processor of an electronic device for updating and using an artificial intelligence learning model according to an embodiment;

[0034] Figure 8 is a detailed block diagram of a learning unit and an acquisition unit according to an embodiment;

[0035] Fig. 9 is an exemplary diagram of an electronic device and an external server learning and determining data in association with each other according to an embodiment;

[0036] Fig.10 is a flowchart of a method for providing information about a user's heart rate through an electronic device according to an embodiment; and

[0037] Fig.11 is a flowchart of a method for grouping a facial region of a user into a plurality of regions including a plurality of pixels of similar colors according to an embodiment. DETAILED DESCRIPTION

[0038] Hereinafter, various exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that the present disclosure is not limited to specific embodiments, but includes various modifications, equivalents and / or alternatives according to the embodiments of the present disclosure. Throughout the accompanying drawings, similar components will be represented by similar reference numerals.

[0039] In the present disclosure, the expressions “have”, “may have”, “include” or “may include” may be used to indicate the presence of features (e.g., components such as values, functions, operations, parts, etc.), and do not exclude the presence of additional features.

[0040] In the present disclosure, the expression "A or B", "at least one of A and / or B", or "one or more of A and / or B", etc. includes all possible combinations of the listed items. For example, "A or B", "at least one of A and B", or "at least one of A or B" includes (1) at least one A, (2) at least one B, and (3) at least one A and at least one B at the same time.

[0041] In addition, the expressions "first", "second", etc. used in the present disclosure may indicate various components regardless of the order and / or importance of the components. The expressions "first", "second", etc. used in the present disclosure may be used to distinguish one component from other components and do not limit the corresponding components.

[0042] It should be understood that an element (e.g., a first element) is "operably or communicatively coupled" to another element (e.g., a second element) / "operably or communicatively coupled to" another element (e.g., a second element) means that any such element can be directly connected to the other element, or can be connected via another element (e.g., a third element). On the other hand, when an element (e.g., a first element) is "directly connected" or "directly accessed" to another element (e.g., a second element), it can be understood that there is no other element (e.g., a third element) between the element and the other element.

[0043] In this document, the expression "configured to" may be used interchangeably with, for example, "suitable for", "capable of", "designed to", "adapted to", "made to", or "capable of". The expression "configured to" does not necessarily mean "specially designed to" in a hardware sense. On the contrary, in some cases, "a device configured to" may indicate that such a device can perform an action together with another device or component. For example, the expression "a processor configured to perform A, B, and C" may indicate a dedicated processor (e.g., an embedded processor) that performs the corresponding actions, or a general-purpose processor (e.g., a central processing unit (CPU) or an application processor (AP)) that can perform the corresponding actions by running one or more software programs stored in a memory device.

[0044] The electronic device according to various example embodiments may include at least one of the following items: for example, but not limited to, a smart phone, a tablet personal computer (PC), a mobile phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP), a Moving Picture Experts Group Stage 1 or Stage 2 (MPEG-1 or MPEG-2) Audio Layer 3 (MP3) player, a medical device, a camera, a wearable device, etc. The wearable device may include at least one of an accessory type (e.g., a watch, a ring, a bracelet, a folding bracelet, a necklace, a pair of glasses, a contact lens, or a head mounted device (HMD)), a fabric or clothing embedded type (e.g., electronic clothes), a body attached type (e.g., a skin pad or a tattoo), a bio-implantable circuit, etc. In some embodiments of the present disclosure, the electronic device may include at least one of the following items: for example, but not limited to, a television, a digital video disc (DVD) player, a speaker, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (e.g., Samsung HomeSync TM 、AppleTV TM or GoogleTV TM ), game consoles (e.g., Xbox TM , PlayStation TM ), electronic dictionaries, electronic keys, cameras, electronic photo frames, etc.

[0045] In another example embodiment, the electronic device may include at least one of the following items: for example, but not limited to, various medical devices (e.g., various portable medical measuring devices (such as blood glucose meters, heart rate meters, blood pressure meters, or temperature measuring devices), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), computed tomography (CT), capture devices, or ultrasonic devices, etc.), navigation systems, global navigation satellite systems (GNSS), event data recorders (EDRs), flight data recorders (FDRs), car infotainment devices, marine electronic equipment (e.g., ship navigation devices, gyrocompasses, etc.), avionics equipment, security devices, car head units, industrial or household robots, drones, automated teller machines (ATMs), store points of sale (POS), Internet of Things (IoT) devices (e.g., light bulbs, various sensors, sprinkler systems, fire alarms, thermostats, street lights, ovens, exercise equipment, hot water tanks, heaters, boilers, etc.), etc.

[0046] In the present disclosure, the term user may refer to a person using an electronic device or an apparatus using an electronic device (eg, an artificial intelligence (AI) electronic device).

[0047] Figure 1is an example diagram illustrating measuring a user's heart rate by an electronic device according to an embodiment.

[0048] The electronic device 100 may be a device that captures an image and measures the heart rate of the user based on an image of the user's face included in the captured image.

[0049] The electronic device 100 may be a device such as a smart phone, a tablet personal computer (PC), a smart television (TV), a smart watch, etc., or a smart medical device capable of measuring a heart rate.

[0050] like Figure 1 As shown in (a) of , if an image is captured, the electronic device 100 may group the face of the user included in the captured image into a plurality of regions including a plurality of pixels of similar colors.

[0051] According to an embodiment, when acquiring a user's facial region in an image frame constituting a captured image, the electronic device 100 may group the user's face into a plurality of regions based on color information and position information of a plurality of pixels constituting the acquired user's facial region.

[0052] The electronic device 100 may group pixels having the same color among adjacent pixels into one group based on color information and position information of a plurality of pixels constituting the face area of ​​the user acquired from the captured image.

[0053] However, the embodiment is not limited thereto, and the electronic device 100 may group pixels having colors included in a predetermined color range among adjacent pixels into one group based on color information and position information of a plurality of pixels constituting the face area of ​​the user.

[0054] The electronic device 100 acquires a color value corresponding to each of the grouped plurality of regions. The electronic device 100 may acquire a color value corresponding to each of the plurality of regions based on color information of pixels included in each of the grouped plurality of regions.

[0055] According to an embodiment, the electronic device 100 may calculate an average value based on color information of pixels included in each of the grouped plurality of regions and may acquire the calculated average value as a color value corresponding to each of the grouped plurality of regions.

[0056] The electronic device 100 may then group the plurality of regions in a predetermined color range into the same group based on a color value corresponding to each of the plurality of regions.

[0057] The electronic device 100 may group a plurality of regions in a predetermined color range into the same group using Gaussian distribution.

[0058] like Figure 1As shown in (b), the electronic device 100 can group areas similar to color A among the multiple areas into a first group, group areas similar to color B into a second group, and group areas similar to color C into a third group based on the color information and position information of each area among the multiple areas constituting the user's face.

[0059] like Figure 1 As shown in (c), the electronic device 100 can obtain pulse signals for the grouped multiple areas based on the grouped color values.

[0060] As described above, when multiple areas are grouped into first to third groups based on the color value of each area in the grouped areas, the electronic device 100 can obtain a first pulse signal based on the color value of each area included in the first group, obtain a second pulse signal based on the color value of each area included in the second group, and obtain a third pulse signal based on the color value of each area included in the third group.

[0061] like Figure 1 As shown in (d) of FIG. 1 , the electronic device 100 may obtain information about the user's heart rate by inputting pulse signals for a plurality of grouped regions into an artificial intelligence learning model. The electronic device 100 may output the obtained information about the user's heart rate, such as Figure 1 As shown in (e).

[0062] Each configuration of the electronic device 100 that provides information about the heart rate of the user by analyzing a region of the user's face included in a captured image will be described in more detail.

[0063] Figure 2 is a block diagram illustrating an electronic device providing information about a user's heart rate according to an embodiment.

[0064] like Figure 2 As shown in , the electronic device 100 includes a capturer 110 , an outputter 120 , and a processor 130 .

[0065] The capturer 110 captures an image using a camera. The captured image may be a moving image or a still image.

[0066] The outputter 120 outputs information about the heart rate of the user acquired based on the facial area of ​​the user included in the image captured by the capturer 110. The outputter 120 may include a Figure 3 The display 121 and the audio output 122 are shown in FIG.

[0067] Therefore, the outputter 120 may output information on the user's heart rate through at least one of the display 121 and the audio outputter 122 .

[0068] The processor 130 controls the operations of the configuration of the electronic device 100 in an overall manner.

[0069] The processor 130 groups the user's face included in the image captured by the capturer 110 into a plurality of regions including a plurality of pixels of similar colors. The processor 130 may then input information about the grouped plurality of regions into an artificial intelligence learning model to acquire information about the user's heart rate.

[0070] The processor 130 then controls the outputter 120 to output the information about the acquired heart rate of the user. Therefore, the outputter 120 may output the information about the heart rate of the user through at least one of the display 121 and the audio outputter 122.

[0071] The processor 130 may group the user's face into a plurality of regions based on color information and position information of a plurality of pixels constituting the user's face, and then acquire a color value corresponding to each of the grouped plurality of regions.

[0072] According to an embodiment, the processor 130 may group pixels having the same color among adjacent pixels into one group based on color information and position information of a plurality of pixels constituting the face area of ​​the user.

[0073] Embodiments are not limited thereto, and the processor 130 may group pixels having colors included in a predetermined color range among adjacent pixels into one group based on color information and position information of a plurality of pixels constituting the user's facial region.

[0074] The processor 130 may calculate an average value based on color information of a plurality of pixels included in each of the grouped plurality of regions and may acquire the calculated average value as a color value corresponding to each of the grouped plurality of regions.

[0075] The processor 130 may group the plurality of regions into the same group based on a color value corresponding to each of the plurality of regions in a predetermined color range.

[0076] The processor 130 may group a plurality of regions in a predetermined color range into the same group using Gaussian distribution.

[0077] The processor 130 may acquire the pulse signals for the plurality of regions grouped into the same group using the color values ​​of the plurality of regions grouped into the same group.

[0078] When pulse signals for a plurality of regions grouped into the same group are acquired, the processor 130 may input the pulse signals for the plurality of regions grouped into the same group into an artificial intelligence learning model to acquire information about the user's heart rate.

[0079] The artificial intelligence learning model may be stored in the memory 170 which will be described later, and the artificial intelligence model will be described in more detail below.

[0080] The processor 130 may acquire the user's facial region from the image captured by the capturer 110 using the following embodiments.

[0081] When an image is captured through the capturer 110, the processor 130 may acquire a facial region of the user within a plurality of image frames constituting the image captured using a support vector machine (SVM) algorithm.

[0082] Processor 130 may use the confidence map to reduce noise at the edges of the user's face.

[0083] According to an embodiment, the processor 130 may reduce noise at an edge of a user's face region using a confidence map based on Equation 1 below.

[0084] Equation 1

[0085] inside mask =distance_transform

[0086] dist max =log 10.5-log 0.5

[0087] inside mask =log(inside mask +0.5)-log 0.5

[0088]

[0089] Result mask =inside mask / inside mask

[0090]

[0091]

[0092] confidence map =[∑skin]×(1-mask w_rate )+(result mask ×mask w_rate )

[0093] When acquiring the user's facial region from a captured image through the above embodiment, the processor 130 may remove a partial region from the previously acquired facial region through a predefined feature point algorithm, and may group the remaining region into multiple regions including multiple pixels of similar colors after removal.

[0094] According to one embodiment, the processor 130 may detect eye, mouth, and neck regions in the acquired user's facial region using a predefined feature point algorithm, and may remove the detected eye, mouth, and neck regions.

[0095] The processor 130 may group the remaining area from which the eyes, the mouth, and the neck are removed in the face area of ​​the user into a plurality of areas including a plurality of pixels of similar colors according to the above-described embodiment.

[0096] According to another embodiment, the processor 130 may detect eye, mouth, neck, and forehead regions of the user when acquiring the user's facial region, and may remove the detected eye, mouth, neck, and forehead regions of the user.

[0097] The processor 130 may group the remaining areas of the user's face after the eyes, mouth, neck, and forehead are removed into a plurality of areas including a plurality of pixels of similar colors.

[0098] According to another embodiment, when acquiring the user's facial region, the processor 130 may detect eye, mouth, neck, and forehead regions of the user, and may remove the detected eye, mouth, neck, and forehead regions.

[0099] The processor 130 may group the region of the user's face from which eyes, mouth, neck, and forehead are removed in the image of some of the remaining regions into a plurality of regions including a plurality of pixels of similar colors.

[0100] Figure 3 is a detailed block diagram of an electronic device providing information about a user's heart rate according to an embodiment.

[0101] As described above, in addition to the configuration of the capturer 110, the outputter 120, and the processor 130, the electronic device 100 may further include an inputter 140, a communicator 150, a sensor 160, and a memory 170. Figure 3 as shown in .

[0102] The inputter 140 is an input device for receiving various user commands and transmitting the commands to the processor 130. The inputter 140 may include a microphone 141, a manipulator 142, a touch inputter 143, and a user inputter 144.

[0103] The microphone 141 may receive a user's voice command, and the manipulator 142 may be implemented as a keyboard including various function keys, number keys, special keys, character keys, and the like.

[0104] When the display 121 is implemented in the form of a touch screen, the touch inputter 143 may be implemented as a touch pad forming an interlayer structure with the display 121. In this example, the touch inputter 143 may receive selection commands for various application-related icons displayed through the display 121.

[0105] The user input 144 may receive an infrared (IR) signal or a radio frequency (RF) signal for controlling the operation of the electronic device 100 from at least one peripheral device (not shown) such as a remote controller.

[0106] The communicator 150 performs data communication with a peripheral device (not shown) such as a smart TV, a smart phone, a tablet PC, a content server (not shown), and a relay terminal device (not shown) for transmitting and receiving data. When the above-mentioned artificial intelligence model is stored in a separate artificial intelligence server (not shown), the communicator 150 may transmit a pulse signal acquired based on the facial area of ​​the user included in the captured image to the artificial intelligence server (not shown), and may receive information about the heart rate of the user based on the pulse signal from the artificial intelligence server (not shown).

[0107] The communicator 150 may include a connector 153, wherein the connector 153 includes at least one of a wireless communication module 152 such as a wireless LAN module, a near field communication module 151, and a wired communication module such as a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB), an Institute of Electrical and Electronics Engineers (IEEE) 1394, and the like.

[0108] The near field communication module 151 may include various near field communication circuits and may be configured to wirelessly perform near field communication with a peripheral device located at a close distance from the electronic device 100. The near field communication module 131 may include at least one of a Bluetooth module, an infrared data association (IrDA) module, a near field communication (NFC) module, a WI-FI module, and a Zigbee module.

[0109] The wireless communication module 152 is a module for connecting to an external network to perform communication according to a wireless communication protocol such as IEEE. The wireless communication module may also include a mobile communication module for connecting to a mobile communication network according to various mobile communication specifications for performing communication, such as the third generation (3G), the third generation partnership project (3GPP), the long term evolution (LTE), etc.

[0110] The communicator 150 may be implemented as various near field communication methods, and may adopt other communication techniques not mentioned in the present disclosure as necessary.

[0111] The connector 153 is configured to provide an interface with various source devices, such as USB 2.0, USB 3.0, HDMI, IEEE1394, etc. The connector 153 may receive content data transmitted from an external server (not shown) through a wired cable connected to the connector 153 according to a control command of the processor 130, or transmit pre-stored content data to an external recordable medium. The connector 153 may receive power from a power source through a wired cable physically connected to the connector 153.

[0112] The sensor 160 may include an accelerometer sensor, a magnetic sensor, a gyro sensor, etc., and sense the motion of the electronic device 100 using various sensors.

[0113] The accelerometer sensor is a sensor for measuring acceleration or vibration intensity of the mobile electronic device 100, and is an essential sensor for various transportation tools (such as vehicles, trains, airplanes, etc.) and control systems (such as robots) in addition to electronic devices (such as smart phones and tablet PCs).

[0114] Magnetic sensors are electronic compasses that can sense orientation using the Earth's magnetic field and can be used for location tracking, three-dimensional (3D) video games, smart phones, radios, global positioning systems (GPS), personal digital assistants (PDAs), navigation devices, etc.

[0115] A gyro sensor is a sensor for applying rotation to an existing accelerometer to identify six-axis directions for more delicate and precise operations.

[0116] As described above, the memory 170 may store an artificial intelligence learning model to acquire information about the user's heart rate based on a pulse signal acquired from the user's facial area.

[0117] The memory 170 may store an operation program for controlling the operation of the electronic device 100 .

[0118] If the electronic device 100 is turned on, the operating program may be a program that is read from the memory 170 and compiled to operate each configuration of the electronic device 100. The memory 170 may be implemented as at least one of a read-only memory (ROM), a random access memory (RAM), or a memory card (e.g., a secure digital (SD) card, a memory stick) that is detachable from the electronic device 100, a nonvolatile memory, a volatile memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0119] As described above, the outputter 120 includes the display 121 and the audio outputter 122 .

[0120] As described above, the display 121 displays information about the user's heart rate acquired through the artificial intelligence learning model. The display 121 may display content, or may display an operation screen including an icon for operating each of a plurality of applications stored in the memory 170 or various user interface (UI) screens for controlling the operation of the electronic device 100, which will be described later.

[0121] The display 121 may be implemented as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, or the like.

[0122] The display 121 may be implemented as a touch screen forming an interlayer structure with the touch inputter 143 receiving a touch command.

[0123] As described above, the audio outputter 122 outputs the information about the user's heart rate acquired through the artificial intelligence learning model in the form of audio. The audio outputter 122 may output audio data included in the content requested by the user or various alarm sounds or voice messages.

[0124] The processor 130 as described above may be a processing device that controls the overall operation of the electronic device 100 or implements control over the overall operation of the electronic device 100 .

[0125] The processor 130 may include a central processing unit (CPU) 133 , a read-only memory ROM 131 , a random access memory (RAM) 132 , and a graphics processing unit (GPU) 134 , and the CPU 133 , ROM 131 , RAM 132 , and GPU 134 may be connected to each other via a bus 135 .

[0126] The CPU 133 accesses the memory 170 and performs booting using an operating system (OS) stored in the memory 170 , and performs various operations using various programs, content data, and the like stored in the memory 170 .

[0127] The GPU 134 may generate a display screen including various objects such as icons, images, texts, etc. The GPU 134 may calculate attribute values ​​such as coordinate values, shapes, sizes, and colors to be displayed by each object according to the layout of the screen based on the received control command, and may generate a display screen including various layouts of the objects based on the calculated attribute values.

[0128] ROM 131 stores one or more instructions for booting the system, etc. When a power-on instruction is input and power is supplied, CPU 133 copies an OS stored in ROM 131 to RAM 134 according to one or more instructions stored in ROM 131, and runs the OS to boot the system. When the booting is completed, CPU 133 copies various application programs stored in storage 170 to RAM 132, runs the application programs copied to RAM 132, and performs various operations.

[0129] The processor 130 may be coupled with each configuration, and may be implemented as a single-chip system (system on chip, SOC or SoC).

[0130] Hereinafter, an artificial intelligence learning model for providing information about a user's heart rate based on a pulse signal acquired based on color information and position information of each of a plurality of pixels constituting a facial area of ​​the user will be described in detail.

[0131] Figure 4 is an example diagram illustrating an artificial intelligence learning model according to an embodiment.

[0132] Reference Figure 4 , the artificial intelligence learning model 400 includes a frequency decomposition layer 410 and a complex number layer 420.

[0133] The frequency decomposition layer 410 obtains periodic attribute information of periodic iteration from the input pulse signal.

[0134] The complex layer 420 converts the periodic attribute information input through the frequency decomposition layer 410 into a value that can be recognized by the artificial intelligence learning model 400.

[0135] As described above, the frequency decomposition layer 410 receives the pulse signals of the plurality of regions grouped into the same group. When the pulse signals of the plurality of regions grouped into the same group are input, the frequency decomposition layer 410 acquires periodic attribute information that repeats periodically from the pulse signals for each group.

[0136] The periodic attribute information may be a complex value.

[0137] When periodic property information as a complex value is input through the frequency decomposition layer 410, the complex number layer 420 converts the value into a value recognizable by the artificial intelligence learning model 400. Here, the recognizable value in the artificial intelligence learning model 400 may be a real value.

[0138] The artificial intelligence learning model 400 may acquire information about the user's heart rate using a transformation value associated with periodic attribute information acquired from the pulse signal for each group through the complex layer 420 .

[0139] Hereinafter, an operation of acquiring a user's facial region from an image captured by the capturer 110 will be described in more detail.

[0140] Figure 5 is a first example diagram of a processor acquiring a user's facial region from a captured image according to an embodiment.

[0141] like Figure 5 As shown in (a) of FIG. 1 , when an image captured by the capturer 110 is input, the processor 130 acquires a facial area of ​​a user within the image input by the above-described embodiment.

[0142] The processor 130 may detect the areas of the eyes, mouth, neck, and forehead within the user's facial region that have been acquired using a predefined feature point algorithm. The processor 130 may then remove the detected areas of the eyes, mouth, neck, and forehead within the user's facial region.

[0143] like Figure 5 As shown in (b), the processor 130 may obtain a facial region of the user from which the eye, mouth, neck, and forehead regions have been removed, and may perform grouping into multiple regions including multiple pixels of similar colors within the facial region of the user from which the eye, mouth, neck, and forehead regions have been removed.

[0144] Figure 6 is a second exemplary diagram illustrating acquiring a user's facial region from a captured image by a processor according to another embodiment.

[0145] like Figure 6 As shown in (a) of FIG. 1 , when an image captured by the capturer 110 is input, the processor 130 may acquire a facial region of a user in the image input by the above-described embodiment.

[0146] The processor 130 may detect the eye, mouth, neck and forehead regions in the pre-acquired user's facial region using a predefined feature point algorithm. The processor 130 then removes the detected eye, mouth, neck and forehead regions from the user's facial region.

[0147] As described above, if a facial region of a user in which the eye, mouth, neck, and forehead regions are removed is obtained, the processor 130 may determine regions to be grouped into a plurality of regions in the facial region of the user in which the eye, mouth, neck, and forehead regions are removed.

[0148] like Figure 6 As shown in (a), the processor 130 determines some areas where the eye, mouth, neck and forehead areas in the user's facial area are removed as areas to be grouped into multiple areas. Here, a part of the area may be a lower area where the area including the mouth is removed.

[0149] Therefore, if Figure 6 As shown in (b), the processor 130 may acquire a lower portion of the user's facial region from which eye, mouth, neck, and forehead regions have been removed, and may perform grouping into a plurality of regions including a plurality of pixels of similar colors within the acquired lower region.

[0150] Hereinafter, the operation of updating and using the artificial intelligence learning model by the processor 130 will be described in more detail.

[0151] Figure 7 is a detailed block diagram of a processor of an electronic device for updating and using an artificial intelligence learning model according to an embodiment.

[0152] like Figure 7 As shown in , the processor 130 may include a learning unit 510 and an acquisition unit 520.

[0153] The learning unit 510 may generate or train an artificial intelligence learning model for acquiring information about the user's heart rate using the learning data.

[0154] The learning data may include at least one of user information, periodic attribute information according to a pulse signal acquired based on a face image of the user, and information on a heart rate according to the periodic attribute information.

[0155] Specifically, the learning unit 510 can generate, train or update an artificial intelligence learning model for obtaining information about the heart rate of a corresponding user by using a pulse signal obtained based on color values ​​of areas grouped in the same group as input data having similar color distribution in a facial area of ​​the user included in a captured image.

[0156] The acquisition unit 520 may acquire information about the user's heart rate by using predetermined data as input data of a pre-learned artificial intelligence learning model.

[0157] The acquisition unit 520 may acquire (or identify, estimate) information about the heart rate of the corresponding user using a pulse signal acquired based on color values ​​of regions grouped in the same group as input data having similar color distribution in a facial region of the user included in a captured image.

[0158] For example, at least one of the learning unit 510 and the acquiring unit 520 may be implemented as a software module or at least one hardware chip form and installed in the electronic device 100 .

[0159] For example, at least one of the learning unit 510 and the acquisition unit 520 can be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or a conventional general-purpose processor (e.g., a CPU or an application processor) or only a graphics processor (e.g., a GPU) and can be installed on various electronic devices as described above.

[0160] In this article, the artificial intelligence dedicated hardware chip is a dedicated processor for probability calculation, and the artificial intelligence dedicated hardware chip has higher parallel processing performance than the existing general-purpose processor, so it can quickly process computing tasks in artificial intelligence (such as machine learning). When the learning unit 510 and the acquisition unit 520 are implemented as software modules (or program modules including instructions), the software modules may be stored in a computer-readable non-temporary computer-readable medium. In this case, the software module may be provided by an operating system (OS) or by a predetermined application. Optionally, some of the software modules may be provided by O / S, and some of the software modules may be provided by a predetermined application.

[0161] In this case, the learning unit 510 and the acquisition unit 520 may be installed on one electronic device 100, or may be installed on separate electronic devices, respectively. For example, one of the learning unit 510 and the acquisition unit 520 may be implemented in the electronic device 100, and the other may be implemented in an external server (not shown). In addition, the learning unit 510 and the acquisition unit 520 may provide the model information constructed by the learning unit 510 to the acquisition unit 520 via wired or wireless communication, and provide the data input to the acquisition unit 520 to the learning unit 510 as additional data.

[0162] Figure 8 is a detailed block diagram of a learning unit and an acquisition unit according to an embodiment.

[0163] Reference Figure 8 (a), according to some embodiments, the learning unit 510 may include a learning data acquisition unit 511 and a model learning unit 514. The learning unit 510 may also selectively implement at least one of a learning data preprocessor 512, a learning data selection unit 513, and a model evaluation unit 515.

[0164] The learning data acquisition unit 511 may acquire the learning data required for the artificial intelligence model. As an embodiment, the learning data acquisition unit 511 may acquire at least one of periodic attribute information of a pulse signal acquired based on an image of the user's face and information about the heart rate based on the periodic attribute information as learning data.

[0165] The learning data may be data collected or tested by the learning unit 510 or a manufacturer of the learning unit 510 .

[0166] The model learning unit 514 can use the learning data to train how to obtain periodic attribute information based on the pulse signal obtained based on the user's facial image or obtain information about the heart rate based on the periodic attribute information. For example, the model learning unit 514 can train the artificial intelligence model through supervised learning using at least a portion of the learning data as a determination criterion.

[0167] Alternatively, the model learning unit 514 may learn by itself using learning data, for example without specific guidance, so that the artificial intelligence model learns through unsupervised learning that detects criteria for determining a situation.

[0168] In addition, the model learning unit 514 can train the artificial intelligence model through reinforcement learning using, for example, feedback as to whether the result of a determined situation according to learning is correct.

[0169] The model learning unit 514 may also use a learning algorithm including, for example, an error back propagation method or gradient descent to enable the artificial intelligence model to learn.

[0170] The model learning unit 514 may determine an artificial intelligence model with a large correlation between the input learning data and the basic learning data as an artificial intelligence model to be learned when there are multiple previously constructed artificial intelligence models. In this case, the basic learning data may be classified in advance according to the type of data, and an AI model may be pre-constructed for each type of data.

[0171] For example, basic learning data may be pre-classified based on various criteria such as the region where the learning data was generated, the time when the learning data was generated, the size of the learning data, the type of learning data, the creator of the learning data, the type of objects within the learning data, and the like.

[0172] When learning the artificial intelligence model, the model learning unit 514 may store the learned artificial intelligence model. In this case, the model learning unit 514 may store the learned artificial intelligence model in the memory 170 of the electronic device 100.

[0173] Alternatively, the model learning unit 514 may store the learned artificial intelligence model in a memory of a server (eg, an AI server) (not shown) connected to the electronic device 100 via a wired or wireless network.

[0174] The learning unit 510 may also implement a learning data preprocessor 512 and a learning data selection unit 513 to improve the response results of the artificial intelligence model or save resources or time required for the generation of the artificial intelligence model.

[0175] The learning data preprocessor 512 may preprocess the data associated with the learning to obtain information about periodic attribute information according to the pulse signal and obtain information about the user's heart rate based on the periodic attribute information.

[0176] The learning data preprocessor 512 may process the acquired data into a predetermined format so that the model learning unit 514 may use the learning-related data to acquire periodic attribute information for each pulse signal and information about the user's heart rate based on the periodic attribute information.

[0177] The learning data selection unit 513 may select data required for learning from the data acquired by the learning data acquisition unit 511 or the data preprocessed by the learning data preprocessor 512. The selected learning data may be provided to the model learning unit 514. The learning data selection unit 513 may select the learning data required for learning from the acquired or preprocessed data according to a predetermined selection criterion. The learning data selection unit 513 may also select the learning data according to a predetermined selection criterion through learning by the model learning unit 514.

[0178] The learning unit 510 may also implement a model evaluation unit 515 to improve the response results of the artificial intelligence model.

[0179] The model evaluation unit 515 may input the evaluation data to the artificial intelligence model, and if the response result output from the evaluation result does not meet the predetermined standard, the model evaluation unit may cause the model learning unit 514 to learn again. In this example, the evaluation data may be predefined data for evaluating the AI ​​learning model.

[0180] For example, in the artificial intelligence learning model recognition result for learning of the evaluation data, the model evaluation unit 515 may evaluate that the recognition result does not meet the predetermined standard when the number or ratio of incorrect evaluation data exceeds a preset threshold.

[0181] When there are multiple learned artificial intelligence learning models, the model evaluation unit 515 may evaluate whether each learned artificial intelligence learning model meets the predetermined criteria, and determine the artificial intelligence learning model that meets the predetermined criteria as the final artificial intelligence learning model. In this example, when there are multiple artificial intelligence learning models that meet the predetermined criteria, the model evaluation unit 515 may determine any one or a predetermined number of models preset in order of high evaluation scores as the final artificial intelligence learning model.

[0182] Reference Figure 8 (b), according to some embodiments, the acquiring unit 520 may include an input data acquiring unit 521 and a providing unit 524.

[0183] In addition, the acquisition unit 520 may also implement at least one of the input data preprocessor 522 , the input data selection unit 523 , and the model updating unit 525 in a selective manner.

[0184] The input data acquisition unit 521 may acquire periodic attribute information based on the pulse signal acquired based on the image of the user's face, and acquire data required for acquiring information about the user's heart rate based on the acquired periodic attribute information. The providing unit 524 applies the data acquired by the input data acquisition unit 521 to the AI ​​model to acquire periodic attribute information based on the pulse signal acquired based on the image of the user's face, and may acquire information about the user's heart rate based on the acquired periodic attribute information.

[0185] The providing unit 524 may apply the data selected by the input data preprocessor 522 or the input data selecting unit 523 to the artificial intelligence learning model to obtain a recognition result. The recognition result may be determined by the artificial intelligence learning model.

[0186] As an embodiment, the providing unit 524 may acquire (estimate) periodic property information from the pulse signal acquired by the input data acquiring unit 521 .

[0187] As another example, the providing unit 524 may acquire (or estimate) information about the heart rate of the user based on periodic attribute information acquired from the pulse signal acquired by the input data acquiring unit 521 .

[0188] The acquisition unit 520 may further include an input data preprocessor 522 and an input data selection unit 523 to improve the recognition result of the AI ​​model or save resources or time for providing the recognition result.

[0189] The input data preprocessor 522 may preprocess the acquired data so that the acquired data may be used for input to the artificial intelligence learning model. The input data preprocessor 522 may process the data in a predefined format so that the providing unit 524 may use the data to acquire periodic attribute information acquired from the pulse signal and information about the user's heart rate based on the periodic attribute information.

[0190] The input data selection unit 523 may select data required for determining a situation from the data acquired by the input data acquisition unit 521 or the data preprocessed by the input data preprocessor 522. The selected data may be provided to the response result providing unit 524. The input data selection unit 523 may select some or all of the acquired or preprocessed data according to a predetermined selection criterion for determining a situation. The input data selection unit 523 may also select data according to a predetermined selection criterion through learning by the model learning unit 524.

[0191] The model updating unit 525 may control the updating of the artificial intelligence model based on the evaluation of the response result provided by the providing unit 524. For example, the model updating unit 525 may provide the response result provided by the providing unit 524 to the model learning unit 524, so that the model learning unit 524 may request further learning or updating of the AI ​​model.

[0192] Fig. 9 is an exemplary diagram in which an electronic device and an external server learn and determine data in association with each other according to an embodiment.

[0193] like Fig. 9 As shown in , the external server S can obtain periodic attribute information from the acquired pulse signal based on color information and position information of the user's facial area included in the captured image, and can learn a standard for obtaining information about the user's heart rate based on the acquired periodic attribute information.

[0194] The electronic device (A) can obtain periodic attribute information from a pulse signal obtained based on color information and position information of a user's facial area by using an artificial intelligence learning model generated based on learning results of a server (S), and can obtain information about the user's heart rate based on the obtained periodic attribute information.

[0195] The model learning unit 514 of the server S may execute Figure 7 The functions of the learning unit 510 shown in FIG. 5A . The model learning unit 514 of the server S may learn a determination criterion (or recognition criterion) for an artificial intelligence learning model.

[0196] The providing unit 514 of the electronic device A can apply the data selected by the input data selection unit 513 to the artificial intelligence learning model generated by the server S to obtain periodic attribute information from the pulse signal obtained based on the color information and position information of the user's facial area, and obtain information about the user's heart rate based on the obtained periodic attribute information.

[0197] Optionally, the providing unit 514 of the electronic device A may receive an artificial intelligence learning model generated by the server S from the server S, use the received artificial intelligence learning model to obtain periodic attribute information from a pulse signal obtained based on color information and position information of the user's facial area, and obtain information about the user's heart rate based on the obtained periodic attribute information.

[0198] An operation of inputting data acquired from the facial region of the user included in the image captured by the electronic device 100 to the artificial intelligence learning model has been described in detail.

[0199] Hereinafter, a method for providing information on a user's heart rate by inputting data acquired from a facial region of the user included in an image captured by the electronic device 100 to an artificial intelligence learning model will be described in detail.

[0200] Fig.10 is a flowchart of a method for providing information about a user's heart rate through an electronic device according to an embodiment.

[0201] like Fig.10 As shown in , in operation S1010, the electronic device 100 may capture an image including a user's face, and acquire a facial region of the user in the captured image.

[0202] In operation S1020, the electronic device 100 may group the acquired facial regions into a plurality of regions including a plurality of pixels of similar colors. Then, in operation S1030, the electronic device 100 may acquire information about the user's heart rate by inputting information about the grouped plurality of regions into an artificial intelligence learning model.

[0203] The electronic device 100 outputs the acquired information on the user's heart rate.

[0204] When capturing an image, the electronic device 100 may use a support vector machine (SVM) algorithm to acquire a facial region of the user in the pre-capture image.

[0205] If the facial region is acquired, the electronic device 100 may remove the regions of eyes, mouth, and neck from the acquired facial region of the user, and may acquire the facial region of the user from which the eyes, mouth, and neck are deleted.

[0206] The electronic device 100 may group the user's facial region from which the eyes, mouth, and neck are removed into a plurality of regions including a plurality of pixels of similar colors.

[0207] According to another aspect, once the user's facial region is acquired in the captured image, the electronic device 100 removes the eye, mouth, neck, and forehead regions in the acquired facial region. The electronic device 100 can then group the facial region into a plurality of regions including a plurality of pixels of similar colors within the user's facial region from which the eye, mouth, neck, and forehead regions have been removed.

[0208] Once the user's facial region is acquired in the captured image, the electronic device 100 removes the eye, mouth, neck, and forehead regions in the acquired facial region. The electronic device 100 then groups the eye, mouth, neck, and forehead regions into a plurality of regions including a plurality of pixels of similar colors in some regions of the user's facial region. Here, some regions may include a region where the mouth region is removed.

[0209] Fig.11 is a flowchart of a method for grouping a facial region of a user into a plurality of regions including a plurality of pixels of similar colors according to an embodiment.

[0210] like Fig.11 As shown in , when a facial region of a user is acquired from a captured image, the electronic device 100 groups the facial region of the user into a plurality of regions based on color information and position information of a plurality of pixels constituting the facial region of the user in operation S1110.

[0211] Thereafter, in operations S1120 and S1130, the electronic device 100 may acquire a color value corresponding to each of the grouped plurality of regions, and may group a plurality of regions within a predetermined color range into the same group based on the color value corresponding to each of the acquired plurality of regions.

[0212] In operation S1140, the electronic device 100 may acquire a pulse signal for the plurality of regions grouped into the same group using color values ​​of the plurality of regions grouped into the same group.

[0213] Through this embodiment, when a pulse signal for the user's facial area is acquired, the electronic device 100 acquires information about the user's heart rate by inputting the acquired pulse signal into an artificial intelligence learning model.

[0214] When a pulse signal is input, the artificial intelligence learning model obtains periodic attribute information of periodic repetition from the pre-input pulse signal through the frequency decomposition layer. Thereafter, the artificial intelligence learning model converts the periodic attribute information obtained from the frequency decomposition layer into a value that can be recognized by the artificial intelligence learning model through the complex layer.

[0215] The periodic attribute information may be a complex value, and the value that can be recognized in the artificial intelligence learning model may be a real value.

[0216] Therefore, the artificial intelligence learning model provides information about the user's heart rate based on the periodic attribute information converted into a value recognizable in the artificial intelligence learning model through the complex layer. Therefore, the electronic device 100 can output the information provided by the artificial intelligence learning model as information about the user's heart rate.

[0217] In addition, the control method of the electronic device 100 as described above may be implemented as at least one operating program for operating the control method of the image forming apparatus as described above, and the operating program may be stored in a non-transitory computer-readable medium.

[0218] A non-temporary readable medium refers to a medium that stores data for a short period of time (such as a register, cache, and memory), but a medium that stores data semi-permanently and can be read by a device. The above program can be stored in various types of recording media that can be read by a terminal, wherein the various types of recording media include random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), register, hard disk, memory card, universal serial bus (USB) memory, compact disk read-only memory (CD-ROM), etc.

[0219] The preferred embodiments have been described.

[0220] Although examples of the present disclosure have been shown and described above, the present disclosure is not limited to the above specific examples, but various modifications may be made by those skilled in the art to which the present disclosure belongs without departing from the scope and spirit of the present disclosure as disclosed in the appended claims. These modifications should also be understood to fall within the scope of the present disclosure.

Claims

1. A method for measuring heart rate using an electronic device, the method comprising: capturing an image including a face of a user, wherein the image includes a plurality of image frames; grouping the user's face into a plurality of first regions including a plurality of pixels of similar colors based on color information and position information of a plurality of pixels constituting the user's face included in the image; Acquire a color value corresponding to each of the grouped plurality of first regions; Based on the acquired color value corresponding to each of the plurality of first regions, grouping the plurality of first regions within a predetermined color range into the same group to obtain a plurality of second regions; acquiring a pulse signal for the plurality of second areas grouped into the same group using a color value of each of the plurality of second areas grouped into the same group; acquiring information about the user's heart rate by inputting pulse signals about a plurality of second areas grouped into the same group into an artificial intelligence learning model; and Output the acquired information about the heart rate.

2. The method of claim 1, wherein: Artificial intelligence learning models include: A frequency decomposition layer configured to obtain periodic attribute information of periodic iteration from an input pulse signal; and The complex layer is configured to convert the periodic attribute information obtained by the frequency decomposition layer into a value that can be recognized by the artificial intelligence learning model.

3. The method of claim 1, wherein the step of capturing comprises: Get the user's facial area in the captured image, The step of obtaining the facial area of ​​the user in the captured image includes: Obtaining the user's facial region in the captured image using a support vector machine (SVM) algorithm; and The eyes, mouth, and neck are removed from the acquired user's facial region.

4. The method of claim 3, wherein: The step of grouping the plurality of pixels corresponding to the user's facial region further includes grouping the image of the remaining region from which the eye, mouth, and neck regions are removed into a plurality of first regions including a plurality of pixels of similar colors.

5. The method of claim 3, wherein: The step of removing further comprises: removing the forehead area from the user's facial area, and The step of grouping the plurality of pixels corresponding to the user's facial region further includes grouping the image of the remaining region from which the eye, mouth, neck, and forehead regions are removed into a plurality of first regions including a plurality of pixels of similar colors.

6. The method of claim 3, wherein: The removing step further includes: removing the forehead area from the user's facial area, and the step of grouping the plurality of pixels corresponding to the user's facial area further includes: grouping images of some areas of the remaining area where the eyes, mouth, neck, and forehead are removed into a plurality of first areas including a plurality of pixels of similar colors, and The some areas include an area where the mouth area is removed.

7. An electronic device comprising: Catcher; an outputter configured to output information about the heart rate; as well as The processor is configured as: grouping the user's face into a plurality of first regions including a plurality of pixels of similar colors based on color information and position information of a plurality of pixels constituting the user's face, wherein the user's face is included in an image captured by a capturer, Obtain a color value corresponding to each of the grouped first regions, Based on the acquired color value corresponding to each of the plurality of first regions, the plurality of first regions within a predetermined color range are grouped into the same group to obtain a plurality of second regions, acquiring a pulse signal for the plurality of second areas grouped into the same group using a color value of each of the plurality of second areas grouped into the same group, acquiring information about the user's heart rate by inputting pulse signals about a plurality of second areas grouped into the same group into an artificial intelligence learning model, and controlling an outputter to output the acquired information about the heart rate, The image includes multiple image frames.

8. The electronic device as claimed in claim 7, wherein: Artificial intelligence learning models include: A frequency decomposition layer configured to obtain periodic attribute information of periodic iteration from an input pulse signal; and The complex layer is configured to convert the periodic attribute information obtained by the frequency decomposition layer into a value that can be recognized by the artificial intelligence learning model.

9. The electronic device as claimed in claim 7, wherein: The processor is further configured to: acquire a facial region of the user in the captured image using a support vector machine (SVM) algorithm, and remove eyes, a mouth, and a neck from the acquired facial region of the user.

10. The electronic device according to claim 9, wherein: The processor is further configured to group the image of the remaining area from which the eye, mouth, and neck areas are removed into a plurality of first areas including a plurality of pixels of similar colors.

11. The electronic device according to claim 9, wherein: The processor is further configured to remove a forehead region from the user's facial region and group images of a remaining region from which the eye, mouth, neck, and forehead regions are removed into a plurality of first regions including a plurality of pixels of similar colors.

Images