Electronic device, server, system and method for providing highly accurate biosignals based on information acquired in a non-contact manner

Abstract

According to various embodiments, an electronic device may be provided that includes a first communication circuit and at least one first processor, the at least one first processor being configured to: acquire, through the communication circuit, a plurality of images including a user's face acquired using a camera of a first external electronic device; acquire, through the communication circuit, while simultaneously acquiring the plurality of images, first data acquired based on a first sensor of the first external electronic device contacted with a first part of the user's body and second data acquired based on a second external electronic device contacted with a second part of the user's body; and acquire a first biosignal of a particular type based on the plurality of images, a second biosignal of the particular type based on the first data, and a third biosignal based on the second data.

Description

[Technical field]

[0001] Various embodiments of the present invention relate to electronic devices, servers, systems and methods of operation for providing highly accurate biosignals based on information acquired in a non-contact manner. [Background technology]

[0002] The most common technology for measuring photoplethysmography (PPG) is to analyze the amount of light that is transmitted through the human body, and is explained by the Beer-Lambert law, which states that the absorbance is proportional to the concentration of the absorbing material and the thickness of the absorbing layer. According to this law, the change in transmitted light results in a signal proportional to the change in the volume of the material through which it passes, so even if the absorbance of the material is unknown, PPG can be used to understand the condition of the heart.

[0003] Recently, a technology using rPPG (remote photoplethysmography) has emerged, which is one step further from the technology using PPG. The most popular technology for grasping signals related to heartbeat using PPG is a technology in which a device with a camera and light attached at close range, like a smartphone, is directly touched to the human body, light is shone on it, and the transmitted light is immediately measured to obtain PPG. Recently, technology related to rPPG (remote photoplethysmography), which grasps changes in the volume of blood vessels from signals obtained from images captured by a camera, is being continuously researched and developed.

[0004] Because rPPG technology does not require contact between the subject and the measurement equipment, it can be used in a variety of locations and devices equipped with cameras, such as airport immigration checkpoints and remote medical care.

[0005] However, since rPPG technology is heavily influenced by noise generated by ambient light and the movement of the subject during the process of photographing the subject with a camera, the key technology for measuring biosignals using rPPG is the technology to extract only signals related to volumetric changes of the subject from the photographed images. Summary of the Invention [Problem to be solved by the invention]

[0006] Remote Photoplethysmography (rPPG) has a lower accuracy than PPG acquired using a contact sensor. According to various embodiments, an electronic device, a server, a system, and an operating method thereof may be provided that provide information on a biosignal having an accuracy corresponding to that of a biosignal acquired in a contact manner based on information acquired in a non-contact manner.

[0007] The problems to be solved by the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those having ordinary skill in the art to which the present invention pertains from the following description. [Means for solving the problem]

[0008] According to various embodiments, an electronic device may be provided that includes a first communication circuit and at least one first processor, the at least one first processor being configured to: acquire, through the communication circuit, a plurality of images including a user's face acquired using a camera of a first external electronic device; acquire, through the communication circuit, while simultaneously acquiring the plurality of images, first data acquired based on a first sensor of the first external electronic device contacted with a first part of the user's body and second data acquired based on a second external electronic device contacted with a second part of the user's body; and acquire a first biosignal of a particular type based on the plurality of images, a second biosignal of the particular type based on the first data, and a third biosignal based on the second data.

[0009] According to various embodiments, a method of operating an electronic device may be provided, the method including: acquiring, through the communication circuit, a plurality of images including a user's face acquired using a camera of a first external electronic device; acquiring, through the communication circuit, first data acquired based on a first sensor of the first external electronic device contacted with a first part of the user's body and second data acquired based on a second external electronic device contacted with a second part of the user's body, simultaneously while acquiring the plurality of images; and acquiring a first biosignal of a specific type based on the plurality of images, a second biosignal of the specific type based on the first data, and a third biosignal based on the second data. Effect of the Invention

[0010] According to various embodiments, the electronic device, server, system, and operation method thereof can provide information on a biosignal based on information acquired in a non-contact manner, the information having an accuracy corresponding to a biosignal acquired in a contact manner. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an example of the configuration of a biosignal measuring system according to various embodiments; [Diagram 2] 1 is a diagram illustrating an example of an electronic device according to various embodiments. [Diagram 3] 1 is a diagram illustrating an example of the configuration of an electronic device and a server (e.g., a learning server and a utilization server) according to various embodiments. [Figure 4] 1 is a diagram illustrating an example of a biosignal measurement module according to various embodiments. [Diagram 5] 1 is a flowchart illustrating an example operation of a learning server to acquire (or collect) data for artificial intelligence model training, according to various embodiments. [Figure 6]1 is a diagram illustrating an example of an operation of an electronic device for simultaneously collecting biosignals in a non-contact manner and a contact manner, according to various embodiments; [Figure 7] 11 is a diagram illustrating an example of an operation of accumulating data for artificial intelligence model learning in a learning server according to various embodiments. [Figure 8] 10 is a flow chart illustrating an example operation of a learning server for acquiring a non-contact bio-signal, according to various embodiments. [Figure 9] 11 is a diagram illustrating an example of an operation of obtaining a difference value between color channels (eg, G and R values, G and B values) for noise reduction according to various embodiments. [Figure 10] 11 is a diagram illustrating an example of an operation for acquiring a characteristic value according to various embodiments. [Figure 11] 11 is a diagram illustrating an example of an operation for acquiring a characteristic value according to various embodiments. [Figure 12] 10 is a flow chart illustrating an example operation of a learning server for acquiring a non-contact bio-signal, according to various embodiments. [Figure 13] 1 is a diagram illustrating an example of a biosignal according to various embodiments; [Figure 14] 11 is a diagram illustrating an example of a time synchronization operation of a biosignal of a learning server according to various embodiments. [Figure 15] A flowchart showing an example of an operation of generating an artificial intelligence model implemented to provide information on a specific type of biosignal with an accuracy corresponding to a contact type based on a specific type of biosignal acquired in a non-contact type manner of a learning server according to various embodiments. [Figure 16] 11 is a diagram illustrating an example of an operation of a learning server for generating an artificial intelligence model according to various embodiments. [Figure 17] 1 is a diagram illustrating an example of at least one artificial intelligence model generated by a learning server according to various embodiments. [Figure 18a]11 is a diagram illustrating another example of at least one artificial intelligence model generated by a learning server according to various embodiments. [Figure 18b] 11 is a diagram illustrating yet another example of at least one artificial intelligence model generated by a learning server according to various embodiments. [Figure 19] 1 is a flowchart illustrating an example operation of generating an artificial intelligence model without time synchronization of a learning server, according to various embodiments. [Figure 20] 11 is a flowchart illustrating an example of an operation of a learning server to generate yet another artificial intelligence model, according to various embodiments. [Figure 21] 11 is a diagram illustrating an example of an operation in which a learning server uses rPPG and PPG to generate an artificial intelligence model according to various embodiments. [Figure 22] 11 is a diagram illustrating an example of an operation in which a learning server uses MPPG and PPG to generate an artificial intelligence model according to various embodiments. [Figure 23] 1 is a diagram illustrating an example of at least one artificial intelligence model generated by a learning server according to various embodiments. [Figure 24] 1 is a flowchart illustrating an example of an operation of using an artificial intelligence model of an electronic device to provide a biometric signal with similar accuracy to a contact-based method, according to various embodiments. [Diagram 25] 1 is a diagram for explaining an operation of providing a biosignal with similar accuracy to a contact type method by using an artificial intelligence model of an electronic device according to various embodiments. [Figure 26] 4 is a flowchart illustrating operations for guiding photography of an electronic device according to various embodiments. [Figure 27] 1 is a diagram illustrating an example of an operation of guiding photography of an electronic device according to various embodiments; [Figure 28] 1 is a flowchart illustrating an example operation of a server (e.g., a utilization server) to utilize an artificial intelligence model, according to various embodiments. [Figure 29] 11 is a diagram illustrating an example of an operation of a server (e.g., a utilization server) using an artificial intelligence model according to various embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The electronic devices according to the various embodiments disclosed herein may be devices of various forms. The electronic devices may include, for example, a portable communication device (e.g., a smartphone), a computing device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronic device. The electronic devices according to the embodiments disclosed herein are not limited to the aforementioned devices.

[0013] Various embodiments and terms used herein are not intended to limit the technical features described in this document to a specific embodiment, but should be understood to include various modifications, equivalents, or alternatives of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the item, unless the relevant context clearly dictates otherwise. In this document, each of the phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B, and C," and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," "first," or "second" may be used simply to distinguish the corresponding component from other corresponding components, and do not limit the corresponding component to other aspects (e.g., importance or order). When a (e.g., first) component is referred to as "coupled" or "connected" to another (e.g., second) component, with or without the terms "functionally" or "communicatively," it means that the component can be coupled to the other component directly (e.g., by wire), wirelessly, or through a third component.

[0014] The term "module" as used in various embodiments herein may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integrated component or the smallest unit or portion of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0015] Various embodiments of this document may be embodied as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., internal memory or external memory) readable by a machine (e.g., an electronic device). For example, a processor (e.g., a processor) of the machine (e.g., an electronic device) may retrieve and execute at least one instruction from the one or more instructions stored in the storage medium. This allows the machine to be operated to perform at least one function according to the retrieved at least one instruction. The one or more instructions may include code generated by a compiler or code that may be executed by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not include a signal (e.g., electromagnetic waves), and this term does not distinguish between data being stored semi-permanently and data being stored temporarily in the storage medium.

[0016] According to an embodiment, the methods according to the various embodiments disclosed herein may be provided in a computer program product. The computer program product may be traded between sellers and buyers as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)) or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store server, or an intermediary server.

[0017] According to various embodiments, each of the components described above (e.g., modules or programs) may include one or more entities, and some of the entities may be located separately in other components. According to various embodiments, one or more of the components described above may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the multiple components in a manner that is the same or similar to that performed by the corresponding component of the multiple components prior to the integration. According to various embodiments, operations performed by modules, programs, or other components may be performed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be performed in a different order or omitted, or one or more other operations may be added.

[0018] According to various embodiments, an electronic device may be provided that includes a first communication circuit and at least one first processor, the at least one first processor being configured to: acquire, through the communication circuit, a plurality of images including a user's face acquired using a camera of a first external electronic device; acquire, through the communication circuit, while simultaneously acquiring the plurality of images, first data acquired based on a first sensor of the first external electronic device contacted with a first part of the user's body and second data acquired based on a second external electronic device contacted with a second part of the user's body; and acquire a first biosignal of a particular type based on the plurality of images, a second biosignal of the particular type based on the first data, and a third biosignal based on the second data.

[0019] According to various embodiments, an electronic device may be provided in which the at least one processor is configured to: further acquire information related to photographing of the first external electronic device acquired by a second sensor of the first external electronic device while acquiring the plurality of images via the communication circuit.

[0020] According to various embodiments, an electronic device may be provided with photography information related to photography of the electronic device, the information including information related to a state related to photography of the first external electronic device and information related to the external environment of the first external electronic device.

[0021] According to various embodiments, an electronic device may be provided in which the at least one processor is configured to: obtain, via the communication circuitry, further personal information related to personal characteristics of the user of the first external electronic device.

[0022] According to various embodiments, an electronic device may be provided in which the at least one processor is configured to generate at least one artificial intelligence model by performing learning based on at least a portion of the first bio-signal, the second bio-signal, the third bio-signal, the imaging information, or the personal information, and the at least one artificial intelligence model is configured to provide a value for the particular type of bio-signal sensed in a contact manner.

[0023] According to various embodiments, an electronic device may be provided in which the at least one artificial intelligence model is configured to output the third biosignal based on receiving at least a portion of the first biosignal, the second biosignal, the imaging information, or the personal information.

[0024] According to various embodiments, an electronic device may be provided, wherein the at least one processor is configured to: perform time synchronization of the first biosignal, the second biosignal, and the third biosignal.

[0025] According to various embodiments, an electronic device may be provided in which the at least one processor is configured to: select, from among the first bio-signal, the second bio-signal, and the third bio-signal, the first bio-signal associated with the face that is closest to the heart as a reference; and synchronize the remaining second bio-signals and each of the second bio-signals to the first bio-signal based on the selected first bio-signal.

[0026] According to various embodiments, an electronic device may be provided in which the at least one processor is configured to: generate the at least one artificial intelligence model in a state in which time synchronization is not performed for the first biosignal, the second biosignal, and the third biosignal.

[0027] According to various embodiments, a method of operating an electronic device may be provided, the method including: acquiring, through the communication circuit, a plurality of images including a user's face acquired using a camera of a first external electronic device; acquiring, through the communication circuit, first data acquired based on a first sensor of the first external electronic device contacted with a first part of the user's body and second data acquired based on a second external electronic device contacted with a second part of the user's body, simultaneously while acquiring the plurality of images; and acquiring a first biosignal of a specific type based on the plurality of images, a second biosignal of the specific type based on the first data, and a third biosignal based on the second data.

[0028] According to various embodiments, a method of operation may be provided that further includes an operation of acquiring information related to photographing of the first external electronic device acquired by a second sensor of the first external electronic device while acquiring the plurality of images through the communication circuit.

[0029] According to various embodiments, an operating method may be provided in which the photographing information related to the photographing of the electronic device includes information related to a state related to photographing of the first external electronic device and information related to an external environment of the first external electronic device.

[0030] According to various embodiments, a method of operation may be provided that further includes the operation of: further acquiring personal information related to a personal characteristic of the user of the first external electronic device via the communication circuit.

[0031] According to various embodiments, an operating method may be provided that further includes an operation of generating at least one artificial intelligence model by performing learning based on at least a portion of the first bio-signal, the second bio-signal, the third bio-signal, the imaging information, or the personal information, wherein the at least one artificial intelligence model is embodied to provide a value for the particular type of bio-signal sensed in a contact manner.

[0032] According to various embodiments, an operating method may be provided in which the at least one artificial intelligence model is configured to output the third biosignal based on receiving at least a portion of the first biosignal, the second biosignal, the imaging information, or the personal information.

[0033] In the following, a biosignal measuring system 1 according to various embodiments will be described.

[0034] According to various embodiments, the biosignal measurement system 1 may be a system embodied to provide a biosignal obtained based on an analysis of a user and / or a specimen (e.g., a part of the user's body such as the face) in a non-contact manner. The biosignal may include photoplethysmography (PPG), oxygen saturation (SPO2), heart rate variability (HRV), electrocardiogram (ECG), electroencephalogram (EEG), electromyogram (EMG), galvanic skin response (GSR), and skin temperature (SKT), but may further include various types of biosignals without being limited to the examples described. In order to improve the accuracy of biosignals acquired in a non-contact manner, the biosignal measurement system 1 can simultaneously collect a plurality of specific types of biosignals in a non-contact manner and in a contact manner, which is more accurate than the non-contact manner, and can utilize an artificial intelligence (AI) model that learns based on the collected plurality of biosignals, and specific examples will be described below.

[0035] 1 is a diagram for explaining an example of the configuration of a biosignal measuring system 1 according to various embodiments. FIG. 1 will be further explained below with reference to FIG.

[0036] FIG. 2 is a diagram illustrating an example of an electronic device 10 according to various embodiments.

[0037] 1, the biosignal measuring system 1 may include an electronic device 10 and a server 20. However, the biosignal measuring system 1 may be embodied to include more devices without being limited to the illustrated and / or described examples.

[0038] According to various embodiments, the electronic device 10 may be an electronic device of a user who desires to measure a biosignal (e.g., remote photoplethysmography, rPPG) in a non-contact manner using the biosignal measurement system 1. For example, the electronic device 200 may include a user terminal such as a smartphone, a wearable device, a head mounted display (HMD) device, etc. as shown in Fig. 2(a), and a user device used in an installed and / or deployed form such as a kiosk, a smart mirror, etc. as shown in Fig. 2(b). The electronic device 10 may be embodied to detect a sample S in a non-contact manner and provide a biosignal based on the detection result. For example, as shown in FIG. 2, the electronic device 10 can acquire a plurality of images (or a video, or a single image) based on photographing a subject S (e.g., the face of a user U), receive a biosignal (e.g., PPG) corresponding to the plurality of images from the server 20, and provide the received biosignal in a form recognizable by the user (e.g., displaying it on a display and / or outputting it in the form of a sound through a speaker).

[0039] According to various embodiments, the sample S may be the face in the case of measuring PPG and the chest in the case of measuring respiratory rate, but is not limited to the described examples and various body parts of the user U may be measured in a non-contact manner.

[0040] According to various embodiments, the server 20 may acquire a biosignal based on a sample S detected in a contactless manner and provide information on the acquired biosignal to the electronic device 10. For example, the server 20 may include a learning server 20a and a utilization server 20b. However, without being limited to the illustrated and / or described examples, the server 20 may be embodied as a single server performing both the functions of the learning server 20a and the utilization server 20b. The learning server 20a may build at least one artificial intelligence model that is trained to provide a biosignal. For example, the learning server 20a may build an artificial intelligence model that is embodied to output a biosignal with an accuracy similar to that of a contact type in response to receiving at least one input of information different from a non-contact biosignal acquired based on a sample S detected in a contactless manner. The artificial intelligence model learned by the learning server 20a may be provided to the utilization server 20b. The utilization server 20b may establish a communication connection with the electronic device 10 and receive information on the sample S acquired by the electronic device 10 in a contactless manner from the electronic device 10. The utilization server 20b can input information about the sample S into the artificial intelligence model, acquire information about the biosignals output from the artificial intelligence model, and transmit the acquired information about the biosignals to the electronic device 10.

[0041] Meanwhile, without being limited to the described example, the electronic device 10 may be embodied in an on-device form so that the electronic device 10 can provide a biosignal without the operation of the server 20.

[0042] Exemplary configurations of electronic device 10 and server 20 according to various embodiments are described below.

[0043] FIG. 3 is a diagram for explaining an example of the configuration of an electronic device 10 and a server 20 (e.g., a learning server 20a and a utilization server 20b) according to various embodiments. In some embodiments, without being limited to the example shown in FIG. 3, at least one of the components may be omitted from the electronic device 20 and the server 20, or one or more other components may be added to the electronic device 20 and the server 20. In some embodiments, some of the components may be embodied in a single integrated circuit. FIG. 3 is further described below with reference to FIG. 4.

[0044] FIG. 4 is a diagram illustrating an example of a biosignal measurement module according to various embodiments.

[0045] Exemplary configurations of electronic device 10, according to various embodiments, are described below.

[0046] According to various embodiments, the electronic device 10 may include a display 11, a camera 12, a first communication circuit 13, a sensor 14, a first memory 17, and a first processor 18. However, without being limited to the illustrated and / or described examples, the electronic device 10 may be embodied to further include various electronic components (e.g., a speaker) and devices provided in a user terminal, and / or to include fewer components. Examples of each configuration are described below.

[0047] According to various embodiments, the display 11 may visually provide information to an outside (e.g., a user) of the electronic device 200. The display 11 may include, for example, a display, a holographic device, or a projector and control circuitry for controlling the corresponding device. According to one embodiment, the display 11 may include touch circuitry configured to sense a touch, or a sensor circuit (e.g., a pressure sensor) configured to measure the strength of a force generated by the touch.

[0048] According to various embodiments, the camera 12 can include an image sensor for capturing images.

[0049] According to various embodiments, the first communication circuit 13 can support the establishment of a wireless communication channel between the electronic device 10 and an external electronic device (e.g., server 20) and the performance of communication through the established communication channel. The first communication circuit 13 operates independently of the first processor 18 and can include one or more communication processors that support wireless communication.

[0050] According to various embodiments, the sensor 14 may include a measurement sensor 15 for acquiring (or sensing) a biosignal in a contact manner, and an environmental sensor 16 for acquiring (or sensing) various types of information related to imaging (imaging information).

[0051] For example, the measurement sensor 15 may include a PPG sensor, an SPO2 sensor, an HRV sensor, an ECG sensor, an EEG sensor, an EMG sensor, a GSR sensor, and / or an SKT sensor, and may further include various types of sensors without being limited to the described examples. As an example, the PPG sensor may be a sensor that emits light in contact with the skin and is embodied to measure a PPG signal based on a change in the amount of light sensitivity of the received light.

[0052] For example, the photographing information acquired by the environmental sensor 16 may include a first environmental sensor (e.g., illuminance sensor 16A, etc.) for measuring information related to the surrounding environment (e.g., light amount, illuminance, temperature, etc.) in which the photograph is taken, and a second environmental sensor (e.g., tilt sensor 16b, motion sensor (not shown), etc.) for measuring information related to the state of the electronic device 10 at the time of photographing (e.g., tilt, movement, position, height, direction). The data acquired by the environmental sensor 16 may be defined as data. At least a part of the information acquired by the environmental sensor 16 may be acquired by an analysis module (not shown) for analyzing the image captured by the camera 12, instead of the environmental sensor 16. For example, the analysis module (not shown) may identify the light amount, illuminance, etc. based on the pixel value (e.g., brightness value) of the image.

[0053] According to various embodiments, the first memory 17 may store various data used by at least one component (e.g., the first processor 18) of the electronic device 210. For example, the first memory 17 may store a predetermined application. Based on the execution of the application, the operation of the electronic device 10, which will be described later, may be performed.

[0054] According to various embodiments, the electronic device 10 may be embodied such that the application acquires additional information. For example, the additional information may include personal information such as the user's gender, year, age, race, BMI index, etc., camera information regarding the parameters of the camera 12 (e.g., focal length, etc.), shooting information indicating the shooting state such as the distance from the subject, shooting video resolution, frames per second (FPS), etc., and image information indicating a characteristic that can be analyzed from the image (or video) (e.g., the direction of light shining on the subject (e.g., front light, back light)). In this case, a part of the additional information (e.g., BMI index) may be acquired based on an artificial intelligence model for calculating the additional information.

[0055] According to various embodiments, the first processor 18 may, for example, execute software to control at least one other component (e.g., hardware or software component) of the electronic device 200 coupled to the first processor 520, and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the first processor 520 may load instructions or data received from another component (e.g., the second communication circuit 540 or the third communication circuit 550) into a volatile memory, process the instructions or data stored in the volatile memory, and store the result data in a non-volatile memory. According to one embodiment, the first processor 520 may include a main processor (e.g., a central processing unit or application processor) and an auxiliary processor (e.g., a graphics processing unit, an image signal processor, a sensor hub processor, or a communication processor) that may operate independently or in conjunction with the main processor. Additionally or generally, the auxiliary processor may be configured to use less power than the main processor or to be dedicated to a designated function. The auxiliary processor may be embodied separately from the main processor or as a part of it.

[0056] Exemplary configurations of server 20 according to various embodiments are described below.

[0057] According to various embodiments, the learning server 20a may include a second communication circuit 21a, a second processor 22a, and a memory 23a. The second communication circuit 21a may be implemented with the first communication circuit 13, the second processor 22a may be implemented with the first processor 18, and the second memory 23a may be implemented with the first memory 17, and therefore, a duplicated description will be omitted. Meanwhile, without being limited to the illustrated and / or described examples, the learning server 20a may be implemented to include more components and / or fewer components. Examples of each configuration will be described below.

[0058] According to various embodiments, the memory 23a may include a database 24a, an artificial intelligence model generation module 25a, and a first biosignal measurement module 26a. The modules 25a, 26a may be implemented in the form of computer readable computer code, programs, software, applications, APIs, and / or instructions, and the second processor 22a of the learning server 20a may be induced to perform a specific operation based on the execution of the modules 25a, 26a.

[0059] According to various embodiments, the database 24a may be implemented to store various types of information for generating an artificial intelligence model. For example, the database 24a may store biosignals measured in a non-contact manner and corresponding biosignals measured in a contact manner acquired by a first biosignal measuring module 26a described below, environmental data measured by the environmental sensor 16, and additional information. The additional information may include personal information, camera information, photographing information, and / or image information, as described above.

[0060] According to various embodiments, the artificial intelligence model generation module 25a may learn an artificial intelligence model capable of providing a biosignal with accuracy similar (or close) to that of a biosignal (e.g., PPG) measured in a contact-type manner based on a biosignal (e.g., rPPG) measured in a non-contact-type manner. An example of the learning of the artificial intelligence model will be described in detail later.

[0061] According to various embodiments, the first biosignal measuring module 26a may be embodied to measure a non-contact biosignal and a contact biosignal based on information received from the electronic device 10. For example, as shown in FIG. 4, the first biosignal measuring module 26a may measure information collected in a non-contact manner (e.g., a plurality of images including the subject S captured by the camera 12).

[0062] ), and a contact acquisition module 423 for acquiring a biosignal (e.g., a contact biosignal) based on information collected in a contact manner (e.g., sensing information collected from the sensor 14).

[0063] Meanwhile, without being limited to the described examples, at least some of the components (e.g., database 24a, artificial intelligence model generation module 25a, and first biosignal measurement module 26a) may be embodied in the electronic device 10. For example, by embodiing the first biosignal measurement module 26a in the electronic device 10, the electronic device 10 can obtain non-contact biosignals and contact biosignals and transmit the plurality of signals to the learning server 20a so that the learning server 20a can learn the artificial intelligence model.

[0064] According to various embodiments, the utilization server 20b may include a third communication circuit 21b, a third processor 22b, and a memory 24b. The third communication circuit 21b may be implemented with the first communication circuit 13, the third processor 22b may be implemented with the first processor 18, and the third memory 24b may be implemented with the first memory 17, and therefore, a duplicated description will be omitted. Meanwhile, without being limited to the illustrated and / or described examples, the utilization server 20b may be implemented to include more components and / or fewer components. Examples of each configuration will be described below.

[0065] According to various embodiments, the third memory 24b may be embodied to store at least one artificial intelligence model 23b, 23c learned by the learning server 20a (e.g., the artificial intelligence model generation module 25a) and a second biosignal measurement module 24b identical to the first biosignal measurement module 26a described above. The utilization server 20b may acquire a specific type of biosignal based on a plurality of images of the sample S detected in a non-contact manner received from the electronic device 10, environmental data (e.g., information sensed by the environmental sensor 14), additional information (e.g., at least a part of personal information, camera information, photographing information, or image information), and the at least one artificial intelligence model 23b, 23c, and transmit the acquired specific type of biosignal to the electronic device 10.

[0066] Meanwhile, without being limited to the described examples, at least a part of the above configurations (e.g., at least one of the artificial intelligence models 23b, 23c and the above-mentioned second biosignal measuring module 24b) may be embodied in the electronic device 10. For example, by embodiing the second biosignal measuring module 24b in the electronic device 10, the electronic device 10 can obtain non-contact biosignals and contact biosignals and transmit the plurality of signals to the utilization server 20b.

[0067] On the other hand, without being limited to the described example, the utilization server 20b may not be embodied, and at least one learned artificial intelligence model 23b, 23c may be used in a form stored in a single server that performs both the functions of the learning server 20a and the utilization server 20b, and / or in the electronic device 10.

[0068] The following describes examples of operations for acquiring (or collecting) data for artificial intelligence model training by the learning server 20a according to various embodiments.

[0069] 5 is a flow chart illustrating an example of operations for acquiring (or collecting) data for artificial intelligence model training of the learning server 20a according to various embodiments. Operations may be performed out of the order of operations shown and / or described, and more operations and / or fewer operations may be performed. FIG. 5 is further described below with reference to FIGS. 6-7.

[0070] 6 is a diagram illustrating an example of an operation of simultaneously collecting biosignals in a non-contact manner and a contact manner of the electronic device 10 according to various embodiments. FIG 7 is a diagram illustrating an example of an operation of storing data for learning an artificial intelligence model in the learning server 20a according to various embodiments.

[0071] According to various embodiments, the learning server 20a (e.g., the second processor 22a) may acquire a first biosignal of a specific type based on photographing the subject S using the camera 12 of the electronic device 10 in operation 501, acquire a second biosignal based on the first contact sensor 15, and acquire a third biosignal of the specific type based on the second contact sensor in operation 503. For example, referring to FIG. 6, the learning server 20a may acquire all (or simultaneously, or during a specific period) of the first biosignal, the second biosignal, and the third biosignal of the specific type that are related to each other using the user U, the electronic device 10 held by the user U, and an external measurement sensor 600 that contacts a part of the body of the user U. The biosignals being related to each other may mean that they are acquired during a specific period in which the correlation between them is higher than a threshold value. For example, the specific type of biosignal may be PPG. 6, the user U can photograph the first subject (e.g., face S1) of the user using the camera 10 of the electronic device 12 while the biosignal measuring device 600 is equipped (or worn) on the third subject (e.g., finger S3 of the first hand) of the user U, and can bring the second subject (e.g., finger S2 of the second hand) into contact with the contact sensor S2 disposed in a specific area (e.g., rear surface) of the electronic device 12. Accordingly, referring to FIG. 7, the learning server 20a can receive a plurality of images of the first subject S1 photographed by the camera 12 of the electronic device 10, and sensing data (e.g., mobile PPG, MPPG) measured by the measurement sensor 15, and can receive sensing data (e.g., PPG) measured by the external measurement sensor 600 from the external measurement sensor 600.

[0072] For example, the non-contact measurement module 421 of the learning server 20a may acquire a specific type of non-contact biosignal based on analyzing a plurality of images of a first sample S1. For example, the specific type of non-contact biosignal may be rPPG. The plurality of images may be images acquired based on at least one parameter of the camera being set to a value within a specific range. For example, the images may be images acquired in a state where the camera parameters are set so that a video is shot with the FPS in the range of 20 to 30 per second.

[0073] For example, the first contact measurement module 423a of the learning server 20a may acquire the specific type of first contact biosignal measured by the electronic device 10 itself based on sensing data received from the measurement sensor 15. The specific type of first contact biosignal measured by the electronic device 10 itself is a PPG, which may be defined as an MPPG (mobile PPG). The first contact measurement module 423a may acquire an MPPG that has already been measured by the electronic device 10 and received from the electronic device 10 to the learning server 20a, but is not limited to the described example and may be embodied to acquire sensing data received from the measurement sensor 15 and measure an MPPG based on analyzing the acquired sensing data.

[0074] For example, the second contact measurement module 423b of the learning server 20a may acquire the specific type of second contact biosignal based on sensing data received from the external measurement device 600. The second specific type of contact biosignal may be a PPG, which may have a relatively high accuracy compared to the accuracy of the above-mentioned MPPG (mobile PPG). The second contact measurement module 423b may acquire a PPG that has already been measured by the external measurement device 600 and received from the external measurement device 600 to the learning server 20a, but may be embodied to acquire sensing data received from the external measurement device 600 and measure a PPG based on analyzing the acquired sensing data without being limited to the above-mentioned example.

[0075] According to various embodiments, the learning server 20a (e.g., the data acquisition module 700) may acquire additional learning information in operation 505. The additional learning information may include additional information including at least a portion of environmental data acquired based on the environmental sensor 16 and personal information, camera information, photographing information, or image information acquired based on an application (not shown). For example, the electronic device 10 may acquire information related to the surrounding environment where the image is captured and information related to the state of the electronic device 10 at the time of capturing the image using the environmental sensor 16, and transmit the information to the learning server 20a. Also, for example, the electronic device 10 may transmit personal information inputted through an execution screen based on the execution of an application to the learning server 20a. Also, for example, the electronic device 10 may transmit at least a portion of camera information acquired based on a set authority based on the execution of an application, and photographing information and image information acquired based on analyzing the video and / or image captured by the camera 12 to the learning server 20a.

[0076] According to various embodiments, the data acquisition module 700 may store the specific types of biosignals (e.g., non-contact biosignals, contact type first biosignals, and contact type second biosignals) and additional learning information in a mutually associated form in the database 24a. At this time, the data acquisition module 700 may be embodied to perform time synchronization of the specific types of biosignals (e.g., non-contact biosignals, contact type first biosignals, and contact type second biosignals), but is not limited to the described examples and time synchronization may not be performed.

[0077] The following describes an example of an operation of acquiring a non-contact bio-signal as at least a part of the operation 501 of the learning server 20a according to various embodiments.

[0078] 8 is a flow chart illustrating an example of an operation of the learning server 20a to acquire a non-contact biosignal according to various embodiments. The operations may be performed out of the order of the operations shown and / or described, and more operations and / or fewer operations may be performed. FIG. 8 is further described below with reference to FIGS. 9 to 11.

[0079] Fig. 9 is a diagram for explaining an example of an operation of acquiring a difference value between color channels (e.g., G value and R value, G value and B value) for noise reduction according to various embodiments. Fig. 10 is a diagram for explaining an example of an operation of acquiring a characteristic value according to various embodiments. Fig. 11 is a diagram for explaining an example of an operation of acquiring a characteristic value according to various embodiments.

[0080] According to various embodiments, the learning server 20a (e.g., the non-contact measurement module 421) may acquire a plurality of images including a specimen in operation 801, and acquire a first biosignal based on the plurality of images in operation 803. For example, the learning server 20a (e.g., the non-contact measurement module 421) may acquire color channel values ​​from the plurality of images including a specimen, and acquire rPPG based on the color channel values. For example, the color channels may refer to the R channel, G channel, and B channel of the RGB color space, respectively, but are not limited to the described and / or illustrated examples, and may refer to channels of other color spaces (e.g., CMY, HSV, etc.).

[0081] At this time, according to various embodiments, referring to FIG. 9, an operation of obtaining rPPG may be performed based on the difference value between color channels (e.g., G values ​​and R values, G values ​​and B values) for noise reduction.

[0082] FIG. 9(a) is a graph showing red channel values ​​extracted in the RGB color space, and FIG. 9(b) is a graph showing green channel values ​​extracted in the RGB color space. Referring to FIG. 9(a) and (b), it can be seen that the extracted color channel values ​​fluctuate over time. At this time, the extracted color channel values ​​may fluctuate due to heartbeats, but may also fluctuate due to the subject's movement or changes in external light intensity. More specifically, the color channel values ​​fluctuate more significantly and slowly due to the subject's movement or changes in external light intensity, while the color channel values ​​fluctuate more significantly and quickly due to the subject's heartbeat. Therefore, since the value fluctuation due to the subject's movement or changes in external light intensity is greater than the fluctuation due to heartbeats, a relative difference between at least two color channel values ​​can be used to reduce this.

[0083] For example, a difference between a green channel value and a red channel value may be used to reduce noise. More specifically, a green channel value and a red channel value acquired in the same image frame may reflect the same movement and the same external light intensity, and a difference between a green channel value and a red channel value in the same frame may reduce noise caused by the movement of a subject and changes in the external light intensity, but is not limited thereto, and noise may be reduced by using a relative difference between at least two color channel values.

[0084] 9(c) is a graph showing the difference between the green channel value and the red channel value. As shown in FIG. 9(c), the difference between the green channel value and the red channel value can reduce noise caused by the subject's movement and changes in the intensity of external light.

[0085] Also, the above-described method of reducing noise may be performed on at least one of the captured image frames, or may be performed on each of a plurality of consecutive image frames.

[0086] Also, although not shown in (c) of FIG. 9, noise can be reduced by using the difference between the Green channel value and the Blue channel value, and noise can also be reduced by using the difference between the Red channel value and the Blue channel value.

[0087] Also, as previously described, at least two color channel values ​​may be selected for determining a difference value to utilize the relative difference between the at least two color channel values ​​to reduce noise.

[0088] At this time, the at least two color channel values ​​may be selected taking into consideration the absorbance of blood.

[0089] According to various embodiments, referring to Figures 10 to 11, the learning server 20a can acquire time-series data (e.g., a first characteristic value and a second characteristic value) for each of the difference value between the Green channel value and the Red channel value and the difference value between the Green channel value and the Blue channel value, acquire a third characteristic value by merging the acquired time-series data, and acquire an rPPG based on the third characteristic value.

[0090] 10(a) is a graph showing color channel values ​​obtained according to an embodiment, more specifically, a graph showing the difference between the Green channel value and the Red channel value. However, this is merely a graph showing the difference between the Green channel value and the Red channel value for convenience of explanation, and is not limited thereto, and may be various color channel values, difference values, processed values, etc.

[0091] Referring to FIG. 10(a), it can be seen that the difference between the Green channel value and the Red channel value (hereinafter, referred to as the "GR value") may not change at a constant rate over time.

[0092] In this case, the GR value may not be constant depending on the subject's movements. For example, when the subject moves little, the change in the GR value may be small, and when the subject moves a lot, the change in the GR value may be large, but is not limited to this.

[0093] In addition, the GR value may not be constant depending on the intensity of external light. For example, when the intensity of external light is weak, the change in the GR value may be small, and when the intensity of external light is strong, the change in the GR value may be large, but is not limited to this.

[0094] Therefore, characteristic values ​​can be extracted in this way to reduce noise caused by the subject's movements, the intensity of external light, and the like.

[0095] Also, a window for the characteristic values ​​may be set for extracting the characteristic values.

[0096] In this case, the window for the characteristic value may mean a predetermined time period or may mean a predetermined number of frames, but is not limited thereto and may mean a window for setting at least a portion of a frame group among a plurality of frames to obtain the characteristic value.

[0097] 10(b) is a schematic diagram for explaining windows for characteristic values, more specifically, a schematic diagram for explaining windows for characteristic values ​​set in 18 image frames obtained by dividing 180 image frames into 10. However, this is merely a diagram for explaining windows for characteristic values ​​set in 18 image frames obtained by dividing 180 image frames into 10 for the sake of convenience of explanation, and windows for characteristic values ​​may be set in various ways and in various numbers without being limited thereto.

[0098] Referring to Fig. 10(b), the acquired image frames may be grouped according to a window for the characteristic value. For example, as shown in Fig. 19(b), 180 image frames may be grouped according to a window for the characteristic value, each group including 18 image frames. More specifically, the first image frame to the 18th image frame may be included in the first image frame group 2210, and the 19th image frame to the 36th image frame may be included in the second image frame group 2220, but is not limited thereto.

[0099] In this case, the characteristic values ​​may be obtained for the image frame groups set by a window for the characteristic values. For example, the characteristic values ​​may be obtained for color channel values ​​for the first image frame group 2210 and for color channel values ​​for the second image frame group 2220.

[0100] Also, for example, if the characteristic value is an average value, the average value of the color channel values ​​for the image frame group may be obtained. More specifically, the average value of the GR values ​​for the 1st to 18th image frames included in the first image frame group 2210 may be obtained, and the average value of the GR values ​​for the 19th to 36th image frames included in the second image frame group 2220 may be obtained, but is not limited thereto.

[0101] Also, for example, if the characteristic value is a standard deviation value, a standard deviation value of color channel values ​​for the image frame group may be obtained. More specifically, a standard deviation value of GR values ​​for the 1st to 18th image frames included in the first image frame group 2210 may be obtained, and a standard deviation value of GR values ​​for the 19th to 36th image frames included in the second image frame group 2220 may be obtained, but is not limited thereto.

[0102] However, the above examples are not limiting and various characteristic values ​​may be obtained for the image frame group.

[0103] Also, the characteristic values ​​may be obtained for at least some of the image frames included in the image frame groups divided by the window for the characteristic values. For example, the characteristic values ​​may be obtained for color channel values ​​for at least some of the 18 image frames included in the first image frame group 2210, and for at least some of the 18 image frames included in the second image frame group 2220.

[0104] Also, for example, when the characteristic value is a deviation value, a deviation value of color channel values ​​for at least some image frames included in the image frame group may be obtained. More specifically, a deviation value of a GR value of a first image frame included in the first image frame group with respect to an average GR value of the first image frame group 2210 may be obtained, and a deviation value of a GR value of a 19th image frame included in the second image frame group with respect to an average GR value of the second image frame group 2220 may be obtained, but is not limited thereto.

[0105] Also, for example, when the characteristic value is a deviation value, a deviation value of color channel values ​​for at least some image frames included in the image frame group may be obtained. More specifically, a deviation value of GR values ​​of a first image frame included in the first image frame group with respect to a GR value average of the first image frame group 2210 may be obtained, and a deviation value of GR values ​​of a second image frame included in the first image frame group 2210 may be obtained, but is not limited thereto.

[0106] Additionally, the obtained characteristic values ​​may be normalized.

[0107] For example, if the characteristic value is a deviation value, the deviation value may be normalized by a standard deviation value. More specifically, if a deviation value of a GR value of a first image frame included in the first image frame group 2210 with respect to an average GR value of the first image frame group 2210 is obtained, the deviation value of the GR value of the first image frame may be normalized by the standard deviation value of the first image frame group 2210, but is not limited thereto and may be normalized in various ways.

[0108] In addition, when normalized in this manner, the magnitude of the change is normalized, so that the change in value due to heartbeat can be better reflected, and noise due to the movement of the subject and noise due to changes in the intensity of external light, etc. can be effectively reduced.

[0109] 11(a) is a graph showing two characteristic values ​​obtained according to an embodiment, more specifically, a first characteristic value obtained based on a GR value and a second characteristic value obtained based on a GB value, but this is merely a specific example shown for convenience of explanation, and the characteristic values ​​may be obtained based on various color channel values, difference values, and processing values.

[0110] At this time, the first characteristic value obtained based on the GR value may be affected by the GR value. For example, when the external light is close to the blue channel, the GR value may not be able to properly reflect the change in blood flow caused by heartbeat.

[0111] Or, for example, a change in blood due to heart beating can be reflected by being influenced by the difference between the absorbance of the green channel and the absorbance of the red channel.

[0112] In addition, the second characteristic value obtained based on the GB value may be affected by the GB value. For example, when the external light is close to the red channel, the GB value may not be able to properly reflect the change in blood flow caused by heartbeat.

[0113] Or, for example, a change in blood due to heartbeat can be reflected by being influenced by the difference between the absorbance of the green channel and the absorbance of the blue channel.

[0114] 11(a), the first characteristic value and the second characteristic value may have a complementary relationship with each other. For example, the second characteristic value may well reflect changes due to heartbeats in a section where the first characteristic value does not well reflect changes due to heartbeats, and vice versa.

[0115] Therefore, the first and second characteristic values ​​can be used to reduce noise caused by changes in wavelength of external light and to better reflect changes in blood due to heartbeat.

[0116] 11(b) is a graph showing a third characteristic value obtained by using the first characteristic value and the second characteristic value, more specifically, a graph showing a third characteristic value obtained by combining the first characteristic value and the second characteristic value, although this is merely specifically shown for convenience of explanation and is not limited thereto.

[0117] Furthermore, the third characteristic value may be obtained based on an operation of the first characteristic value and the second characteristic value. For example, the third characteristic value may be obtained based on an addition operation of the first characteristic value and the second characteristic value, but is not limited thereto, and may be obtained based on various operations such as a difference operation, a product operation, etc.

[0118] Also, the third characteristic value may be obtained by applying various weights to the first characteristic value and the second characteristic value, for example, but not limited to, based on the following Equation 1:

[0119] (Number 1) 3rd characteristic value = a·1st characteristic value + b·2nd characteristic value

[0120] Also, referring to (a) and (b) of Figures 11, the third characteristic value can better reflect changes in blood due to heartbeat than the first and second characteristic values, and can reduce noise caused by changes in wavelength of external light.

[0121] The following describes an example of an operation of performing time synchronization on multiple biosignals as at least a part of the operation 505 of the learning server 20a according to various embodiments.

[0122] FIG. 12 is a flow chart illustrating an example of an operation of the learning server 20a to acquire a non-contact biosignal according to various embodiments. The operations may be performed out of the order of the operations shown and / or described, and more operations and / or fewer operations may be performed. FIG. 12 is further described below with reference to FIG. 13-FIG. 14.

[0123] 13 is a diagram illustrating an example of a biological signal according to various embodiments. FIG 14 is a diagram illustrating an example of a time synchronization operation of a biological signal of the learning server 20a according to various embodiments.

[0124] According to various embodiments, the learning server 20a (e.g., the data acquisition module 700) may perform time synchronization between a plurality of biosignals in operation 1201, and store the time-synchronized plurality of biosignals in operation 1203. For example, referring to FIG. 13, the plurality of biosignals 1301, 1303 may have different magnitudes (or intensities) (or different patterns) with respect to time based on different positions of the specimens S1, S2 at which the plurality of biosignals 1301, 1301 are measured. For example, the second biosignal 1303 acquired by the measurement sensor 15 based on contact of the specimen S2 (e.g., the finger of the first hand) relatively far from the heart may be delayed by a specific time td from the first biosignal 1301 acquired based on photographing the specimen S1 (e.g., the face) close to the heart of the user U. Accordingly, the learning server 20a may time-synchronize the plurality of biosignals 1301, 1303 based on the specific time td for more sophisticated learning. 14, the learning server 20a may delete (or remove) the signal 1303a corresponding to a specific time td of the second biosignal 1303, or may shift the time of the second biosignal 1303 forward by a specific time td. Also, for example, the learning server 20a may perform an operation of shifting the first biosignal 1303 backward by a specific time td (e.g., time shifting). Meanwhile, although not shown, time synchronization with respect to a third biosignal measured by an external measurement sensor 600 may also be performed.

[0125] According to various embodiments, the learning server 20a may select a reference biosignal from among the plurality of biosignals 1301, 1303, and time-synchronize the remaining biosignals based on the specific time td identified based on the selected reference biosignal. For example, the learning server 20a may select a first biosignal 1301 associated with a specimen S1 closest to the heart from among the plurality of biosignals 1301, 1303 as a reference signal. The learning server 20a may identify the specific time td based on a distance difference between a specimen S2 associated with the remaining second biosignal 1303 (and / or a biosignal measured by the external measurement sensor 600) and a specimen S2 corresponding to the first reference signal 1301, and perform the above-mentioned time synchronization operation. Meanwhile, although not shown, time synchronization for a third biosignal measured by the external measurement sensor 600 may also be performed in this manner.

[0126] According to various embodiments, the learning server 20a may determine the specific time td based on the user's personal characteristic information. The learning server 20a may store information on a plurality of delay times and identify the specific time td corresponding to the user's personal characteristic information from among the plurality of delay times. For example, the taller the user is, the longer the specific time td may be selected.

[0127] According to various embodiments, the learning server 20a may implement an artificial intelligence model for the time synchronization and perform time synchronization based on the implemented artificial intelligence model.

[0128] Hereinafter, an example of an operation of generating an AI model that is embodied to provide information on a specific type of biosignal having an accuracy corresponding to a contact type based on a specific type of biosignal acquired in a non-contact type by the learning server 20a according to various embodiments will be described. Hereinafter, an example in which the specific type of biosignal is PPG will be described, but the present invention is not limited to the described example, and AI models for measuring various types of biosignals may be embodied.

[0129] 15 is a flowchart showing an example of an operation of generating an artificial intelligence model implemented to provide information on a specific type of biosignal with an accuracy corresponding to a contact type based on a specific type of biosignal acquired in a non-contact type manner by the learning server 20a according to various embodiments. The operations may be performed out of the order of the operations shown and / or described, and more and / or fewer operations may be performed. FIG. 15 will be further described below with reference to FIGS. 16 to 18b.

[0130] FIG 16 is a diagram illustrating an example of an operation of generating an artificial intelligence model of the learning server 20a according to various embodiments. FIG 17 is a diagram illustrating an example of at least one artificial intelligence model generated by the learning server 20a according to various embodiments. FIG 18a is a diagram illustrating another example of at least one artificial intelligence model generated by the learning server 20a according to various embodiments. FIG 18b is a diagram illustrating yet another example of at least one artificial intelligence model generated by the learning server 20a according to various embodiments.

[0131] According to various embodiments, the learning server 20a may acquire a first biosignal of a specific type based on photographing the subject S using the camera 12 of the electronic device 10 in operation 1501, acquire a second biosignal based on the first contact sensor 15, acquire a third biosignal of the specific type based on the second contact sensor in operation 1503, and acquire the third biosignal of the specific type based on the second contact sensor in operation 1505. Operations 1501 to 1505 of the learning server 20a may be implemented in the same manner as operations 501 to 505 of the learning server 20a described above, and therefore repeated description will be omitted.

[0132] According to various embodiments, the learning server 20a may acquire at least one artificial intelligence model for acquiring the specific type of biosignal based on the plurality of biosignals and the additional learning information in operation 1507. For example, referring to FIG. 16, the artificial intelligence model generation module 25a may generate at least one artificial intelligence model 1700a-1700f and 1800 based on the plurality of biosignals (MPPG 1720, rPPG 1740, PPG 1750) and the additional learning information 1730 (e.g., additional information 1731 and environmental data 1733) stored in the database 24a by the data acquisition module 700. At this time, the PPG data stored in the database 24a may be time-synchronized with each other, but may not be time-synchronized without being limited to the example described above. The at least one artificial intelligence model 1700a to 1700f and 1800 may be implemented to provide a PPG estimation value with an accuracy corresponding to a PPG acquired in a contact manner based on information on a sample acquired in a non-contact manner (e.g., a plurality of images including a face). For example, the artificial intelligence model generation module 25a may implement an artificial intelligence model through supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, etc. Also, the artificial intelligence model generation module 25a according to an embodiment may implement an artificial intelligence model using an artificial neural network (ANN). For example, the artificial intelligence model generation module 25a may use, but is not limited to, a feedforward neural network, a radial basis function network, or a Kohonen self-organizing network, etc. Also, the artificial intelligence model generation module 25a according to an embodiment may implement an artificial intelligence model using a deep neural network (DNN).For example, the artificial intelligence model generation module 25a may use, but is not limited to, a convolutional neural network (CNN), a recurrent neural network (RNN), a Long Short Term Memory Network (LSTM), or Gated Recurrent Units (GRUs).

[0133] According to various embodiments, the learning server 20a may be embodied to learn models 1700a, 1700c, and 1700e for acquiring a non-contact PPG (rPPG) and models 1700b, 1700d, and 1700f for acquiring a contact PPG. Accordingly, a PPG 1750 may be acquired by inputting image information 1710 (e.g., a plurality of images) to the rPPG acquisition models 1700a, 1700c, and 1700e, and outputting the rPPG 1740, which is input to a second PPG acquisition model 1700b, 1700d, and 1700f.

[0134] In one embodiment, referring to FIG. 17(a), the learning server 20a may be embodied to learn an artificial intelligence model for acquiring a non-contact PPG (rPPG) (e.g., a first rPPG acquisition model 1700a) and an artificial intelligence model for acquiring a contact PPG (e.g., a first PPG acquisition model 1700b). For example, the learning server 20a may acquire the first rPPG acquisition model 1700a by learning the image information 1710 captured by the camera 12 and the additional learning information 1730 (e.g., additional information 1731 and environmental data 1733) among the mutually related information stored in the database 24a as input data and the rPPG 1740 as output data. The first rPPG acquisition model 1700a may be embodied to output the rPPG 1740 when the image information 1710 and the additional learning information 1730 (e.g., additional information 1731 and environmental data 1733) are input. Also, for example, the learning server 20a may acquire a first PPG acquisition model 1700b by learning using the rPPG 1740 and MPPG 1720 as input data and the PPG 1750 as output data from among the information stored in the database 24a. The second PPG acquisition model 1700b may be embodied to output a PPG 1750 when the rPPG 1740 and the MPPG 1720 are input. The rPPG 1740 output by inputting the image information 1710 and the additional learning information 1730 (e.g., additional information 1731, environmental data 1733) to the first rPPG acquisition model 1700a, and the MPPG 1720 acquired by the electronic device 10 to the second PPG acquisition model 1700b may be acquired as an output PPG 1750.

[0135] Also, in one embodiment, referring to FIG. 17(b), the learning server 20a may be embodied to learn a model for acquiring a non-contact PPG (rPPG) (e.g., a second rPPG acquisition model 1700c) and an artificial intelligence model for acquiring a contact PPG (e.g., a second PPG acquisition model 1700d). For example, the learning server 20a may acquire the second rPPG acquisition model 1700c by learning the image information 1710 captured by the camera 12 as input data and the rPPG 1740 as output data among the mutually related information stored in the database 24a. The second rPPG acquisition model 1700c may be embodied to output the rPPG 1740 when the image information 1710 is input. Meanwhile, without being limited to the described example, the second rPPG acquisition model 1700c may be software and / or an algorithm for acquiring a third characteristic value based on the GR and GB described above in FIG. 9. Also, for example, the learning server 20a may acquire a second PPG acquisition model 1700d by learning the rPPG 1740, the additional learning information 1730, and the MPPG 1720 from among the information stored in the database 24a as input data and the PPG 1750 as output data. The second PPG acquisition model 1700d may be embodied to output the PPG 1750 when the rPPG 1740, the additional learning information 1730, and the MPPG 1720 are input. The rPPG 1740 output by inputting the image information 1710 to the second rPPG acquisition model 1700c, and the MPPG 1720 acquired by the electronic device 10 and the additional learning information 1730 (e.g., additional information 1731, environmental data 1733) may be input to the second PPG acquisition model 1700d to acquire the PPG 1750 output.

[0136] Also, in one embodiment, referring to FIG. 17(c), the learning server 20a may be embodied to learn an artificial intelligence model for acquiring a non-contact PPG (rPPG) (e.g., a third rPPG acquisition model 1700e) and an artificial intelligence model for acquiring a contact PPG (e.g., a third PPG acquisition model 1700f). For example, the learning server 20a may acquire the third rPPG acquisition model 1700e by learning the image information 1710 captured by the camera 12, the MPPG 1720, and the additional learning information 1730 among the mutually related information stored in the database 24a as input data and the rPPG 1740 as output data. The third rPPG acquisition model 1700e may be embodied to output the rPPG 1740 when the image information 1710, the MPPG 1720, and the additional learning information 1730 (e.g., additional information 1731, environmental data 1733) are input. Also, for example, the learning server 20a may acquire a third PPG acquisition model 1700d by learning with the rPPG 1740 as input data and the PPG 1750 as output data from among the information stored in the database 24a. The third PPG acquisition model 1700f may be embodied to output a PPG 1750 when the rPPG 1740 is input. The rPPG 1740 output by inputting the image information 1710, the MPPG 1720, and the additional learning information 1730 (e.g., additional information 1731, environmental data 1733) to the third rPPG acquisition model 1700e may be input to the third PPG acquisition model 1700f to acquire a PPG 1750.

[0137] According to various embodiments, the learning server 20a may learn a single integrated artificial intelligence model (e.g., first integrated artificial intelligence model 1800) for acquiring a PPG. For example, referring to FIG. 18a, the learning server 20a may acquire the first integrated artificial intelligence model 1800 by learning the image information 1710, the MPPG 1720, and the additional learning information 1730 as input data and the PPG 1740 as output data. The first-1 integrated artificial intelligence model 1800a may be embodied to output a PPG 1750 when the image information 1710, the MPPG 1720, and the additional learning information 1730 (e.g., additional information 1731, environmental data 1733) are input. Meanwhile, without being limited to the described examples, the 1-1 integrated artificial intelligence model 1800a may be embodied to output rPPG 1720 when image information 1710, MPPG 1720, and additional learning information 1730 (e.g., additional information 1731, environmental data 1733) are input.

[0138] According to various embodiments, the learning server 20a may learn a single integrated artificial intelligence model (e.g., the first integrated artificial intelligence model 1800) for acquiring a PPG. For example, referring to FIG. 18b, the learning server 20a may acquire the first-second integrated artificial intelligence model 1800b by learning the image information 1710, the PPG 1750, and the additional learning information 1730 as input data and the rPPG 1740 or the MPPG 1720 as output data. The first-second integrated artificial intelligence model 1800b may be embodied to output the MPPG 1720 or the rPPG 1740 when the image information 1710, the PPG 1750, and the additional learning information 1730 (e.g., the additional information 1731, the environmental data 1733) are input.

[0139] The following describes example operations for generating an artificial intelligence model without time synchronization as at least part of the 1507 operations of the learning server 20a according to various embodiments.

[0140] 19 is a flowchart illustrating an example of operations for generating an artificial intelligence model without time synchronization of the learning server 20a, according to various embodiments. Operations may be performed out of the order of operations shown and / or described, and more operations and / or fewer operations may be performed.

[0141] According to various embodiments, the learning server 20a may acquire a plurality of biosignals (e.g., rPPG, MPPG, PPG) as training data without performing time synchronization between the plurality of different biosignals (e.g., rPPG, MPPG, PPG) in operation 1901. Accordingly, the learning server 20a may perform an operation of learning the at least one artificial intelligence model 1700a-1700f and 1800 described above based on the plurality of biosignals (e.g., rPPG, MPPG, PPG) without performing time synchronization. Accordingly, the at least one artificial intelligence model 1700a-1700f and 1800 may be embodied to output information on PPG with high accuracy even if information acquired by the electronic device 10 in the future is input without time synchronization.

[0142] The following is a diagram for explaining an example of an operation of generating another artificial intelligence model of the learning server 20a according to various embodiments. In the following, the specific type of biosignal is described as PPG, but the present invention is not limited to the described example, and artificial intelligence models for measuring various types of biosignals may be implemented.

[0143] According to various embodiments, the time difference td between the multiple vital signs (eg, rPPG, MPPG, PPG) may be utilized to measure physical information such as blood pressure.

[0144] FIG. 20 is a flowchart illustrating an example of another artificial intelligence model generation operation of the learning server 20a according to various embodiments. The operations may be performed out of the order of the operations shown and / or described, and more operations and / or fewer operations may be performed. FIG. 20 is further described below with reference to FIGS. 21 to 23.

[0145] FIG 21 is a diagram for explaining an example of an operation in which the learning server 20a uses rPPG and PPG to generate an AI model according to various embodiments. FIG 22 is a diagram for explaining an example of an operation in which the learning server 20a uses MPPG and PPG to generate an AI model according to various embodiments. FIG 23 is a diagram for explaining an example of at least one AI model generated by the learning server 20a according to various embodiments.

[0146] According to various embodiments, the learning server 20a may acquire a first biosignal of a particular type based on photographing the subject S using the camera 12 of the electronic device 10 in operation 2001, acquire a second biosignal based on a contact sensor (e.g., the external measurement sensor 600, the contact sensor 15) in operation 2003, and acquire the additional learning information in operation 2005. For example, the learning server 20a may acquire rPPG and PPG as shown in Fig. 21, or acquire rPPG and MPPG as shown in Fig. 22, instead of acquiring all of rPPG, PPG, and MPPG for generating an artificial intelligence model. Other overlapping descriptions will be omitted.

[0147] According to various embodiments, the learning server 20a may acquire at least one model 2300a, 2300b, 2300c, 2300d, 2300e for acquiring the specific type of biosignal based on at least a portion of the plurality of biosignals and the additional learning information in operation 2007. For example, the learning server 20a may generate at least one artificial intelligence model embodied to output information on MPPG or information on PPG as a result.

[0148] Also, in one embodiment, referring to FIG. 23(a), the learning server 20a may be embodied to learn an artificial intelligence model (e.g., fourth rPPG acquisition model 2300a) for acquiring a non-contact PPG (rPPG) and an artificial intelligence model (e.g., fourth PPG acquisition model 2300b) for acquiring a PPG 2301 (e.g., MPPG or PPG). For example, the learning server 20a may acquire the fourth rPPG acquisition model 2300a by learning the image information 1710 captured by the camera 12 as input data and the rPPG 1740 as output data among the mutually related information stored in the database 24a. The fourth rPPG acquisition model 2300a may be embodied to output the rPPG 1740 when the image information 1710 is input. Meanwhile, without being limited to the described example, the fourth rPPG acquisition model 2300a may be software and / or algorithm for acquiring a third characteristic value based on the GR and GB described above in FIG. 9. Also, for example, the learning server 20a may acquire a fourth PPG acquisition model 2300b by learning the rPPG 1740 and the additional learning information 1730 as input data and the PPG 2301 (e.g., MPPG or PPG) as output data from among the information stored in the database 24a. The fourth PPG acquisition model 2300b may be embodied to output the PPG 2301 (e.g., MPPG or PPG) when the rPPG 1740 and the additional learning information 1730 (e.g., additional information 1731, environmental data 1733) are input. A PPG 2301 (e.g., MPPG or PPG) can be acquired by inputting image information 1710 into the fourth rPPG acquisition model 2300a, outputting the rPPG 1740, together with additional learning information 1730 (e.g., additional information 1731, environmental data 1733) into the fourth PPG acquisition model 2300b.

[0149] Also, in one embodiment, referring to FIG. 23(a), the learning server 20a may be implemented to learn an artificial intelligence model (e.g., fourth rPPG acquisition model 2300a) for acquiring a non-contact PPG (rPPG) and an artificial intelligence model (e.g., fourth PPG acquisition model 2300b) for acquiring a PPG 2301 (e.g., MPPG or PPG). For example, the learning server 20a may acquire the fourth rPPG acquisition model 2300a by learning using image information 1710 captured by the camera 12 as input data and rPPG 1740 as output data among the mutually related information stored in the database 24a. The fourth rPPG acquisition model 2300a may be implemented to output rPPG 1740 when image information 1710 is input. Also, for example, the learning server 20a may acquire a fourth PPG acquisition model 2300b by learning the rPPG 1740 and the additional learning information 1730 as input data and the PPG 2301 (e.g., MPPG or PPG) as output data from among the information stored in the database 24a. The fourth PPG acquisition model 2300b may be embodied to output the PPG 2301 (e.g., MPPG or PPG) when the rPPG 1740 is input. The rPPG 1740 output by inputting the image information 1710 to the fourth rPPG acquisition model 2300a may be input to the fourth PPG acquisition model 2300b to acquire the PPG 2301 (e.g., MPPG or PPG).

[0150] Also, in one embodiment, referring to FIG. 23(b), the learning server 20a may be embodied to learn an artificial intelligence model (e.g., fifth rPPG acquisition model 2300c) for acquiring a non-contact PPG (rPPG) and an artificial intelligence model (e.g., fifth PPG acquisition model 2300d) for acquiring a PPG 2301 (e.g., MPPG or PPG). For example, the learning server 20a may acquire the fifth rPPG acquisition model 2300c by learning the image information 1710 captured by the camera 12 and the additional learning information 1730 as input data and the rPPG 1740 as output data among the mutually related information stored in the database 24a. The fifth rPPG acquisition model 2300c may be embodied to output the rPPG 1740 when the image information 1710 and the additional learning information 1730 (e.g., additional information 1731, environmental data 1733) are input. Also, for example, the learning server 20a may acquire a fifth PPG acquisition model 2300d by learning with the rPPG 1740 as input data and the PPG 2301 (e.g., MPPG or PPG) as output data from the information stored in the database 24a. The fifth PPG acquisition model 2300d may be embodied to output the PPG 2301 (e.g., MPPG or PPG) when the rPPG 1740 is input. The fifth rPPG acquisition model 2300c may receive the image information 1710, and the rPPG 1740 may be input to the fifth PPG acquisition model 2300d to acquire the PPG 2301 (e.g., MPPG or PPG).

[0151] According to various embodiments, the learning server 20a may learn a single integrated artificial intelligence model (e.g., the second integrated artificial intelligence model 2300e) for acquiring a PPG. For example, referring to FIG. 23(c), the learning server 20a may acquire the second integrated artificial intelligence model 2300e by learning the image information 1710, the MPPG 1720, and the additional learning information 1730 as input data and the PPG 2301 (e.g., MPPG or PPG) as output data. The second integrated artificial intelligence model 2300e may be embodied to output the PPG 2301 (e.g., MPPG or PPG) when the image information 1710, the MPPG 1720, and the additional learning information 1730 (e.g., the additional information 1731, the environmental data 1733) are input.

[0152] The following is a diagram for explaining an example of an operation of providing a biosignal with similar accuracy to that of a contact type based on information acquired in a non-contact type using an artificial intelligence model of the electronic device 10 according to various embodiments. In the following, an example in which the specific type of biosignal is PPG will be described, but the present invention is not limited to the described example, and artificial intelligence models for measuring various types of biosignals may be embodied.

[0153] 24 is a flow chart illustrating an example of operations for providing a biosignal with similar accuracy to a contact-based method using an artificial intelligence model of the electronic device 10, according to various embodiments. Operations may be performed out of the order of the operations shown and / or described, and more operations and / or fewer operations may be performed. FIG. 24 is further described below with reference to FIG. 25.

[0154] FIG. 25 is a diagram for explaining the operation of providing a biosignal with similar accuracy to the contact type by using an artificial intelligence model of the electronic device 10 according to various embodiments.

[0155] According to various embodiments, the electronic device 10 may execute an application in operation 2401. For example, the application may be an application implemented to provide information on a biosignal with the accuracy of a contact type based on acquiring information on a part of a body of a user U (i.e., a sample) in a contactless manner and / or bioinformation (e.g., blood pressure, blood glucose, etc.) analyzed based on the biosignal.

[0156] According to various embodiments, the electronic device 10 may acquire additional learning information in operation 2403. For example, the electronic device 10 may acquire personal information as additional learning information. An application execution screen for inputting information on the personal information (e.g., gender, year, age, race, etc.) may be displayed, and user characteristic information inputted through the execution screen may be stored and / or transmitted to the server 20 (e.g., the usage server 20b). The application execution screen may be an execution screen provided at the time of user subscription and / or an execution screen for inputting the user's personal information. Meanwhile, without being limited to the described examples, the electronic device 10 may acquire camera information, shooting information, and / or image information as additional learning information.

[0157] According to various embodiments, the electronic device 10 may acquire a plurality of images using the camera 12 of the electronic device 10 in ACT 2405, and may acquire sensing data using the contact sensor 15 of the electronic device 10 in ACT 2407. For example, as illustrated in FIG. 21, the electronic device 10 may display an application execution screen 2501 for guiding a user to touch a part of the body (e.g., a finger) to the contact sensor 15 while photographing the user's face using the camera 12. The execution screen 2501 may include an area 2501a in which the user's face is displayed, an area 2501b in which guiding text (e.g., "Touch the sensor with your finger to photograph your face") is displayed, and an area 2501c in which information on the user's photographing environment (e.g., the distance, position, inclination, illuminance, etc. between the electronic device 10 and the face) is displayed. Accordingly, the electronic device 10 can obtain a plurality of images including the user's face using the camera 12, obtain sensing data using the contact sensor 15 disposed on the rear surface, and obtain additional learning information (e.g., environmental data obtained by the environmental sensor 16, and additional information) during shooting. Meanwhile, as described above, the additional information can include personal information, camera information, shooting information, and / or image information. The electronic device 10 can transmit the obtained information to the server 20 (e.g., the utilization server 20b).

[0158] According to various embodiments, an application may be implemented such that, when executed, it drives and has authority over the camera 12, the contact sensor 15, and the environmental sensors 14 of the electronic device 10.

[0159] According to various embodiments, the electronic device 10 may acquire a specific type of biosignal based on the plurality of images, the sensing data, and the additional learning information in ACT 2409, and may acquire at least one bioinformation corresponding to the specific type of biosignal in ACT 2411. For example, the utilization server 20b may acquire a PPG that is finally output as a response to inputting received information into at least one learned artificial intelligence model (e.g., artificial intelligence models 1700a-1700f in FIG. 17, artificial intelligence models 1800a-1800b in FIG. 18a-FIG. 18b, and artificial intelligence models 2300a-2300e in FIG. 23) as described above. As a result, the PPG may have the accuracy of a contact type method. The utilization server 20b may also acquire physical information of the user (e.g., blood pressure, heart rate information) based on the PPG. As a result, the electronic device 10 can receive information on the PPG and / or the user's physical information (e.g., blood pressure) from the utilization server 20b and display them on the application execution screen 2503. For example, the execution screen 2503 can include information on heart rate 2503a and information on blood pressure 2503b.

[0160] The following is a diagram illustrating an example of an operation of guiding photography, which is at least a part of the operation 2403 of the electronic device 10 according to various embodiments.

[0161] 26 is a flow chart illustrating operations for guiding photography of the electronic device 10, according to various embodiments. Operations may be performed out of the order of operations shown and / or described, and more operations and / or fewer operations may be performed. FIG. 26 is further described below with reference to FIG. 27.

[0162] FIG. 27 is a diagram illustrating an example of an operation of guiding photography of the electronic device 10 according to various embodiments.

[0163] According to various embodiments, the electronic device 10 may display an execution screen of an application for taking pictures in operation 2601. For example, as shown in FIG 27, the electronic device 10 may provide an execution screen 2701 of the application for taking pictures.

[0164] According to various embodiments, the electronic device 10 may determine whether a particular condition is satisfied in operation 2603, and if the particular condition is satisfied (2603-Y), perform shooting in a state where at least one camera parameter (e.g., shutter speed, FPS, shooting resolution, etc.) is set to a particular value in operation 2605. For example, in at least a part of the operation of determining whether a particular condition is satisfied, the electronic device 10 may determine whether information related to the surrounding environment to be shot (e.g., illuminance, etc.) and / or information related to the state of the electronic device 10 at the time of shooting (e.g., position, weather, etc.) satisfies a particular condition using the environmental sensor 16 of the electronic device 10. For example, as illustrated in FIG. 27, the electronic device 10 may determine whether the position of the electronic device 10 satisfies a particular position (e.g., second position). When the electronic device 10 is in the particular position (e.g., second position), the electronic device 10 may perform shooting using the camera 12. Accordingly, the deviation of the image acquired in the non-contact manner is reduced, and the accuracy of the PPG can be improved.

[0165] According to various embodiments, as shown in FIG. 27, the electronic device 10 may update and provide information on a location (e.g., a first location and a second location) on an execution screen 2701 of an application for photographing, thereby allowing a user to recognize the location.

[0166] According to various embodiments, at least one camera parameter (e.g., shutter speed, FPS, etc.) may be set to a specific value during the shooting. For example, the camera parameter may be set so that the video is shot with the FPS in the range of 20 to 30 per second.

[0167] Below are diagrams for explaining examples of operations of a server 20 (e.g., utilization server 20b) using an artificial intelligence model according to various embodiments.

[0168] FIG. 28 is a flow chart illustrating an example of an operation of a server 20 (e.g., utilization server 20b) to utilize an artificial intelligence model, according to various embodiments. Operations may be performed out of the order of operations shown and / or described, and more operations and / or fewer operations may be performed. FIG. 28 is further described below with reference to FIG. 29.

[0169] FIG. 29 is a diagram illustrating an example of an operation of a server 20 (e.g., utilization server 20b) using an artificial intelligence model according to various embodiments.

[0170] According to various embodiments, the utilization server 20b may acquire at least one input data corresponding to each of a plurality of artificial intelligence models in operation 2801, and acquire a plurality of specific types of biosignals in response to inputting the at least one input data to each of the plurality of artificial intelligence models in operation 2803. For example, the utilization server 20b may store at least one of the learned artificial intelligence models described above (e.g., artificial intelligence models 1700a-1700f in FIG. 17, artificial intelligence models 1800a-1800b in FIG. 18a-FIG. 18b, and artificial intelligence models 2300a-2300e in FIG. 23). At this time, the utilization server 20b can obtain multiple PPGs to be output by inputting input data related to each artificial intelligence model (e.g., artificial intelligence models 1700a to 1700f of FIG. 17, artificial intelligence models 1800a to 1800b of FIG. 18a to FIG. 18b, artificial intelligence models 2300a to 2300e of FIG. 23) based on the image information 1710, MPPG 1720, and / or additional learning information 1730 received from the electronic device 10.

[0171] According to various embodiments, the utilization server 20b may acquire a specific bio-signal of the specific type based on the plurality of bio-signals of the specific type in operation 2805. In one embodiment, the utilization server 20b may select a specific PPG determined to be the most reliable among the plurality of PPGs, and provide information and / or physical information for the specific PPG to the electronic device 10. In another embodiment, the utilization server 20b may acquire information for a specific PPG by performing a predetermined calculation (e.g., averaging) based on the plurality of PPGs, and provide information and / or physical information for the specific PPG to the electronic device 10. In another embodiment, the utilization server 20b may select a specific PPG output by an artificial intelligence model that is most suitable for the user from the plurality of PPGs, and provide information and / or physical information for the specific PPG to the electronic device 10.

Claims

1. 1. An electronic device comprising: Communications circuits; and at least one processor; The at least one processor: acquiring a plurality of images including a face of a user captured using a camera of a first external electronic device through the communication circuit; acquiring, via the communication circuit, photographing information related to photographing acquired by the first external electronic device while acquiring the plurality of images; acquiring, through the communication circuit, a first biosignal of a particular type acquired based on a first sensor of the first external electronic device contacted with a first finger of the first hand of the user, and a second biosignal of the particular type acquired based on a second sensor of a second external electronic device contacted with a second finger corresponding to the first finger of the second hand of the user, which are acquired simultaneously while acquiring the plurality of images; Obtaining personal information related to the user's personal characteristics through the communication circuit; and acquiring at least one artificial intelligence model by performing learning using the plurality of images, the information related to the photographing, the first biosignal, the second biosignal, and the personal information as training data; The at least one artificial intelligence model is configured to provide information regarding the second bio-signal in response to receiving the plurality of images acquired using a camera of the first external electronic device, information related to the shooting, information regarding the first bio-signal acquired based on the first sensor of the first external electronic device, and input of the personal information, the electronic device.

2. The electronic device of claim 1 , wherein the photographing information comprises information related to a photographing-related condition of the first external electronic device and information related to an external environment of the first external electronic device.

3. The at least one processor:

3. An electronic device according to claim 1 or 2, configured to perform time synchronization of the first and second vital signs.

4. The at least one processor: The electronic device of claim 3 , configured to perform time synchronization of the first and second vital signs with reference to a vital sign identified based on the plurality of images.

5. The at least one processor: The electronic device of claim 3 , configured to generate the at least one artificial intelligence model without the time synchronization being performed.

6. 1. A method of operating an electronic device, comprising: acquiring, via a communication circuit of the electronic device, a plurality of images including a face of a user, the plurality of images being acquired using a camera of a first external electronic device; acquiring, via the communication circuitry, photographic information related to photographing acquired by the first external electronic device during acquisition of the plurality of images; acquiring, through the communication circuit, a first biosignal of a particular type acquired based on a first sensor of the first external electronic device contacted with a first finger of the first hand of the user and a second biosignal of the particular type acquired based on a second sensor of a second external electronic device contacted with a second finger corresponding to the first finger of the second hand of the user, the first biosignal being acquired simultaneously during the acquisition of the plurality of images; obtaining, via said communication circuitry, personal information related to a personal characteristic of said user; and acquiring at least one artificial intelligence model by performing learning using the plurality of images, the information related to the photographing, the first biosignal, the second biosignal, and the personal information as training data; An operating method, wherein the at least one artificial intelligence model is configured to provide information regarding the second bio-signal in response to receiving the plurality of images acquired using a camera of the first external electronic device, information related to the shooting, information regarding the first bio-signal acquired based on the first sensor of the first external electronic device, and input of the personal information.

7. The method of claim 6 , wherein the photographing information related to the photographing of the electronic device includes information related to a state related to the photographing of the first external electronic device and information related to an external environment of the first external electronic device.

Images