Method for obtaining biometric data by using wearable device and optical device

Abstract

Provided according to an embodiment is a wearable device comprising: an optical sensor that irradiates a plurality of different regions of a user's body with light and receives a plurality of reflected lights reflected from the plurality of different regions of the user's body; and a control unit that performs computation to obtain PPG data on the basis of a plurality of photocurrent signals generated by the plurality of reflected lights, wherein the control unit determines, on the basis of the plurality of photocurrent signals generated by the plurality of reflected lights, a photocurrent signal having the smallest noise and a photocurrent signal having the largest noise, and obtains PPG data on the basis of photocurrent signals obtained by performing certain computations on the photocurrent signal having the smallest noise and the photocurrent signal having the largest noise.

Description

Method for acquiring biometric data using wearable devices and optical devices

[0001] The present disclosure relates to a method for obtaining biometric data using a wearable device and an optical device, and more particularly, to a method for obtaining biometric data using a wearable device and an optical device capable of obtaining PPG data with reduced noise.

[0002] As modern people's interest in health grows, the healthcare industry is developing, and with it, the demand for wearable devices that can naturally monitor health conditions is increasing.

[0003] Monitoring a user's health using wearable devices requires highly sensitive sensors capable of sensing a variety of biometric data. Sensors applicable to wearable devices include contact electrode sensors, optical sensors, and temperature sensors.

[0004] In particular, optical sensors that can sense health-related data such as heart rate, oxygen saturation, and sleep quality in real time by irradiating light on human skin and sensing the reflected light are becoming increasingly important in terms of utilizing wearable devices.

[0005] Optical sensors embedded in wearable devices must make good contact with the skin to effectively illuminate it with light. If light is emitted from the optical sensor without proper contact, the light cannot reach the skin, potentially reducing the sensing efficiency of biometric data.

[0006] Additionally, when a user moves while wearing a wearable device, motion artifacts, noise caused by the relative movement between the optical sensor on the wearable device and the user's skin, can occur when sensing biometric data. To obtain higher-quality biometric data using optical sensors, the impact of these motion artifacts must be minimized.

[0007] Meanwhile, hearable devices (a portmanteau of hear and wearable, meaning wearable devices focused on hearing) that have been in the spotlight recently are significantly smaller than the most common wearable devices, such as smartwatches or VR devices, and because they necessarily include an audio output device, there are limitations to additionally installing a separate skin detection sensor in addition to the heart rate sensor.

[0008] According to various embodiments of the present disclosure, there is provided a wearable device capable of obtaining high-quality PPG data with reduced motion artifacts by utilizing an optical sensor including a plurality of optical channels, and a method for obtaining biometric data using an optical device.

[0009] According to various embodiments of the present disclosure, there is provided a wearable device and a method for obtaining biometric data using an optical device, which increase energy consumption efficiency by selectively driving some of a plurality of light-emitting elements included in an optical sensor.

[0010] However, the technical problems that the various embodiments of the present disclosure seek to achieve are not limited to the technical problems described above, and other technical problems may exist.

[0011] One example is,

[0012] A wearable device is provided, comprising: an optical sensor that irradiates light to a plurality of different areas of a user's body and receives a plurality of reflected lights reflected from the plurality of different areas of the user's body; and a control unit that performs an operation to obtain PPG data based on a plurality of photocurrent signals by the plurality of reflected lights, wherein the control unit determines a photocurrent signal with the smallest noise and a photocurrent signal with the largest noise based on the plurality of photocurrent signals by the plurality of reflected lights, and obtains PPG data based on the photocurrent signal obtained by performing a predetermined operation on the photocurrent signal with the smallest noise and the photocurrent signal with the largest noise.

[0013] In another aspect, the control unit can obtain PPG data based on a photocurrent signal obtained by subtracting the photocurrent signal with the largest noise from the photocurrent signal with the smallest noise.

[0014] In another aspect, the control unit can obtain PPG data based on a photocurrent signal obtained by removing a signal corresponding to the PPG frequency range from the photocurrent signal with the greatest noise and subtracting the photocurrent signal obtained by removing the signal corresponding to the PPG frequency range from the photocurrent signal with the greatest noise from the photocurrent signal with the smallest noise.

[0015] In another aspect, the control unit may obtain a plurality of frequency signals by frequency-converting a plurality of photocurrent signals by the plurality of reflected lights, determine a selected frequency range having a maximum power spectral density (PSD) for each of the plurality of frequency signals, and determine a frequency signal having a largest PSD in the selected frequency range and a frequency signal having a smallest PSD in the selected frequency range among the plurality of frequency signals.

[0016] In another aspect, the control unit may obtain a noise frequency signal by removing a signal corresponding to a selected frequency range of a frequency signal having the largest PSD in the selected frequency range from a frequency signal having the smallest PSD in the selected frequency range, obtain a reference photocurrent signal by inversely frequency-converting the frequency signal having the largest PSD in the selected frequency range, obtain a noise photocurrent signal by inversely frequency-converting the noise frequency signal, and obtain PPG data based on a target photocurrent signal obtained by subtracting the noise photocurrent signal from the reference photocurrent signal.

[0017] In another aspect, the control unit can ensemble process each of the plurality of photocurrent signals by the plurality of reflected lights, and determine the photocurrent signal with the smallest noise and the photocurrent signal with the largest noise based on the plurality of ensemble-processed photocurrent signals.

[0018] In another aspect, the control unit can normalize each of the plurality of ensemble-processed photocurrent signals, and determine the photocurrent signal with the smallest noise and the photocurrent signal with the largest noise based on the normalized plurality of photocurrent signals.

[0019] In another aspect, the optical sensor includes a plurality of light-emitting elements corresponding to the plurality of areas, and the control unit determines a selected light-emitting element that emits light that is the basis of the photocurrent signal with the smallest noise among the plurality of light-emitting elements, and can selectively drive the selected light-emitting element among the plurality of light-emitting elements.

[0020] One example is,

[0021] A method for obtaining biometric data using an optical device is provided, comprising the steps of irradiating light to a plurality of different areas of a user's body and receiving a plurality of reflected lights reflected from each of the plurality of different areas of the user's body, determining a photocurrent signal with the least noise and a photocurrent signal with the greatest noise based on a plurality of photocurrent signals by the plurality of reflected lights, and obtaining PPG data based on the photocurrent signals obtained by performing a predetermined operation on the photocurrent signal with the least noise and the photocurrent signal with the greatest noise.

[0022] In another aspect, the step of determining the photocurrent signal with the smallest noise and the photocurrent signal with the largest noise may include the step of frequency-converting the plurality of photocurrent signals by the plurality of reflected lights to obtain a plurality of frequency signals, the step of determining a selected frequency range having a maximum power spectral density (PSD) for each of the plurality of frequency signals, and the step of determining a frequency signal having the largest PSD in the selected frequency range and a frequency signal having the smallest PSD in the selected frequency range among the plurality of frequency signals.

[0023] In another aspect, the step of obtaining the PPG data may include the step of obtaining a noise frequency signal by removing a signal corresponding to a selected frequency range of a frequency signal having the largest PSD in the selected frequency range from a frequency signal having the smallest PSD in the selected frequency range, the step of obtaining a reference photocurrent signal by inversely frequency-converting the frequency signal having the largest PSD in the selected frequency range, the step of obtaining a noise photocurrent signal by inversely frequency-converting the noise frequency signal, and the step of obtaining PPG data based on a target photocurrent signal obtained by subtracting the noise photocurrent signal from the reference photocurrent signal.

[0024] According to various embodiments of the present disclosure, a wearable device and a method for obtaining biometric data using an optical device can be provided, which can obtain high-quality PPG data with reduced motion artifacts by performing a predetermined operation on a plurality of photocurrent signals from an optical sensor including a plurality of optical channels.

[0025] According to various embodiments of the present disclosure, a wearable device and a method for acquiring biometric data using an optical device can be provided, which increase energy consumption efficiency by selectively driving a light-emitting element among a plurality of light-emitting elements included in an optical sensor, which is the basis for acquiring high-quality PPG data with reduced motion artifacts.

[0026] However, the effects that can be obtained through various embodiments of the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood from the description below.

[0027] FIG. 1 is a conceptual diagram of a system for acquiring a user's biometric data using a wearable device including an optical sensor according to one embodiment.

[0028] Figure 2 is a block diagram illustrating the functions of a server according to one embodiment.

[0029] FIG. 3 is a block diagram illustrating the structure of a wearable device according to an embodiment.

[0030] FIG. 4 is for explaining the operation of an optical sensor according to one embodiment.

[0031] FIG. 5 is a diagram illustrating a user wearing bone conduction earphones according to one embodiment.

[0032] FIG. 6 is a diagram illustrating a multi-channel optical sensor included in a bone conduction earphone according to one embodiment arranged to correspond to the user's skin.

[0033] Figure 7 is an internal block diagram of an electronic device according to one embodiment.

[0034] Fig. 8 is a flowchart of a method for acquiring biometric data using an optical device according to one embodiment.

[0035] FIG. 9 is for explaining an exemplary form of a plurality of photocurrent signals according to one embodiment.

[0036] Fig. 10 is a flowchart for explaining the steps of determining the photocurrent signal with the smallest noise and the photocurrent signal with the largest noise in Fig. 8.

[0037] FIG. 11 is a diagram illustrating a method for determining a selected frequency region having a maximum power spectral density for each of a plurality of photocurrent signals according to one embodiment.

[0038] Figure 12 is a flowchart for explaining the steps for obtaining PPG data of Figure 8.

[0039] FIG. 13 is for explaining a method of obtaining a noise frequency signal according to one embodiment.

[0040] FIG. 14 is a diagram illustrating a method for obtaining PPG data by performing a predetermined operation on a reference photocurrent signal and a noise photocurrent signal according to one embodiment.

[0041] FIG. 15 is for explaining a method of ensemble processing a photocurrent signal according to one embodiment.

[0042] The present invention is capable of various modifications and embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, as well as the methods for achieving them, will become clear with reference to the embodiments described in detail below together with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms. In the following embodiments, terms such as first, second, etc. are not used in a limiting sense but are used for the purpose of distinguishing one component from another. Furthermore, the singular expression includes plural expressions unless the context clearly indicates otherwise. Furthermore, terms such as "include" or "have" indicate the presence of a feature or component described in the specification, and do not preemptively exclude the possibility that one or more other features or components may be added. Furthermore, in the drawings, the sizes of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to what is shown.

[0043] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals and redundant descriptions thereof will be omitted.

[0044] FIG. 1 is a conceptual diagram of a system (1000) for acquiring biometric data of a user using a wearable device including an optical sensor according to one embodiment. FIG. 2 is a block diagram for explaining the function of a server (100) according to one embodiment. FIG. 3 is a block diagram for explaining the structure of a wearable device (200) according to one embodiment. FIG. 4 is for explaining the operation of an optical sensor (211) according to one embodiment. FIG. 5 is for explaining an appearance of a user wearing bone conduction earphones (201) according to one embodiment. FIG. 6 is for explaining an appearance of a multi-channel (MC) of an optical sensor (211) included in a bone conduction earphone (201) according to one embodiment being arranged to correspond to the user's skin. FIG. 7 is an internal block diagram of an electronic device (300) according to one embodiment. FIG. 8 is a flowchart of a method (S100) for acquiring biometric data using an optical device according to one embodiment. FIG. 9 is for explaining an exemplary form of a plurality of photocurrent signals according to one embodiment. FIG. 10 is a flowchart for explaining a step (S103) of determining a photocurrent signal with the smallest noise and a photocurrent signal with the largest noise of FIG. 8. FIG. 11 is a flowchart for explaining a method of determining a selected frequency region having a maximum power spectral density for each of a plurality of photocurrent signals according to one embodiment. FIG. 12 is a flowchart for explaining a step (S105) of obtaining PPG data of FIG. 8. FIG. 13 is a flowchart for explaining a method of obtaining a noise frequency signal according to one embodiment. FIG. 14 is a flowchart for explaining a method of obtaining PPG data by performing a predetermined operation on a reference photocurrent signal and a noise photocurrent signal according to one embodiment. FIG. 15 is a flowchart for explaining a method of ensemble processing a photocurrent signal according to one embodiment.

[0045] Referring to FIG. 1, a user's biometric data acquisition system (1000) according to one embodiment may include a server (100), a wearable device (200), and an electronic device (300). The server (100), the wearable device (200), and the electronic device (300) may transmit and receive data to and from each other via a network.

[0046] A system (1000) according to one embodiment can measure various types of biometric data, such as heart rate data and body temperature data, of a user measured by a sensor unit (210) included in a wearable device (200), and provide a service for guiding a customized exercise program to the user based on the measured biometric data. Here, the heart rate data may be referred to as PPG data.

[0047] A wearable device (200) may be an electronic device that can be worn on a user's body, such as clothing or accessories. For example, the wearable device (200) may include a smartwatch, a smartband, smart glasses, etc. In addition, the wearable device (200) may include a hearable device, such as bone conduction earphones (201) or completely wireless earphones (202), as illustrated in FIG. 1.

[0048] Here, hearable device is a compound word of hear and wearable, and can refer to a wearable device focused on hearing that provides various convenient functions such as voice recognition, connection with voice recognition artificial intelligence, music playback, and phone calls.

[0049] The system (1000) can obtain sensing data based on a predetermined biosensor included in a wearable device (200), and provide the user with biofeedback content generated based on the user's bio-information, exercise amount information, posture information, etc. calculated based on the obtained sensing data.

[0050] The network according to the embodiment means a connection structure that enables information exchange between each node, such as a server (100), a wearable device (200), and / or an electronic device (300), and examples of such a network include, but are not limited to, a 3GPP (3rd Generation Partnership Project) network, an LTE (Long Term Evolution) network, a WIMAX (World Interoperability for Microwave Access) network, the Internet, a LAN (Local Area Network), a Wireless LAN (Wireless Local Area Network), a WAN (Wide Area Network), a PAN (Personal Area Network), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, a DMB (Digital Multimedia Broadcasting) network, and the like.

[0051] Hereinafter, the server (100), wearable device (200), and electronic device (300) implementing the system (1000) will be described in detail with reference to the attached drawings.

[0052] -Server (100)

[0053] A server (100) according to one embodiment can perform a series of processes to provide a user's biometric data acquisition environment.

[0054] In detail, in the embodiment, the server (100) can provide a user's biometric data acquisition environment by exchanging data necessary to drive a user's biometric data acquisition process with an external device, such as a wearable device (200) and an electronic device (300).

[0055] In more detail, in an embodiment, the server (100) may provide an environment in which an application (311) can operate on an electronic device (300) (in an embodiment, a mobile type computing device and / or a desktop type computing device, etc.).

[0056] To this end, the server (100) may include application programs, data and / or commands for the application (311) to operate, and may transmit and receive various data based thereon with the external device.

[0057] Additionally, in the embodiment, the server (000) can perform various deep learning for obtaining biometric data in conjunction with a deep learning neural network.

[0058] Here, the deep learning neural network according to the embodiment may include a convolutional neural network (CNN), an R-CNN (Regions with CNN features), a Fast R-CNN, a Faster R-CNN, a Mask R-CNN, etc., and may include any deep learning neural network that includes an algorithm capable of performing the embodiment described below, and the embodiment of the present invention does not limit or restrict such deep learning neural network itself.

[0059] At this time, depending on the embodiment, the deep learning neural network may be installed directly on the server (100) or may operate as a device separate from the server (100) to perform deep learning for the exercise program provision service.

[0060] In addition, in the embodiment, the server (100) can read out a predetermined deep learning neural network driving program constructed to perform the deep learning from memory and perform deep learning according to the read out predetermined deep learning neural network system.

[0061] The server (100) can store and manage at least one or more sensing data, user body information, user status information, user exercise information, test results, heart rate information, biofeedback content, user exercise ability and / or exercise program, etc.

[0062] However, in the embodiment of the present invention, the functional operations that the server (100) can perform are not limited to those described above, and other functional operations may be performed.

[0063] Meanwhile, referring further to FIG. 1, in the embodiment, the server (100) as described above may be implemented as a computing device including at least one processor for data processing and at least one memory for storing various application programs, data and / or commands for obtaining biometric data.

[0064] Additionally, the memory may include a program area and a data area. Here, the program area according to the embodiment may be linked between the operating system (OS) that boots the server and functional elements, and the data area may store data generated according to the use of the server (100).

[0065] In an embodiment, such memory may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and may also be web storage that performs the storage function of the memory on the internet.

[0066] Meanwhile, the at least one processor may perform various operations for acquiring biometric data. The at least one processor may be a system-on-chip (SOC) suitable for a server, including a central processing unit (CPU) and / or a graphics processing unit (GPU), and may execute an operating system (OS) and / or application programs stored in memory.

[0067] Additionally, at least one processor may be implemented using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0068] In addition, referring to FIG. 2, at least one processor of the server (100) can perform the functions of a signal conversion unit (11), a maximum PSD area extraction unit (12), a signal extraction unit (13), a signal filtering unit (14), a signal subtraction unit (15), a PPG data extraction unit (16), and a signal preprocessing unit (17).

[0069] The signal conversion unit (11) can convert a signal received from the wearable device (200) through a network into a predetermined format. For example, the wearable device (200) can irradiate light onto the user's skin, and some of the light irradiated onto the user's skin can be absorbed by blood vessels within the skin, and the remaining portion can be reflected. The wearable device (200) can receive the reflected light reflected from the surface of the user's skin or the blood vessels within the skin, and generate a photocurrent signal by the reflected light. The wearable device (200) can transmit the photocurrent signal by the reflected light to the signal conversion unit (11) through the network.

[0070] The signal conversion unit (11) can obtain a frequency signal by frequency converting the photocurrent signal from the wearable device (200). For example, the signal conversion unit (11) can perform a fast Fourier transform (FFT) on the photocurrent signal.

[0071] In addition, the signal conversion unit (11) can obtain a photocurrent signal by performing inverse frequency conversion on a signal on which a predetermined operation has been performed on a frequency signal. For example, the signal conversion unit (11) can perform inverse FFT conversion on a signal on which a predetermined operation has been performed on a frequency signal. Here, the predetermined operation may refer to a series of processes performed by the maximum PSD area extraction unit (12), the signal extraction unit (13), and the signal filtering unit (14) described below.

[0072] The maximum PSD region extraction unit (12) can determine a selected frequency region having the maximum power spectral density (PSD) for the frequency signal obtained by the signal conversion unit (11).

[0073] For example, the frequency signal acquired by the signal conversion unit (11) may have various intensity values ​​for each frequency. In this case, the frequency signal may have PSDs of various sizes for each frequency range. The maximum PSD area extraction unit (12) may determine the frequency range with the largest PSD. Here, the frequency range exhibiting the largest PSD may be referred to as the selected frequency range.

[0074] In addition, the maximum PSD area extraction unit (12) can determine a selection frequency having a maximum intensity value for the frequency signal acquired by the signal conversion unit (11). For example, the frequency signal acquired by the signal conversion unit (11) can have various intensity values ​​for each frequency, and the maximum PSD area extraction unit (12) can determine a frequency having the largest intensity of the frequency signal. Here, the frequency representing the frequency signal having the largest intensity value can be referred to as a selection frequency.

[0075] The signal extraction unit (13) can determine the photocurrent signal with the least noise and the photocurrent signal with the greatest noise based on a plurality of photocurrent signals transmitted from the wearable device (200) through the network. Here, noise may refer to a motion artifact caused by the user's movement.

[0076] For example, the wearable device (200) may include a sensor unit (210) that can irradiate light to multiple different areas of the user's body, receive multiple reflected lights reflected from the multiple different areas, and generate multiple photocurrent signals by the multiple reflected lights. The wearable device (200) may transmit multiple photocurrent signals by the multiple reflected lights to a signal conversion unit (11). The signal conversion unit (11) may convert the multiple photocurrent signals into multiple frequency signals, and the maximum PSD region extraction unit (12) may determine a selected frequency region having the maximum PSD for each of the multiple frequency signals.

[0077] In this case, the signal extraction unit (13) can determine a frequency signal having the largest PSD in the selected frequency range and a frequency signal having the smallest PSD in the selected frequency range from among a plurality of frequency signals for which a selected frequency range having the maximum PSD is determined. Here, the photocurrent signal corresponding to the frequency signal having the largest PSD in the selected frequency range may be the photocurrent signal having the smallest noise, and the photocurrent signal corresponding to the frequency signal having the smallest PSD in the selected frequency range may be the photocurrent signal having the largest noise.

[0078] In this way, the signal extraction unit (13) can determine the photocurrent signal with the smallest noise and the photocurrent signal with the largest noise from the plurality of photocurrent signals based on the size of the PSD in the selected frequency range of the plurality of frequency signals generated by frequency conversion of the plurality of photocurrent signals.

[0079] The signal filtering unit (14) can perform a predetermined filtering on the photocurrent signal with the greatest noise determined by the signal extraction unit (13). For example, the signal filtering unit (14) can perform a predetermined filtering on the frequency signal with the smallest PSD in the selected frequency range corresponding to the photocurrent signal with the greatest noise.

[0080] For example, the signal filtering unit (14) can obtain a noise frequency signal by removing a signal corresponding to the selected frequency range of the frequency signal having the largest PSD in the selected frequency range from the frequency signal having the smallest PSD in the selected frequency range.

[0081] Here, the signal corresponding to the selected frequency range of the frequency signal having the largest PSD in the selected frequency range may be the signal having the greatest influence on determining the PPG data characteristics among the frequency signals having the largest PSD in the selected frequency range.

[0082] In this way, by removing the signal corresponding to the frequency range of the signal that has the greatest influence on determining the PPG data characteristics from the frequency signal with the smallest PSD in the selected frequency range, the PPG data characteristics can be minimized, thereby obtaining a noise frequency signal with highlighted noise characteristics.

[0083] The signal subtraction unit (15) can subtract the photocurrent signal with the greatest noise from the photocurrent signal with the least noise.

[0084] For example, the signal subtraction unit (15) can perform a subtraction operation on a plurality of photocurrent signals obtained by inversely converting some of the plurality of frequency signals by the signal conversion unit (11).

[0085] For example, the signal conversion unit (11) can obtain a reference photocurrent signal by inversely converting the frequency signal having the largest PSD in the selected frequency range, and can obtain a noise photocurrent signal by inversely converting the noise frequency signal.

[0086] The signal subtraction unit (15) can obtain a target photocurrent signal by subtracting a noise photocurrent signal from the reference photocurrent signal obtained by the signal conversion unit (11).

[0087] The PPG data extraction unit (16) can obtain PPG data based on the photocurrent signal obtained by the signal subtraction unit (15) by subtracting the photocurrent signal with the greatest noise from the photocurrent signal with the least noise.

[0088] For example, the PPG data extraction unit (16) can obtain PPG data based on the target photocurrent signal obtained by the signal subtraction unit (15) by utilizing a PPG data extraction algorithm.

[0089] Here, the target photocurrent signal is a signal obtained by subtracting a noise photocurrent signal obtained by inversely converting a noise frequency signal with the strongest noise characteristic from a reference photocurrent signal obtained by inversely converting a frequency signal with the largest PSD in a selected frequency range with the strongest PPG data characteristic, and thus may be a signal in which the PPG data characteristic appears more distinctly.

[0090] In this way, when the PPG data extraction unit (16) acquires PPG data based on a target photocurrent signal in which the PPG data characteristics are more distinct than those of the reference photocurrent signal, it is possible to acquire high-quality PPG data with reduced noise influence compared to when the PPG data is acquired directly based on the photocurrent signal acquired through the sensor unit (210) of the wearable device (200).

[0091] The signal preprocessing unit (17) can perform preprocessing on multiple photocurrent signals generated by multiple reflected lights. For example, the signal preprocessing unit (17) can perform at least one preprocessing among smoothing processing, high-frequency noise filtering, baseline drift correction, ensemble processing, and normalization.

[0092] Smoothing processing extracts only the pattern of the photocurrent signal caused by the reflected light reflected from the user's body, which may mean removing residual peaks contained in the reflected light.

[0093] The signal preprocessing unit (17) can perform smoothing processing on the photocurrent signal generated by reflected light using a known smoothing processing technique. For example, the signal preprocessing unit (17) can perform smoothing processing on the photocurrent signal generated by reflected light using a plurality of low-pass filters.

[0094] High-frequency noise filtering can mean removing high-frequency noise, which is noise caused by environmental factors, electric field interference, motion artifacts, etc., in addition to signals representing biological changes.

[0095] The signal preprocessing unit (17) can perform high-frequency noise filtering on the photocurrent signal caused by reflected light using a known high-frequency noise filtering technique. For example, the signal preprocessing unit (17) can perform high-frequency noise filtering on the photocurrent signal caused by reflected light using a low-pass filter, a Kalman filter, or the like.

[0096] Baseline drift correction can refer to compensating for baseline drift, which occurs when the reference line for data, which should be constant, fluctuates due to environmental interference, motion artifacts, etc. Baseline drift can interfere with extracting desired information from received data.

[0097] The signal preprocessing unit (17) can utilize, for example, filtering techniques, baseline correction algorithms, and appropriate calibration operations to perform baseline drift correction. This can minimize baseline fluctuations and extract meaningful information from the original biometric data.

[0098] Ensemble processing may be a process of dividing a periodic signal into periods and summing the divided signals corresponding to each period to obtain a signal with further enhanced periodic characteristics of the periodic signal. For example, in the ensemble processing process, a given periodic signal may be divided into a first divided signal of a first period, a second divided signal of a second period, and a third divided signal of a third period, and the first to third divided signals may be summed to obtain an ensemble signal. The ensemble signal may be a signal with enhanced periodic characteristics of the given periodic signal.

[0099] Meanwhile, when ensemble processing is performed on a noise signal with weak periodic characteristics, a signal with reduced noise influence can be obtained. This is because noise signals do not have periodic repeating signal patterns, so even if multiple divided signals divided by period are added together, the periodic characteristics are not enhanced.

[0100] The signal preprocessing unit (17) can ensemble process the photocurrent signal, and accordingly, the periodic characteristics of the photocurrent signal can be further enhanced.

[0101] Normalization may be a preprocessing method used to standardize the intensities of multiple signals to facilitate comparison. For example, the signal preprocessing unit (17) may normalize a photocurrent signal using a MinMax Scaler. However, this is not a limitation, and the signal preprocessing unit (17) may normalize the input signal using various methods.

[0102] Meanwhile, at least one processor of the server (100) can selectively drive a plurality of light-emitting elements included in the optical sensor (211) of the wearable device (200).

[0103] For example, at least one processor of the server (100) can determine, through a series of processes, a selected light-emitting element that emits light that is the basis of a photocurrent signal with the lowest noise and the strongest PPG data characteristics among a plurality of light-emitting elements.

[0104] For example, among the plurality of light-emitting elements, a first light-emitting element may be controlled to irradiate light to a first area of ​​the user's body at a first time point, and a second light-emitting element may be controlled to irradiate light to a second area of ​​the user's body that is different from the first area at a second time point that is different from the first time point.

[0105] At least one processor of the server (100) can perform a predetermined operation on a first photocurrent signal by a first reflected light from a first region and a second photocurrent signal by a second reflected light to determine a photocurrent signal having the smallest noise and the strongest PPG data characteristics, and can determine a selected light-emitting element based on this determination.

[0106] Additionally, at least one processor of the server (100) can selectively drive a selected light-emitting element from among a plurality of light-emitting elements to emit light, and control the remaining light-emitting elements so that they are not driven. In this way, by controlling only some of the plurality of light-emitting elements to be driven, the energy consumption efficiency of the wearable device (200) can be improved.

[0107] In the above description, it has been described that the server (100) according to the embodiment of the present invention performs the functional operation as described above, but depending on the embodiment, at least a part of the functional operation performed by the server (100) may be performed by an external device (e.g., a wearable device (200), an electronic device (300), etc.), and at least a part of the functional operation performed by the external device may be further performed by the server (100), and various other embodiments may be possible.

[0108] -Wearable devices (200)

[0109] A wearable device (200) according to one embodiment may be an electronic device that can be worn on a user's body and can be linked with an application (311) that provides biofeedback content installed in an electronic device (300).

[0110] The wearable device (200) may take various forms, such as a smart watch, a smart band, or smart glasses. In addition, the wearable device (200) may include a hearable device, such as a bone conduction earphone (201) or a completely wireless earphone (202), as illustrated in FIG. 1.

[0111] Hereinafter, the wearable device (200) is described as being implemented as a bone conduction earphone (201), but the wearable device (200) can be implemented as any device that is connected to an electronic device (300) and can be worn on the user's body.

[0112] Referring to FIG. 3, from a functional perspective, the wearable device (200) may include a sensor unit (210), an input unit (220), an output unit (230), a battery (240), an interface unit (250), a storage unit (260), a communication unit (270), and / or a control unit (280). These components may be configured to be included, for example, within the housing of the wearable device (200).

[0113] The sensor unit (210) may include various types of biosensors, such as an optical sensor that senses PPG (Photoplethysmogram) data and a body temperature sensor. In addition, the sensor unit (210) may further include various sensors, such as a position sensor (IMU), an audio sensor, a distance sensor, a proximity sensor, and a contact sensor.

[0114] In one embodiment, the sensor unit (210), as shown in FIG. 4, may include an optical sensor (211) for collecting PPG data of a user wearing a wearable device (200).

[0115] The optical sensor (211) may be a sensor that measures the amount of blood flowing in peripheral blood vessels by irradiating green light or red light onto the user's skin using a green light source or a red light source, receiving light transmitted or reflected from the skin using a light-receiving element, and measuring the user's pulse or the like based on the light-receiving signal.

[0116] Referring to FIG. 4, the optical sensor (211) may include a light emitting unit (21) that irradiates light onto the user's body and a light receiving unit (22) that receives reflected light reflected from the user's body.

[0117] The light emitting unit (21) can irradiate light to a plurality of different regions (A1, A2, A3) of the user's body. For example, the light emitting unit (21) can irradiate a first light (L1) to a first region (A1) of the user's body, a second light (L2) to a second region (A2) of the user's body, and a third light (L3) to a third region (A3) of the user's body. Each of the plurality of different regions (A1, A2, A3) may be a region with a different blood vessel distribution within the user's body.

[0118] For example, the light emitting unit (21) may include a plurality of light emitting elements. Each of the plurality of light emitting elements may include an LED. The first light emitting element of the light emitting unit (21) may be controlled to irradiate a first light (L1) to a first area (A1), the second light emitting element of the light emitting unit (21) may be controlled to irradiate a second light (L2) to a second area (A2), and the third light emitting element of the light emitting unit (21) may be controlled to irradiate a third light (L3) to a third area (A3).

[0119] The light receiving unit (22) may include at least one light receiving element that receives a plurality of reflected lights (R1, R2, R3) reflected from a plurality of different areas (A1, A2, A3) of the user's body. Here, the plurality of reflected lights (R1, R2, R3) may include a first reflected light (R1) reflected from a first area (A1), a second reflected light (R2) reflected from a second area (A2), and a third reflected light (R3) reflected from a third area (A3).

[0120] At least one light-receiving element included in the light-receiving unit (22) may include a photodiode. At least one light-receiving element of the light-receiving unit (22) may generate a plurality of photocurrent signals (S1, S2, S3) based on a plurality of reflected lights (R1, R2, R3) reflected from a plurality of different regions (A1, A2, A3) of the user's body. The plurality of photocurrent signals (S1, S2, S3) may be transmitted to the control unit (280). The control unit (280) may transmit a driving control signal (C1) based on the plurality of photocurrent signals (S1, S2, S3) to the light-emitting unit (21), and the operation of the light-emitting unit (21) may be controlled based on the driving control signal (C1).

[0121] For example, referring to FIG. 5, a bone conduction earphone (201) may be worn on a user's ear, and in this case, a multi-channel (MC) composed of a plurality of light-emitting elements of an optical sensor (211) included in the bone conduction earphone (201) may be positioned to correspond to an area close to the user's ear.

[0122] Referring to FIG. 6, a first blood vessel (V1) and a second blood vessel (V2) may be formed to extend vertically around the user's ear (E). In addition, countless capillaries may be distributed around each of the first blood vessel (V1) and the second blood vessel (V2). In this case, the multi-channel (MC) of the bone conduction earphone (201) may be positioned to correspond to a portion of the first blood vessel (V1) and the second blood vessel (V2). For example, the first sub-channel (CH1) included in the multi-channel (MC) may correspond to a portion of the first blood vessel (V1), the second sub-channel (CH2) may correspond to a portion of the second blood vessel (V2), the third sub-channel (CH3) may correspond to another portion of the first blood vessel (V1), and the fourth sub-channel (CH4) may correspond to another portion of the second blood vessel (V2).

[0123] The position sensor (IMU) can detect at least one of the movement, acceleration, and / or tilt of the wearable device (200). For example, the position sensor (IMU) may be composed of a combination of various position sensors, such as an accelerometer, a gyroscope, and a magnetometer. Such a position sensor (IMU) may also be referred to as a motion sensor.

[0124] In detail, in one embodiment, the position sensor can measure the user's movement based on the acceleration sensor. Furthermore, in another embodiment, the position sensor can measure the user's posture by obtaining the difference in inclination between the user's left and right sides.

[0125] Additionally, the position sensor (IMU) can recognize spatial information about the physical space around the wearable device (200) in conjunction with the GPS of the communication unit (270).

[0126] The audio sensor can recognize sounds around the wearable device (200).

[0127] In detail, the audio sensor may include a microphone capable of detecting voice input from a user using the wearable device (200).

[0128] The distance sensor can measure the distance between the wearable device (200) and the electronic device (300).

[0129] The proximity sensor can detect an electronic device (300) and / or a user's body in proximity to the wearable device (200).

[0130] The contact sensor can detect an object and / or the user's body in contact with the wearable device (200).

[0131] That is, the wearable device (200) according to the embodiment can obtain sensing data including at least one of heart rate data, oxygen saturation data, location data, distance data, and / or posture data based on the sensor unit (210) including the plurality of sensors described above.

[0132] The input unit (220) can detect a user's input (e.g., a gesture, a voice command, a touch input, a button operation, or another type of input).

[0133] In detail, the input unit (220) may include a predetermined pressure sensor (e.g., a button) and / or a touch sensor for detecting a user's input.

[0134] Additionally, the input unit (220) may be configured in the form of a touch screen unit and / or a touch screen panel. Specifically, it may be configured in one of a resistive film method, an electrostatic capacitance method, an optical method, and an ultrasonic method, but it is preferable to use an electrostatic capacitance method.

[0135] Additionally, the input unit (220) can obtain a predetermined control signal for controlling the wearable device (200) and / or the electronic device (300) based on the touch sensor.

[0136] In detail, the input unit (220) can transmit a signal including the number of detected touches to the control unit (180). Accordingly, the control unit (180) can execute a predetermined process matched to the signal. For example, if the user inputs one short touch, a process for pausing playback while listening to music can be executed. Additionally, if the user inputs two short touches, a process for playing the next song while listening to music can be executed.

[0137] The output unit (230) may include a predetermined audio output device (hereinafter, speaker).

[0138] In detail, the output unit (230) may include an internal speaker that transmits sound to a user wearing the wearable device (200) and an external speaker that transmits sound to the outside of the earphones.

[0139] The internal speaker may provide sounds including biofeedback content and / or music. Additionally, the external speaker may provide sounds including a predetermined beep tone in case of loss.

[0140] Additionally, the output unit (230) may include a predetermined lighting and / or vibration module. For example, the lighting and / or vibration module may operate when a specific event, such as pairing and / or loss, occurs.

[0141] The battery (240) is implemented in the form of a rechargeable secondary battery and may include a wired charging module and / or a wireless charging module.

[0142] In detail, the battery (240) can supply a predetermined amount of power from the power supply of a digital device when connected to the digital device via a wire (e.g., USB cable, etc.) based on a wired charging module.

[0143] Additionally, the battery (240) can supply a predetermined amount of power from the power supply of a predetermined digital device when wirelessly connected to the digital device based on the wireless charging module.

[0144] The interface unit (250) can connect the wearable device (200) to one or more other devices so that they can communicate with each other. In detail, the interface unit (250) can include wired and / or wireless communication devices compatible with one or more different communication protocols.

[0145] Through this interface unit (250), the wearable device (200) can be connected to multiple input / output devices (e.g., electronic devices (300)).

[0146] The storage unit (260) can store one or more of various application programs, data, and commands for creating and providing a biofeedback content provision environment.

[0147] Additionally, the storage unit (260) may include a program area and a data area.

[0148] Here, the program area according to the embodiment may be linked between the operating system (OS) that boots the wearable device (200) and functional elements, and the data area may store data generated according to the use of the wearable device (200).

[0149] Additionally, the storage unit (260) may include at least one non-transitory computer-readable storage medium and one or more temporary computer-readable storage medium.

[0150] Additionally, the storage unit (260) can store certain sensing data obtained from the sensor unit (210).

[0151] The communication unit (270) may include one or more devices for communicating with external devices. This communication unit (270) may communicate via a wireless network.

[0152] In detail, the communication unit (270) can communicate with an electronic device (300) that stores a content source for implementing a biofeedback content provision environment, and can communicate with various user input components, such as a controller that receives user input.

[0153] In an embodiment, the communication unit (270) can transmit and receive various data (e.g., biosignals and / or sensing data) related to a biofeedback content provision service to and from another terminal and / or an external server (e.g., an electronic device (300) and / or a server (100)).

[0154] This communication unit (270) can wirelessly transmit and receive data with at least one of a base station, an external terminal, and an arbitrary server on a mobile communication network constructed through a communication device (e.g., a communication chip that performs Bluetooth communication) that can perform technical standards or communication methods for mobile communication (e.g., LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G NR (New Radio), WIFI) or short-range communication methods.

[0155] In an embodiment, the communication unit (270) can transmit and receive predetermined data with the electronic device (300) through short-range communication using Bluetooth.

[0156] The control unit (280) may include at least one processor capable of executing commands of an application stored in the electronic device (300) to perform various tasks for providing a biofeedback content provision environment.

[0157] Additionally, in the embodiment, the control unit (280) can control the overall operation of the components of the wearable device (200) to provide a biofeedback providing environment.

[0158] In detail, in an embodiment, the control unit (280) may include an embedded processor that connects the optical sensor (211) of the sensor unit (210) to the analog front-end and then controls an analog-to-digital converter (ADC) to obtain a PPG signal.

[0159] This control unit (280) may be a system on chip (SOC) suitable for the wearable device (200), and may execute an operating system (OS) and / or application programs stored in the storage unit (260), and control each component mounted on the wearable device (200).

[0160] In addition, the control unit (280) can communicate with each component internally via a system bus and can include one or more predetermined bus structures including a local bus.

[0161] In addition, the control unit (280) may be implemented by including at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0162] In addition, the control unit (280) can control the wearable device (200) by exchanging data with the electronic device (300) in response to signals received from the sensor unit (210) and / or the input unit (220).

[0163] A wearable device (200) including the above-described components can transmit sensing data including at least one or more of biometric data such as heart rate data and oxygen saturation data and at least one or more of location data, distance data, and / or posture data to an electronic device (300) according to an embodiment, and such various types of data can be stored in a memory (310) of the electronic device (300).

[0164] -Electronic devices (300)

[0165] An electronic device (300) according to one embodiment may be a computing device having an application (311) installed thereon that provides a bio-data acquisition environment and bio-feedback content.

[0166] In detail, from a hardware perspective, the electronic device (300) may include a mobile type computing device and / or a desktop type computing device on which an application is installed.

[0167] In the embodiment, the user is a user who carries a wearable device (200) and an electronic device (300) and exercises. For convenience of explanation, the electronic device (300) is described below as a mobile type computing device.

[0168] Here, the mobile type computing device may be a mobile device such as a smart phone or tablet PC on which an application is installed.

[0169] For example, mobile type computing devices may include smart phones, mobile phones, digital broadcasting devices, personal digital assistants (PDAs), portable multimedia players (PMPs), tablet PCs, etc.

[0170] Additionally, according to an embodiment, the electronic device (300) may further include a predetermined server computing device that provides a biofeedback content provision environment.

[0171] Referring to FIG. 7, from a functional perspective, the electronic device (300) may include a memory (310), a processor assembly (320), a communication module (330), an interface module (340), an input system (350), a sensor system (360), and a display system (370). These components may be configured to be included within the housing of the electronic device (300).

[0172] In detail, in the memory (310), an application (311) is stored, and the application (311) can store one or more of various application programs, data, and commands for providing a biofeedback content provision environment.

[0173] Additionally, the memory (310) may include a program area and a data area.

[0174] Here, the program area according to the embodiment may be linked between the operating system (OS) that boots the electronic device (300) and functional elements, and the data area may store data generated according to the use of the electronic device (300).

[0175] Additionally, the memory (310) may include at least one non-transitory computer-readable storage medium and one or more temporary computer-readable storage medium.

[0176] For example, the memory (310) may be a variety of storage devices such as a ROM, EPROM, flash drive, hard drive, etc., and may include web storage that performs the storage function of the memory (310) on the Internet.

[0177] The processor assembly (320) may include at least one processor capable of executing instructions of an application (311) stored in the memory (310) to perform various tasks for creating a biofeedback content provision environment.

[0178] In an embodiment, the processor assembly (320) can control the overall operation of the components through an application (311) of the memory (310) to provide a biofeedback content provision environment.

[0179] The processor assembly (320) can execute an application (311) to provide the user with biofeedback content that guides a customized exercise program based on various types of biometric data, such as the user's heart rate and body temperature, based on the user's data measured by the sensor unit (210) included in the wearable device (200).

[0180] Here, biofeedback content according to one embodiment may mean predetermined content generated by an electronic device (300) to guide customized exercise according to the user's condition based on user data such as body temperature data, PPG data, and movement data based on data sensed in real time through a sensor unit (210) of a wearable device (200).

[0181] In one embodiment, such biofeedback content may include audio coaching content (hereinafter, audio coaching) output to a wearable device (200) and visual coaching content output to an electronic device (300).

[0182] For example, in one embodiment, data of biofeedback content generated by the processor assembly (320) may be transmitted to the wearable device (200) through the communication module (330), and the control unit (280) of the wearable device (200) may control audio coaching content to be output through the output unit (230) based on the data of the received biofeedback content.

[0183] Accordingly, by outputting audio coaching content as sound by the wearable device (200) according to one embodiment, the wearable device (200) can provide biofeedback content to the user based on the user's sensing data acquired in real time.

[0184] The visual coaching content output to the electronic device (300) may be content displayed through the display system (370) of the electronic device (300).

[0185] The processor assembly (320) may be a system on chip (SOC) suitable for an electronic device (300) including a central processing unit (CPU) and / or a graphics processing unit (GPU), and may execute an operating system (OS) and / or application programs stored in a memory (310) and control each component mounted on the electronic device (300).

[0186] Additionally, the processor assembly (320) may communicate with each component internally via a system bus and may include one or more predetermined bus structures including a local bus.

[0187] Additionally, the processor assembly (320) may be implemented by including at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0188] The communication module (330) may include one or more devices for communicating with external devices. The communication module (330) may communicate via a wireless network.

[0189] In detail, the communication module (330) can communicate with a server (100) that stores a content source for implementing a biofeedback content provision environment, and can communicate with various user input components, such as a controller that receives user input.

[0190] In an embodiment, the communication module (330) can transmit and receive various data related to the biofeedback content provision environment to and from other electronic devices and / or external servers.

[0191] These communication modules (330) can wirelessly transmit and receive data with at least one of a base station, an external terminal, and an arbitrary server on a mobile communication network constructed through a communication device capable of performing technical standards or communication methods for mobile communication (e.g., LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G NR (New Radio), WIFI) or short-range communication methods.

[0192] The interface module (340) can communicatively connect the electronic device (300) to one or more other devices. In detail, the interface module (340) can include wired and / or wireless communication devices compatible with one or more different communication protocols.

[0193] Through this interface module (340), the electronic device (300) can be connected to multiple input / output devices.

[0194] For example, the interface module (340) can be connected to an audio output device such as a headset port or speaker to output audio.

[0195] As an example, the audio output device is described as being connected via an interface module (340), but an embodiment in which it is installed inside an electronic device (300) may also be included.

[0196] Additionally, for example, the interface module (340) may be connected to an input device such as a keyboard and / or mouse to obtain user input.

[0197] Such an interface module (340) may be configured to include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, an earphone port, a power amplifier, an RF circuit, a transceiver, and other communication circuits.

[0198] The input system (350) can detect user input (e.g., a gesture, a voice command, the operation of a button, or other type of input).

[0199] In detail, the input system (350) may include a predetermined button, a touch sensor, and / or an image sensor that receives user motion input.

[0200] Additionally, the input system (350) can be connected to an external controller through an interface module (340) to receive user input.

[0201] The sensor system (360) may include an image sensor (361), a position sensor (IMU, 363), and an audio sensor (365). In addition, the sensor system (360) may further include various sensors such as a distance sensor, a proximity sensor, and a contact sensor.

[0202] Here, the image sensor (361) can capture images and / or videos of the physical space around the electronic device (300).

[0203] The image sensor (361) can capture an image by photographing the direction in which it is positioned on the front or / and back of the electronic device (300), and can capture a physical space through a camera positioned toward the outside of the electronic device (300).

[0204] This image sensor (361) may include an image sensor device and an image processing module. In detail, the image sensor (361) may process still images or moving images obtained by an image sensor device (e.g., CMOS or CCD).

[0205] The position sensor (IMU, 363) can detect at least one of the movement and acceleration of the electronic device (300). For example, it can be formed by a combination of various position sensors such as an accelerometer, a gyroscope, and a magnetometer. Such a position sensor (IMU) may be referred to as a motion sensor hereinafter.

[0206] Additionally, the position sensor (IMU, 363) can recognize spatial information about the physical space around the electronic device (300) in conjunction with the GPS of the communication module (330).

[0207] The audio sensor (365) can recognize sounds around the electronic device (300).

[0208] In detail, the audio sensor (365) may include a microphone capable of detecting voice input from a user using the electronic device (300).

[0209] The display system (370) can output various information related to the biofeedback content provision environment as graphic images.

[0210] As an example, the display system (370) can display various user interfaces (as an example, a membership registration interface) for a profit environment.

[0211] Such displays may include at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, and an e-ink display.

[0212] The components may be arranged within the housing of such an electronic device (300), and the user interface may include a touch sensor (373) on a display (371) configured to receive user touch input.

[0213] In detail, the display system (370) may include a display (371) that outputs an image and a touch sensor (373) that detects a user's touch input.

[0214] For example, the display (371) may be implemented as a touch screen by forming a mutual layer structure with the touch sensor (373) or forming an integral structure. Such a touch screen may function as a user input unit that provides an input interface between the electronic device (300) and the user, and at the same time, provide an output interface between the electronic device (300) and the user.

[0215] An electronic device (300) including the above-described components may store at least one sensing data, user body information, user status information, user exercise information, user exercise ability, and / or exercise program in a memory (310) according to an embodiment.

[0216] -Method for obtaining biometric data (S100)

[0217] Referring to FIG. 8, a method (S100) for obtaining biometric data using an optical device according to one embodiment may include a step (S101) of irradiating light to a plurality of different regions (A1, A2, A3) of a user's body and receiving a plurality of reflected lights (R1, R2, R3) reflected from each of the plurality of different regions (A1, A2, A3) of the user's body, a step (S103) of determining a photocurrent signal with the smallest noise and a photocurrent signal with the largest noise based on a plurality of photocurrent signals (S1, S2, S3) by the plurality of reflected lights (R1, R2, R3), and a step (S105) of obtaining PPG data based on the photocurrent signals obtained by performing a predetermined operation on the photocurrent signal with the smallest noise and the photocurrent signal with the largest noise.

[0218] The method (S100) for obtaining biometric data using an optical device may be performed by at least one processor included in the server (100). However, the present invention is not limited thereto, and at least a part of the method (S100) may be performed by the control unit (280) of the wearable device (200) or the processor assembly (320) of the electronic device (300), and another part may be performed by at least one processor of the server (100).

[0219] For example, at least one processor included in the server (100), the control unit (280) of the wearable device (200), and at least one processor assembly (320) included in the electronic device (300) may execute at least one command stored in the memory included in the server (100), the storage unit (260) of the wearable device (200), or the memory (310) of the electronic device (300), thereby performing the method (S100) for obtaining biometric data using an optical device.

[0220] Hereinafter, it is described that at least one processor of the server (100) performs the method (S100).

[0221] In step (S101), at least one processor of the server (100) can control the optical sensor (211) of the wearable device (200) to irradiate light to a plurality of different areas (A1, A2, A3) of the user's body. For example, referring to FIG. 4, a first light (L1) can be irradiated from the light emitting unit (21) to a first area (A1) of the user's body, a second light (L2) can be irradiated to a second area (A2) of the user's body, and a third light (L3) can be irradiated to a third area (A3) of the user's body.

[0222] FIG. 4 illustrates irradiating first to third lights (L1, L2, L3) to three areas (A1, A2, A3) of the user's body, but is not limited thereto, and the optical sensor (211) may be controlled to irradiate light to two or fewer areas or four or more areas of the user's body.

[0223] For example, the light emitting unit (21) may include a plurality of light emitting elements. Each of the plurality of light emitting elements may include an LED. In step (S101), the first light emitting element of the light emitting unit (21) may be controlled to irradiate a first light (L1) to a first area (A1), the second light emitting element of the light emitting unit (21) may be controlled to irradiate a second light (L2) to a second area (A2), and the third light emitting element of the light emitting unit (21) may be controlled to irradiate a third light (L3) to a third area (A3).

[0224] In addition, in step (S101), the light receiving unit (22) of the wearable device (200) can receive a plurality of reflected lights (R1, R2, R3) reflected from a plurality of different areas (A1, A2, A3) of the user's body. Here, the plurality of reflected lights (R1, R2, R3) can include a first reflected light (R1) reflected from a first area (A1), a second reflected light (R2) reflected from a second area (A2), and a third reflected light (R3) reflected from a third area (A3).

[0225] Furthermore, in step (S101), a plurality of photocurrent signals (S1, S2, S3) can be generated based on a plurality of reflected lights (R1, R2, R3) reflected from a plurality of different areas (A1, A2, A3) of the user's body by the light receiving unit (22), and the plurality of photocurrent signals (S1, S2, S3) can be transmitted to the control unit (280) of the wearable device (200), at least one processor of the server (100), and the processor assembly (320) of the electronic device (300).

[0226] The multiple photocurrent signals (S1, S2, S3) may have different waveforms. For example, referring to FIG. 9, the first photocurrent signal (S1) may have a waveform of a periodic pattern in which the characteristics of the PPG signal are relatively clearly visible, the second photocurrent signal (S2) may have a waveform of a periodic pattern in which the characteristics of the PPG signal are mixed with noise, and the third photocurrent signal (S3) may have a waveform in which the characteristics of the PPG signal are hardly visible and corresponds to noise of an aperiodic pattern.

[0227] In step (S103), at least one processor of the server (100) can analyze a plurality of photocurrent signals (S1, S2, S3) to determine a photocurrent signal with the least noise and a photocurrent signal with the greatest noise. Here, the photocurrent signal with the least noise means a signal with the strongest PPG data characteristics, and noise may mean a motion artifact caused by the user's movement.

[0228] At least one processor of the server (100) can convert a plurality of photocurrent signals (S1, S2, S3) into a plurality of frequency signals, and through analysis of the plurality of frequency signals, can determine a photocurrent signal with the least noise and a photocurrent signal with the greatest noise.

[0229] For example, referring to FIGS. 10 and 11, step (S103) may include a step (S1031) of frequency-converting a plurality of photocurrent signals (S1, S2, S3) by a plurality of reflected lights (R1, R2, R3) to obtain a plurality of frequency signals (FS1, FS2, FS3), a step (S1033) of determining a selected frequency range having a maximum power spectral density (PSD) for each of the plurality of frequency signals (FS1, FS2, FS3), and a step (S1035) of determining a frequency signal having a largest PSD in the selected frequency range and a frequency signal having a smallest PSD in the selected frequency range among the plurality of frequency signals (FS1, FS2, FS3).

[0230] In step (S1031), at least one processor of the server (100) may frequency convert the plurality of photocurrent signals (S1, S2, S3) to obtain a plurality of frequency signals (FS1, FS2, FS3). For example, at least one processor of the server (100) may perform a fast Fourier transform (FFT) on the plurality of photocurrent signals (S1, S2, S3).

[0231] Referring to FIG. 11, for example, a first photocurrent signal (S1) may be frequency converted to generate a first frequency signal (FS1), a second photocurrent signal (S2) may be frequency converted to generate a second frequency signal (FS2), and a third photocurrent signal (S3) may be frequency converted to generate a third frequency signal (FS3).

[0232] Each of the first to third frequency signals (FS1, FS2, FS3) may have different waveforms depending on the characteristics of the areas where the plurality of reflected lights (R1, R2, R3) that form the basis of the first to third frequency signals (FS1, FS2, FS3) are reflected.

[0233] For example, some of the first to third frequency signals (FS1, FS2, FS3) may exhibit signals of relatively strong intensity in a PPG frequency range corresponding to PPG data, others may exhibit signals of relatively strong intensity in an oxygen saturation frequency range corresponding to oxygen saturation data, and still others may exhibit signals of relatively strong intensity in a noise frequency corresponding to noise.

[0234] In step (S1033), at least one processor of the server (100) can determine a selected frequency range having a maximum PSD for each of the plurality of frequency signals (FS1, FS2, FS3). Here, the selected frequency range may mean a predetermined frequency band or a specific frequency. Hereinafter, a case in which the selected frequency range is a predetermined frequency band will be described first, and a case in which the selected frequency range means a specific frequency will be described.

[0235] Referring to (a) of FIG. 11, for example, at least one processor of the server (100) may determine a selection frequency region having a maximum PSD for the first frequency signal (FS1) as the region w1 to w2. That is, at least one processor of the server (100) may determine the region w1 to w2, which is a specific frequency region having a maximum graph area (K1) in the graph of FIG. 11 (a), as a selection frequency region for the first frequency signal (FS1).

[0236] Referring to (b) of FIG. 11, for example, at least one processor of the server (100) may determine a selection frequency region having a maximum PSD for the second frequency signal (FS2) as the region w1 to w2. That is, at least one processor of the server (100) may determine the region w1 to w2, which is a specific frequency region having a maximum graph area (K2) in the graph of FIG. 11 (b), as a selection frequency region for the second frequency signal (FS2).

[0237] Referring to (c) of FIG. 11, for example, at least one processor of the server (100) may determine the selected frequency region having the maximum PSD for the third frequency signal (FS3) as the region w3 to w4. That is, at least one processor of the server (100) may determine the region w3 to w4, which is a specific frequency region having the maximum graph area (K3) in the graph of FIG. 11 (c), as the selected frequency region for the third frequency signal (FS3).

[0238] Here, the selected frequency ranges of the first frequency signal (FS1) and the second frequency signal (FS2) may be the same as the w1 to w2 range. In addition, the w3 to w4 range, which is the selected frequency range of the third frequency signal (FS3), may be different from the w1 to w2 range, which is the selected frequency range of the first frequency signal (FS1) and the second frequency signal (FS2).

[0239] This may mean that even if the waveforms of the first frequency signal (FS1) and the second frequency signal (FS2) are different, their main frequency characteristics may be similar. For example, although the first frequency signal (FS1) and the second frequency signal (FS2) have somewhat different waveforms, both signals may have the characteristics of a PPG signal as their main characteristics.

[0240] Additionally, the third frequency signal (FS3) may have the characteristics of a motion artifact signal rather than the characteristics of a PPG signal as its main characteristic.

[0241] Meanwhile, in step (S1033), at least one processor of the server (100) can determine a selection frequency having a maximum intensity value for each of the plurality of frequency signals (FS1, FS2, FS3).

[0242] Referring to (a) of FIG. 11, for example, at least one processor of the server (100) can determine a selection frequency having a maximum intensity value for the first frequency signal (FS1) in the region w1 to w2, which is a selection frequency region having a maximum PSD. That is, at least one processor of the server (100) can determine a specific frequency as a selection frequency in the region w1 to w2, which is a selection frequency region including a maximum intensity value in the graph of (a) of FIG. 11.

[0243] Referring to (b) of FIG. 11, for example, at least one processor of the server (100) can determine a selection frequency having a maximum intensity value for the second frequency signal (FS2) in the region w1 to w2, which is a selection frequency region having a maximum PSD. That is, at least one processor of the server (100) can determine a specific frequency as a selection frequency in the region w1 to w2, which is a selection frequency region including a maximum intensity value in the graph of (b) of FIG. 11.

[0244] Referring to (c) of FIG. 11, for example, at least one processor of the server (100) can determine a selection frequency having a maximum intensity value for the third frequency signal (FS3) in the region w3 to w4, which is a selection frequency region having a maximum PSD. That is, at least one processor of the server (100) can determine a specific frequency as a selection frequency in the region w3 to w4, which is a selection frequency region including a maximum intensity value in the graph (c) of FIG. 11.

[0245] In step (S1035), at least one processor of the server (100) can determine a frequency signal having the largest PSD in a selected frequency range and a frequency signal having the smallest PSD in a selected frequency range among a plurality of frequency signals (FS1, FS2, FS3).

[0246] Here, the frequency signal with the largest PSD in the selected frequency range may be the frequency signal with the strongest PPG data characteristics among the multiple frequency signals. Furthermore, the frequency signal with the smallest PSD in the selected frequency range may be the frequency signal with the weakest PPG data characteristics and the strongest noise characteristics among the multiple frequency signals.

[0247] That is, the frequency signal with the largest PSD in the selected frequency range can correspond to the photocurrent signal with the smallest noise, and the frequency signal with the smallest PSD in the selected frequency range can correspond to the photocurrent signal with the largest noise.

[0248] For example, at least one processor of the server (100) can compare the PSD in the selected frequency range (w1 to w2 range) of the first frequency signal (FS1), the PSD in the selected frequency range (w1 to w2 range) of the second frequency signal (FS2), and the PSD in the selected frequency range (w3 to w4 range) of the third frequency signal (FS3).

[0249] In this case, for example, at least one processor of the server (100) may determine that the PSD in the selected frequency range (w1 to w2 range) of the first frequency signal (FS1) is the largest, and the PSD in the selected frequency range (w3 to w4 range) of the third frequency signal (FS3) is the smallest.

[0250] Accordingly, at least one processor of the server (100) can determine the first frequency signal (FS1) as the frequency signal with the largest PSD in the selected frequency range (w1 to w2 range), and determine the third frequency signal (FS3) as the frequency signal with the smallest PSD in the selected frequency range (w3 to w4 range).

[0251] Here, the first frequency signal (FS1), which is the frequency signal with the largest PSD in the selected frequency range, may be the frequency signal with the strongest PPG data characteristic among the plurality of frequency signals. In addition, the third frequency signal (FS3), which is the frequency signal with the smallest PSD in the selected frequency range, may be the frequency signal with the weakest PPG data characteristic and the strongest noise characteristic among the plurality of frequency signals.

[0252] Meanwhile, in step (S1035), at least one processor of the server (100) can determine a frequency signal having the largest intensity value at the selected frequency and a frequency signal having the smallest intensity value at the selected frequency among a plurality of frequency signals (FS1, FS2, FS3).

[0253] For example, at least one processor of the server (100) can compare the intensity value at the selected frequency of the first frequency signal (FS1), the intensity value at the selected frequency of the second frequency signal (FS2), and the intensity value at the selected frequency of the third frequency signal (FS3).

[0254] In this case, for example, at least one processor of the server (100) may determine that the intensity value at the selected frequency of the first frequency signal (FS1) is the largest, and the intensity value at the selected frequency of the third frequency signal (FS3) is the smallest.

[0255] Accordingly, at least one processor of the server (100) can determine the first frequency signal (FS1) as the frequency signal having the largest intensity value at the selected frequency, and determine the third frequency signal (FS3) as the frequency signal having the smallest intensity value at the selected frequency.

[0256] In step (S105), at least one processor of the server (100) can obtain PPG data based on the obtained photocurrent signal by performing a predetermined operation on the photocurrent signal with the smallest noise and the photocurrent signal with the largest noise.

[0257] For example, at least one processor of the server (100) can obtain a target photocurrent signal by performing a predetermined operation on a frequency signal having the largest PSD in the selected frequency range determined in step (S103) and a frequency signal having the smallest PSD in the selected frequency range.

[0258] For example, referring to FIG. 12, step (S105) may include a step (S1051) of obtaining a noise frequency signal by removing a signal corresponding to a selected frequency range of a frequency signal having the largest PSD in the selected frequency range from a frequency signal having the smallest PSD in the selected frequency range, a step (S1053) of obtaining a reference photocurrent signal by inversely frequency-converting the frequency signal having the largest PSD in the selected frequency range, and a step (S1054) of obtaining a noise photocurrent signal by inversely frequency-converting the noise frequency signal, and a step (S1055) of obtaining PPG data based on a target photocurrent signal obtained by subtracting the noise photocurrent signal from the reference photocurrent signal.

[0259] In step (S1051), for example, at least one processor of the server (100) can obtain a noise frequency signal by removing a signal corresponding to the selected frequency range of the first frequency signal (FS1) determined as the frequency signal with the largest PSD in the selected frequency range from the third frequency signal (FS3) determined as the frequency signal with the smallest PSD in the selected frequency range in step (S103).

[0260] For example, referring to FIG. 13, in step (S1051), at least one processor of the server (100) can obtain a noise frequency signal (NS1) by removing a signal corresponding to a selected frequency range of the first frequency signal (FS1) from the third frequency signal (FS3) to the w1 to w2 range.

[0261] In this case, at least one processor of the server (100) can obtain a noise frequency signal (NS1) by performing filtering to remove a signal corresponding to the w1 to w2 region for the third frequency signal (FS3). For the filtering, a filter circuit capable of filtering a portion of a frequency signal converted from a photocurrent signal from an optical sensor (211) can be utilized.

[0262] In step (S1053), at least one processor of the server (100) may perform inverse frequency transformation on a frequency signal having the largest PSD in a selected frequency range to obtain a reference photocurrent signal, and may perform inverse frequency transformation on a noise frequency signal to obtain a noise photocurrent signal. For example, at least one processor of the server (100) may perform inverse FFT transformation on a frequency signal having the largest PSD in a selected frequency range and a noise frequency signal.

[0263] For example, at least one processor of the server (100) may obtain a reference photocurrent signal by inversely frequency-converting the first frequency signal (FS1) determined as the frequency signal with the largest PSD in the selected frequency range in step (S103). In addition, at least one processor of the server (100) may obtain a noise photocurrent signal by inversely frequency-converting the noise frequency signal (NS1) obtained in step (S1051).

[0264] For example, referring to FIG. 14, a first frequency signal (FS1) may be inversely converted into a frequency by a signal conversion unit (11) of at least one processor of a server (100) to obtain a reference photocurrent signal, and a noise frequency signal (NS1) may be inversely converted into a frequency by a noise photocurrent signal.

[0265] In step (S1055), at least one processor of the server (100) can obtain PPG data based on a target photocurrent signal obtained by subtracting a noise photocurrent signal from a reference photocurrent signal.

[0266] For example, referring to FIG. 14, a noise photocurrent signal can be subtracted from a reference photocurrent signal by a signal subtraction unit (15) of at least one processor of a server (100). Accordingly, a target photocurrent signal with a weakened noise characteristic can be obtained from the reference photocurrent signal.

[0267] Additionally, referring to FIG. 14, PPG data can be acquired based on a target photocurrent signal by a PPG data extraction unit (16) of at least one processor of a server (100).

[0268] Meanwhile, the method (S100) may further include a step of ensemble processing each of the plurality of photocurrent signals by the plurality of reflected lights before step (S103).

[0269] For example, referring to FIG. 15, at least one processor of the server (100) can obtain a first ensemble photocurrent signal (E1) by adding a plurality of sub-photocurrent signals (Sub1, Sub2, Sub3) obtained by dividing the first photocurrent signal (S1) into regular periods.

[0270] The first ensemble photocurrent signal (E1) may be a signal in which the main characteristics of the first photocurrent signal (S1) are further enhanced. For example, the first ensemble photocurrent signal (E1) may be a signal in which the PPG data characteristics of the first photocurrent signal (S1) are further enhanced.

[0271] Similarly, at least one processor of the server (100) can ensemble process each of the second photocurrent signal (S2) and the third photocurrent signal (S3).

[0272] In this case, in step (S103), a photocurrent signal with the smallest noise and a photocurrent signal with the largest noise can be determined based on a plurality of ensemble-processed photocurrent signals.

[0273] In addition, the method (S100) may further include a step of normalizing each of the ensemble-processed plurality of photocurrent signals after the step of ensemble-processing each of the plurality of photocurrent signals by the plurality of reflected lights. In this case, in step (S103), a series of processes (S1031, S1033, S1035) are performed on the normalized plurality of photocurrent signals, thereby determining the photocurrent signal with the least noise and the photocurrent signal with the greatest noise.

[0274] The embodiments of the present invention described above may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. Hardware devices may be changed into one or more software modules to perform processing according to the present invention, and vice versa.

[0275] The specific implementations described in the present invention are exemplary embodiments and do not limit the scope of the present invention in any way. For the sake of brevity, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the lines connecting or connecting members between components illustrated in the drawings are merely representative of functional connections and / or physical or circuit connections, and may be replaced or represented as various additional functional connections, physical connections, or circuit connections in an actual device. In addition, unless specifically mentioned as “essential,” “important,” etc., a component may not be absolutely necessary for the application of the present invention.

[0276] Although the detailed description of the present invention has been described with reference to preferred embodiments of the present invention, it will be understood by those skilled in the art or having ordinary knowledge in the art that various modifications and changes can be made to the present invention without departing from the spirit and technical scope of the present invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

[0277] The present invention has industrial applicability in that it can provide a wearable device that can provide more accurate bio-information by utilizing PPG data with reduced noise due to reduced motion artifacts by performing a predetermined operation on a plurality of photocurrent signals from an optical sensor including a plurality of optical channels.

Claims

1. An optical sensor that irradiates light to multiple different areas of the user's body and receives multiple reflected lights reflected from multiple different areas of the user's body; and A control unit that performs an operation to obtain PPG data based on a plurality of photocurrent signals by the plurality of reflected lights; The above control unit, Based on the plurality of photocurrent signals by the plurality of reflected lights, the photocurrent signal with the smallest noise and the photocurrent signal with the largest noise are determined, A wearable device that obtains PPG data based on a photocurrent signal obtained by performing a predetermined operation on a photocurrent signal with the smallest noise and a photocurrent signal with the largest noise.

2. In paragraph 1, The above control unit, A wearable device that obtains PPG data based on a photocurrent signal obtained by subtracting a photocurrent signal with the greatest noise from a photocurrent signal with the smallest noise.

3. In paragraph 1, The above control unit, Remove the signal corresponding to the PPG frequency range from the photocurrent signal with the largest noise, A wearable device that obtains PPG data based on a photocurrent signal obtained by subtracting a photocurrent signal having the smallest noise from a photocurrent signal having the largest noise, thereby removing a signal corresponding to the PPG frequency range.

4. In paragraph 1, The above control unit, By frequency converting a plurality of photocurrent signals by the plurality of reflected lights, a plurality of frequency signals are obtained, For each of the plurality of frequency signals, a selected frequency range having a maximum power spectral density (PSD) is determined, A wearable device that determines a frequency signal having the largest PSD in the selected frequency range and a frequency signal having the smallest PSD in the selected frequency range among the plurality of frequency signals.

5. In paragraph 4, The above control unit, A noise frequency signal is obtained by removing a signal corresponding to a selected frequency range of a frequency signal having the largest PSD in the selected frequency range from a frequency signal having the smallest PSD in the selected frequency range, A reference photocurrent signal is obtained by inversely converting the frequency signal having the largest PSD in the above-mentioned selected frequency range, and a noise photocurrent signal is obtained by inversely converting the noise frequency signal. A wearable device that obtains PPG data based on a target photocurrent signal obtained by subtracting the noise photocurrent signal from the reference photocurrent signal.

6. In paragraph 1, The above control unit, Ensemble processing of each of the plurality of photocurrent signals by the plurality of reflected lights, A wearable device that determines a photocurrent signal with the smallest noise and a photocurrent signal with the largest noise based on a plurality of ensemble-processed photocurrent signals.

7. In paragraph 6, The above control unit, Normalizing each of the plurality of photocurrent signals processed in the above ensemble, A wearable device that determines a photocurrent signal with the smallest noise and a photocurrent signal with the largest noise based on the normalized plurality of photocurrent signals.

8. In paragraph 1, The optical sensor includes a plurality of light-emitting elements corresponding to the plurality of areas, The above control unit, A wearable device that determines a selected light-emitting element that emits light that is the basis of a photocurrent signal with the smallest noise among the plurality of light-emitting elements, and selectively drives the selected light-emitting element among the plurality of light-emitting elements.

9. A step of irradiating light to multiple different areas of the user's body and receiving multiple reflected lights reflected from each of the multiple different areas of the user's body; A step of determining a photocurrent signal with the smallest noise and a photocurrent signal with the largest noise based on a plurality of photocurrent signals by the plurality of reflected lights; and A method for obtaining biometric data using an optical device, comprising: a step of obtaining PPG data based on a photocurrent signal obtained by performing a predetermined operation on a photocurrent signal with the smallest noise and a photocurrent signal with the largest noise; 10. In paragraph 9, The step of determining the photocurrent signal with the smallest noise and the photocurrent signal with the largest noise is: A step of obtaining a plurality of frequency signals by frequency converting a plurality of photocurrent signals by the plurality of reflected lights; A step of determining a selected frequency range having a maximum power spectral density (PSD) for each of the plurality of frequency signals; and A method for obtaining biometric data using an optical device, comprising: a step of determining a frequency signal having the largest PSD in the selected frequency range and a frequency signal having the smallest PSD in the selected frequency range among the plurality of frequency signals; 11. In paragraph 10, The step of acquiring the above PPG data is: A step of obtaining a noise frequency signal by removing a signal corresponding to a selected frequency range of a frequency signal having the largest PSD in the selected frequency range from a frequency signal having the smallest PSD in the selected frequency range; A step of obtaining a reference photocurrent signal by inversely converting the frequency signal having the largest PSD in the selected frequency range, and obtaining a noise photocurrent signal by inversely converting the noise frequency signal; and A method for obtaining biometric data using an optical device, comprising: a step of obtaining PPG data based on a target photocurrent signal obtained by subtracting the noise photocurrent signal from the reference photocurrent signal;

Images