Optical-type skin detection-based biometric data acquisition method, optical-type skin detection-based biometric data sensor module, and wearable device including optical-type skin detection-based biometric data sensor module

Abstract

An embodiment provides an optical-type skin detection-based biometric data acquisition method including the steps of: irradiating an object with first output light of a first wavelength range and second output light of a second wavelength range different from the first wavelength range by using a light-emitting unit; receiving first reflected light and second reflected light generated from the first output light and the second output light reflected from the object, respectively, by using a light receiving unit; determining the proximity of the object on the basis of first reflected light data for the first reflected light; determining whether the object is a target skin on the basis of second reflected light data for the second reflected light when it is determined that the object is close; and obtaining biometric data for the target skin by processing the second reflected light data when it is determined that the object is the target skin.

Description

A method for acquiring biometric data based on optical skin detection, a biometric data sensor module based on optical skin detection, and a wearable device including a biometric data sensor module based on optical skin detection

[0001] The present disclosure relates to an optical skin detection-based biometric data acquisition method, an optical skin detection-based biometric data sensor module, and a wearable device including the optical skin detection-based biometric data sensor module, and more particularly, to an optical skin detection-based biometric data acquisition method, an optical skin detection-based biometric data sensor module, and a wearable device including the optical skin detection-based biometric data sensor module, which quickly detect an object using light of a first wavelength range and determine whether the detected object is skin using light of a second wavelength range.

[0002] As modern people's interest in health grows, the healthcare industry is developing, and with it, the demand for wearable devices that can naturally monitor health conditions is increasing.

[0003] Monitoring a user's health using wearable devices requires highly sensitive sensors capable of sensing a variety of biometric data. Sensors applicable to wearable devices include contact electrode sensors, optical sensors, and temperature sensors.

[0004] In particular, optical sensors that can sense health-related data such as heart rate, oxygen saturation, and sleep quality in real time by irradiating light on human skin and sensing the reflected light are becoming increasingly important in terms of utilizing wearable devices.

[0005] Optical sensors embedded in wearable devices must make good contact with the skin to effectively illuminate it with light. If light is emitted from the optical sensor without proper contact, the light cannot reach the skin, reducing the sensing efficiency of biometric data.

[0006] To prevent these problems, wearable devices require a device that detects contact with the skin. Conventional methods for detecting skin include a method using current flow between electrodes, a method of flowing microcurrent through the human body, a method of measuring impedance changes in internal circuits, and a method of detecting microscopic pressure applied to the skin.

[0007] However, according to the conventional method, even when an object other than a human body is in contact, the contact area is recognized as a human body, and problems such as the optical sensor being continuously activated even when the user is not using it, causing the battery to discharge, etc. occurred.

[0008] Meanwhile, hearable devices (a portmanteau of hear and wearable, meaning wearable devices focused on hearing) that have been in the spotlight recently are significantly smaller than the most common wearable devices, such as smartwatches or VR devices, and because they necessarily include an audio output device, there are limitations to additionally installing a separate skin detection sensor in addition to the heart rate sensor.

[0009] According to various embodiments of the present disclosure, there is provided an optical skin detection-based biometric data acquisition method capable of determining whether an object is close and whether the object is skin using a single optical sensor, an optical skin detection-based biometric data sensor module, and a wearable device including the optical skin detection-based biometric data sensor module.

[0010] According to various embodiments of the present disclosure, there is provided an optical skin detection-based biometric data acquisition method capable of quickly determining whether an object is close and determining whether the close object is a target skin, an optical skin detection-based biometric data sensor module, and a wearable device including the optical skin detection-based biometric data sensor module.

[0011] However, the technical problems to be solved by the present invention and embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

[0012] One example is,

[0013] A method for obtaining biometric data using a sensor including a light-emitting unit and a light-receiving unit is provided, comprising: a step of irradiating an object with a first output light of a first wavelength range and a second output light of a second wavelength range different from the first wavelength range using the light-emitting unit; a step of receiving a first reflection light and a second reflection light, which are respectively reflected from the object by the first output light and the second output light, using the light-receiving unit; a step of determining whether the object is in proximity based on first reflection light data for the first reflection light; a step of determining whether the object is target skin based on second reflection light data for the second reflection light when the object is determined to be in proximity; and a step of obtaining biometric data for the target skin by processing the second reflection light data when the object is determined to be the target skin.

[0014] In another aspect, the step of determining whether the object is close may include the step of generating data on the rate of change in the intensity of the first reflected light based on data on the intensity of the first reflected light, the step of measuring a first-first point in time at which the value of the rate of change in the intensity of the first reflected light reaches a first threshold value, and the step of determining that the object is close at the first-first point in time.

[0015] In another aspect, the step of determining whether the object is close may further include the step of measuring a first-second time point at which the value of the change rate of the intensity of the first reflected light reaches a second threshold value smaller than the first threshold value based on the change rate data of the intensity of the first reflected light, and the step of determining that the object has disappeared at the first-second time point.

[0016] In another aspect, the first threshold value may have a positive value and the second threshold value may have a negative value.

[0017] In another aspect, the optical skin detection-based biometric data acquisition method may further include a step of extracting a change pattern of a change rate of intensity of the first reflected light after the first-1 time point, and a step of comparing the extracted change pattern of the change rate of intensity of the first reflected light with a preset reference pattern to determine whether the sensor is worn on the object.

[0018] In another aspect, the step of determining whether the object is the target skin may include the step of measuring the intensity of the second reflected light at a second time point after a predetermined stabilization time has passed from the first-first time point, the step of determining whether the intensity of the second reflected light at the second time point is within a predetermined range, and the step of determining that the object is the target skin when the intensity of the second reflected light at the second time point is determined to be within the predetermined range.

[0019] In another aspect, the optical skin detection-based biometric data acquisition method further includes, after the step of receiving the first reflected light and the second reflected light, a step of performing preprocessing on the first reflected light data and the second reflected light data, and in the step of determining whether the object is in proximity, the proximity of the object is determined based on the preprocessed first reflected light data, and in the step of acquiring biometric data for the target skin, the biometric data for the target skin can be acquired based on the preprocessed second reflected light data.

[0020] In another aspect, the preprocessing may include at least one of real-time high-frequency noise filtering and baseline drift correction.

[0021] In another aspect, the first output light may be infrared (IR) light in a wavelength band of 850 nm to 1050 nm, and the second output light may be red (RED) light in a wavelength band of 620 nm to 750 nm.

[0022] One example is,

[0023] The present invention provides an optical skin detection-based biometric data sensor module, comprising: a light emitting unit that irradiates a first output light of a first wavelength range and a second output light of a second wavelength range different from the first wavelength range to an object; a light receiving unit that receives first reflected light and second reflected light reflected from the object by the first output light and the second output light; and a control unit that determines whether the object is close based on first reflected light data for the first reflected light, determines whether the object is target skin based on second reflected light data for the second reflected light when the object is determined to be close, and processes the second reflected light data when the object is determined to be the target skin to obtain biometric data for the target skin.

[0024] One example is,

[0025] A device comprising: a first ear unit including a first audio output module, a second ear unit including a second audio output module, an optical sensor provided in at least one of the first ear unit and the second ear unit, and a control unit for performing skin detection to determine whether an object is in proximity and whether the object is skin based on data acquired through the optical sensor, wherein the control unit controls the optical sensor so that a first output light of a first wavelength range and a second output light of a second wavelength range different from the first wavelength range are irradiated to the object, receives signals by first reflected light and second reflected light respectively reflected from the object by the first output light and the second output light received by the optical sensor, determines whether the object is in proximity based on first reflected light data for the first reflected light, and when the object is determined to be in proximity, determines whether the object is target skin based on second reflected light data for the second reflected light, and when the object is determined to be the target skin, processes the second reflected light data to obtain biometric data for the target skin. A wearable device including a skin detection-based biometric data sensor is provided.

[0026] In another aspect, the control unit can control activation of the wearable device depending on whether the object determined to be in proximity is the target skin.

[0027] According to various embodiments of the present disclosure, an optical skin detection-based biometric data acquisition method capable of utilizing a compact sensor module by performing a determination of proximity of an object and a determination of whether the object is skin based on a single optical sensor, an optical skin detection-based biometric data sensor module, and a wearable device including an optical skin detection-based biometric data sensor module can be provided.

[0028] According to various embodiments of the present disclosure, an optical skin detection-based biometric data acquisition method capable of quickly determining whether an object is close by utilizing change rate data of light intensity received through an optical sensor and determining whether the close object is target skin by comparing the light intensity received through the optical sensor with reference data, an optical skin detection-based biometric data sensor module, and a wearable device including an optical skin detection-based biometric data sensor module can be provided.

[0029] According to various embodiments of the present disclosure, an optical skin detection-based biometric data acquisition method capable of controlling activation of the wearable device depending on whether an object determined to be close is target skin, an optical skin detection-based biometric data sensor module, and a wearable device including the optical skin detection-based biometric data sensor module can be provided.

[0030] However, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood from the description below.

[0031] FIG. 1 is a conceptual diagram of an optical skin detection-based biometric data acquisition system according to one embodiment of the present disclosure.

[0032] FIG. 2 schematically illustrates the structure of a server according to one embodiment of the present disclosure.

[0033] Figure 3 is an internal block diagram of a wearable device according to one embodiment.

[0034] Figure 4 is for explaining the structure of a wearable device according to one embodiment.

[0035] FIG. 5 is for explaining the structure of an optical sensor according to one embodiment.

[0036] Figure 6 is an internal block diagram of a user terminal according to one embodiment.

[0037] Fig. 7 is a flowchart of a method for acquiring biometric data based on optical skin detection according to one embodiment.

[0038] FIG. 8 is a flowchart of a step for determining whether an object is in proximity that can be included in the method of FIG. 7.

[0039] FIG. 9 is for explaining a preprocessing process for data received through an optical sensor according to one embodiment.

[0040] FIG. 10 is a diagram illustrating a process for generating data on the rate of change in intensity of reflected light according to one embodiment.

[0041] FIG. 11 is a diagram illustrating a process for determining whether an object is close according to one embodiment.

[0042] FIG. 12 is a flowchart of a step for determining whether a nearby object that can be included in the method of FIG. 7 is target skin.

[0043] FIGS. 13 and 14 illustrate a process for determining whether a nearby object is a target skin according to one embodiment.

[0044] Fig. 15 is a flowchart of a method for acquiring biometric data based on optical skin detection according to another embodiment.

[0045] Fig. 16 is a flowchart of a method for acquiring biometric data based on optical skin detection according to another embodiment.

[0046] The present invention is capable of various modifications and embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, as well as the methods for achieving them, will become clear with reference to the embodiments described in detail below together with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms. In the following embodiments, the terms "first," "second," etc. are not used in a limiting sense but are used for the purpose of distinguishing one component from another. Furthermore, the singular expression includes the plural expression unless the context clearly indicates otherwise. Furthermore, terms such as "include" or "have" indicate the presence of a feature or component described in the specification, and do not preemptively exclude the possibility that one or more other features or components may be added. Furthermore, in the drawings, the sizes of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to what is shown.

[0047] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals and redundant descriptions thereof will be omitted.

[0048] FIG. 1 is a conceptual diagram of an optical skin detection-based biometric data acquisition system (1000) according to one embodiment of the present disclosure. FIG. 2 schematically illustrates the structure of a server (100) according to one embodiment of the present disclosure. FIG. 3 is an internal block diagram of a wearable device (200) according to one embodiment. FIG. 4 is for explaining the structure of a wearable device (200) according to one embodiment. FIG. 5 is for explaining the structure of an optical sensor (211) according to one embodiment. FIG. 6 is an internal block diagram of a user terminal (300) according to one embodiment. FIG. 7 is a flowchart of an optical skin detection-based biometric data acquisition method (S100) according to one embodiment. FIG. 8 is a flowchart of a step (S105) of determining whether an object is close, which can be included in the method of FIG. 7. FIG. 9 is for explaining a preprocessing process for data received through an optical sensor (211) according to one embodiment. FIG. 10 is a flowchart illustrating a process for generating data on the rate of change in the intensity of reflected light according to one embodiment. FIG. 11 is a flowchart illustrating a process for determining whether an object is close according to one embodiment. FIG. 12 is a flowchart illustrating a step (S107) for determining whether a close object is target skin, which may be included in the method of FIG. 7. FIG. 13 and FIG. 14 are flowcharts illustrating a process for determining whether a close object is target skin according to one embodiment. FIG. 15 is a flowchart illustrating a method (S200) for obtaining biometric data based on skin detection using an optical method according to another embodiment. FIG. 16 is a flowchart illustrating a method (S300) for obtaining biometric data based on skin detection using an optical method according to another embodiment.

[0049] Referring to FIG. 1, an optical skin detection-based biometric data acquisition system (1000) according to one embodiment may include a server (100), a wearable device (200), and a user terminal (300). The server (100), the wearable device (200), and the user terminal (300) may transmit and receive data to each other via a network.

[0050] An optical skin detection-based bio-data acquisition system (1000) according to one embodiment can measure bio-data such as a user's heart rate and body temperature measured by a sensor unit (210) included in a wearable device (200), and provide a service to the user that provides bio-feedback content based on the measured bio-data.

[0051] A wearable device (200) may be implemented in the form of a hearable device that can be worn on a user's ear. Here, a hearable device is a compound word of hear and wearable, and may refer to a wearable device focused on hearing that provides various convenient functions such as voice recognition, linkage with voice recognition artificial intelligence, music playback, and phone calls.

[0052] A wearable device (200) according to various embodiments may obtain sensing data (e.g., heart rate data (value)) based on a predetermined biosensor, and provide biofeedback content generated based on the amount of exercise, posture, etc. calculated based on the obtained sensing data to a user.

[0053] The network according to the embodiment means a connection structure that enables information exchange between each node, such as a server (100), a wearable device (200), and a user terminal (300), and examples of such a network include, but are not limited to, a 3GPP (3rd Generation Partnership Project) network, an LTE (Long Term Evolution) network, a WIMAX (World Interoperability for Microwave Access) network, the Internet, a LAN (Local Area Network), a Wireless LAN (Wireless Local Area Network), a WAN (Wide Area Network), a PAN (Personal Area Network), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, a DMB (Digital Multimedia Broadcasting) network, etc.

[0054] Hereinafter, the server (100), wearable device (200), and user terminal (300) implementing the system (1000) will be described in detail with reference to the attached drawings.

[0055] -Server (100)

[0056] A server (100) according to one embodiment can perform a series of processes to provide an environment for acquiring biometric data based on optical skin detection.

[0057] In detail, in the embodiment, the server (100) can exchange data necessary to enable an optical skin detection-based biometric data acquisition process to be driven in an external device such as a user terminal (300), thereby performing a predetermined operation on the acquired raw data to acquire biometric data, and provide biometric data-based biofeedback content to the user.

[0058] In more detail, in an embodiment, the server (100) may provide an environment in which an application (311) can operate on an external device (in an embodiment, a mobile type computing device and / or a desktop type computing device, etc.).

[0059] To this end, the server (100) may include application programs, data and / or commands for the operation of the application (311) of the genetic terminal (300), and may transmit and receive various data based thereon with the external device.

[0060] Additionally, in the embodiment, the server (100) can perform various deep learning for providing biofeedback content in conjunction with a deep learning neural network.

[0061] Here, the deep learning neural network according to the embodiment may include a convolutional neural network (CNN), an R-CNN (Regions with CNN features), a Fast R-CNN, a Faster R-CNN, a Mask R-CNN, etc., and may include any deep learning neural network that includes an algorithm capable of performing the embodiment described below, and the embodiment of the present invention does not limit or restrict such deep learning neural network itself.

[0062] At this time, depending on the embodiment, the deep learning neural network may be installed directly on the server (100) or may operate as a device separate from the server (100) to perform deep learning for providing the biofeedback content.

[0063] In addition, in the embodiment, the server (100) can read out a predetermined deep learning neural network driving program constructed to perform the deep learning from the memory (130) and perform the deep learning described below according to the read out predetermined deep learning neural network system.

[0064] In addition, in the embodiment, the server (100) can store and manage various application programs, commands and / or data for implementing an optical skin detection-based biometric data acquisition environment.

[0065] In an embodiment, the server (100) can store and manage at least one or more sensing data, user body information, user status information, user exercise information, test results, heart rate information, biofeedback content, user exercise ability and / or exercise program, etc.

[0066] However, in one embodiment of the present disclosure, the functional operations that the server (100) can perform are not limited to those described above, and other functional operations may be performed.

[0067] Meanwhile, referring further to FIG. 1, in the embodiment, the server (100) as described above may be implemented as a computing device including at least one processor module (110: Processor Module) for data processing, at least one communication module (120: Communication Module) for data exchange with an external device, and at least one memory (130: Memory Module) for storing various application programs, data, and / or commands for providing an environment for obtaining biometric data based on optical skin detection.

[0068] Here, the memory (130) can store one or more of an operating system (OS), various application programs, data, and commands for providing an environment for acquiring biometric data based on optical skin detection.

[0069] Additionally, the memory (130) may include a program area and a data area.

[0070] Here, the program area according to the embodiment may be linked between the operating system (OS) that boots the server and functional elements, and the data area may store data generated according to the use of the server.

[0071] In an embodiment, the memory (130) may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and may also be web storage that performs the storage function of the memory (130) on the Internet.

[0072] Additionally, the memory (130) may be a removable recording medium on the server (100).

[0073] Meanwhile, the processor module (110) can control the overall operation of each unit described above in order to implement an optical skin detection-based biometric data acquisition environment.

[0074] This processor module (110) may be a system on chip (SOC) suitable for a server including a central processing unit (CPU) and / or a graphics processing unit (GPU), and may execute an operating system (OS) and / or application programs stored in a memory (130) and control each component mounted on the server.

[0075] In addition, the processor module (110) can communicate with each component internally via a system bus and can include one or more predetermined bus structures including a local bus.

[0076] Additionally, the processor module (110) may be implemented using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0077] In addition, referring to FIG. 2, the processor module (110) of the server (100) can perform the functions of a data preprocessing unit (11), a change rate data generation unit (12), a proximity determination unit (13), a target skin determination unit (14), and a bio-information extraction unit (15).

[0078] The data preprocessing unit (11) can perform preprocessing on data acquired by the sensor unit (210) of the wearable device (200) to more accurately determine the proximity of an object and acquire biometric data. The data preprocessing unit (11) can perform preprocessing processes such as smoothing processing, high-frequency noise filtering, and baseline drift correction.

[0079] Smoothing processing extracts only data about the main pattern contained in the reflected light from the object, which may mean removing residual peaks contained in the reflected light.

[0080] For example, the optical sensor (211) included in the sensor unit (210) of the wearable device (200) can obtain first reflected light data based on the first reflected light reflected from an object as the first output light, and can obtain second reflected light data based on the second reflected light reflected from an object as the second output light. Referring to (a) of Fig. 9, the first reflected light data can be expressed as a first curve (c1), and the second reflected light data can be expressed as a second curve (c2).

[0081] Here, the first output light may be infrared (IR) light in a wavelength band of 850 nm to 1050 nm, and the second output light may be red (RED) light in a wavelength band of 620 nm to 750 nm.

[0082] The data preprocessing unit (11) can perform smoothing processing on the first reflected light data of the first curve (c1) and the second reflected light data of the second curve (c2), respectively. Referring to (b) of Fig. 9, the result data of the smoothing processing on the first reflected light data can be expressed as the third curve (c3), and the result data of the smoothing processing on the second reflected light data can be expressed as the fourth curve (c4).

[0083] In this case, the data preprocessing unit (11) may perform smoothing processing on the first reflected light data and the second reflected light data using a known smoothing processing technique. For example, the data preprocessing unit (11) may perform smoothing processing on the first reflected light data and the second reflected light data using a plurality of low-pass filters.

[0084] High-frequency noise filtering can mean removing high-frequency noise, which is noise caused by environmental factors, electric field interference, motion artifacts, etc., in addition to signals representing biological changes.

[0085] The data preprocessing unit (11) can perform high-frequency noise filtering on the first reflected light data and the second reflected light data using a known high-frequency noise filtering technique. For example, the data preprocessing unit (11) can perform high-frequency noise filtering on the first reflected light data and the second reflected light data using a low-pass filter, a Kalman filter, or the like.

[0086] Baseline drift correction can refer to compensating for baseline drift, which occurs when the reference line for data, which should be constant, fluctuates due to environmental interference, motion artifacts, etc. Baseline drift can interfere with extracting desired information from received data.

[0087] The data preprocessing unit (11) can perform baseline drift correction on the first reflected light data of the first curve (c1) and the second reflected light data of the second curve (c2), respectively.

[0088] The data preprocessing unit (11) can utilize, for example, filtering techniques, baseline correction algorithms, and appropriate calibration operations to perform baseline drift correction. This can minimize baseline fluctuations and extract meaningful information from the original biometric data.

[0089] The change rate data generation unit (12) can perform a change rate data generation task for data acquired by the sensor unit (210) of the wearable device (200). The change rate data generation unit (12) can generate change rate data of the intensity of reflected light based on data of the intensity of reflected light received by the optical sensor (211) included in the sensor unit (210) of the wearable device (200).

[0090] For example, the change rate data generation unit (12) can generate change rate data of the intensity of the first reflected light by performing a differentiation operation on the result data smoothed by the data preprocessing unit (11) for the first reflected light data. Referring to Fig. 10, the change rate data of the intensity of the first reflected light can be expressed as a fifth curve (c5).

[0091] The proximity determination unit (13) can determine whether an object is in proximity based on first reflected light data for the first reflected light. For example, the proximity determination unit (13) can determine whether an object is in proximity based on rate of change data of the intensity of the first reflected light generated by the rate of change data generation unit (12).

[0092] For example, the proximity determination unit (13) can determine that an object is approaching when the value of the rate of change in the intensity of the first reflected light reaches an arbitrary threshold value, and can determine that the object has disappeared when the value of the rate of change in the intensity of the first reflected light reaches another arbitrary threshold value.

[0093] A specific method for determining whether an object is in proximity by comparing the value of the rate of change in the intensity of the first reflected light with a plurality of threshold values ​​by the proximity determination unit (13) will be described later with reference to FIG. 7.

[0094] In addition, the proximity determination unit (13) can extract a change pattern of the rate of change in the intensity of the first reflected light and use this to determine whether the optical sensor (211) is worn on a nearby object.

[0095] For example, when a user wears an optical sensor (211) on a specific body part, there may be a specific change pattern in the rate of change in the intensity of the first reflected light corresponding to the user. The proximity determination unit (13) can determine whether the optical sensor (211) is worn on a nearby object by extracting the change pattern in the rate of change in the intensity of the first reflected light and comparing it with the specific change pattern.

[0096] In this way, by utilizing data on the rate of change in the intensity of reflected light to determine whether an object is close, the proximity of an object can be determined more quickly.

[0097] The target skin determination unit (14) can determine whether a nearby object is target skin based on second reflection light data for the second reflection light.

[0098] Here, the target skin may be the area of ​​the user's skin from which biometric data is to be acquired. For example, if biometric data is to be acquired from the user's ear, the target skin may be the skin of the user's ear.

[0099] The target skin determination unit (14) can determine whether an adjacent object is target skin based on the result data smoothed by the data preprocessing unit (11) for the second reflected light data.

[0100] For example, the target skin determination unit (14) determines whether the intensity of the second reflected light at a specific point in time is within a predetermined range, and if it is determined that the intensity of the second reflected light at the specific point in time is within the predetermined range, it can determine that the nearby object is the target skin.

[0101] A specific method for determining that a nearby object is target skin based on the result of the target skin determination unit (14) determining whether the intensity of the second reflected light is within a predetermined range will be described later with reference to FIGS. 12 to 14.

[0102] The biometric information extraction unit (15) can obtain biometric data on the target skin by processing the second reflected light data for the second reflected light. For example, the biometric information extraction unit (15) can obtain the user's heart rate data based on the second reflected light data.

[0103] For example, if a nearby object is determined to be the target skin by the target skin determination unit (14), the biometric information extraction unit (15) can obtain biometric data for the target skin based on the result data obtained by performing preprocessing such as high-frequency noise filtering and baseline drift correction on the second reflected light data by the data preprocessing unit (11).

[0104] In the above description, it has been described that the server (100) according to the embodiment of the present invention performs the functional operation as described above, but depending on the embodiment, at least a part of the functional operation performed by the server (100) may be performed by an external device (e.g., a wearable device (200), a user terminal (300), etc.), and at least a part of the functional operation performed by the external device may be further performed by the server (100), and various other embodiments may be possible.

[0105] -Wearable devices (200)

[0106] A wearable device (200) according to one embodiment may be a device capable of interworking with an optical skin detection-based biometric data acquisition application (hereinafter, “application”) that provides optical skin detection-based biometric data installed in a user terminal (300). The wearable device (200) may include various types of devices such as a smart watch, a head-up display device, smart glasses, and smart jewelry.

[0107] For example, the wearable device (200) may include wireless earphones that a user can wear on his or her ears to receive sound.

[0108] In detail, from a hardware perspective, the wearable device (200) may include hearable devices such as a Bluetooth headset, neckband earphones, bone conduction earphones, and TWS (True Wireless Stereo, completely wireless earphones), which are distinguished by their external appearance. In addition, the wearable device (200) may include a canal-type or open-type hearable device, which are distinguished by the shape of the part that comes into contact with the ear.

[0109] In this disclosure, a wearable device (200) is described based on being implemented as a kernel-type TWS, but the wearable device (200) can be implemented as any device that is connected to a user terminal (300) and can be worn on the user's ear.

[0110] Referring to FIG. 1, a wearable device (200) according to one embodiment may include a left ear unit (201) and a right ear unit (202). In addition, the wearable device (200) may further include a charging case (not shown) that accommodates and charges the left ear unit (201) and the right ear unit (202).

[0111] In this case, various components, excluding a predetermined biosensor included in the sensor unit (210) among the components described below, may be included equally within the left ear unit (201) and the right ear unit (202). In the following, the components included in each ear unit are described together, but it is preferable to understand that each component is configured and operated within a single ear unit in which it is placed.

[0112] Referring to FIG. 3, from a functional perspective, the wearable device (200) may include a sensor unit (210), an input unit (220), an output unit (230), a battery (240), an interface unit (250), a storage unit (260), a communication unit (270), and / or a control unit (280). These components may be configured to be included, for example, within the ear unit and / or charging case of the wearable device (200).

[0113] The sensor unit (210) may include various types of biosensors, such as optical sensors and body temperature sensors. In this case, the optical sensor may be used to acquire PPG (Photoplethysmogram) data. In addition, the sensor unit (210) may further include various sensors, such as a position sensor (IMU), audio sensor, distance sensor, proximity sensor, and contact sensor.

[0114] In one embodiment, the sensor unit (210), referring to FIGS. 4 and 5, may include an optical sensor (211) for collecting biometric data (e.g., photoplethysmography and / or oxygen saturation (SpO2)) of a user wearing the wearable device (200). In this case, the optical sensor (211) may be provided in the left ear unit (201). However, the present invention is not limited thereto, and the optical sensor (211) may also be provided in the right ear unit (202).

[0115] Referring to FIG. 5, the optical sensor (211) may include a light emitting unit (21) that irradiates light and a light receiving unit (22) that receives reflected light reflected from an object by the light emitting unit (21).

[0116] The light emitting unit (21) can, for example, irradiate an object with a first output light of a first wavelength range and a second output light of a second wavelength range different from the first wavelength range. Here, the first output light may be infrared (IR) light of a wavelength range of 850 nm to 1050 nm, and the second output light may be red (RED) light of a wavelength range of 620 nm to 750 nm.

[0117] The light-emitting unit (21) may include, for example, a first light-emitting element that irradiates a first output light to an object and a second light-emitting element that irradiates a second output light to the object. The first light-emitting element and the second light-emitting element may include, for example, an LED.

[0118] The light receiving unit (22) can receive the first reflected light and the second reflected light reflected from the object, which are the first output light and the second output light. For example, the light receiving unit (22) can include a light receiving element that receives the first reflected light and the second reflected light. Here, the light receiving element can include, for example, a photodiode.

[0119] The optical sensor (211) may be installed at a location corresponding to a predetermined measurement location on the user's ear. For example, the optical sensor (211) may be installed at a location corresponding to an area with a high microvascular density on the ear.

[0120] For example, referring to FIG. 4, the left ear unit (201) may include a housing (h1) and an optical sensor (211). In addition, the left ear unit (201) may further include an audio output module (not shown) provided inside the housing (h1) to provide sound.

[0121] The housing (h1) is supported by the user's ear and may include a window (w1) formed to face a specific area of ​​the user's ear when the housing (h1) is supported by the user's ear. An optical sensor (211) may be provided inside the housing (h1) to measure bio-data about the user's ear through the window (w1). Here, the window (w1) may be an opening formed in a portion of the housing (h1), and thus, the optical sensor (211) provided inside the housing (h1) may be exposed to the outside of the housing (h1) through the window (w1).

[0122] For example, the window (w1) faces the upper portion of the conchae of the user's ear, and the optical sensor (211) provided inside the housing (h1) can measure PPG data for the user's ear through the window (w1). For example, the optical sensor (211) can irradiate light to the user's ear through the window (w1) and receive reflected light reflected from the user's ear. In this case, the optical sensor (211) can irradiate light to the upper portion of the conchae of the user's ear through the window (w1) and receive reflected light reflected from the upper portion of the conchae.

[0123] Meanwhile, the left ear unit (201) may be provided with an extension portion (p1) extending downward from one end of the housing (h1) and an ear tip (e1) formed to be inserted into the external auditory canal of the user's ear at the other end of the housing (h1). In this case, the extension portion (p1) and the housing (h1) may be formed to be inclined at a predetermined angle with respect to each other.

[0124] The extension portion (p1) may be formed to extend downward from the user's ear when the left ear unit (201) is mounted on the user's ear. The extension portion (p1) may contact the user's ear to provide support to the left ear unit (201) so that the left ear unit (201) is stably mounted on the user's ear.

[0125] The ear tip (e1) is formed to be inserted into the user's external auditory canal and may include a sound hole through which sound is transmitted from an audio output module provided inside the housing (h1).

[0126] The input unit (220) can detect a user's input (e.g., a gesture, a voice command, a touch input, a button operation, or another type of input).

[0127] In detail, the input unit (220) may include a predetermined pressure sensor (e.g., a button) and / or a touch sensor for detecting a user's input.

[0128] In addition, the input unit (220) may be configured in the form of a touch screen unit and / or a touch screen panel. Specifically, it may be configured in one of a resistive film method, an electrostatic capacitance method, an optical method, and an ultrasonic method, but it is preferable to use an electrostatic capacitance method.

[0129] Additionally, the input unit (220) can obtain a predetermined control signal for controlling the wearable device (200) and / or the user terminal (300) based on the touch sensor.

[0130] In detail, the input unit (220) can transmit a signal including the number of detected touches to the control unit (280). Accordingly, the control unit (280) can execute a predetermined process matched to the signal. For example, if the user inputs one short touch, a process for pausing playback while listening to music can be executed. Additionally, if the user inputs two short touches, a process for playing the next song while listening to music can be executed.

[0131] The output unit (230) may include a predetermined audio output device (hereinafter, speaker).

[0132] In detail, the output unit (230) may include an internal speaker that transmits sound into the ear of a user wearing the wearable device (200) and an external speaker that transmits sound to the outside of the earphone.

[0133] The internal speaker may provide sounds including biofeedback content and / or music. Additionally, the external speaker may provide sounds including a predetermined beep tone in case of loss.

[0134] Additionally, the output unit (230) may include a predetermined lighting and / or vibration module. For example, the lighting and / or vibration module may operate when a specific event, such as pairing and / or loss, occurs.

[0135] The battery (240) is implemented in the form of a rechargeable secondary battery and may include a wired charging module and / or a wireless charging module.

[0136] In detail, the battery (240) can supply a predetermined amount of power from the power supply of a digital device when connected to the digital device via a wire (e.g., USB cable, etc.) based on a wired charging module.

[0137] Additionally, the battery (240) can supply a predetermined amount of power from the power supply of a predetermined digital device when wirelessly connected to the digital device based on the wireless charging module.

[0138] The interface unit (250) can connect the wearable device (200) to one or more other devices so that they can communicate with each other. In detail, the interface unit (250) can include wired and / or wireless communication devices compatible with one or more different communication protocols.

[0139] Through this interface unit (250), the wearable device (200) can be connected to multiple input / output devices (e.g., user terminals (300)).

[0140] The storage unit (260) can store one or more of various application programs, data, and commands for creating and providing an environment for acquiring biometric data based on optical skin detection.

[0141] Additionally, the storage unit (260) may include a program area and a data area.

[0142] Here, the program area according to the embodiment may be linked between the operating system (OS) that boots the wearable device (200) and functional elements, and the data area may store data generated according to the use of the wearable device (200).

[0143] Additionally, the storage unit (260) may include at least one non-transitory computer-readable storage medium and one or more temporary computer-readable storage medium.

[0144] Additionally, the storage unit (260) can store predetermined biometric data obtained from the sensor unit (210).

[0145] The communication unit (270) may include one or more devices for communicating with external devices. This communication unit (270) may communicate via a wireless network.

[0146] In detail, the communication unit (270) can communicate with a user terminal (300) that stores a content source for implementing an optical skin detection-based biometric data acquisition environment, and can communicate with various user input components such as a controller that receives user input.

[0147] In an embodiment, the communication unit (270) can transmit and receive various data (e.g., biosignals and / or sensing data) related to an optical skin detection-based biometric data acquisition environment to and from another terminal and / or an external server (e.g., a user terminal (300) and / or a server (100)).

[0148] This communication unit (270) can wirelessly transmit and receive data with at least one of a base station, an external terminal, and an arbitrary server on a mobile communication network constructed through a communication device (e.g., a communication chip that performs Bluetooth communication) that can perform technical standards or communication methods for mobile communication (e.g., LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G NR (New Radio), WIFI) or short-range communication methods.

[0149] In an embodiment, the communication unit (270) can transmit and receive predetermined data with the user terminal (300) through short-range communication using Bluetooth.

[0150] Additionally, the communication unit (270) can transmit and receive a predetermined signal between the left ear unit (201) and the right ear unit (202).

[0151] The control unit (280) may include at least one processor capable of executing commands of an application stored in the user terminal (300) to perform various tasks for creating an environment for acquiring biometric data based on optical skin detection.

[0152] Additionally, in the embodiment, the control unit (280) can control the overall operation of the components of the wearable device (200) to provide an environment for acquiring biometric data based on optical skin detection.

[0153] In detail, in an embodiment, the control unit (280) may include an embedded processor that connects the optical sensor (211) of the sensor unit (210) to the analog front-end and then controls an analog-to-digital converter (ADC) to obtain a PPG signal.

[0154] This control unit (280) may be a system on chip (SOC) suitable for the wearable device (200), and may execute an operating system (OS) and / or application programs stored in the storage unit (260), and control each component mounted on the wearable device (200).

[0155] In addition, the control unit (280) can communicate with each component internally via a system bus and can include one or more predetermined bus structures including a local bus.

[0156] In addition, the control unit (280) may be implemented by including at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0157] In addition, the control unit (280) can control the wearable device (200) by exchanging data with the user terminal (300) in response to signals received from the sensor unit (210) and / or the input unit (220).

[0158] A wearable device (200) including the above-described components can transmit sensing data including at least one or more bio-data such as heart rate data and oxygen saturation data and at least one or more of location data, distance data and / or posture data to a user terminal (300) according to an embodiment, and such various data can be stored in a memory (310) of the user terminal (300).

[0159] -User terminal (300)

[0160] A user terminal (300) according to one embodiment may be a computing device having an application (hereinafter, “application”) installed thereon that provides an optical skin detection-based biometric data acquisition environment.

[0161] In detail, from a hardware perspective, the user terminal (300) may include a mobile type computing device and / or a desktop type computing device on which an application is installed.

[0162] In the embodiment, the user is a user who exercises while carrying a wearable device (200) and a user terminal (300). For convenience of explanation, the user terminal (300) is described below as a mobile type computing device.

[0163] Here, the mobile type computing device may be a mobile device such as a smart phone or tablet PC on which an application is installed.

[0164] For example, mobile type computing devices may include smart phones, mobile phones, digital broadcasting devices, personal digital assistants (PDAs), portable multimedia players (PMPs), tablet PCs, etc.

[0165] Additionally, according to an embodiment, the user terminal (300) may further include a server computing device that provides an environment for acquiring biometric data based on optical skin detection.

[0166] Referring to FIG. 6, from a functional perspective, the user terminal (300) may include a memory (310), a processor assembly (320), a communication processor (330), an interface module (340), an input system (350), a sensor system (360), and a display system (370). These components may be configured to be included within the housing of the user terminal (300).

[0167] In detail, in the memory (310), an application (311) is stored, and the application (311) can store one or more of various application programs, data, and commands for providing an environment for acquiring biometric data based on optical skin detection.

[0168] Additionally, the memory (310) may include a program area and a data area.

[0169] Here, the program area according to the embodiment may be linked between the operating system (OS) that boots the user terminal (300) and functional elements, and the data area may store data generated according to the use of the user terminal (300).

[0170] Additionally, the memory (310) may include at least one non-transitory computer-readable storage medium and one or more temporary computer-readable storage medium.

[0171] For example, the memory (310) may be a variety of storage devices such as a ROM, EPROM, flash drive, hard drive, etc., and may include web storage that performs the storage function of the memory (310) on the Internet.

[0172] The processor assembly (320) may include at least one processor capable of executing commands of an application (311) stored in a memory (310) to perform various tasks for creating an optical skin detection-based biometric data acquisition environment.

[0173] In an embodiment, the processor assembly (320) can control the overall operation of the components through the application (311) of the memory (310) to provide an environment for acquiring biometric data based on optical skin detection.

[0174] This processor assembly (320) may be a system on chip (SOC) suitable for a user terminal (300) including a central processing unit (CPU) and / or a graphics processing unit (GPU), and may execute an operating system (OS) and / or application programs stored in a memory (310) and control each component mounted on the user terminal (300).

[0175] Additionally, the processor assembly (320) may communicate with each component internally via a system bus and may include one or more predetermined bus structures including a local bus.

[0176] Additionally, the processor assembly (320) may be implemented by including at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0177] The communication processor (330) may include one or more devices for communicating with external devices. The communication processor (330) may communicate via a wireless network.

[0178] In detail, the communication processor (330) can communicate with a server (100) that stores a content source for implementing an optical skin detection-based biometric data acquisition environment, and can communicate with various user input components, such as a controller that receives user input.

[0179] In an embodiment, the communication processor (330) can transmit and receive various data related to an optical skin detection-based biometric data acquisition environment to and from other user terminals and / or external servers.

[0180] This communication processor (330) can wirelessly transmit and receive data with at least one of a base station, an external terminal, and an arbitrary server on a mobile communication network constructed through a communication device capable of performing technical standards or communication methods for mobile communication (e.g., LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G NR (New Radio), WIFI) or short-range communication methods.

[0181] The interface module (340) can connect the user terminal (300) to one or more other devices so that they can communicate with each other. In detail, the interface module (340) can include wired and / or wireless communication devices compatible with one or more different communication protocols.

[0182] Through this interface module (340), the user terminal (300) can be connected to multiple input / output devices.

[0183] For example, the interface module (340) can be connected to an audio output device such as a headset port or speaker to output audio.

[0184] As an example, the audio output device is described as being connected through an interface module (340), but an embodiment in which it is installed inside a user terminal (300) may also be included.

[0185] Additionally, for example, the interface module (340) may be connected to an input device such as a keyboard and / or mouse to obtain user input.

[0186] Such an interface module (340) may be configured to include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, an earphone port, a power amplifier, an RF circuit, a transceiver, and other communication circuits.

[0187] The input system (350) can detect user input (e.g., a gesture, a voice command, the operation of a button, or other type of input).

[0188] In detail, the input system (350) may include a predetermined button, a touch sensor, and / or an image sensor that receives user motion input.

[0189] Additionally, the input system (350) can be connected to an external controller through an interface module (340) to receive user input.

[0190] The sensor system (360) may include various sensors such as an image sensor (361), a position sensor (IMU, 363), an audio sensor (365), a distance sensor, a proximity sensor, and a contact sensor.

[0191] Here, the image sensor (361) can capture images and / or videos of the physical space around the user terminal (300).

[0192] The image sensor (361) can capture an image by photographing the direction in which it is positioned on the front or / and rear of the user terminal (300), and can capture a physical space through a camera positioned toward the outside of the user terminal (300).

[0193] This image sensor (361) may include an image sensor device and an image processing module. In detail, the image sensor (361) may process still images or moving images obtained by an image sensor device (e.g., CMOS or CCD).

[0194] The position sensor (IMU, 363) can detect at least one of the movement and acceleration of the user terminal (300). For example, it can be formed by a combination of various position sensors, such as an accelerometer, a gyroscope, and a magnetometer. Such a position sensor (IMU) may be referred to as a motion sensor hereinafter.

[0195] Additionally, the position sensor (IMU, 363) can recognize spatial information about the physical space around the user terminal (300) in conjunction with the GPS of the communication processor (330).

[0196] The audio sensor (365) can recognize sounds around the user terminal (300).

[0197] In detail, the audio sensor (365) may include a microphone capable of detecting a voice input of a user using the user terminal (300).

[0198] The display system (370) can output various information related to the optical skin detection-based biometric data acquisition environment as graphic images.

[0199] As an example, the display system (370) can display various user interfaces (as an example, a membership registration interface) for an optical skin detection-based biometric data acquisition environment.

[0200] Such displays may include at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, and an e-ink display.

[0201] The above components may be arranged within the housing of the user terminal (300), and the user interface may include a touch sensor (373) on a display (371) configured to receive user touch input.

[0202] In detail, the display system (370) may include a display (371) that outputs an image and a touch sensor (373) that detects a user's touch input.

[0203] For example, the display (371) may be implemented as a touch screen by forming a mutual layer structure with the touch sensor (373) or forming an integral structure. Such a touch screen may function as a user input unit that provides an input interface between the user terminal (300) and the user, and at the same time, provide an output interface between the user terminal (300) and the user.

[0204] The user terminal (300) including the above-described components may store at least one or more sensing data, user body information, user status information, user exercise information, user exercise ability, and / or exercise program in the memory (310) according to an embodiment.

[0205] - Optical skin detection-based biometric data acquisition method (S100, S200, S300)

[0206] Referring to FIG. 7, an optical skin detection-based biometric data acquisition method (S100) according to one embodiment may include a step (S101) of irradiating an object with a first output light of a first wavelength range and a second output light of a second wavelength range different from the first wavelength range, a step (S103) of receiving a first reflection light and a second reflection light reflected from the object by the first output light and the second output light, respectively, a step (S105) of determining whether the object is close, a step (S107) of determining whether the close object is target skin, and a step (S109) of processing the second reflection light data to acquire biometric data for the target skin.

[0207] The optical skin detection-based biometric data acquisition method (S100) may be performed by a processor module (110) included in a server (100). However, the present invention is not limited thereto, and at least a part of the method (S100) may be performed by a control unit (280) of a wearable device (200) or a processor assembly (320) of a user terminal (300), and another part may be performed by a processor module (110) of the server (100).

[0208] For example, at least one of a processor module (110) included in a server (100), a control unit (280) of a wearable device (200), and a processor assembly (320) included in a user terminal (300) may execute at least one command stored in a memory (130) included in the server (100), a storage unit (260) of a wearable device (200), or a memory (310) of a user terminal (300), thereby performing an optical skin detection-based biometric data acquisition method (S100, S200, S300).

[0209] Hereinafter, the processor module (110) of the server (100) is described as performing the methods (S100, S200, S300).

[0210] In the step (S101) of irradiating the first output light and the second output light to an object, the processor module (110) of the server (100) can control the light emitting unit (21) of the optical sensor (211) to irradiate the first output light of the first wavelength range and the second output light of the second wavelength range different from the first wavelength range to the object.

[0211] In this case, the processor module (110) can control the first light-emitting element included in the light-emitting unit (21) to irradiate the first output light to the object, and control the second light-emitting element included in the light-emitting unit (21) to irradiate the second output light to the object. Here, the first light-emitting element and the second light-emitting element can be provided spaced apart from each other.

[0212] The processor module (110) can control the first light-emitting element and the second light-emitting element so that the first output light and the second output light are irradiated to the object simultaneously. However, the present invention is not limited thereto, and the processor module (110) can also control the first light-emitting element and the second light-emitting element so that the first output light and the second output light are irradiated to the object at different times.

[0213] The first output light may be infrared (IR) light in a wavelength band of 850 nm to 1050 nm, and the second output light may be red (RED) light in a wavelength band of 620 nm to 750 nm.

[0214] Red light is visible and has a high power consumption for its light output, whereas infrared light is invisible and can have a low power consumption for its light output.

[0215] Infrared light exhibits significant variations in reflectance for most objects, making it useful for determining proximity to an object. However, red light exhibits minimal variations in reflectance for inorganic objects and significant variations in reflectance for organic matter, making it useful for obtaining data on organic matter.

[0216] Accordingly, in various embodiments of the present disclosure, proximity to an object can be determined based on the first output light, which is infrared light, and proximity to an object can be determined based on the second output light, which is red light, whether the proximity object is target skin.

[0217] In the step (S103) of receiving the first reflected light and the second reflected light, the processor module (110) of the server (100) can receive a signal by the first reflected light and the second reflected light received by the light receiving unit (22) of the optical sensor (211).

[0218] For example, in step (S101), the first output light and the second output light irradiated on the object can be received by the light receiving unit (22) as the first reflected light and the second reflected light reflected by the object, and the photoelectric conversion signal generated by the light receiving unit (22) can be received by the processor module (110) of the server (100).

[0219] In the step (S105) of determining whether an object is close, the processor module (110) of the server (100) can determine whether an object is close based on the first reflected light data for the first reflected light.

[0220] For example, in step (S105), the processor module (110) of the server (100) can determine whether an object is approaching based on the change rate data of the intensity of the first reflected light. Referring to FIG. 8, step (S105) may include, for example, a step (S1051) of generating change rate data of the intensity of the first reflected light based on the data of the intensity of the first reflected light, a step (S1053) of measuring a 1-1 time point (t1) at which the value of the change rate of the intensity of the first reflected light reaches a first threshold value (k1), and a step (S1055) of determining that an object is approaching at the 1-1 time point (t1).

[0221] In addition, step (S105) may further include a step (S1057) of measuring a first-second time point (t2) at which the value of the change rate of the intensity of the first reflected light reaches a second threshold value (k2) smaller than the first threshold value (k1) based on the change rate data of the intensity of the first reflected light, and a step (S1059) of determining that the object has disappeared at the first-second time point (t2).

[0222] For example, referring to (a) of FIG. 10, the smoothed result data for the first reflected light data for the first reflected light received by the light receiving unit (22) may be expressed as a third curve (c3), and the smoothed result data for the second reflected light data for the second reflected light may be expressed as a fourth curve (c4).

[0223] In step (S1051), the processor module (110) of the server (100) may perform a differentiation operation on the smoothed result data for the first reflected light data to generate the change rate data of the intensity of the first reflected light. Referring to (b) of FIG. 10, the change rate data of the intensity of the first reflected light may be expressed as a fifth curve (c5).

[0224] In step (S1053), referring to (a) of FIG. 11, the processor module (110) of the server (100) can measure the first-first point in time (t1) at which the value of the rate of change in the intensity of the first reflected light reaches the first threshold value (k1). The first threshold value (k1) is a value corresponding to a reference value of the rate of change in the intensity of the reflected light reflected from a nearby object, and can be a criterion for determining proximity.

[0225] In step (S1055), the processor module (110) of the server (100) may determine that an object is approaching at the first-first time point (t1).

[0226] In step (S1057), referring to FIG. 11, the processor module (110) of the server (100) can measure a 1-2 time point (t2) at which the value of the rate of change in the intensity of the first reflected light reaches a second threshold value (k2) that is less than the first threshold value (k1) after the 1-1 time point (t1).

[0227] In this case, the first threshold value (k1) may have a positive value, and the second threshold value (k2) may have a negative value. The second threshold value (k2) is a value corresponding to a reference value of the rate of change in the intensity of reflected light reflected from an object in the process of disappearing, and may be a criterion for determining whether the object disappears.

[0228] Until the 1-2 time point (t2) at which the value of the rate of change in the intensity of the first reflected light reaches the second threshold value (k2) after the 1-1 time point (t1) is measured, the processor module (110) of the server (100) can maintain the determination that the object is in a close state.

[0229] At step (S1059), the processor module (110) of the server (100) may determine that the object has disappeared at the first-second time point (t2).

[0230] Through steps (S1051) to (S1059), the processor module (110) of the server (100) in step (S105) can determine that an object is approaching in the section between the 1-1 time point (t1) at which the value of the change rate of the intensity of the first reflected light reaches the first threshold value (k1) and the 1-2 time point (t2) at which the value of the change rate of the intensity of the first reflected light reaches the second threshold value (k2).

[0231] The processor module (110) can generate a signal corresponding to the proximity state of an object during the period between the 1st-1 time point (t1) and the 1st-2 time point (t2), and data according to this signal can be expressed as the 6th curve (c6) of (b) of FIG. 11.

[0232] In step (S105), if it is determined that the object is close, it can proceed to step (S107), and if it is determined that the object is not close, it can return to step (S101).

[0233] In the step (S107) of determining whether the nearby object is the target skin, the processor module (110) of the server (100) can determine whether the nearby object is the target skin based on the second reflected light data for the second reflected light.

[0234] Here, the target skin may be the area of ​​the user's skin from which biometric data is to be acquired. For example, if biometric data is to be acquired from the user's ear, the target skin may be the skin of the user's ear.

[0235] For example, referring to FIGS. 12 and 13, step (S107) may include a step (S1071) of measuring the intensity of the second reflected light at a second time point (t3) after a predetermined stabilization time (Δt) has passed from a first-first time point (t1) when the object is close, a step (S1073) of determining whether the intensity of the second reflected light at the second time point (t3) is within a predetermined range, and a step (S1075) of determining that the object is the target skin when it is determined that the intensity of the second reflected light at the second time point (t3) is within the predetermined range.

[0236] Additionally, step (S107) may further include a step (S1077) of determining that the object is not the target skin when it is determined that the intensity of the second reflected light at the second time point (t3) is not within the predetermined range.

[0237] In step (S1071), the processor module (110) of the server (100) can measure the intensity of the second reflected light at a second time point (t3) after a predetermined stabilization time (Δt) has passed from the first-first time point (t1) when the object approaches. For example, referring to (a) of FIG. 13, in step (S1071), the processor module (110) of the server (100) can measure the value (q1) of the intensity of the second reflected light at the second time point (t3) based on the result data smoothed for the second reflected light data.

[0238] Here, the stabilization time (Δt) may be the time required for the second reflected light data to transition from a transient state to a steady state during the object's approach. For example, the stabilization time (Δt) may be 0.6 seconds or less.

[0239] In step (S1073), the processor module (110) of the server (100) can determine whether the value (q1) of the intensity of the second reflected light measured at the second time point (t3) is within a predetermined range.

[0240] For example, the processor module (110) of the server (100) can determine whether the value (q1) of the intensity of the second reflected light measured at the second time point (t3) is included between the third threshold value (k3) and the fourth threshold value (k4). Here, the third threshold value (k3) may be an upper limit of the predetermined range, and the fourth threshold value (k4) may be a lower limit of the predetermined range.

[0241] For example, a predetermined range corresponding to a third threshold value (k3) and a fourth threshold value (k4) may be a range of the intensity of the second reflected light when the second reflected light is light reflected from the user's target skin (e.g., the skin of the ear). Accordingly, when the value (q1) of the intensity of the second reflected light measured at the second time point (t3) is included between the third threshold value (k3) and the fourth threshold value (k4), information that the object on which the second reflected light is reflected is the target skin can be obtained based on the value (q1) of the intensity of the second reflected light.

[0242] In step (S1075), the processor module (110) of the server (100) can determine that the nearby object is the target skin if it is determined that the value (q1) of the intensity of the second reflected light measured at the second time point (t3) is within a predetermined range.

[0243] For example, if the processor module (110) of the server (100) determines that the value (q1) of the intensity of the second reflected light measured at the second time point (t3) is included between the third threshold value (k3) and the fourth threshold value (k4), the processor module (110) of the server (100) may determine that the object determined to be close based on the first reflected light data is the target skin.

[0244] However, in contrast, referring to FIG. 14, the value (q2) of the intensity of any third reflected light different from the second reflected light at the second time point (t3) may not be included between the third threshold value (k3) and the fourth threshold value (k4).

[0245] Here, the third reflected light is red light with a wavelength of 620 nm to 750 nm, similar to the second reflected light, but may be light reflected from the skin of another part of the user's body rather than the target skin.

[0246] In this case, it may be determined that the value (q2) of the intensity of the third reflected light measured at the second time point (t3) in step (S1073) is not included between the third threshold value (k3) and the fourth threshold value (k4), and accordingly, the object determined to be close based on the first reflected light data in step (S1077) may be determined not to be the target skin.

[0247] Meanwhile, the processor module (110) can stop the task of determining whether an object close to the first-second time point (t2), which is the time point at which the object is determined to have disappeared, is the target skin.

[0248] In addition, the processor module (110) of the server (100) can maintain the determination that the nearby object is the target skin when the value (q1) of the intensity of the second reflected light is maintained constant from the second time point (t3) to before the first-second time point (t2).

[0249] Furthermore, the processor module (110) can generate a signal indicating that a nearby object is the target skin during the period between the second time point (t3) and the first-second time point (t2), and data according to this signal can be expressed as the seventh curve (c7) of (c) of FIG. 13.

[0250] In the step (S109) of acquiring biometric data, if it is determined in step (S107) that the nearby object is the target skin, the processor module (110) of the server (100) can process the second reflected light data to acquire biometric data for the target skin.

[0251] For example, referring to (a) of FIG. 13, in step (S109), the processor module (110) of the server (100) may process the smoothed result data for the second reflected light data to obtain biometric data for the target skin. For example, the processor module (110) of the server (100) may process the smoothed result data for the second reflected light data to obtain the user's heart rate data.

[0252] Additionally, the method (S100) may further include a step of controlling activation of the wearable device (200) depending on whether the proximity object is the target skin after step (S107).

[0253] For example, if it is determined in step (S107) that the nearby object is the target skin, the processor module (110) of the server (100) can transmit a control signal to activate the wearable device (200) to the linked wearable device (200).

[0254] Additionally, if it is determined in step (S107) that the nearby object is not the target skin, the processor module (110) of the server (100) can transmit a control signal to the linked wearable device (200) to deactivate the wearable device (200).

[0255] In this case, the control unit (280) of the wearable device (200) can control the activation of the wearable device (200) based on a control signal received from the server (100).

[0256] Meanwhile, referring to FIG. 15, an optical skin detection-based biometric data acquisition method (S200) is provided according to another embodiment that is substantially the same as the method (S100) of FIG. 7, except that it further includes a step (S205) of performing preprocessing on data for the first reflected light and the second reflected light after receiving the first reflected light and the second reflected light.

[0257] In explaining Fig. 15, any content overlapping with Figs. 7 to 14 is omitted.

[0258] Referring to FIG. 15, the method (S200) may include a step (S201) of irradiating an object with a first output light of a first wavelength range and a second output light of a second wavelength range different from the first wavelength range, a step (S203) of receiving a first reflection light and a second reflection light reflected from the object by the first output light and the second output light, respectively, a step (S205) of performing preprocessing on data on the first reflection light and the second reflection light, a step (S207) of determining whether an object is close, a step (S209) of determining whether the close object is target skin, and a step (S211) of processing the second reflection light data to obtain biometric data on the target skin.

[0259] In the step of performing preprocessing (S205), the processor module (110) of the server (100) may perform preprocessing on the first reflected light data and the second reflected light data acquired by the optical sensor. Here, the preprocessing may include smoothing processing, high-frequency noise filtering, baseline drift correction, etc.

[0260] In step (S205), the proximity of an object can be determined more accurately by determining whether the object is in proximity based on the preprocessed first reflected light data. In addition, more accurate biometric data can be obtained by acquiring biometric data based on the preprocessed second reflected light data.

[0261] Furthermore, referring to FIG. 16, another embodiment of an optical skin detection-based biometric data acquisition method (S300) is provided, which is substantially the same as the method (S100) of FIG. 7, except that it further includes a step (S307) of determining whether an optical sensor (211) is worn on a nearby object after determining whether an object is nearby.

[0262] In explaining Fig. 16, any content overlapping with Figs. 7 to 14 is omitted.

[0263] Referring to FIG. 16, the method (S300) may include a step (S301) of irradiating an object with a first output light of a first wavelength range and a second output light of a second wavelength range different from the first wavelength range, a step (S303) of receiving a first reflection light and a second reflection light respectively reflected from the object by the first output light and the second output light, a step (S305) of determining whether the object is close, a step (S307) of determining whether the sensor (211) is worn on the close object, a step (S309) of determining whether the close object is target skin, and a step (S311) of processing the second reflection light data to obtain biometric data for the target skin.

[0264] In step (S307), referring to FIG. 11, the processor module (110) of the server (100) can extract a change pattern of the rate of change in the intensity of the first reflected light after the first-first time point (t1). In addition, the processor module (110) can determine whether the optical sensor (211) is worn on a nearby object by comparing the extracted change pattern of the rate of change in the intensity of the first reflected light with a preset reference pattern.

[0265] For example, the preset reference pattern may be a unique change pattern of the rate of change in the intensity of the first reflected light reflected by the target skin (e.g., the skin of the user's ear) when the optical sensor (211) is worn on the target skin of the user. Accordingly, the processor module (110) may determine whether the extracted first reflected light is reflected by the target skin in the process of wearing the optical sensor (211) on the target skin based on the similarity between the extracted change pattern of the rate of change in the intensity of the first reflected light and the reference pattern. Here, data regarding the reference pattern may be stored in advance in at least one of, for example, the memory module (110) of the server (100), the storage unit (260) of the wearable device (200), and the memory (310) of the user terminal (300).

[0266] For example, if the similarity between the extracted change pattern of the change rate of the intensity of the first reflected light and the reference pattern is greater than a standard value (e.g., 90% similarity), the processor module (110) determines that the first reflected light is light reflected by the target skin during the process of the optical sensor (211) being worn on the target skin, and accordingly, determines that the optical sensor (211) is worn on an object that is the target skin.

[0267] In step (S307), if it is determined that the optical sensor (211) is worn on the object that is the target skin, the process proceeds to step (S309), and if it is determined that the optical sensor (211) is not worn on the object that is the target skin, the process returns to step (S301).

[0268] The embodiments of the present invention described above may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. Hardware devices may be changed into one or more software modules to perform processing according to the present invention, and vice versa.

[0269] The specific implementations described in the present invention are exemplary embodiments and do not limit the scope of the present invention in any way. For the sake of brevity, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the lines connecting or connecting members between components illustrated in the drawings are merely representative of functional connections and / or physical or circuit connections, and may be replaced or represented as various additional functional connections, physical connections, or circuit connections in an actual device. In addition, unless specifically mentioned as “essential,” “important,” etc., a component may not be absolutely necessary for the application of the present invention.

[0270] Although the detailed description of the present invention has been described with reference to preferred embodiments of the present invention, it will be understood by those skilled in the art or having ordinary knowledge in the art that various modifications and changes can be made to the present invention without departing from the spirit and technical scope of the present invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

[0271] The present invention has industrial applicability in that it can provide a wearable device with a compact sensor module that can perform judgments on the proximity of an object and whether the object is skin based on a single optical sensor by quickly detecting an object using light of a first wavelength range and determining whether the detected object is skin using light of a second wavelength range.

Claims

1. A method for obtaining biometric data using a sensor including a light-emitting unit and a light-receiving unit, A step of irradiating an object with a first output light of a first wavelength range and a second output light of a second wavelength range different from the first wavelength range using the above light emitting unit; A step of using the light receiving unit to receive the first reflected light and the second reflected light, which are respectively reflected from the object, using the first output light and the second output light; A step of determining whether the object is close based on first reflected light data for the first reflected light; If the object is determined to be close, a step of determining whether the object is the target skin based on second reflection data for the second reflection; and A method for obtaining biometric data based on skin detection using an optical method, comprising: a step of obtaining biometric data for the target skin by processing the second reflected light data when the object is determined to be the target skin; 2. In paragraph 1, The step of determining whether the above object is close is: A step of generating data on the rate of change in the intensity of the first reflected light based on data on the intensity of the first reflected light; A step of measuring a first-first point in time when the value of the rate of change in the intensity of the first reflected light reaches a first threshold value; and A method for obtaining biometric data based on skin detection using an optical method, comprising: a step of determining that the object is close at the first point in time; 3. In paragraph 2, A step of measuring a first-second point in time at which the value of the change rate of the intensity of the first reflected light reaches a second threshold value that is less than the first threshold value based on the change rate data of the intensity of the first reflected light; and A method for obtaining biometric data based on skin detection using an optical method, further comprising: a step of determining that the object has disappeared at the first or second time point; 4. In paragraph 3, A method for acquiring biometric data based on optical skin detection, wherein the first threshold value has a positive value and the second threshold value has a negative value.

5. In paragraph 2, A step of extracting a change pattern of the rate of change in the intensity of the first reflected light after the first-1 point in time; and A method for obtaining biometric data based on skin detection using an optical method, further comprising: a step of comparing a change pattern of a change rate of the intensity of the extracted first reflected light with a preset reference pattern to determine whether the sensor is worn on the object; 6. In paragraph 2, The step of determining whether the above object is the target skin is: A step of measuring the intensity of the second reflected light at a second point in time after a predetermined stabilization time has passed from the first point in time; A step of determining whether the intensity of the second reflected light is within a predetermined range at the second point in time; and A method for obtaining biometric data based on skin detection using an optical method, comprising: a step of determining that the object is target skin when the intensity of the second reflected light at the second point in time is determined to be within the predetermined range; 7. In paragraph 1, After the step of receiving the first reflected light and the second reflected light, A step of performing preprocessing on the first reflected light data and the second reflected light data; further comprising, In the step of determining whether the object is close, whether the object is close is determined based on the preprocessed first reflected light data, A method for acquiring biometric data based on skin detection using an optical method, wherein in the step of acquiring biometric data for the target skin, the biometric data for the target skin is acquired based on the preprocessed second reflected light data.

8. In paragraph 7, A method for acquiring biometric data using an optical skin detection method, wherein the above preprocessing includes at least one of real-time high-frequency noise filtering and baseline drift correction.

9. In paragraph 1, A method for obtaining biometric data based on skin detection using an optical method, wherein the first output light is infrared (IR) light having a wavelength band of 850 nm to 1050 nm, and the second output light is red (RED) light having a wavelength band of 620 nm to 750 nm.

10. A light emitting unit that irradiates a first output light of a first wavelength range and a second output light of a second wavelength range different from the first wavelength range to an object; A light receiving unit that receives the first reflected light and the second reflected light reflected from the object, the first output light and the second output light; and An optical skin detection-based biometric data sensor module comprising: a control unit that determines whether an object is close based on first reflection data for the first reflection light, determines whether the object is target skin based on second reflection data for the second reflection light when the object is determined to be close, and processes the second reflection data to obtain biometric data for the target skin when the object is determined to be the target skin; 11. A first ear unit including a first audio output module; A second ear unit comprising a second audio output module; An optical sensor provided in at least one of the first ear unit and the second ear unit; A control unit that performs skin detection to determine whether an object is close based on data acquired through the optical sensor and determines whether the close object is skin; The above control unit, Control the optical sensor so that a first output light of a first wavelength range and a second output light of a second wavelength range different from the first wavelength range are irradiated to an object, The first output light and the second output light received by the optical sensor receive signals by the first reflected light and the second reflected light reflected from the object, respectively, Determine whether the object is close based on the first reflected light data for the first reflected light, If the object is determined to be close, it is determined whether the object is the target skin based on the second reflection data for the second reflection, A wearable device including an optical skin detection-based biometric data sensor that processes the second reflected light data to obtain biometric data for the target skin when the object is determined to be the target skin.

12. In paragraph 11, A wearable device including an optical skin detection-based biometric data sensor, wherein the control unit controls activation of the wearable device depending on whether the object determined to be close is the target skin.

Images