Device for estimating biological information

Abstract

A device for estimating biological information is provided. The device for estimating biological information according to an embodiment of the present disclosure includes: a first sensor configured to obtain a fingerprint image of a user's finger; a second sensor configured to measure a force or pressure applied by the finger; and a processor configured to: obtain a first eigenvalue related to a pulse wave and a second eigenvalue related to skin elasticity based on the fingerprint image; obtain a third eigenvalue based on the force or pressure; and estimate the user's biological information based on the first eigenvalue, the second eigenvalue, and the third eigenvalue.

Description

[0001] This application is based on and claims priority to Korean Patent Application No. 10-2020-0080145, filed with the Korean Intellectual Property Office on Jun. 30, 2020, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] The following description relates to a device and method for estimating biological information and a technique for cuffless blood pressure estimation. Background Art

[0003] Generally, methods for non-invasively measuring blood pressure without harming the human body include a method of measuring blood pressure by measuring cuff-based pressure and a method of estimating blood pressure by measuring a pulse wave without using a cuff.

[0004] The Korotkoff-sound method is one of the cuff-based blood pressure measurement methods, in which the pressure in a cuff wrapped around the upper arm is increased, and blood pressure is measured by listening to the sounds generated in the blood vessels with a stethoscope while decreasing the pressure. Another cuff-based blood pressure measurement method is the oscillometric method using an automatic machine, in which the cuff is wrapped around the upper arm, the pressure in the cuff is increased, the pressure in the cuff is continuously measured while gradually decreasing the cuff pressure, and blood pressure is measured based on points with large changes in the pressure signal.

[0005] Cuffless blood pressure measurement methods generally include a method of estimating blood pressure by calculating the pulse transit time (PTT) and a pulse wave analysis (PWA) method of estimating blood pressure by analyzing a pulse waveform. Summary of the Invention

[0006] According to an aspect of an exemplary embodiment, a device for estimating biological information of a user may include: a first sensor configured to obtain a fingerprint image of the user's finger; a second sensor configured to measure a force or pressure applied by the finger; and a processor configured to: obtain a first eigenvalue related to a pulse wave and a second eigenvalue related to skin elasticity based on the fingerprint image; obtain a third eigenvalue based on the force or pressure; and estimate the user's biological information based on the first eigenvalue, the second eigenvalue, and the third eigenvalue.

[0007] The first sensor may include: one or more light sources configured to emit light onto the finger; and one or more complementary metal oxide semiconductor (CMOS) image sensors.

[0008] The processor may obtain a pulse wave signal based on the fingerprint image and obtain the first eigenvalue based on at least one of a maximum amplitude value and a minimum amplitude value of the pulse wave signal.

[0009] The processor can obtain the change in the distance between the ridges or valleys of the fingerprint from the fingerprint image based on the finger pressing the first sensor, and obtain the second eigenvalue based on the change in the distance.

[0010] The processor can obtain the first average value of the distance between the ridges or valleys at one or more points of the first fingerprint image continuously obtained in a predetermined time period and the second average value of the distance at one or more points of the second fingerprint image, and obtain the difference between the first average value and the second average value as the change in the distance.

[0011] The processor can generate a curve graph of the fingerprint pattern change based on the change in the distance, and obtain the second eigenvalue based on the curve graph of the fingerprint pattern change.

[0012] The processor can obtain at least one of the maximum slope value, the minimum slope value, and the average slope value of each predefined unit interval as the second eigenvalue.

[0013] The processor can generate a differential curve graph by differentiating the curve graph of the fingerprint pattern change, and obtain the second eigenvalue by using the differential curve graph.

[0014] The processor can obtain the force value or pressure value at the time point corresponding to the first eigenvalue as the third eigenvalue.

[0015] The processor can combine the first eigenvalue, the second eigenvalue, and the third eigenvalue, and estimate the biological information by applying a predefined estimation model to the result of combining the first eigenvalue, the second eigenvalue, and the third eigenvalue.

[0016] The processor can apply the first weight to the first eigenvalue to obtain the first weighted value; apply the second weight to the second eigenvalue to obtain the second weighted value; apply the third weight to the third eigenvalue to obtain the third weighted value; and combine the first weighted value, the second weighted value, and the third weighted value.

[0017] Each of the first weight value, the second weight value, and the third weight value can be a predefined fixed value, or a value adjusted by at least considering the user characteristics or the type of biological information.

[0018] When the fingerprint image is obtained from the finger, the processor can control the output interface to output guiding information to guide the user about the pressure between the finger and the first sensor.

[0019] The biological information can include one or more of blood pressure, vascular age, arterial stiffness, aortic pressure waveform, vascular compliance, pressure index, and degree of fatigue.

[0020] A method for estimating a user's biological information may include: obtaining a fingerprint image of the user's finger; measuring the force or pressure exerted by the finger; obtaining a first eigenvalue related to a pulse wave based on the fingerprint image; obtaining a second eigenvalue related to skin elasticity based on the fingerprint image; obtaining a third eigenvalue based on the force or pressure; and estimating the biological information based on the first eigenvalue, the second eigenvalue, and the third eigenvalue.

[0021] The step of obtaining the first eigenvalue may include: obtaining a pulse wave signal from the fingerprint image; and obtaining the first eigenvalue based on at least one of a maximum amplitude value and a minimum amplitude value of the pulse wave signal.

[0022] The step of obtaining the second eigenvalue may include: obtaining a change in the distance between ridges or valleys of the fingerprint from the fingerprint image based on the finger pressing a first sensor; and obtaining the second eigenvalue based on the obtained change in distance.

[0023] The step of obtaining the second eigenvalue may include: obtaining a first average value of a first distance between ridges or valleys at one or more points of a first fingerprint image continuously obtained in a predetermined time period; obtaining a second average value of a second distance between ridges or valleys at one or more points of a second fingerprint image; and obtaining a difference between the first average value and the second average value as the change in distance.

[0024] The step of obtaining the second eigenvalue may include: generating a curve graph of fingerprint pattern change based on the change in distance; and obtaining the second eigenvalue based on the curve graph of fingerprint pattern change.

[0025] The step of obtaining the second eigenvalue may include: obtaining at least one of a maximum slope value, a minimum slope value, and an average slope value for each predefined unit interval as the second eigenvalue.

[0026] The step of obtaining the third eigenvalue may include: obtaining a force value or a pressure value at a time point corresponding to the first eigenvalue as the third eigenvalue.

[0027] The step of estimating the biological information may include: combining the first eigenvalue, the second eigenvalue, and the third eigenvalue; and estimating the biological information by applying a predefined estimation model to the result of combining the first eigenvalue, the second eigenvalue, and the third eigenvalue.

[0028] The method may include: guiding the user regarding the pressure between the finger and the first sensor when the fingerprint image is obtained from the finger.

[0029] According to an aspect of an example embodiment, a device for estimating a user's biological information may include: a first sensor configured to obtain a fingerprint image of the user's finger; a second sensor configured to measure a force or pressure applied by the finger; and a processor configured to: obtain two or more eigenvalue based on at least one of the fingerprint image and the force or pressure; and estimate first biological information and second biological information based on the two or more eigenvalue.

[0030] The first biological information may be blood pressure, and the processor may obtain a first eigenvalue and a second eigenvalue based on the fingerprint image; obtain a third eigenvalue based on the first eigenvalue and the force or pressure; and estimate the blood pressure by combining the first eigenvalue and the second eigenvalue.

[0031] The second biological information may be at least one of skin elasticity and skin age, and the processor may obtain a second eigenvalue based on the fingerprint image; and estimate at least one of skin elasticity and skin age based on the second eigenvalue.

[0032] The processor may obtain a change in a distance between ridges or valleys of the fingerprint from the fingerprint image based on the finger pressing the first sensor; and obtain a second eigenvalue based on the change in the distance.

[0033] When a change in the second eigenvalue shows an increasing or decreasing trend compared to the second eigenvalue obtained at a reference time, the processor may estimate that the skin elasticity decreases or increases and the skin age increases or decreases.

[0034] According to an aspect of an example embodiment, a device for estimating a user's biological information may include: a sensor configured to: obtain a fingerprint image of a finger based on the user's finger pressing the sensor; and a processor configured to: obtain an eigenvalue from the fingerprint image based on a fingerprint pattern change based on the finger pressing the sensor; and estimate at least one of skin elasticity and skin age based on the eigenvalue.

[0035] The sensor may measure a force or pressure applied by the finger, and the processor may obtain a change in a distance between ridges or valleys of the fingerprint from the fingerprint image based on the finger pressing the sensor; generate a curve graph of the fingerprint pattern change by plotting the change in the distance with respect to the force or pressure; and obtain an eigenvalue by using the curve graph of the fingerprint pattern change. Description of the Drawings

[0036] The above and other aspects, features, and advantages of specific embodiments of the present disclosure will become clearer from the following description in conjunction with the accompanying drawings, where:

[0037] Figure 1 and Figure 2 is a block diagram showing an example of a device for estimating biological information;

[0038] Figure 3A and Figure 3B is a diagram showing an example of the configuration of a processor of a device for estimating biological information;

[0039] Figures 4A to 6C is a diagram illustrating an example of obtaining eigenvalue;

[0040] Figure 7 is a flowchart showing a method for estimating biological information according to an embodiment of the present disclosure;

[0041] Figure 8 is a flowchart showing a method for estimating biological information according to another embodiment of the present disclosure;

[0042] Figure 9 is a diagram showing a wearable device according to an embodiment of the present disclosure; and

[0043] Figure 10 is a diagram showing an example of an intelligent device to which a device for estimating biological information is applied. DETAILED DESCRIPTION

[0044] Details of the exemplary embodiments are included in the following detailed description and the drawings. The advantages and features of the present disclosure and the method of realizing the present disclosure will be more clearly understood from the embodiments described in detail below with reference to the drawings. Throughout the drawings and the detailed description, unless otherwise described, the same reference numerals will be understood to represent the same elements, features, and structures.

[0045] It will be understood that although the terms "first", "second", etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used to distinguish one element from another. In addition, unless the context clearly indicates otherwise, the singular form of the terms is also intended to include the plural form of the terms. It will also be understood that unless explicitly described to the contrary, when an element is referred to as "including" another element, the element is not intended to exclude one or more other elements, but also includes one or more other elements. In the following description, terms (such as, "unit" and "module") indicate a unit for processing at least one function or operation, and the unit can be implemented by using hardware, software, or a combination thereof.

[0046] Hereinafter, embodiments of a device and a method for estimating biological information will be described in detail with reference to the drawings.

[0047] Figure 1 and Figure 2 is a block diagram showing an example of a device for estimating biological information.

[0048] Devices 100 and 200 for estimating biological information according to embodiments of the present disclosure may be installed in medical devices used in professional medical institutions, smartwatches worn on the wrist, various types of wearable devices (such as smart belt-type wearable devices, earphone-type wearable devices, headband-type wearable devices, etc.), or mobile devices (such as smartphones, tablet personal computers (PCs), etc.), but are not limited thereto.

[0049] Referring Figure 1 and Figure 2 , devices 100 and 200 for estimating biological information include a sensor unit 110 and a processor 120.

[0050] The sensor unit 110 may include a first sensor 111 for obtaining a fingerprint image when a finger touches the sensor unit 110 and a second sensor 112 for measuring force or pressure when the finger touches and presses the first sensor 111.

[0051] The first sensor 111 may include one or more light sources for emitting light onto the finger and one or more detectors for detecting light scattered or reflected from the finger. The light sources may include light-emitting diodes (LEDs), laser diodes (LDs), phosphors, etc., but are not limited thereto. The detectors may include complementary metal-oxide-semiconductor (CMOS) image sensors (CISs). However, the detectors are not limited thereto and may include charge-coupled device (CCD) image sensors, photodiodes, phototransistors, etc. The multiple light sources may emit light of the same wavelength or different wavelengths. The multiple detectors may be located at different distances from the light sources.

[0052] The second sensor 112 may include a force sensor, an array of force sensors, a pressure sensor, an airbag-type pressure sensor, a pressure sensor including a combination of a force sensor and an area sensor, etc.

[0053] For example, the first sensor 111 may include a finger contact surface for placing the finger. The finger contact surface may be formed as a smooth curved surface, but is not limited thereto. In this case, the second sensor 112 may be disposed at the lower end of the first sensor 111 such that when the finger is placed on the finger contact surface of the first sensor 111 and the pressing force or pressure is gradually increased or decreased, the second sensor 112 may measure the pressing force or pressure.

[0054] In another example, in the case where the second sensor 112 is a pressure sensor including a combination of a force sensor and an area sensor, the area sensor, the first sensor 111, and the force sensor are arranged in a stacked structure, and the finger contact surface may be formed on the area sensor. When the finger contacts the finger contact surface of the area sensor and changes the pressing force, the area sensor may obtain the contact area of the finger, and the force sensor may measure the pressing force.

[0055] The processor 120 may be electrically connected to the sensor unit 110. The processor 120 may control the sensor unit 110 in response to a user's request and may receive data from the sensor unit 110. In addition, the processor 120 may estimate biometric information by using the data received from the sensor unit 110.

[0056] For example, the processor 120 may obtain one or more eigenvalue based on a fingerprint image of a finger obtained by the first sensor 111 and a force or pressure measured by the second sensor 112, and may estimate biometric information by using the obtained eigenvalue. In this case, the biometric information may include, but is not limited to, heart rate, blood pressure, vascular age, arterial stiffness, aortic pressure waveform, vascular compliance, pressure index, degree of fatigue, skin elasticity, skin age, etc. For the sake of convenience of explanation, blood pressure, skin elasticity, and skin age will be described as examples below.

[0057] For example, the processor 120 may obtain a pulse wave signal from a fingerprint image of a finger and may obtain a first eigenvalue related to the pulse wave by using the obtained pulse wave signal. In addition, the processor 120 may obtain a second eigenvalue related to skin elasticity from a fingerprint image of a finger and may obtain a third eigenvalue based on the force or pressure of the finger. In addition, the processor 120 may combine two or more of the first eigenvalue, the second eigenvalue, and the third eigenvalue, and may estimate blood pressure by applying a predefined blood pressure estimation model to the combined value. Alternatively, the processor 120 may estimate skin elasticity and / or skin age by using the second eigenvalue.

[0058] Referring to Figure 2 , the device 200 for estimating biometric information according to another embodiment of the present disclosure may further include an output interface 210, a storage device 220, and a communication interface 230 in addition to the sensor unit 110 and the processor 120. The sensor unit 110 and the processor 120 have been described above, so the following description will focus on the output interface 210, the storage device 220, and the communication interface 230.

[0059] Based on receiving a request for estimating biometric information, the processor 120 may obtain a reference pressure by referring to the storage device 220, and may generate guidance information for guiding the user to keep the pressure of the user's finger pressing the sensor unit 110 within the reference pressure range. In addition, based on the sensor unit 110 measuring force and pressure, the processor 120 may generate guidance information for guiding the actual force or pressure applied to the sensor unit 110 by the finger. In this case, the guidance information may include information for guiding the object to gradually increase the pressure when the object contacts and presses the sensor unit 110, or conversely, information for guiding the object to gradually decrease the pressure when the object initially applies a contact pressure greater than or equal to a predetermined threshold to the sensor unit 110.

[0060] The output interface 210 may output the fingerprint image and / or force / pressure obtained by the sensor unit 110, and the estimated biometric information value and / or guidance information obtained by the processor 120. For example, the output interface 210 may visually output the data processed by the sensor unit 110 or the processor 120 through the display module, or may non-visually output information such as sound, vibration, and touch through the speaker module, the haptic module, etc. In this case, the display area may be divided into two or more areas, where the output interface 210 may output the fingerprint image, the pulse wave signal, the force or pressure, etc. for estimating biometric information in the form of various curves in the first area; and the output interface 210 may output the estimated biometric information value in the second area together with the information. In this case, if the estimated biometric information value falls outside the normal range, the output interface 210 may output warning information in various ways (such as highlighting the abnormal value in red, etc., displaying the abnormal value together with the normal range, outputting a voice warning message, adjusting the vibration intensity, etc.).

[0061] The storage device 220 may store the results processed by the sensor unit 110 and the processor 120. In addition, the storage device 220 may store various reference information for estimating biometric information. For example, the reference information may include user characteristic information (such as the user's age, gender, health status, etc.). In addition, the reference information may include a biometric information estimation model, a standard for estimating biometric information, a reference pressure, etc., but is not limited thereto.

[0062] In this case, the storage device 220 may include at least one storage medium such as a flash memory, a hard disk memory, a multimedia card micro memory, a card memory (such as a secure digital (SD) memory, an extreme digital (XD) memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a programmable read only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk, etc., but is not limited thereto.

[0063] The communication interface 230 can communicate with an external device under the control of the processor 120 by using wired or wireless communication technologies, and can send various data to the external device and receive various data from the external device. For example, the communication interface 230 can send the biological information estimation result to the external device, and can receive various reference information for estimating biological information from the external device. In this case, the external device can include a cuff-type blood pressure measurement device and an information processing device (such as a smart phone, a tablet PC, a desktop computer, a laptop computer, etc.).

[0064] In this case, examples of the communication technologies can include Bluetooth communication, Bluetooth Low Energy (BLE) communication, Near Field Communication (NFC), Wireless Local Area Network (WLAN) communication, Zigbee communication, Infrared Data Association (IrDA) communication, Wi-Fi Direct (WFD) communication, Ultra Wideband (UWB) communication, Ant+ communication, Wi-Fi communication, Radio Frequency Identification (RFID) communication, 3G communication, 4G communication, 5G communication, etc. However, this is only exemplary and is not intended to limit.

[0065] Figure 3A is a diagram showing an example of the configuration of a processor of a device for estimating biological information. Figures 4A to 6C is a diagram illustrating an example of obtaining a feature value.

[0066] Referring to Figure 3A , the processor 300a according to an embodiment of the present disclosure includes a first feature value acquirer 310, a second feature value acquirer 320, a third feature value acquirer 330, and an estimator 340.

[0067] Based on the fingerprint image received from the first sensor 111, the first feature value acquirer 310 can obtain a pulse wave signal based on the fingerprint image. For example, the first feature value acquirer 310 can obtain a pulse wave signal based on the change in the intensity of the fingerprint images continuously obtained within a predetermined time period. When the force or pressure of the finger pressing the sensor unit 110 gradually increases or decreases, the amplitude value of the pulse wave signal obtained from the fingerprint image has a pattern of gradually increasing or decreasing. The first feature value acquirer 310 can obtain a first feature value by analyzing the waveform of the increase or decrease of the pulse wave signal. In this case, the first feature value can include the maximum amplitude value and / or the minimum amplitude value of the pulse wave signal. However, the first feature value is not limited thereto, and can include the amplitude values of points related to the propagation wave and the reflection wave, a partial area of the waveform of the pulse wave signal, etc.

[0068] The second eigenvalue acquirer 320 can acquire a second eigenvalue related to skin elasticity based on the fingerprint image received from the first sensor 111. When the user touches the sensor unit 110 with a finger and increases or decreases the force or pressure with which the finger presses the sensor unit 110, the fingerprint pattern of the finger is displayed in the fingerprint image according to the change in skin elasticity. Therefore, based on the change in the fingerprint pattern displayed in the fingerprint image, the second eigenvalue acquirer 320 can acquire a second eigenvalue related to skin elasticity.

[0069] For example, the second eigenvalue acquirer 320 can acquire the change in the distance between the ridges or valleys of the fingerprint or the degree of pressing of the fingerprint pattern, and can acquire a second eigenvalue related to skin elasticity based on the acquired change in the distance between the ridges or valleys or the acquired degree of pressing of the fingerprint pattern. In this case, in order to reflect the characteristic that the degree of pressing of the fingerprint pattern changes according to the position of the fingerprint, the second eigenvalue acquirer 320 can acquire the distance between the ridges or valleys at two or more points of the fingerprint image, and can acquire the second eigenvalue based on the change in a statistical value (such as the average value of the acquired distances).

[0070] The third eigenvalue acquirer 330 can acquire a third eigenvalue based on the force or pressure received from the second sensor 112. For example, the third eigenvalue acquirer 330 can acquire the force or pressure value at the time point when the first eigenvalue acquirer 310 acquires the first eigenvalue as the third eigenvalue. However, the third eigenvalue is not limited thereto.

[0071] Figures 4A to 6C is a diagram illustrating an example of acquiring eigenvalues.

[0072] Figure 4A is a diagram showing an example of a pulse wave signal obtained based on the change in the intensity of continuously acquired fingerprint images when the user touches the sensor unit 110 with a finger for a predetermined period of time and gradually increases the pressing force of the finger. By using the obtained pulse wave signal, the first eigenvalue acquirer 310 can acquire, for example, the maximum amplitude value of the pulse wave signal as the first eigenvalue.

[0073] Figure 4B is a diagram illustrating an example of the third eigenvalue acquired by the third eigenvalue acquirer 330 and showing an oscillogram (OW) diagram, which shows the relationship between the change in the amplitude of the pulse wave signal and the change in the force or pressure measured by the second sensor within a predetermined period of time. The third eigenvalue acquirer 330 can extract the peak-to-peak points of the pulse wave signal waveform by subtracting the negative (-) amplitude value in3 from the positive (+) amplitude value in2 of the waveform envelope in1 at each measurement time of the pulse wave signal, and can obtain the oscillogram (OW) envelope by plotting the peak-to-peak amplitude at each measurement time against the force / pressure value at the same time point.

[0074] Based on the first eigenvalue acquirer 310, the maximum amplitude value MA is obtained as the first eigenvalue. The third eigenvalue acquirer 330 can obtain the force / pressure value MP at the time point corresponding to the maximum amplitude value MA as the third eigenvalue. However, the third eigenvalue is not limited thereto, and the third eigenvalue acquirer 330 can also obtain the force / pressure values DP and SP at the points to the left and right of MP corresponding to the amplitude values having a preset ratio (e.g., 0.5 to 0.7) with the maximum amplitude value MA, etc., as additional third eigenvalues.

[0075] Figures 5A to 5C It is a diagram illustrating an example in which the second eigenvalue acquirer 320 obtains a second eigenvalue from a fingerprint image.

[0076] Referring to Figure 5A , among the fingerprint images continuously measured within a predetermined time period, there is a change in the fingerprint pattern between the first fingerprint image 51a measured at the first time and the second fingerprint image 51b measured at the second time. That is, as the measurement progresses from the first time to the second time, if the force applied by the finger to the sensor unit 110 gradually increases, the finger gradually presses, so that the distance between the ridges or valleys of the fingerprint is changed as shown in Figure 5A .

[0077] Figure 5B It is a diagram illustrating the change in the distance between the ridges or valleys of the fingerprint and showing an enlarged view of the first fingerprint area 52a in the first fingerprint image 51a and the second fingerprint area 52b corresponding to the first fingerprint area 52a in the second fingerprint image 51b. As shown in Figure 5B , at the first time when the finger applies a weak force, the pressing degree of the fingerprint is relatively low, so that in the first fingerprint area 52a, the heights of the first valley V1 and the second valley V2 are relatively high, and the distance IN1 between the ridge R between the first valley V1 and the second valley V2 is relatively small. On the contrary, at the second time when the finger applies a relatively strong force, the pressing degree of the fingerprint is relatively high, so that in the second fingerprint area 52b, the heights of the first valley V1 and the second valley V2 are relatively low, and the distance IN2 between the ridge R between the first valley V1 and the second valley V2 is relatively large.

[0078] As described above, by using fingerprint images continuously obtained within a predetermined period of time, the second eigenvalue acquirer 320 can acquire the change in the ridge distance between valleys or the change in the valley distance between ridges. In this case, in order to reflect the characteristics of fingerprints at multiple positions on the finger, the second eigenvalue acquirer 320 can acquire the change between the average value of the distances at two or more points LN1 and LN2 of the first fingerprint image 51a and the average value of the distances between two or more corresponding points LN1 and LN2 of the second fingerprint image 51b as the change in the ridge distance or the valley distance. In one example, the second eigenvalue acquirer 320 can acquire the difference between the average value of the distances at two or more points LN1 and LN2 of the first fingerprint image 51a and the average value of the distances between two or more corresponding points LN1 and LN2 of the second fingerprint image 51b as the change in the ridge distance or the valley distance.

[0079] Figure 5C A graph showing the change in the fingerprint pattern is shown in the top view, and this graph shows the relationship between the change in the force with which the finger presses the sensor unit 110 and the change in the distance between valleys or ridges obtained based on the fingerprint image; and Figure 5C A graph showing the relationship between the change in the force with which the finger presses and the change in the actual area of the finger obtained by the area sensor is shown in the bottom view. As Figure 5C shown, when the user gradually increases the force with which the finger presses, the area of the finger increases non-linearly and saturates within a predetermined period of time. The trend of the change in the finger area changes according to individual characteristics (such as the user's gender, age, skin elasticity of the finger, etc.). As Figure 5C shown, the change in the distance with respect to the force of the finger shows a trend similar to the change in the finger area, so that the second eigenvalue acquirer 320 can acquire a second eigenvalue related to the skin elasticity of the finger based on the change in the distance between valleys or ridges.

[0080] Referring to Figure 6A , by plotting the change in the distance between valleys or ridges with respect to the force / pressure of the finger pressing at each time point on the time axis, the second eigenvalue acquirer 320 can generate a graph (top view) showing the change in the fingerprint pattern of the distance and the force / pressure. In addition, the second eigenvalue acquirer 320 can perform multi-dimensional equation curve fitting (middle view) on the fingerprint pattern change graph (top view), and can generate a differential graph (bottom view) by performing differentiation based on the force / pressure. The second eigenvalue acquirer 320 can acquire the second eigenvalue by using the generated differential graph (bottom view).

[0081] Referring to Figure 6B, based on the differential curve graph, the second eigenvalue acquirer 320 can acquire the slope value AMax at the point with the largest distance change and / or the slope value AMin at the point with the smallest distance change as the second eigenvalue. In addition, the second eigenvalue acquirer 320 can divide the force / pressure axis into predefined unit intervals, and can acquire the average slope value of each unit interval as the second eigenvalue. Figure 6B Eight unit intervals N1, N2, N3, N4, N5, N6, N7, and N8 divided in units of 1000 mN are shown. For example, the second eigenvalue acquirer 320 can acquire the average value (about 0.5) between the differential value (about 0.8) at the starting point (the point with a force / pressure value of 0) of the first unit interval N1 and the differential value (about 0.2) at the ending point (the point with a force / pressure value of 1000) of the first unit interval N1 as the average slope value AN1. In this way, the second eigenvalue acquirer 320 can acquire the average slope value of each of the eight unit intervals N1, N2, N3, N4, N5, N6, N7, and N8.

[0082] Return reference Figure 3A , the estimator 340 can estimate the biological information by combining the acquired first eigenvalue, second eigenvalue, and third eigenvalue. In this case, as shown in Equation 1 below, the estimator 340 can estimate the biological information by applying a predefined biological information estimation model to the values. The biological information estimation model can be represented in various forms of linear or non-linear combination functions (such as addition, subtraction, division, multiplication, logarithmic values, regression equations, etc.), without specific limitations. For example, Equation 1 below represents a simple linear function.

[0083] [Equation 1]

[0084] y = af1 + bf2 + cf3 + d

[0085] Here, y represents the biological information to be acquired (e.g., diastolic blood pressure, systolic blood pressure, mean arterial pressure, etc.); f1 represents the first eigenvalue, f2 represents the second eigenvalue, and f3 represents the third eigenvalue. In addition, a, b, c, and d represent coefficients for weighting each of the eigenvalues, and can be predefined fixed values that can be generally applied to multiple users according to the type of biological information, or can be adjusted for each user according to user characteristics, etc. Here, f1 can be any one of multiple first eigenvalues, or a combination of two or more; f2 can be any one of multiple second eigenvalues, or a combination of two or more; f3 can be any one of multiple third eigenvalues, or a combination of two or more. In this case, the criteria for combining the eigenvalues can be defined differently according to the type of biological information to be acquired, and can be appropriately defined for each user according to user characteristics.

[0086] Figure 3B FIG. is a diagram showing another example of the configuration of a processor of a device for estimating biological information.

[0087] Referring to Figure 3B , according to an embodiment, the processor 300b includes a first eigenvalue acquirer 310, a second eigenvalue acquirer 320, a third eigenvalue acquirer 330, a first estimator 341, and a second estimator 342. The first eigenvalue acquirer 310, the second eigenvalue acquirer 320, and the third eigenvalue acquirer 330 are described above with reference to Figure 3A .

[0088] The first estimator 341 can estimate the first biological information by using any one of the first eigenvalue, the second eigenvalue, and the third eigenvalue, or a combination of two or more of them. In this case, the first biological information may be blood pressure.

[0089] The second estimator 342 can estimate the second biological information based on the second eigenvalue. In this case, the second biological information may be skin elasticity and / or skin age. For example, Figure 6C shows the change in the distance between valleys or ridges for each age group when the force is similarly changed by a finger. For example, Figure 6C shows the overall average change mr of multiple subjects, the average change m20 of subjects in their 20s, the average change m30 of subjects in their 30s, the average change m40 of subjects in their 40s, and the average change m50 of subjects in their 50s. As Figure 6C shown, as the age increases, there is a greater distance change for the same contact force. As a result, overall, as the age increases, the skin elasticity decreases and the skin age increases.

[0090] The second estimator 342 can estimate the skin elasticity and / or skin age of each user by considering general skin characteristics. For example, the second estimator 342 can estimate the skin elasticity and / or skin age by considering the change trend of the second feature obtained from a specific user compared with the reference second eigenvalue obtained at a reference time. The following description gives some examples of the present disclosure, but the present disclosure is not limited thereto.

[0091] For example, the second estimator 342 can estimate the skin elasticity and / or skin age by applying a predefined estimation model to the change trend of the second eigenvalue obtained from a specific user compared with the reference second eigenvalue obtained based on the average change of multiple subjects. In this case, the estimation model can be defined as various linear or non-linear combination functions that use the difference between the reference second eigenvalue and the second eigenvalue obtained from the user as an input. However, the estimation model is not limited thereto.

[0092] In another example, reference second eigenvalue(s) may be obtained for each age group, gender, occupation, and health condition, or the reference second eigenvalue(s) may be divided into various groups based on their combination. Based on the second eigenvalue(s) obtained for a specific user, the second estimator 342 may compare the second eigenvalue(s) obtained for the user with the reference second eigenvalue(s) for each group, and may estimate the user's skin elasticity and / or skin age by using the skin elasticity and / or skin age of the corresponding group.

[0093] In another example, the second estimator 342 may relatively estimate that since the change in the second eigenvalue(s) obtained at the current time shows an increasing / decreasing trend compared to the reference second eigenvalue(s) obtained from a specific user at a reference time, the skin elasticity decreases / increases and the skin age increases / decreases.

[0094] In addition, Figure 3B the processor 300b in the embodiment of may include a mode controller. The mode controller may operate in a mode for estimating first biological information, a mode for estimating second biological information, and a mode for estimating both first biological information and second biological information.

[0095] For example, if the mode for estimating first biological information is set to the basic mode, or in response to a user's request for estimating first biological information, the mode controller may estimate the first biological information by driving the first eigenvalue acquirer 310, the second eigenvalue acquirer 320, the third eigenvalue acquirer 330, and the first estimator 341. In another example, if the mode for estimating second biological information is set to the basic mode, or in response to a user's request for estimating second biological information, the mode controller may estimate the second biological information by driving the second eigenvalue acquirer 320 and the second estimator 342. In yet another example, if the mode for estimating both first biological information and second biological information is set to the basic mode, or in response to a user's request for estimating both first biological information and second biological information, the mode controller may estimate both first biological information and second biological information by driving the first eigenvalue acquirer 310, the second eigenvalue acquirer 320, the third eigenvalue acquirer 330, the first estimator 341, and the second estimator 342.

[0096] Figure 7 is a flowchart showing a method for estimating biological information according to an embodiment of the present disclosure. Figure 7 The method of is Figure 1 by the device 100 for estimating biological information of Figure 2 and the device 200 for estimating biological information of execute an example of the method for estimating biological information, which has been described in detail above, so it will be briefly described below.

[0097] In response to a request for estimating biometric information, devices 100 and 200 for estimating biometric information may obtain a fingerprint image of a finger in operation 710. The request for estimating biometric information may be received at a predetermined interval or from a user or an external device. In this case, based on the received request for estimating biometric information, devices 100 and 200 for estimating biometric information may guide the force or pressure applied by the user's finger to the sensor unit.

[0098] Then, when the fingerprint image of the finger is obtained, devices 100 and 200 for estimating biometric information may measure the force or pressure applied by the finger to the sensor unit in operation 720. The user may change the pressure between the finger and the sensor unit by gradually increasing the pressing force when placing the finger on the sensor unit, or by gradually decreasing the pressing force when pressing the sensor unit with a force greater than or equal to a threshold value. Alternatively, when the finger contacts the sensor unit, the finger may be pressed by an external force to change the pressure between the finger and the sensor unit.

[0099] Subsequently, devices 100 and 200 for estimating biometric information may obtain a first eigenvalue based on the fingerprint image in operation 730. For example, devices 100 and 200 for estimating biometric information may obtain a pulse wave signal based on the change in the intensity of continuously obtained fingerprint images, and may obtain the maximum amplitude value of the pulse wave signal as the first eigenvalue.

[0100] Next, devices 100 and 200 for estimating biometric information may obtain a second eigenvalue based on the fingerprint image in operation 740. For example, devices 100 and 200 for estimating biometric information may obtain the distance between valleys or ridges of the fingerprint based on the fingerprint image, and may obtain the second eigenvalue based on the change in the distance obtained within a predetermined time period. For example, the second eigenvalue may include the slope value at the point where the change in distance is the largest, the slope value at the point where the change in distance is the smallest, and / or the average slope value obtained in each unit interval when the force / pressure axis is divided into predetermined unit intervals in a graph showing the change in the fingerprint pattern, where the graph of the fingerprint pattern change shows the relationship between the change in distance and the force / pressure. In this case, operations 730 and 740 do not have to be performed in chronological order, but may be performed simultaneously.

[0101] Then, devices 100 and 200 for estimating biometric information may obtain a third eigenvalue based on the first eigenvalue obtained in operation 730 and the force / pressure obtained in operation 720 in operation 750. For example, devices 100 and 200 for estimating biometric information may obtain the force / pressure value at the time point corresponding to the first eigenvalue as the third eigenvalue.

[0102] Subsequently, devices 100 and 200 for estimating biological information may estimate biological information by combining the first eigenvalue, the second eigenvalue, and the third eigenvalue in operation 760. For example, devices 100 and 200 for estimating biological information may estimate biological information (e.g., blood pressure) by applying predefined weights to each of the first eigenvalue, the second eigenvalue, and the third eigenvalue and linearly or non-linearly combining the weighted values.

[0103] Next, devices 100 and 200 for estimating biological information may output a biological information estimation result in operation 770. In this case, devices 100 and 200 for estimating biological information may output the biological information estimation result by appropriately using a display module, a speaker module, a haptic module through vibration, touch, etc.

[0104] Figure 8 is a flowchart showing a method for estimating biological information according to another embodiment of the present disclosure. Figure 8 The method of Figure 1 Devices 100 for measuring biological information and Figure 2 Devices 200 for measuring biological information execute an example of a method for estimating biological information, which will be briefly described below.

[0105] In response to a request for estimating biological information, devices 100 and 200 for estimating biological information may obtain a fingerprint image of a finger in operation 810.

[0106] Then, when a fingerprint image of the finger is obtained, devices 100 and 200 for estimating biological information may measure the force or pressure applied by the finger to the sensor unit in operation 820.

[0107] Subsequently, devices 100 and 200 for estimating biological information may obtain a first eigenvalue based on the fingerprint image in operation 830 and obtain a second eigenvalue in operation 840, and may obtain a third eigenvalue based on the first eigenvalue obtained in operation 830 and the force / pressure obtained in operation 820 in operation 850.

[0108] Next, devices 100 and 200 for estimating biological information may estimate a first biological information by combining the first eigenvalue, the second eigenvalue, and the third eigenvalue in operation 860, and may estimate a second biological information based on the second eigenvalue obtained in operation 840 in operation 870. In this case, the first biological information may be blood pressure, and the second biological information may be skin elasticity and / or skin age.

[0109] Then, devices 100 and 200 for estimating biological information may output a biological information estimation result in operation 880.

[0110] Figure 9 FIG. is an example diagram of a wearable device showing an example of wearable devices 100 and 200 to which devices for estimating biological information are applied. As Figure 9 shown, the wearable device may be a smartwatch or a smart band wearable device worn on the wrist, but is not limited thereto. Examples of wearable devices may include smart rings, smart necklaces, smart glasses, etc.

[0111] Referring Figure 9 , the wearable device 900 includes a main body 910 and a band 930.

[0112] The band 930 connected to both ends of the main body 910 may be flexible so as to bend around the user's wrist. The band 930 may be composed of a first band and a second band that are separated from each other. Respective ends of the first band and the second band are connected to the main body 910, and their other ends may be connected to each other via a connecting device. In this case, the connecting device may be formed as a magnetic connection, a Velcro connection, a pin connection, etc., but is not limited thereto. In addition, the band 930 is not limited thereto, and may be integrally formed as a non-detachable band.

[0113] In this case, air may be injected into the band 930, or the band 930 may be provided with an airbag so that the band 930 may have elasticity according to a change in pressure applied to the wrist, and may transmit the pressure change of the wrist to the main body 910.

[0114] A battery may be embedded in the main body 910 or the band 930 to supply power to the wearable device 900.

[0115] In addition, the main body 910 may include a sensor unit 920 mounted on one side thereof. The sensor unit 920 may include a first sensor for obtaining a fingerprint image of a finger and a second sensor for measuring the force / pressure applied to the first sensor by the finger. The first sensor may include a light source and a CMOS image sensor (CIS). In addition, the second sensor may include a force sensor, a pressure sensor, an area sensor, etc.

[0116] A processor may be mounted in the main body 910. The processor may estimate biological information based on the obtained fingerprint image of the finger and the obtained force / pressure. For example, the processor may obtain a first eigenvalue related to a pulse wave based on the fingerprint image of the finger, may obtain a second eigenvalue related to skin elasticity, and may obtain a third eigenvalue based on the force / pressure. In addition, the processor may estimate biological information (such as blood pressure, skin elasticity, skin age) by combining the first eigenvalue, the second eigenvalue, and the third eigenvalue.

[0117] Based on a request for estimating biological information received from a user, the processor can guide the user to apply force / pressure through a display, and based on the estimated biological information, the processor can provide the user with a biological information estimation result through the display. The display can be mounted on the front surface of the main body 910, output guiding information and / or biological information estimation results, and receive a touch input from the user and send the touch input to the processor.

[0118] In addition, the main body 910 may include a storage device for storing information processed by the processor, reference information for estimating biological information, and the like.

[0119] In addition, the main body 910 may include a manipulator 940 that receives a control command from the user and sends the received control command to the processor. The manipulator 940 can be mounted on the side of the main body 910 and may have a function of inputting a command for turning on / off the wearable device 900.

[0120] In addition, the wearable device 900 may include a communication interface for sending and receiving various data with an external device, and various other modules for additional functions provided by the wearable device 900.

[0121] Figure 10 FIG. is a diagram showing an example of a smart device to which devices 100 and 200 for estimating biological information are applied. In this case, the smart device can be a smart phone, a tablet PC, or the like.

[0122] Referring to Figure 10 , the smart device 1000 includes a main body 1010 and a sensor unit 1030 mounted on one surface of the main body 1010. The sensor unit 1030 may include a first sensor including one or more light sources 1031 and a detector 1032, and a second sensor provided at the lower end of the first sensor and measuring the force / pressure applied to the first sensor by a finger. In this case, the detector 1032 may include a CMOS image sensor (CIS) 1032.

[0123] As Figure 10 shown, the main body 1010 may include an image sensor 1020. The image sensor 1020 captures various images and can obtain a fingerprint image of a finger when the finger touches the sensor unit 1030. If the CMOS image sensor 1032 is mounted in the first sensor of the sensor unit 1030, the image sensor 1020 may be omitted.

[0124] The processor may be mounted in the main body 1010 and may estimate biological information (such as blood pressure, skin elasticity, or skin age) based on the fingerprint image and force / pressure of the finger obtained by the sensor unit 1030.

[0125] In addition, a display, a communication interface, etc. may be installed in the main body 1010 to output and provide the biological information processed by the processor to the user, or to send the biological information to an external device. Various other modules for performing various functions may be installed in the main body 1010.

[0126] Embodiments of the present disclosure may be implemented by computer-readable code written on a non-transitory computer-readable medium and executed by a processor. The non-transitory computer-readable medium may be any type of recording device that stores data in a computer-readable manner.

[0127] Examples of the non-transitory computer-readable medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and carrier waves (e.g., data transmission via the Internet). The non-transitory computer-readable medium may be distributed over a plurality of computer systems connected to a network such that the computer-readable code is written thereto and executed therefrom in a decentralized manner. Functional programs, codes, and code segments for implementing the embodiments of the present disclosure may be obtained by ordinary programmers in the field to which the present disclosure pertains.

[0128] The present disclosure has been described with respect to example embodiments. However, it will be apparent to those skilled in the art that various changes and modifications can be made without changing the technical concept and features of the present disclosure. Therefore, it is clear that the above embodiments are illustrative in all respects and are not intended to limit the present disclosure.

Claims

1. A device for estimating a user's biological information, the device comprising: A first sensor configured to obtain a fingerprint image of the user's finger; A second sensor configured to measure the force exerted by the finger when the finger is placed on the finger contact surface of the first sensor; And A processor configured to: Obtain a first eigenvalue related to a pulse wave and a second eigenvalue related to skin elasticity based on the fingerprint image; Obtain a third eigenvalue based on the force; And Estimate the user's biological information based on the first eigenvalue, the second eigenvalue, and the third eigenvalue, wherein the processor is further configured to: Obtain a change in the distance between ridges or valleys of the fingerprint from the fingerprint image based on the finger pressing the first sensor; and Obtain the second eigenvalue based on the change in distance.

2. The device according to claim 1, wherein, The first sensor includes: One or more light sources configured to emit light onto the finger; and One or more complementary metal oxide semiconductor image sensors.

3. The device according to claim 1, wherein, The processor is further configured to: Obtain a pulse wave signal based on the fingerprint image; and Obtain the first eigenvalue based on at least one of the maximum amplitude value and the minimum amplitude value of the pulse wave signal.

4. The device according to claim 1, wherein, The processor is further configured to: Obtain a first average value of the distance between ridges or valleys at one or more points of the first fingerprint image, and a second average value of the distance at one or more points of the second fingerprint image, wherein the first fingerprint image and the second fingerprint image are continuously obtained in a predetermined time period; and Obtain the difference between the first average value and the second average value as the change in distance.

5. The device according to claim 1, wherein, The processor is further configured to: Generate a curve graph of the change in the fingerprint pattern based on the change in distance; and Obtain the second eigenvalue based on the curve graph of the change in the fingerprint pattern.

6. The apparatus according to claim 5, wherein, The processor is further configured to: Obtain at least one of a maximum slope value, a minimum slope value, and an average slope value in each predefined unit interval as the second eigenvalue.

7. The device according to claim 6, wherein, The processor is further configured to: Generate a differential curve graph by differentiating the curve graph of the change in the fingerprint pattern; and Obtain the second eigenvalue by using the differential curve graph.

8. The device according to any one of claims 1 to 7, wherein, The processor is further configured to: Obtain the force value at the time point corresponding to the first eigenvalue as the third eigenvalue.

9. The device according to any one of claims 1 to 7, wherein, The processor is further configured to: Combine the first eigenvalue, the second eigenvalue, and the third eigenvalue; and Estimate the biological information by applying a predefined estimation model to the result of combining the first eigenvalue, the second eigenvalue, and the third eigenvalue.

10. The device according to claim 9, wherein, The processor is further configured to: Apply a first weight to the first eigenvalue to obtain a first weighted value; Apply a second weight to the second eigenvalue to obtain a second weighted value; Apply a third weight to the third eigenvalue to obtain a third weighted value; And Combine the first weighted value, the second weighted value, and the third weighted value.

11. The apparatus according to claim 10, wherein, Each of the first weight, the second weight, and the third weight is a predefined fixed value, or a value adjusted by at least considering the user characteristics or the type of biological information.

12. The device according to any one of claims 1 to 7, wherein, The processor is further configured to: When the fingerprint image is obtained from the finger, control the output interface to output guiding information to guide the user to adjust the force between the finger and the first sensor.

13. The device according to any one of claims 1 to 7, wherein The biological information includes one or more of heart rate, blood pressure, vascular age, arterial stiffness, aortic pressure waveform, vascular compliance, pressure index, degree of fatigue, skin elasticity, and skin age.

14. A computer-readable storage medium storing a program, which when executed by a processor, causes the processor to execute a method for estimating a user's biological information, the method comprising: Obtaining a fingerprint image of the user's finger through a first sensor; Measuring, through a second sensor, the force exerted by the finger when the finger is placed on the finger contact surface of the first sensor; Obtaining a first eigenvalue related to a pulse wave based on the fingerprint image; Obtaining a second eigenvalue related to skin elasticity based on the fingerprint image; Obtaining a third eigenvalue based on the force; and Estimating the biological information based on the first eigenvalue, the second eigenvalue, and the third eigenvalue, wherein the step of obtaining the second eigenvalue includes: Obtaining a change in the distance between ridges or valleys of the fingerprint from the fingerprint image based on the finger pressing the first sensor; and Obtaining the second eigenvalue based on the obtained change in distance.

15. The computer-readable storage medium according to claim 14, wherein, The step of obtaining the first eigenvalue includes: Obtaining a pulse wave signal from the fingerprint image; and Obtaining the first eigenvalue based on at least one of the maximum amplitude value and the minimum amplitude value of the pulse wave signal.

16. The computer-readable storage medium according to claim 14, wherein, The step of obtaining the second eigenvalue includes: Obtaining a first average value of a first distance between ridges or valleys at one or more points of a first fingerprint image; Obtaining a second average value of a second distance between ridges or valleys at one or more points of a second fingerprint image; and Obtaining the difference between the first average value and the second average value as the change in distance, wherein the first fingerprint image and the second fingerprint image are continuously obtained within a predetermined time period.

17. The computer-readable storage medium according to claim 16, wherein, The step of obtaining the second eigenvalue includes: Generating a curve graph of fingerprint pattern change based on the change in distance; and Obtaining the second eigenvalue based on the curve graph of fingerprint pattern change.

18. The computer-readable storage medium according to claim 17, wherein, The step of obtaining the second eigenvalue includes: obtaining at least one of a maximum slope value, a minimum slope value, and an average slope value for each predefined unit interval as the second eigenvalue.

19. The computer-readable storage medium according to any one of claims 14 to 18, wherein, The step of obtaining the third eigenvalue includes: obtaining the force value at the time point corresponding to the first eigenvalue as the third eigenvalue.

20. The computer-readable storage medium according to any one of claims 14 to 18, wherein, The step of estimating the biological information includes: Combining the first eigenvalue, the second eigenvalue, and the third eigenvalue; and Estimating the biological information by applying a predefined estimation model to the result of combining the first eigenvalue, the second eigenvalue, and the third eigenvalue.

21. The computer-readable storage medium according to any one of claims 14 to 18, further comprising: Guiding the user to adjust the force between the finger and the first sensor when the fingerprint image is obtained from the finger.

Images