Device and method for providing biometric information

Abstract

A device for providing biometric information according to an embodiment of the present invention comprises: a collection unit for collecting imaging information related to a user; a facial area detection unit for detecting a facial area of the user on the basis of the image capture information; an image-of-interest extraction unit for extracting an image of interest including at least one region of interest for the facial area; a color space adjustment unit for adjusting color space information of the image of interest; a peak value detection unit for detecting at least one peak value for the image of interest the color space information of which has been adjusted; and a biometric information calculation unit for calculating biometric information of the user on the basis of the at least one peak value.

Description

Biometric information provision device and method

[0001] The present invention relates to a device and method for providing biometric information using a robot.

[0002] Recent robot development is being reoriented towards the development of companion robots that are closely related to and can deeply penetrate human lives, rather than simply mechanical computing devices as in the past. These companion robots are called interaction robots because they interact with humans.

[0003] However, most of the interactions performed by currently released interactive robots, such as smart speakers and toy robots, are simple and passive actions that receive input from users and output responses accordingly. Therefore, it is difficult for robots to recognize users and actively provide biometric information (health information) of recognized users. It is more appropriate to view them as simple computing devices, so differentiation from them is becoming more important.

[0004] In particular, older adults who spend a significant amount of time at home often develop chronic or sudden health problems due to aging. In these situations, family members or caregivers provide care, but ongoing care requires significant time and money.

[0005] However, as mentioned above, the currently developed robot has a problem in that it provides biometric information about the user only through various inputs received from the user who is the object of care.

[0006] Currently, care robots developed for this purpose offer healthcare services, emergency support, and media playback services that measure the user's biometric information and provide results. However, these services are often provided only after the user has requested a specific service. Therefore, it is difficult for robots to actively collect and provide biometric information, recognize emergency situations, and provide appropriate services in response to the situation.

[0007] The present invention provides a biometric information providing device and method for actively collecting and providing biometric information of a user using a robot.

[0008] In addition, the present invention seeks to provide a biometric information providing device and method capable of calculating and providing biometric information based on photographed information collected by moving to a location where a user can be recognized through a robot or recognizing the face of a recognized user.

[0009] In addition, the present invention seeks to provide a device and method for providing highly accurate biometric information using an artificial intelligence model.

[0010] However, the technical tasks that this embodiment seeks to accomplish are not limited to the technical tasks described above, and other technical tasks may exist.

[0011] As a means for achieving the above-described technical task, a device for providing biometric information according to one embodiment of the present invention includes a collection unit for collecting photographing information related to a user; a facial region detection unit for detecting a facial region of the user based on the photographing information; an image of interest extraction unit for extracting an image of interest including at least one region of interest for the facial region; a color space adjustment unit for adjusting color space information for the image of interest; a peak value detection unit for detecting at least one peak value for the image of interest for which the color space information is adjusted; and a biometric information calculation unit for calculating biometric information for the user based on the at least one peak value.

[0012] In addition, a method for providing biometric information according to another embodiment of the present invention includes the steps of collecting photographing information related to a user; detecting a facial region of the user based on the photographing information; extracting an image of interest including at least one region of interest for the facial region; adjusting color space information for the image of interest; detecting at least one peak value for the image of interest having adjusted color space information; and calculating biometric information for the user based on the at least one peak value.

[0013] The above-described problem-solving methods are merely exemplary and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, additional embodiments may exist, as described in the drawings and detailed description of the invention.

[0014] According to any one of the above-described problem solving means of the present invention, a robot can be used to actively and accurately recognize a user and provide biometric information based on photographed information of the recognized user.

[0015] In addition, since the present invention uses only an RGB (Red, Green, Blue) camera without using any special equipment, it enables real-time health monitoring of users who need to periodically check the biometric information of elderly people or patients, enabling immediate response to emergency situations or deteriorating health.

[0016] In addition, the present invention can collect biometric information by automatically controlling the shooting direction and angle regardless of the location or situation of an elderly person or patient using a robot.

[0017] In addition, the present invention can quickly and accurately respond to emergency situations by using a robot to move to the elderly or patient and control the shooting direction and angle before acquiring biometric information even in the event of an emergency situation involving the elderly or patient.

[0018] Figure 1 is a configuration diagram of a biometric information providing device according to one embodiment of the present invention.

[0019] Figure 2 is a configuration diagram of a direction change unit according to one embodiment of the present invention.

[0020] FIG. 3 is a drawing for explaining a first form in which a biometric information providing device according to one embodiment of the present invention is implemented.

[0021] FIG. 4 is a drawing for explaining a second form in which a biometric information providing device according to one embodiment of the present invention is implemented.

[0022] FIG. 5 is a drawing for explaining a method for a collection unit to collect shooting information according to one embodiment of the present invention.

[0023] FIG. 6 is a drawing for explaining a method for a collection unit (110) to collect shooting information in an environment where the illuminance is below a preset value according to one embodiment of the present invention.

[0024] FIG. 7 is a drawing for explaining a method for setting a region of interest for a facial area according to one embodiment of the present invention.

[0025] FIG. 8 is a diagram for explaining a method for adjusting color space information for an image of interest according to one embodiment of the present invention.

[0026] FIG. 9 is a diagram illustrating a method for detecting at least one peak value for an image of interest with adjusted color space information according to one embodiment of the present invention.

[0027] FIG. 10 is a diagram illustrating a method for calculating biometric information including at least one of an average heart rate and an average respiration rate for a user based on at least one peak value according to one embodiment of the present invention.

[0028] FIG. 11 is a diagram illustrating a method for calculating biometric information including at least one of the highest blood pressure and the lowest blood pressure for a user based on at least one peak value according to one embodiment of the present invention.

[0029] FIG. 12 is a diagram illustrating a method for generating biometric information including at least one of heart health information and stress information for a user based on at least one peak value according to one embodiment of the present invention.

[0030] Figure 13 is a flowchart for explaining a biometric information providing method according to one embodiment of the present invention.

[0031] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement them. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, irrelevant parts have been omitted for clarity of description, and similar reference numerals have been used throughout the specification to indicate similar elements.

[0032] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element in between. Furthermore, when a part is said to "include" a component, this should be understood to mean that, unless specifically stated to the contrary, it may include other components rather than excluding them, and does not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0033] In this specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Furthermore, a single unit may be realized using two or more pieces of hardware, and two or more units may be realized by a single piece of hardware.

[0034] Some of the operations or functions described herein as being performed by a terminal or device may instead be performed by a server connected to the terminal or device. Similarly, some of the operations or functions described as being performed by a server may also be performed by a terminal or device connected to the server.

[0035] Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings.

[0036] Figure 1 is a configuration diagram of a biometric information providing device (100) according to one embodiment of the present invention.

[0037] The biometric information providing device (100) of FIG. 1 and each component of the biometric information providing device (100) can be connected to each other through a network, and can also be connected to a server (e.g., a health management server) or an external device (e.g., a guardian device) through a network. For example, as illustrated in FIG. 1, the biometric information providing device (100) and the server can be connected to the network simultaneously or at time intervals. In addition, the network refers to a connection structure that enables information exchange between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), the Internet (WWW: World Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television communication networks, etc. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth, infrared, ultrasonic, visible light communication (VLC), and LiFi.

[0038] Referring to FIG. 1, a biometric information providing device (100) may include a collection unit (110), a facial area detection unit (120), an image of interest extraction unit (130), a color space adjustment unit (140), a peak value detection unit (150), and a biometric information calculation unit (160).

[0039] The collection unit (110) can capture images of the user or the environment surrounding the user via a visual sensor, such as a camera. The visual sensor may include a camera that generates image information of typical RGB pixels. In an embodiment, the visual sensor may further include image capture devices, such as an infrared camera for capturing objects in dark environments and a depth camera capable of generating depth information regarding the distance to the subject.

[0040] The facial region detection unit (120) can detect the user's facial region based on the collected photographed information. The facial region detection unit (120) can include a preset deep learning-based algorithm, and input the collected photographed information into the algorithm to recognize single or multiple user faces in the photographed information, detect the facial region, and calculate the facial recognition rate. Through this, the biometric information providing device (100) can calculate and provide biometric information for single or multiple users.

[0041] When a user is recognized, the biometric information providing device (100) outputs at least one of a guidance voice and a shooting guideline corresponding to the user's facial recognition rate calculated through the facial area detection unit (120), reduces or enlarges the shooting screen based on the facial recognition rate, and sets the rotation rate of the shooting information based on the facial recognition rate.

[0042] In one embodiment, the biometric information providing device (100) may further include a speaker unit (not shown) that outputs voice. The speaker unit may output various sounds including voice. Here, voice corresponds to auditory information output by the biometric information providing device (100) to interact with the user, and the type of voice can be set in various ways through a media-only application installed on the user's terminal (not shown) or direct control of the biometric information providing device (100). For example, the type of voice output through the speaker unit may be selected from among various voices, such as a male voice, a female voice, an adult voice, and a child's voice, and even the type of language, such as Korean, English, Japanese, and French, may be selected.

[0043] The biometric information providing device (100) may further include at least one of a moving unit (180 in FIG. 3) controlled to move to a location where the user can be recognized and a direction changing unit (170 in FIG. 2) that changes the shooting direction of the collection unit (110) based on the user's facial recognition rate.

[0044] For example, the moving unit may be provided on one side of the biometric information providing device (100) and may be configured to move the biometric information providing device (100). The moving unit may include a motor and a plurality of wheels that can provide a means for moving within a specific space by a control unit to be described later, and these may be combined to perform functions of driving, changing direction, and rotating the biometric information providing device (100).

[0045] Meanwhile, referring to FIGS. 1 and 2, the direction change unit may include a tilting unit (171) that rotates the collection unit (110) in a vertical direction, a panning unit (172) that rotates the collection unit (110) in a horizontal direction, and an elevating unit (173) that controls the shooting height of the collection unit (110).

[0046] The control unit may control at least one of the moving unit and the direction change unit based on a facial recognition rate, a preset time (e.g., an alarm time set by the user), a preset event (e.g., an emergency situation, etc.), or user input (e.g., voice input for movement, movement input via a linked application, etc.). This will be described with reference to FIGS. 3 and 4.

[0047] FIG. 3 is a drawing for explaining a first form (310) in which a biometric information providing device according to one embodiment of the present invention is implemented, and FIG. 4 is a drawing for explaining a second form (410) in which a biometric information providing device according to one embodiment of the present invention is implemented.

[0048] Referring to FIG. 3, the bio-information providing device can be implemented in a first form (310) including a moving part (180) and a direction changing part (170).

[0049] Referring to FIG. 3, the biometric information providing device implemented in the first form (310) can control the moving part (180) to move to a place where the user is recognized, and when the user is recognized, can control the direction changing part (170) to change the shooting direction of the collecting part based on the user's facial recognition rate.

[0050] In one embodiment, the biometric information providing device implemented in the first form (310) can output an alarm when a preset time has elapsed. After the alarm is output, or based on the captured image information collected while the moving unit (180) is controlled, the user can be recognized. Once the user is recognized, the biometric information providing device can control at least one of the tilting unit, the panning unit, and the lifting unit based on the facial recognition rate to change the capturing direction of the collecting unit.

[0051] In one embodiment, the biometric information providing device implemented in the first form (310) can detect a fall or emergency situation of the user through photographed information or can detect the user's emergency situation based on an emergency signal received from an external device. When the biometric information providing device detects the user's emergency situation, the device can move to the user through a moving unit, recognize the user's face through control of a direction changing unit, collect biometric information, and transmit the biometric information to a guardian device or an emergency rescue agency.

[0052] Referring to FIG. 4, the bio-information providing device can be implemented in a second form (410) including a direction changing unit (170).

[0053] Referring to FIG. 4, the biometric information providing device implemented in the second form (410) can control the direction changing unit (170) to change the shooting direction of the collection unit based on the user's facial recognition rate when the user is recognized.

[0054] In one embodiment, a biometric information providing device implemented in the second form (410) can output an alarm at a preset time. After the alarm is output, or based on the collected photographic information, the user can be recognized.

[0055] When a user is recognized, the biometric information providing device can change the shooting direction of the collection unit by controlling at least one of the tilting unit, the panning unit, and the lifting unit based on the facial recognition rate.

[0056] Referring back to FIG. 1, when a user is recognized, the collection unit (110) outputs at least one of a guidance voice and a shooting guideline corresponding to the user's facial recognition rate, reduces or enlarges the shooting screen based on the facial recognition rate, and sets the rotation rate of the shooting information based on the facial recognition rate. This will be described with reference to FIG. 5.

[0057] FIG. 5 is a drawing for explaining a method for a collection unit to collect shooting information according to one embodiment of the present invention.

[0058] Referring to FIG. 5, if the facial recognition rate is below a preset level, the biometric information providing device may output guidance voices corresponding to the facial recognition rate through a speaker unit (not shown) to improve the facial recognition rate. Here, the biometric information providing device may include a display unit (190) that provides the photographing information and the photographing screen collected through the collection unit.

[0059]

[0060] *Referring to (a) of FIG. 5, if the biometric information providing device determines that the user's face is not located in the shooting guideline (510) provided through the display unit (190) based on the facial recognition rate, it can output a guidance voice and guidance message requesting that the face be located in the shooting guideline (510) through the display unit (190) and the speaker unit, respectively.

[0061] Referring to (b) of FIG. 5, if the biometric information providing device determines that the user's face is recognized as being smaller than the size of the shooting guideline (510) provided through the display unit (190) based on the facial recognition rate, the device may output a guidance voice and guidance message requesting the user to bring the face closer to the collection unit and position the face on the shooting guideline (510) through the display unit (190) and the speaker unit, respectively.

[0062] Referring to (c) of FIG. 5, if the biometric information providing device determines that the user's face is recognized as being larger than the size of the shooting guideline (510) provided through the display unit (190) based on the facial recognition rate, the device may output a guidance voice and guidance message requesting to move the face away from the collection unit and position the face on the shooting guideline (510) through the display unit (190) and the speaker unit, respectively.

[0063] Referring to (d) of FIG. 5, if the biometric information providing device determines that the user's face is recognized in accordance with the shooting guidelines (510) provided through the display unit (190) based on the facial recognition rate, it can output a guidance voice and guidance message indicating that biometric information is being measured from the recognized face and a measurement rate through the display unit (190) and the speaker unit, respectively.

[0064] As described above, the biometric information providing device can output a shooting guideline (510) corresponding to the facial recognition rate and a guidance voice and guidance message through the display unit (190).

[0065] In one embodiment, the biometric information providing device may reduce or enlarge the captured screen based on the image size of the recognized user (e.g., the pixel size of the object recognized as the user) when the facial recognition rate is below a preset level.

[0066] In one embodiment, when the facial recognition rate of the collected photographed information is below a preset level, the biometric information providing device can improve the facial recognition rate by setting the rotation rate of the collected photographed information based on the facial recognition rate.

[0067] When the facial recognition rate of the collected photographed information is below a preset level, the biometric information providing device sets the rotation rate of the photographed information until the facial recognition rate reaches the preset level, and changes the shooting direction of the collection unit based on the set rotation rate to collect photographed information with an improved facial recognition rate.

[0068] In one embodiment, the display unit (190) is a type of liquid crystal display device, and can output one or more of text, images, and videos of predetermined information. Here, the predetermined information can include status information of the biometric information providing device, such as communication status strength information, remaining battery level information, and wireless Internet ON / OFF information. The display unit (190) can display media content that the biometric information providing device, which will be described later, displays in response to biometric information, and can also display information that converts voice output by the biometric information providing device through the speaker into text. For example, if the biometric information providing device sets a wake-up alarm while providing an appropriate service to the user through situational awareness, the text corresponding to this can be output on the display unit (190) as "Wake up at 7:00 AM."

[0069] The text output on the display unit (190) may be one of the previously described pieces of information repeatedly output, or multiple pieces of information may be output alternately, or specific information may be set as default and output. For example, status information of the biometric information providing device, such as communication status strength information, remaining battery level information, and wireless Internet ON / OFF information, may be set as small text at the top or bottom of the display unit (190) by default and continuously output, and other information may be output alternately.

[0070] Meanwhile, it is said that the display unit (190) can output one or more of images and videos. In this case, it is preferable to implement the display unit (190) as a liquid crystal display device with a high resolution and a large size, rather than a case where only text is output, in order to improve visibility. The display unit (190) of FIG. 1 should be viewed as an example.

[0071] Meanwhile, a method for a biometric information providing device to secure a user's facial recognition rate in a low-light environment will be described with reference to FIG. 6 together with FIG. 1.

[0072] Figure 6 is a diagram illustrating a method for a collection unit to collect photographic information in an environment where the illuminance is below a preset value, according to one embodiment of the present invention. The collection unit can recognize that the environment in which the user is currently being photographed is dark based on illuminance values ​​extracted from a light sensor or the collected photographic information. In other words, if the illuminance of the environment in which the user is being photographed is below a preset value, the collection unit can adjust the illuminance environment as follows.

[0073] Referring to FIG. 6, in order to secure a user's face recognition rate in a low-illuminance environment as illustrated in (a), the biometric information providing device separates a shooting guideline (510) including a face on the display unit (190) and an area outside thereof from the moment the shooting information is collected when the illuminance is below a preset value or from the time the collection unit operates to collect the shooting information, as illustrated in (b), outputs a shooting image of the face as is in the area inside the shooting guideline (510), outputs a preset color (e.g., white) in the area outside the shooting guideline (510), and sets the brightness value outside the shooting guideline (510) to a preset brightness value (e.g., maximum value).

[0074] By setting the color and brightness values ​​of the exterior of the photographing guideline (510) as described above, the exterior of the photographing guideline (510) can serve as a kind of external lighting, thereby securing a user's facial recognition rate in low-light environments. Furthermore, the biometric information providing device can overcome the problem of difficulty in collecting photographing information in dark environments, which is a drawback of conventional technologies.

[0075] Referring back to FIG. 1, the image of interest extraction unit (130) can extract an image of interest that includes at least one region of interest (ROI) in the facial area. For example, the ROI may be an area of ​​the face with a high distribution of blood vessels, such as the forehead, chin, left and right cheeks, and nose, making it easy to detect biometric information. This will be described with reference to FIG. 7.

[0076] FIG. 7 is a drawing for explaining a method for setting a region of interest (710) for a facial area according to one embodiment of the present invention.

[0077] Referring to FIG. 7, the interest image extraction unit can set at least one region of interest (710) from the facial region based on a facial recognition model that recognizes the position and size of a face from an input image and outputs a facial region, and an interest region (710) output model that outputs a region of interest (710) from the facial region, and extract an interest image including the set at least one region of interest (710).

[0078] For example, a face recognition model can identify the location of a face within an image and generate a bounding box corresponding to that location. A face recognition model can utilize a detection model based on a convolutional neural network (CNN), and object detection algorithms such as RCNN, Fast RCNN, YOLO, Single Shot Detector (SSD), Retina-Net, and Pyramid Net can be utilized.

[0079] A facial recognition model can extract the location of at least one human face within an image and generate a bounding box as output. The bounding box generated by the facial recognition model can be used as input data for a region of interest (710) output model.

[0080] To improve the accuracy of the facial recognition model, the robot can be fed with information about the user's height, either pre-entered or estimated using a skeletal model. This allows the biometric information provider to control the moving or directional elements to orient the collection unit toward the user's face, enabling accurate recognition of the user's face even at close range while standing.

[0081] Meanwhile, the output model of the region of interest (710) can receive a bounding box generated through a facial recognition model using, for example, a deep learning algorithm, and set each region of interest (710) of the face by distinguishing them, and can extract an image of interest including the region of interest (710) by maintaining the set region of interest (710) even if a person moves for each frame of the shooting information.

[0082] The color space control unit can control the color space information for the image of interest. For example, the color space control unit can amplify the color space information for the image of interest based on a color magnification algorithm. This will be described with reference to FIG. 8.

[0083] FIG. 8 is a diagram for explaining a method for adjusting color space information for an image of interest according to one embodiment of the present invention.

[0084] The color space control unit can amplify the color space information of the image of interest to further highlight the changes in blood flow distributed on the face of the image of interest using a color amplification algorithm. The color amplification algorithm can be RGB, YC b C r , HSV, etc., can be applied to all color spaces.

[0085] Specifically, Fig. 8 illustrates an example of using a color amplification algorithm to visualize a human pulse. Fig. 8 (a) illustrates four frames of an original image sequence, (b) illustrates four frame images in which color space information for the four frame images of (a) is amplified to create an amplified pulse signal, and (c) illustrates scan lines (top) for an input image (a) and scan lines (bottom) for an output image (b) that are structured in time series. Periodic color space information amplification can be confirmed through (c). No change is observed in the scan line sequence (top) of (a) in (c), but a change can be observed in the expanded scan line sequence (bottom) of (b).

[0086] Referring back to FIG. 1, the peak value detection unit (150) can detect at least one peak value for an image of interest whose color space information has been adjusted. For example, the peak value detection unit (150) can perform noise filtering on the image of interest whose color space information has been adjusted, and apply a bandpass filter to extract at least one of pulse and respiration. This will be described with reference to FIG. 9.

[0087] FIG. 9 is a diagram illustrating a method for detecting at least one peak value for an image of interest with adjusted color space information according to one embodiment of the present invention.

[0088] Figure 9 (a) shows the image of interest with adjusted color space information before (blue line) and after (red line) performing noise filtering to remove noise caused by user movement and external environment.

[0089] Figure 9 (b) shows time information representing a signal in which noise for each frame obtained over a preset period of time is filtered, and (c) shows the time information converted into frequency information, and is the result of applying a band-pass filter to show the maximum peak value for each pulse and respiration for each region of interest.

[0090] Referring to (c) of Fig. 9, the peak value detection unit can extract at least one of pulse and respiration for each region of interest based on the maximum peak value.

[0091] Specifically, the peak value detection unit can detect at least one peak value for each region of interest from frequency information of an image of interest to which noise filtering and a band-pass filter are applied. Here, at least one peak value is related to at least one of a heart rate and a respiration rate, and the at least one peak value can include at least one maximum peak value for each region of interest.

[0092] Referring back to FIG. 1, the biometric information generation unit (160) can generate biometric information about a user based on at least one peak value. This will be described with reference to FIGS. 10 to 12.

[0093] FIG. 10 is a diagram for explaining a method for calculating bio-information including at least one of an average heart rate and an average respiration rate for a user based on at least one peak value according to one embodiment of the present invention, FIG. 11 is a diagram for explaining a method for calculating bio-information including at least one of a maximum blood pressure and a minimum blood pressure for a user based on at least one peak value according to one embodiment of the present invention, and FIG. 12 is a diagram for explaining a method for calculating bio-information including at least one of heart health information and stress information for a user based on at least one peak value according to one embodiment of the present invention.

[0094] First, referring to FIG. 10, the biometric information generating unit can generate biometric information including at least one of an average heart rate and an average respiration rate for the user based on at least one peak value.

[0095] Referring to FIG. 10, the biometric information generation unit may generate a heart rate and a respiration rate for each region of interest based on at least one peak value for each region of interest. Thereafter, the biometric information generation unit may use the average value of the generated heart rate and respiration rate for each region of interest to generate an average heart rate and an average respiration rate, and may provide the generated average heart rate and average respiration rate to the user as biometric information.

[0096] Referring to FIG. 11, the biometric information generating unit may generate biometric information based on an artificial intelligence algorithm that calculates a PTT (Pulse Transit Time) by comparing at least one peak value, and receives at least one of the PTT and heart rate as input and outputs at least one of the maximum blood pressure and the minimum blood pressure for the user. More specifically, the artificial intelligence algorithm may include a machine learning algorithm. Here, the biometric information generated based on the machine learning algorithm may include at least one of the maximum blood pressure and the minimum blood pressure for the user.

[0097] Referring to (a) of Fig. 11, the bio-information calculation unit can calculate PTT by comparing at least one peak value for each region of interest. Referring to (b), the bio-information calculation unit can detect the maximum peak time of an rPPG (Remote Photoplethysmography) signal for each at least one region of interest. Thereafter, the bio-information calculation unit (160) calculates PTT by comparing the maximum peak values ​​for each of a preset number (e.g., 2) of regions of interest, and inputs the calculated PTT and heart rate as input data to a machine learning algorithm to output at least one of the highest blood pressure and the lowest blood pressure.

[0098] Referring to FIG. 12, the bio-information generating unit can generate an average HRV (Heart Rate Variability) from at least one peak value based on at least one PTT generated through comparison between at least one peak value, and generate bio-information including heart health information for the user based on the average HRV.

[0099] Meanwhile, the bio-information production unit can produce bio-information including stress information about the user based on the produced ratio of low-frequency components and high-frequency components based on the average HRV.

[0100] Thereafter, the biometric information providing device can provide biometric information including at least one of average heart rate, average respiration rate, maximum blood pressure, minimum blood pressure, heart health information, and stress information calculated through the biometric information calculating unit through the display unit or speaker unit.

[0101] Figure 13 is a flowchart for explaining a biometric information providing method according to one embodiment of the present invention.

[0102] Referring to FIG. 13, a method for providing biometric information according to an embodiment of the present invention may include a step of collecting photographing information related to a user (S10), a step of detecting a facial area of ​​the user based on the photographing information (S20), a step of extracting an image of interest including at least one region of interest for the facial area (S30), a step of adjusting color space information for the image of interest (S40), a step of detecting at least one peak value for the image of interest having adjusted color space information (S50), and a step of calculating biometric information for the user based on the at least one peak value (S60).

[0103] Meanwhile, the method for providing biometric information may further include at least one of a step of moving to a place where the user can be recognized and a step of changing the shooting direction to collect shooting information based on the user's facial recognition rate, which are performed before the step (S10) of collecting shooting information related to the user.

[0104] The step of changing the shooting direction for collecting shooting information based on the user's facial recognition rate may include a step of rotating the shooting direction horizontally, a step of rotating the shooting direction vertically, and a step of controlling the shooting height for the shooting direction.

[0105] Meanwhile, the method for providing biometric information may further include a step of performing at least one of a step of moving to a place where a user can be recognized based on a facial recognition rate, a preset time, a preset event, or a user input, and a step of changing a shooting direction.

[0106] Meanwhile, the step of collecting shooting information (S10) may include a step of outputting at least one of a guidance voice and a shooting guideline corresponding to the user's facial recognition rate when the user is recognized, a step of reducing or enlarging the shooting screen based on the facial recognition rate, and a step of setting the rotation rate of the shooting information based on the facial recognition rate.

[0107] In an embodiment, a biometric information providing method can calculate biometric information about a user and provide it to the user.

[0108] The above biometric information provision method can be further divided into additional steps or combined into fewer steps, as described in the embodiments described above with reference to FIGS. 1 through 12. Furthermore, some steps may be omitted as needed, or the order of the steps may be reversed.

[0109] The above-described method for providing biometric information can also be implemented in the form of a computer program stored in a computer-readable recording medium executed by a computer or a recording medium including commands executable by a computer.

[0110] A computer-readable recording medium may be any available medium that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, a computer-readable recording medium may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0111] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0112] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

1. In a device that provides biometric information, A collection unit that collects shooting information related to the user; A facial area detection unit that detects the user's facial area based on the above shooting information; An image of interest extraction unit for extracting an image of interest including at least one region of interest for the above facial region; A color space adjustment unit for adjusting color space information for the image of interest; A peak value detection unit for detecting at least one peak value for an image of interest for which the color space information is adjusted; and A biometric information generating unit that generates biometric information about the user based on at least one peak value A biometric information providing device, comprising:

2. In paragraph 1, A moving part controlled to move to a place where the user can be recognized; and A direction changing unit that changes the shooting direction of the collecting unit based on the facial recognition rate of the user. A biometric information providing device, comprising at least one of:

3. In paragraph 2, The above direction change part A tilting unit that rotates the above collecting unit in a vertical direction; A panning unit that rotates the above collecting unit horizontally; and Elevator for controlling the shooting height of the above collection unit A biometric information providing device comprising:

4. In paragraph 2, A control unit that controls at least one of the moving unit and the direction changing unit based on the facial recognition rate, preset time, preset event, or user input. A biometric information providing device further comprising:

5. In paragraph 1, When the user is recognized, the collection unit outputs at least one of a guidance voice and a shooting guideline corresponding to the user's facial recognition rate. Reduce or enlarge the shooting screen based on the above facial recognition rate, A biometric information providing device that sets the rotation rate of the photographing information based on the facial recognition rate.

6. In paragraph 1, A biometric information providing device, wherein the collecting unit outputs a preset color to an external area of ​​a shooting guideline output through a display if the illuminance of the environment in which the user is photographed is below a preset value, and sets the brightness value of the external area to a preset brightness value.

7. In paragraph 1, A biometric information providing device, wherein the image of interest extraction unit sets at least one region of interest from the facial region based on a facial recognition model that recognizes the position and size of a face from an input image and outputs a facial region and a region of interest output model that outputs a region of interest from the facial region, and extracts the image of interest including the set at least one region of interest.

8. In paragraph 1, A bio-information providing device, wherein the color space control unit amplifies the color space information for the image of interest based on a color magnification algorithm.

9. In paragraph 1, The above peak value detection unit performs noise filtering on the image of interest for which the color space information is adjusted, A device for providing biometric information, which applies a bandpass filter to extract at least one of pulse and respiration.

10. In paragraph 9, The above peak value detection unit detects at least one peak value for each region of interest from the frequency information of the image of interest to which the noise filtering and the band-pass filter are applied. A biometric information providing device, wherein at least one of the above peak values ​​is related to at least one of a heart rate and a respiratory rate.

11. In Article 10, A biometric information providing device, wherein the biometric information calculating unit calculates the biometric information including at least one of an average heart rate and an average breathing rate for the user based on the at least one peak value.

12. In paragraph 11, The above bio-information generating unit calculates at least one PTT (Pulse Transit Time) through a comparison between at least one peak value, A biometric information providing device that calculates the biometric information based on an artificial intelligence algorithm that receives at least one of the above PTT and at least one of the above heart rate and outputs at least one of the highest blood pressure and the lowest blood pressure for the user.

13. In paragraph 12, Calculating an average HRV (Heart Rate Variability) from the at least one peak value based on the at least one PTT, A biometric information providing device that calculates biometric information including heart health information about the user based on the average HRV.

14. In paragraph 13, The above bio-information generating unit calculates the ratio of low-frequency components and high-frequency components based on the average HRV, A biometric information providing device that calculates biometric information including stress information about the user based on the calculated ratio.

15. In a method for providing biometric information using a biometric information providing device, A step of collecting user-related shooting information; A step of detecting the user's facial area based on the above shooting information; A step of extracting an image of interest including at least one region of interest for the above facial region; A step of adjusting color space information for the above image of interest; A step of detecting at least one peak value for an image of interest for which the color space information is adjusted; and A step of calculating biometric information about the user based on at least one peak value. A method for providing biometric information, comprising:

16. In paragraph 15, A step of moving to a place where the above user can be recognized; and A step of changing the shooting direction for collecting the shooting information based on the facial recognition rate of the user A method for providing biometric information, comprising at least one of:

17. In paragraph 16, The steps for changing the above shooting direction are A step of rotating the above shooting direction horizontally; a step of rotating the above shooting direction vertically; and Step for controlling the shooting height for the above shooting direction A method for providing biometric information, comprising:

18. In paragraph 16, A step of performing at least one of the steps of moving to a place where the user can be recognized based on the facial recognition rate, preset time, preset event or user input and the step of changing the shooting direction. A method for providing biometric information, comprising:

19. In paragraph 15, The steps for collecting the above shooting information are: When the user is recognized, a step of outputting at least one of a guidance voice and a shooting guideline corresponding to the facial recognition rate of the user; A step of reducing or enlarging the shooting screen based on the above facial recognition rate; and A step for setting the rotation rate of the photographing information based on the above facial recognition rate. A method for providing biometric information, comprising:

Images