Biological information measuring apparatus, biological information measuring method, and recording medium

Abstract

A biological information measuring apparatus includes: an initial setting section which causes a first image obtained by the photographing section photographing a living body and a first figure indicating a position within the first image to be superimposition-displayed on the display section, and corrects the position based on a first instruction input to the input section to move the position to a first position of a feature point included in the first image, and obtains a corrected position; a measurement preparation section which causes a second image obtained by the photographing section photographing the living body and a second figure indicating the corrected position to be superimposition-displayed on the display section, and causes the living body to align a second position of the feature point included in the second image with the corrected position; and a measurement section which measures biological information of the living body from a third image obtained by the photographing section photographing the living body in response to a signal indicating that the second position and the corrected position have been aligned being input.

Description

Technical Field

[0001] This disclosure relates to biological information measurement devices, biological information measurement methods, and recording media. Background Technology

[0002] When measuring biometric information based on dynamic images of the face captured by a biometric measurement device, the accuracy can sometimes be improved by measuring the information from a dynamic image within a region of interest (ROI) that strongly reflects the biometric information being measured, rather than based on the entire dynamic image. Similarly, when measuring vascular characteristics such as pulsation, hemoglobin concentration, and blood pressure using a biometric measurement device, the accuracy can sometimes be improved by measuring the vascular characteristics from a dynamic image within an ROI that strongly reflects the measured vascular characteristics.

[0003] Figure 15A This is an example diagram illustrating the state of setting an appropriate ROI in the face region.

[0004] When a biometric measurement device detects the location of a Region of Interest (ROI) in a dynamic image obtained by photographing a face, it performs the following steps, for example.

[0005] First, such as Figure 15A As shown, the biometric information measuring device overlays a dynamic image 801 of the face and a guide 802 onto a display or similar surface. The user of the biometric information measuring device refers to the display or similar surface to align the position of the outline of the face region 803 with the position of the guide 802. Thus, the biometric information measuring device is able to capture an image of the face.

[0006] Next, as Figure 15A As shown, the biometric information measuring device uses a face detection algorithm to detect the location of a specific facial region in the guide 802 from the obtained dynamic image 801, and uses this location as a feature point 804. This specific region includes the eyes, nose, and mouth, etc.

[0007] Next, as Figure 15A As shown, the bio-information measurement device calculates the position of ROI 805 in such a way that the relative position of ROI 805 with respect to the detected feature point 804 is an appropriate relative position.

[0008] Patent Document 1 discloses a photographic aid device that enables a subject to identify a suitable photographic position. In this device, feature points on the face in a photographic image are extracted. Furthermore, reference points are set at the locations where the feature points exist in an average face model. Additionally, image information is output, showing the superposition of a line connecting the current position of the extracted feature points and a line connecting the reference position of the set reference points on a plane. Thus, the subject can easily identify the suitable position of the face and the orientation of the face for photographing, and can move the face to the identified position to align the face with the correct orientation. This results in a high-precision frontal photographic image (paragraphs 0010, 0056, 0059, 0061, and 0073).

[0009] Existing technical documents

[0010] Patent documents

[0011] Patent Document 1: Japanese Patent Application Publication No. 2010-219645 Summary of the Invention

[0012] The technical problem to be solved by the present invention

[0013] In a biometric measurement device, it is desirable to calculate the position of the Region of Interest (ROI) 805 so that the position of the ROI 805 within the facial region 803 does not deviate during each biometric measurement. This is because the position of the ROI 805 must be the position of the region that strongly reflects biometric information such as vascular characteristics to be measured; therefore, when the reproducibility of the ROI 805 position is low, the measurement accuracy decreases.

[0014] Ideally, whenever facial personality and biometric information are measured, the biometric information measuring device should detect feature points 804 at the same location in the facial region 803, and the Region of Interest (ROI) 805 should be located at the same location in the facial region 803. Furthermore, it is desirable that the positions of feature points 804 and ROI 805 do not deviate during biometric information measurement. However, these things are difficult to achieve.

[0015] Further explain the reasons.

[0016] Figure 15B , Figure 15C and Figure 15D This is an example diagram illustrating the state of an inappropriate ROI set in the face region.

[0017] It is difficult to ensure that the position of feature point 804 is completely consistent each time it is detected. Therefore, even when feature point 804 is detected from a dynamic image 801 of the same person's face, the position of the detected feature point 804 within the face region 803 may differ each time it is detected. Moreover, the detection of feature point 804 may sometimes fail due to changes in facial features such as hairstyle, weight loss, or aging. Furthermore, there are advantages and disadvantages in the algorithm for detecting feature point 804. Therefore, there are people with faces where feature point 804 is easily detected and people with faces where feature point 804 is difficult to detect.

[0018] When the detected feature point 804 is used for typical face recognition or detection, even if the position of the detected feature point 804 deviates slightly from the position of the actual feature point, it does not cause a major problem. Therefore, in the prior art, the deviation of the position of the detected feature point 804 from the position of the actual feature point is not a significant issue. However, when the detected feature point 804 is used to measure biological information, high accuracy in the position of the detected feature point 804 is required. For example, when the detected feature point 804 is used to measure vascular characteristics, since facial blood vessels are fine and have a complex distribution, it is preferable that the deviation between the position of the set ROI 805 and the position of the actual blood vessel image be as small as possible. However, for example, as... Figure 15B As shown, even if a detected feature point 804 deviates to the right from the actual feature point position, the position of the set ROI 805 within the face image 801 also deviates, and the position of the set ROI 805 deviates from the actual blood vessel image position.

[0019] Furthermore, to prevent the positions of feature points 804 and ROI 805 from deviating from their proper positions during the measurement of biological information, face detection is performed to make the bounding boxes follow the face region 803. However, when the face region 803 moves rapidly, it is difficult to perform face detection in real time so that the bounding boxes completely follow the face region 803. Therefore, even when the dynamic image 801 of the face is a dynamic image of the same person's face, it is sometimes impossible to make feature points 804 and ROI 805 follow the face region 803, and the positions of feature points 804 and ROI 805 deviate from their proper positions.

[0020] The shape and size of the face, as well as the position and shape of the eyes, nose, and mouth, vary from person to person. Therefore, the relative position of ROI 805 with respect to feature point 804 also varies from person to person. Therefore, as... Figure 15DAs shown, ROI 805 may sometimes cover undesirable areas such as the outline of face area 803 and eye area 806. Alternatively, when ROI 805 is intended to avoid obstruction areas such as glasses, masks, hair, and uneven areas of the face, it may sometimes cover obstruction areas due to its position being off-target.

[0021] Therefore, for these reasons, the accuracy of biological information measurements can sometimes be low.

[0022] One aspect of this disclosure was made in view of the above-mentioned problems. The object of one aspect of this disclosure is to provide a biological information measuring device capable of, for example, improving the accuracy of biological information measurement.

[0023] Solution for solving the problem

[0024] A biological information measuring device according to one aspect of this disclosure includes: an imaging unit; a display unit; an input unit; an initial setting unit that overlays a first image of a biological object captured by the imaging unit and a first graphic representing a position within the first image on the display unit, and corrects the position and obtains a corrected position based on a first instruction input to the input unit to move the position to a first position of a feature point contained in the first image; a measurement preparation unit that overlays a second image of a biological object captured by the imaging unit and a second graphic representing the corrected position on the display unit, and aligns a second position of a feature point contained in the second image with the corrected position of the biological object; and a measurement unit that measures biological information of the biological object based on a third image of the biological object captured by the imaging unit in response to an input signal indicating that the second position and the corrected position are aligned.

[0025] Another aspect of the present disclosure of a method for measuring biological information includes: a) overlaying a first image of a photograph of a biological organism and a first graphic representing a position within the first image, and correcting the position and obtaining a corrected position based on a first instruction indicating that the position be moved to a first position of a feature point contained in the first image; b) overlaying a second image of a photograph of a biological organism and a second graphic representing the corrected position, and aligning a second position of a feature point contained in the second image with the corrected position; and c) measuring biological information of the biological organism based on a third image of a photograph of the biological organism in response to an input signal indicating that the second position and the corrected position are aligned.

[0026] Another aspect of this disclosure is a storage medium storing a program for causing a computer to perform the following steps: a) overlaying a first image of an organism and a first graphic representing a position within the first image, and correcting the position and obtaining a corrected position based on a first instruction instructing the position to be moved to a first position of a feature point contained in the first image; b) overlaying a second image of an organism and a second graphic representing the corrected position, and aligning a second position of a feature point contained in the second image with the corrected position; and c) measuring biometric information of the organism based on a third image of the organism in response to an input signal indicating that the second position and the corrected position are aligned. Attached Figure Description

[0027] Figure 1 This is a block diagram of the biological information measuring device according to the first embodiment.

[0028] Figure 2 This is a schematic diagram illustrating an image obtained by the biological information measurement device of the first embodiment, registered feature points registered in the biological information measurement device, and the bounding box and region of interest (ROI) set by the biological information measurement device.

[0029] Figure 3 This is a block diagram of the control unit included in the biological information measuring device of the first embodiment.

[0030] Figure 4 This is a flowchart illustrating the process of initial setting up the biological information measuring device according to the first embodiment.

[0031] Figure 5 This is a schematic illustration of an example of a screen displayed during the initial setup of the biological information measuring device according to the first embodiment.

[0032] Figure 6 This is a schematic illustration of an example of a screen displayed during the initial setup of the biological information measuring device according to the first embodiment.

[0033] Figure 7 This is a flowchart illustrating the process of preparing for measurement and handling during measurement using the biological information measurement device of the first embodiment.

[0034] Figure 8 This is a schematic illustration of an example of a screen displayed during measurement preparation by the biological information measuring device of the first embodiment.

[0035] Figure 9This is a schematic illustration of an example of a screen displayed during measurement preparation by the biological information measuring device of the first embodiment.

[0036] Figure 10 The image is schematically illustrated by the biological information measuring device of the first variant of the first embodiment, and the diagram shows the registered feature points registered in the biological information measuring device.

[0037] Figure 11 This is an example of a screen displayed during the initial setup of the biological information measuring device according to the second embodiment.

[0038] Figure 12 This is an example of a screen displayed during the initial setup of the biological information measuring device according to the second embodiment.

[0039] Figure 13 This is an example of a screen displayed during the initial setup of the biological information measuring device according to the third embodiment.

[0040] Figure 14 This is a schematic illustration of an example of a screen displayed during measurement preparation by the biological information measuring device of the fourth embodiment.

[0041] Figure 15A This is an example diagram illustrating the state of setting an appropriate ROI in the face region.

[0042] Figure 15B This is an example diagram illustrating the state of an inappropriate ROI set in the face region.

[0043] Figure 15C This is an example diagram illustrating the state of an inappropriate ROI set in the face region.

[0044] Figure 15D This is an example diagram illustrating the state of an inappropriate ROI set in the face region. Detailed Implementation

[0045] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Furthermore, in the accompanying drawings, the same or equivalent elements will be given the same reference numerals, and repeated descriptions will be omitted.

[0046] 1. First Implementation Method

[0047] 1.1 Overview of Biological Information Measurement Devices

[0048] Figure 1 This is a block diagram of the biological information measuring device according to the first embodiment.

[0049] Figure 1The biometric information measuring device 1 of the first embodiment illustrated estimates the positions of feature points contained in the first image 21 obtained by photographing the organism 901, and performs an initial setting to allow the user 902 to correct the estimated feature point positions. Furthermore, the biometric information measuring device 1 performs measurement preparation to allow the user 902 to match the positions of feature points contained in the second image 22 obtained by photographing the organism 901 with the corrected feature point positions. Additionally, the biometric information measuring device 1 measures the biometric information 31 of the organism 901 based on the third image 23 obtained by photographing the organism 901.

[0050] The organism 901 photographed is a human body. However, the organism 901 photographed could also be an animal body.

[0051] The biometric information measuring device 1 captures images of the face 903. However, the biometric information measuring device 1 can also capture images of areas other than the face 903. For example, the biometric information measuring device 1 can also capture images of the neck, wrist, palm, etc.

[0052] User 902 is a person who possesses the photographed organism 901. However, User 902 may sometimes be a person other than a person or animal who possesses the photographed organism 901. For example, if the photographed organism 901 is the body of an infant, User 902 may be the infant's parent. Alternatively, if the photographed organism 901 is the body of an animal, User 902 may be the animal's owner.

[0053] 1.2 The composition of the biological information measurement device is as follows Figure 1 As shown, the biological information measuring device 1 includes a control unit 11, an imaging unit 12, a display unit 13, an input unit 14, an output unit 15, a sound input unit 16, a storage unit 17, a detection unit 18, and a measuring unit 19.

[0054] The control unit 11, the imaging unit 12, the display unit 13, the input unit 14, the output unit 15, the sound input unit 16, the storage unit 17, the detection unit 18, and the measurement unit 19 are electrically connected to each other. Thus, the control unit 11, the imaging unit 12, the display unit 13, the input unit 14, the output unit 15, the sound input unit 16, the storage unit 17, the detection unit 18, and the measurement unit 19 can exchange information with each other.

[0055] The control unit 11 controls the entire biological information measuring device 1, and controls the imaging unit 12, display unit 13, input unit 14, output unit 15, sound input unit 16, storage unit 17, detection unit 18 and measuring unit 19 included in the biological information measuring device 1.

[0056] The control unit 11 includes processors, etc. A processor can be a central processing unit (CPU), a graphics processing unit (GPU), etc.

[0057] The imaging unit 12 captures images of the organism 901 and sends them to the control unit 11. The acquired images are moving images. The image acquired during initial setup is the first image 21. The image acquired during measurement preparation is the second image 22. The image acquired during measurement is the third image 23.

[0058] The camera unit 12 is located in a place where faces 903 can be captured. This place may include a personal computer, a monitor, a mirror, a washbasin, etc.

[0059] The imaging unit 12 is a camera, etc. The camera includes a lens and an image sensor. The lens causes light arriving from the organism 901 being photographed to form an image on the image sensor. The image sensor converts the imaged light into an image signal. The image sensor is a complementary metal-oxide-semiconductor (CMOS) image sensor, a charge-coupled device (CCD) image sensor, etc.

[0060] Display unit 13 displays the acquired image. The displayed image is a moving image captured during the process. Additionally, display unit 13 displays various information. This information includes graphs showing the positions of feature points within the image, graphs showing the region of interest (ROI) defined within the image, measured biological information 31, notifications to user 902, date and time, etc. Biological information 31 includes charts showing vital signs and changes in vital signs over time.

[0061] Display unit 13 includes a monitor, etc.

[0062] User 902 inputs information to input unit 14. By inputting information indicating the content of the instruction into input unit 14, user 902 is able to provide instructions to biological information measuring device 1.

[0063] The input unit 14 includes a keyboard, mouse, touch panel, buttons, dials, sliders, etc.

[0064] The output unit 15 outputs information to the user 902. The output information is a notification to the user 902, etc. The output unit 15 outputs information through text, sound, etc.

[0065] Output section 15 includes a display, speakers, etc.

[0066] User 902 inputs sound into sound input unit 16. By inputting sound into sound input unit 16, user 902 is able to send a signal to biological information measuring device 1.

[0067] The sound input section 16 includes a microphone, etc.

[0068] Storage unit 17 stores the acquired images. Additionally, storage unit 17 stores various types of information. This stored information includes information pre-stored by the manufacturer in the biometric information measuring device 1 and information subsequently stored by the user 902 in the biometric information measuring device 1. The former includes information such as face detection algorithms and measurement programs. The latter includes information such as the daily changes in the measured biometric information 31 and data related to corrections described later.

[0069] Storage unit 17 includes random access memory (RAM), read-only memory (ROM), hard disk drive, solid-state drive, etc.

[0070] The measurement unit 19 measures biological information 31 of the organism 901 based on the acquired third image 23. The measurement unit 19 measures biological information 31 based on the image within the Region of Interest (ROI) contained in the third image 23. For example, the measurement unit 19 measures biological information 31 based on the brightness of the image within the ROI and the temporal changes in that brightness. The measured biological information 31 includes the state of blood vessels, blood pressure, blood flow, heart rate, etc. By measuring biological information 31 from a dynamic image, the measurement unit 19 can measure the temporal changes in vital signs.

[0071] Figure 2 It is a schematic diagram illustrating the image obtained by the biological information measurement device of the first embodiment, the registered feature points registered in the biological information measurement device, and the bounding box and ROI set by the biological information measurement device.

[0072] During the measurement of biological information 31, the detection unit 18 makes... Figure 2 The ROI62 shown follows Figure 2 The resulting image shows a facial region 61, and the movement of ROI 62 is made to match the movement of facial region 61. The method by which ROI 62 follows facial region 61 varies depending on the algorithm. The algorithm is selected from those suitable for the measured biological information 31.

[0073] In the first embodiment, the detection unit 18 is based on Figure 2 The registration feature point 63 shown in the figure is set. Figure 2The bounding box 64 is shown in the figure, and the ROI 62 is set based on the registered feature point 63 and the bounding box 64. The detection unit 18 determines the initial position of the bounding box 64 based on the position of the registered feature point 63, and determines the initial position of the ROI 62 based on the position of the registered feature point 63 and the initial position of the bounding box 64. For example, if the registered feature point 63 is the left and right ends of the eye region, nose region, and mouth region, the detection unit 18 sets the area that includes the face region 61, which is estimated based on the registered feature point 63, as the bounding box 64, and sets the area that includes the cheek region, which is estimated based on the registered feature point 63, as the ROI 62. In addition, the detection unit 18 maintains the relative positions of the registered feature point 63, the bounding box 64, and the ROI 62 during the measurement of the biological information 31. In addition, the detection unit 18 makes the bounding box 64 follow the face region 61. The detection unit 18 detects image features within the bounding box 64, such as edges (contours), and detects movement of these detected features. It then moves the bounding box 64 accordingly to track the face region 61. As a result, the detection unit 18 causes the ROI 62 to follow the face region 61. This allows for the elimination of... Figure 15C The problem shown is that feature points 804 and ROI 805 cannot follow the face region 803 and their positions deviate from the proper positions.

[0074] The control unit 11, the detection unit 18, and the measurement unit 19 are configured by executing a biometric information measurement program 41, consisting of a face detection algorithm and a measurement program, on a computer assembled in the biometric information measurement device 1. All or part of the control unit 11, the detection unit 18, and the measurement unit 19 may also be configured using dedicated circuitry.

[0075] The biological information measurement program 41 is stored in the storage unit 17. The biological information measurement program 41 can be installed in a computer either by writing the biological information measurement program 41 received via the network 911 into the storage unit 17, or by writing the biological information measurement program 41 read from an external storage medium 912 such as an optical disc, magnetic disk, or Universal Serial Bus (USB) memory into the storage unit 17. This disclosure also relates to the biological information measurement program 41 stored in a computer-readable, non-transitory storage medium such as the storage unit 17 or an external storage medium.

[0076] 1.3 Control Department

[0077] Figure 3 This is a block diagram of the control unit included in the biological information measuring device of the first embodiment.

[0078] like Figure 3As shown, the control unit 11 includes an initial setting unit 51 and a measurement preparation unit 52.

[0079] The initial setting unit 51 performs initial settings. The measurement preparation unit 52 performs measurement preparation based on the initial settings each time a measurement is performed after initial settings.

[0080] 1.4 Initial Settings

[0081] Figure 4 This is a flowchart illustrating the process of initial setting up the biological information measuring device according to the first embodiment. Figure 5 and Figure 6 This is a schematic illustration of an example of a screen displayed during the initial setup of the biological information measuring device according to the first embodiment.

[0082] The biological information measuring device 1 and the user 902 initiate the initial setting by inputting an initial setting start instruction to the input unit 14 in a coordinated manner. The biological information measuring device 1 then executes the initial setting process. Figure 4 Steps S61 to S67 are shown.

[0083] In step S61, user 902 inputs identification information 32 for identifying the photographed organism 901 into input unit 14. This allows initial setting unit 51 to perform initial settings for each organism 901. Consequently, daily logs of organism information 31 for each organism 901 can be stored in the database. The input identification information 32 includes identifiers (IDs), names, etc. The identification information 32 for identifying the organism 901 can also be the result of facial recognition of the first image 21.

[0084] In the next step S62, the imaging unit 12 captures the face 903 to obtain the first image 21.

[0085] like Figure 5 As shown, the initial setting unit 51 displays the first image 21 and the guide 72 overlapping on the display unit 13, and the user 902 moves the face 903 so that the outline of the face region 81 is approximately consistent with the guide 72.

[0086] The initial setting unit 51 can also display information that is easy to align, such as the grid overlapping with the first image 21 and the coordinates of the face region 81, on the display unit 13.

[0087] In the next step S63, the detection unit 18 detects feature points included in the first image 21 and estimates the estimated positions of the detected feature points. The detected feature points are six points formed by the left and right ends of the eye region, nose region, and mouth region. However, the detected feature points are determined by the algorithm used to detect the feature points and the biometric information 31 being measured, and are appropriately set by the manufacturer. Therefore, the detected feature points are sometimes points other than the six points formed by the left and right ends of the eye region, nose region, and mouth region. In addition, the detected feature points are sometimes more than two, less than five, or more than seven. Furthermore, the more feature points detected, the more accurate the measurement of biometric information 31 can be improved. When the biometric information measuring device 1 captures images of the neck, wrist, or palm, the detected feature points are blood vessel regions. The estimated positions of the estimated feature points may deviate from the correct positions of the feature points.

[0088] In the next step S64, as Figure 5 As shown, the initial setting unit 51 displays the first image 21, the blank asterisk 71, and the guide 72 overlapping on the display unit 13.

[0089] The blank asterisk 71 indicates a position within the first image 21. The initial setting unit 51 sets the estimated position of the feature point to the position indicated by the blank asterisk 71, and places the blank asterisk 71 at the estimated position of the feature point. The blank asterisk 71 is merely an example of a first graphic indicating a position within the first image 21. Therefore, the blank asterisk 71 can be replaced with other graphics. For example, the blank asterisk 71 can be replaced with a point placed at the estimated position of the feature point, a wireframe formed by lines connecting the estimated positions of the feature points to each other, etc. As mentioned above, the estimated position of the feature point may deviate from the correct position of the feature point. Therefore, as... Figure 5 As shown, the position indicated by the blank asterisk 71, that is, the position configured with the blank asterisk 71, may deviate from the correct position of the feature point. The position indicated by the blank asterisk 71 can be a position determined without using the algorithm used to detect the feature point.

[0090] In the next step S65, user 902, referring to the overlaid first image 21 and the blank asterisk 71, inputs a first instruction 33 into the input unit 14. This first instruction 33 instructs the user to move the position represented by the blank asterisk 71 (i.e., the position where the blank asterisk 71 is located) to the correct position, i.e., the first position, of the feature point contained in the first image 21. The user inputs the first instruction 33 by operating the cursor keys to move the blank asterisk 71 in the direction corresponding to the operated cursor key, or by operating a mouse, touch panel, or other indicating device to drag the blank asterisk 71.

[0091] The first image 21, which is displayed overlapping the blank asterisk 71, is a static image. Therefore, it is easy to indicate that the position indicated by the blank asterisk 71 should be moved to the correct position of the feature points contained in the first image 21.

[0092] The initial setting unit 51, based on the input first instruction 33, corrects the position indicated by the blank asterisk 71 and obtains the corrected position. Therefore, as Figure 6 As shown, the position indicated by the blank asterisk 71 is corrected to the correct position of the feature point, and is corrected to the position corresponding to the shape of the face and the position and size of the eyes, nose and mouth.

[0093] The detection unit 18 sets the bounding box 64 and the region of interest (ROI) 62 based on the corrected position. Based on the corrected position, the detection unit 18 determines the initial range of the bounding box 64 set within the third image 23, and also determines the initial range of the ROI 62 set within the third image 23. The ROI 62 is set in the cheek region. However, the region where the ROI 62 is set is determined by the measured biometric information 31 and is appropriately set by the manufacturer. Therefore, sometimes the ROI 62 is set in an area outside the cheek region.

[0094] In the next step S66, the initial setting unit 51 establishes an association with and registers the identification information 32, the corrected position, the relative position of the bounding box 64 with respect to the corrected position, the relative position of the ROI 62 with respect to the corrected position, and the first image 21. Therefore, when measuring the bio-information 31 of the organism 901 identified by the identification information 32, the corrected position, the relative position of the bounding box 64, the relative position of the ROI 62, and the first image 21 associated with the identification information 32 can be reused. The initial setting unit 51 establishes an association with and registers the registered information by associating it with the storage unit 17 and storing it. The corrected position is represented by the positional relationship between the corrected positions, the coordinates of the corrected position, etc. Points with corrected positions are stored in the storage unit 17 as registered feature points 63.

[0095] The algorithm used to detect feature points can be corrected. For example, the algorithm used to detect feature points can also be corrected so that a specific value is added when estimating the estimated location of the feature points based on the first image 21 obtained by photographing a specific organism 901.

[0096] In the next step S67, user 902 emits an input sound 34 and inputs the input sound 34 into the sound input unit 16. The input sound 34 is a phrase that user 902 likes, such as "OK!" or "Start!".

[0097] The initial setting unit 51 establishes an association between the identification information 32, the input voice 34, and the phrase, and registers them.

[0098] Before the user 902 first measures the biological information measurement device 1 for biological information 31, the user 902 performs initial settings on the biological information measurement device 1. Additionally, the user 902 also performs initial settings on the biological information measurement device 1 when facial contours change due to changes in hairstyle, age, etc. This eliminates the need for [further settings]. Figure 15B The problem shown is that feature point 804 cannot be properly detected and its position deviates from the actual position of the feature point, and the position of the set ROI 805 is deviated within the face region 803.

[0099] 1.5 Measurement preparation and measurement

[0100] Figure 7 This is a flowchart illustrating the process of preparing for measurement and handling during measurement using the biological information measurement device of the first embodiment. Figure 8 and Figure 9 This is a schematic illustration of an example of a screen displayed during measurement preparation by the biological information measuring device of the first embodiment.

[0101] The bio-information measuring device 1 and the user 902 input a measurement start instruction to the input unit 14 in a coordinated manner to begin measurement preparation. The bio-information measuring device 1, having begun measurement preparation, executes... Figure 7 Steps S101 to S107 are shown.

[0102] In step S101, user 902 inputs identification information 35 to input unit 14 for identifying the photographed organism 901. As a result, measurement preparation unit 52 can perform measurement preparation suitable for the organism 901 identified by the input identification information 35. The input identification information 35 may be an ID, name, etc. The identification information 35 for identifying the organism 901 may also be the result of facial recognition of the second image 22.

[0103] In the next step S102, the imaging unit 12 captures a face 903 to obtain a second image 22. The obtained second image 22 is a moving image.

[0104] In the next step S103, as Figure 8 As shown, the measurement preparation unit 52 displays the second image 22, the black asterisk 111, and the guide 112 superimposed on the display unit 13.

[0105] The black asterisk 111 indicates the corrected position, i.e., the position of the registered feature point 63. The black asterisk 111 is merely an example of the second graphic representing the corrected position. Therefore, the black asterisk 111 can be replaced with other graphics. For example, the black asterisk 111 can be replaced with a wireframe consisting of points arranged at the corrected position, lines connecting the corrected positions to each other, etc.

[0106] The corrected position, indicated by the black asterisk 111, is the corrected position registered by associating it with the identification information 32, which corresponds to the identification information 35 input in step S101. Therefore, when the measurement preparation unit 52 inputs the identification information 35, which corresponds to the identification information 32 input during initial setup, into the input unit 14, it displays the second image 22 and the black asterisk 111, representing the corrected position registered by associating it with the identification information 32, superimposed on the display unit 13.

[0107] User 902 moves face 903 so that the outline of face region 121 is roughly consistent with guide 112.

[0108] The measurement preparation unit 52 can also display alignment information such as grid and face coordinates on the display unit 13, overlapping with the second image 22.

[0109] In the next step S104, user 902 refers to the overlaid second image 22 and the black asterisk 111, aligning the positions of the feature points contained in the second image 22 with the corrected positions indicated by the black asterisk 111. This roughly reproduces the state when the ROI 62 and bounding box 64 were set. At this time, user 902 moves the face 903 for alignment. Preferably, user 902 performs alignment simultaneously, ensuring that the positions of all feature points are aligned with their corresponding corrected positions.

[0110] In the next step S105, after alignment, user 902, without moving face 903 further, emits a sound 36 equivalent to the input sound 34 registered during initial setup, inputting sound 36 to the sound input unit 16. The input sound 36 is merely an example of a signal indicating that the position of the feature point included in the second image 22 is aligned with the correction position indicated by the black asterisk 111. However, when sound 36 is a signal, compared to when a hand movement operation such as pressing a button is a signal, it is possible to suppress body movements that occur when the signal is input to the biometric information measuring device 1. Therefore, the signal of sound 36 can be replaced with other signals that can suppress body movements generated when input to the biometric information measuring device 1. For example, the signal of sound 36 can be replaced with a signal such as a gaze.

[0111] In the next step S106, the measurement preparation unit 52 waits until a set time has elapsed since the input sound 36. This time is the time required to suppress the movement of the mouth muscles by emitting the sound 36.

[0112] In the next step S107, the measurement preparation unit 52 causes the measurement unit 19 to begin measuring the biological information 31. The measurement unit 19 measures the biological information 31 in response to the input of the sound 36, but measures the biological information 31 after a set time has elapsed since the sound 36 was input.

[0113] The camera unit 12 captures a face 903 to obtain a third image 23. The obtained third image 23 is a moving image.

[0114] The measurement unit 19 measures the biological information 31 of the organism 901 based on the obtained third image 23.

[0115] During the measurement of biometric information 31 by the measurement unit 19, the detection unit 18 causes the bounding box 64 to follow the face region 61 of the image, which is the face 903 included in the third image 23. If the distance from the initial range to the bounding box 64 is greater than or equal to a first reference, the detection unit 18 causes the output unit 15 to output a first warning 37. The output first warning 37 is a warning indicating excessive face movement, such as a message like "The face is moving." If the distance from the initial range to the bounding box 64 is greater than or equal to a second reference, the detection unit 18 causes the output unit 15 to output a second warning 38. The output second warning 38 is a warning prompting remeasurement, such as a message like "Please remeasure." The first and second references are given by thresholds, etc., and are predetermined by the manufacturer based on the measured biometric information 31. The second reference is greater than the first reference.

[0116] After completing the measurement for the set time and the set number of times, the measurement unit 19 performs actions such as sending the measured organism information 31 to the organism information analysis unit (not shown) and then ends the processing.

[0117] 1.6 Variation Example

[0118] Figure 10 The image is schematically illustrated by the biological information measuring device of the first variant of the first embodiment, and the diagram shows the registered feature points registered in the biological information measuring device.

[0119] In the first embodiment, the detected feature points are six points comprising the left and right ends of the eye region, nose region, and mouth region. However, as... Figure 10 As shown, the detected feature points can be multiple points 131 arranged along the contour lines of the eye region, nose region, and mouth region.

[0120] 2. Second Implementation Method

[0121] The differences between the second embodiment and the first embodiment will be explained below. Regarding points not described, the second embodiment also employs the same configuration as the first embodiment.

[0122] Figure 11 and Figure 12 This is an example of a screen displayed during the initial setup of the biological information measuring device according to the second embodiment.

[0123] In the second embodiment, in step S64, as Figure 11 As shown, the initial setting unit 51 displays the first image 21, the blank asterisk 71, the guide 72, and the rectangle 73 overlapping on the display unit 13.

[0124] Rectangle 73 represents ROI 62 within the first image 21. Figure 11 The middle image shows the overlap between ROI 62 and the outline of face region 81 and eye region 82. Rectangle 73 is merely an example of a third graphic representing ROI 62 within the first image 21. Therefore, rectangle 73 can be replaced with other graphics.

[0125] In addition, in the second embodiment, in step S65, the user 902 refers to the overlapping first image 21 and the rectangle 73 and inputs a second instruction 39 to the input unit 14, which instructs the ROI 62 shown by the rectangle 73 to move or deform.

[0126] The initial setting unit 51, based on the input second instruction 39, corrects the ROI 62 shown by the rectangular frame 73 and obtains the corrected ROI 62. The correction of ROI 62 involves correcting its position and / or shape. Thus, as... Figure 12 As shown, the ROI 62 represented by the rectangle 73 is corrected to not overlap with the outline of the face region 81 and the eye region 82.

[0127] In addition, in the second embodiment, in step S107, the measurement unit 19 measures the organism information 31 based on the image in the corrected ROI 62 contained in the third image 23.

[0128] Therefore, it is possible to eliminate Figure 15D As shown, ROI62 covers unwanted areas such as the outline of the face region 803 and the eye region 806, which leads to a decrease in the accuracy of biometric information measurement.

[0129] In step S65, the initial setting unit 51 may also cause the output unit 15 to output suggestions that help with correction. The output suggestions may be, for example, messages such as "Please try to select a flat area that does not overlap with obstacles such as glasses, hair, or unevenness of the face."

[0130] 3. Third Implementation Method

[0131] The differences between the third embodiment and the second embodiment will be explained below. Regarding points not described, the third embodiment also employs the same configuration as the second embodiment.

[0132] Figure 13 This is an example of a screen displayed during the initial setup of the biological information measuring device according to the third embodiment.

[0133] In the third embodiment, in step S64, as Figure 13 As shown, the initial setting unit 51 displays the first image 21, the blank asterisk 71, the guide 72, and the rectangle 73 overlapping in the first area 151 of the display unit 13. Simultaneously, the initial setting unit 51 displays a template image 181 overlapping the template image 161, the blank asterisk 171, the guide 172, and the rectangle 173 in the second area 152 of the display unit 13.

[0134] The template image 161 is prepared by the manufacturer. The template image 161 can be either a realistic image or an illustration. The realistic image can also be an image of an organism other than the photographed organism 901.

[0135] The blank asterisk 171 indicates the position of a feature point contained in the template image 161. The blank asterisk 171 is merely an example of a graphic representation of the position of a feature point contained in the template image 161. Therefore, the blank asterisk 171 can be replaced with other graphics. For example, the blank asterisk 171 can be replaced with a wireframe consisting of points positioned at the presumed locations of the feature points, lines connecting the positions of the feature points to each other, etc. The position indicated by the blank asterisk 171 does not deviate from the position of the feature point contained in the template image 161.

[0136] Rectangle 173 represents the appropriate ROI 62 within the template image 161. Rectangle 173 may also be displayed if ROI 62 correction is performed, or not displayed if ROI 62 correction is not performed.

[0137] The first zone 151 and the second zone 152 are arranged side by side.

[0138] In addition, in the third embodiment, in step S65, the user 902 refers to the displayed template image 161 and inputs a first instruction 33 to the input unit 14. The first instruction 33 instructs the position indicated by the blank asterisk 71 to be moved to the correct position of the feature point contained in the first image 21, and inputs a second instruction 39 to the input unit 14. The second instruction 39 instructs the ROI 62 indicated by the rectangle 73 to be moved or deformed.

[0139] In the third embodiment, by using the reference template image 181, the user 902 can input appropriate first instruction 33 and second instruction 39.

[0140] The location and number of feature points and ROI62 are not limited to Figure 13 The location and quantity shown are not determined by the biological information 31 measured.

[0141] 4. Fourth Implementation Method

[0142] The differences between the fourth embodiment and the first embodiment will be explained below. Regarding points not described, the fourth embodiment also employs the same structure as the first embodiment.

[0143] Figure 14 This is a schematic illustration of an example of a screen displayed during measurement preparation by the biological information measuring device of the fourth embodiment.

[0144] In the fourth embodiment, in step S103, as Figure 14 As shown, the measurement preparation unit 52 displays the second image 22, the black asterisk 111, and the guide 112 overlapping in the first area 151 of the display unit 13. Simultaneously, the measurement preparation unit 52 displays a template image 211 overlapping the first image 21, the black asterisk 201, and the guide 202 in the second area 152 of the display unit 13.

[0145] The first image 21 is the image obtained during the initial setup.

[0146] The black asterisk 201 indicates the corrected position. The black asterisk 201 is merely an example of a graphic representing the corrected position. Therefore, the black asterisk 201 can be replaced with other graphics. For example, the black asterisk 201 can be replaced with a wireframe consisting of points positioned at the corrected positions, lines connecting the corrected positions to each other, etc. The position indicated by the black asterisk 201 does not deviate from the position of the feature points contained in the first image 21.

[0147] The first zone 151 and the second zone 152 are arranged side by side.

[0148] In addition, in the fourth embodiment, in step S104, user 902 refers to template image 211 and aligns the position of feature points included in second image 22 with the corrected position indicated by black asterisk 111.

[0149] In the fourth embodiment, by using the reference template image 211, the user 902 can perform proper alignment.

[0150] The location and number of feature points are not limited to Figure 14 The location and quantity shown are not determined by the biological information 31 measured.

[0151] The present invention is not limited to the above embodiments, and can be replaced by a structure that is substantially the same as the structure shown in the above embodiments, a structure that achieves the same effect, or a structure that can achieve the same purpose.

[0152] Explanation of reference numerals in the attached figures

[0153] 1. Biometric Information Measurement Device; 11. Control Unit; 12. Imaging Unit; 13. Display Unit; 14. Input Unit; 15. Output Unit; 16. Sound Input Unit; 17. Storage Unit; 18. Detection Unit; 19. Measurement Unit; 21. First Image; 22. Second Image; 23. Third Image; 31. Biometric Information; 32. Recognition Information; 33. First Indication; 34. Input Sound; 35. Recognition Information; 36. Sound; 37. First Warning; 38. Second Warning; 41. Biometric Information Measurement Program; 51. Initial Setting Unit; 52. Measurement Preparation Unit; 61. Facial Region; 62. Region of Interest (ROI); 63. Registered Feature Points. 71 Blank Asterisk, 72 Guide, 73 Rectangle, 81 Face Region, 82 Eye Region, 111 Black Asterisk, 112 Guide, 121 Face Region, 131 Point, 151 First Region, 152 Second Region, 161 Template Image, 171 Blank Asterisk, 172 Guide, 173 Rectangle, 181 Template Image, 801 Animated Image, 802 Guide, 803 Face Region, 804 Feature Point, 805 ROI, 806 Eye Region, 901 Organism, 902 User, 903 Face, 911 Network, 912 External Storage Medium

Claims

1. A biological information measuring device, characterized in that, include: Filming Department; Display section; Input section; An initial setting unit overlays a first image of a living organism captured by the imaging unit and a first graphic representing a position within the first image onto the display unit, and corrects the position and obtains the corrected position based on a first instruction input to the input unit to move the position to a first position of a feature point contained in the first image. The measurement preparation unit overlays a second image of the organism captured by the imaging unit and a second graphic representing the corrected position on the display unit, so that the organism aligns the second position of the feature points contained in the second image with the corrected position. as well as The measuring unit, in response to an input signal indicating that the second position is aligned with the corrected position, measures biological information of the organism based on a third image of the organism captured by the imaging unit. The initial setting unit overlays the first image and a third graphic representing a region of interest within the first image onto the display unit, and corrects the region of interest based on a second instruction input to the input unit to move or deform the region of interest, thereby obtaining a corrected region of interest. The measurement unit measures biological information based on the image of the corrected region of interest contained in the third image.

2. The biological information measuring device as described in claim 1, characterized in that, include: Output section; as well as The detection unit determines an initial range of a range set within the third image based on the corrected position, makes the range follow the image of the organism contained in the third image, and outputs a warning when the distance from the initial range to the range becomes above a reference.

3. The biological information measuring device as described in claim 1 or 2, characterized in that, Including the voice input section, The initial setting unit registers the input sounds that are input to the sound input unit. The signal being input means that the sound equivalent to the input sound is input to the sound input unit.

4. The biological information measuring device as described in claim 1 or 2, characterized in that, The measuring unit measures the biological information after a set time has elapsed since the signal was input.

5. The biological information measuring device as described in claim 1 or 2, characterized in that, The measurement preparation unit displays the second image and the second graphic overlapping on the display unit, while simultaneously displaying a first template image formed by overlapping the first image and the graphic representing the corrected position on the display unit.

6. The biological information measuring device as described in claim 1 or 2, characterized in that, The initial setting unit displays the first image and the first graphic overlapping on the display unit, while simultaneously displaying a second template image formed by overlapping a first template image and a graphic indicating the positions of feature points contained in the first template image on the display unit.

7. The biological information measuring device as described in claim 1 or 2, characterized in that, The initial setting unit displays the first image and the first graphic overlapping on the display unit, while simultaneously displaying a third template image formed by overlapping a second template image and a graphic representing an appropriate region of interest within the second template image on the display unit.

8. The biological information measuring device as described in claim 1 or 2, characterized in that, This includes a detection unit that estimates the positions of feature points contained in the first image. The initial setting unit sets the estimated position to the given position.

9. The biological information measuring device as described in claim 1 or 2, characterized in that, The first image is a static image. The second image is a dynamic image.

10. The biological information measuring device as described in claim 1 or 2, characterized in that, The initial setting unit will identify the organism's identification information, the corrected position, and the first image, establish a connection, and register them. When information corresponding to the identification information is input to the input unit, the measurement preparation unit causes the second image and the second graphic to be superimposed and displayed on the display unit.

11. A method for measuring biological information, characterized in that, include: a) A process of overlaying a first image obtained by photographing an organism and a first graphic representing a position within the first image, and correcting the position based on a first instruction that instructs the position to be moved to a first position of a feature point contained in the first image, and obtaining the corrected position. b) A process of overlaying a second image obtained from photographing an organism and a second graphic representing the corrected position, and aligning the organism with the second position of a feature point contained in the second image and the corrected position; c) A process of measuring biological information of an organism based on a third image obtained by capturing the organism in response to an input signal indicating that the second position is aligned with the corrected position; d) The process of overlaying the first image and a third graphic representing the region of interest within the first image, and correcting the region of interest based on a second instruction for instructing the region of interest to move or deform, and obtaining the corrected region of interest; as well as e) The process of measuring biological information based on the image of the corrected region of interest contained in the third image.

12. A recording medium storing a program that causes a computer to execute: a) A process of overlaying a first image obtained by photographing an organism and a first graphic representing a position within the first image, and correcting the position based on a first instruction that instructs the position to be moved to a first position of a feature point contained in the first image, and obtaining the corrected position. b) A process of overlaying a second image obtained from photographing an organism and a second graphic representing the corrected position, and aligning the organism with the second position of a feature point contained in the second image and the corrected position; c) A process of measuring biological information of an organism based on a third image obtained by capturing the organism in response to an input signal indicating that the second position is aligned with the corrected position; d) The process of overlaying the first image and a third graphic representing the region of interest within the first image, and correcting the region of interest based on a second instruction for instructing the region of interest to move or deform, and obtaining the corrected region of interest; as well as e) The process of measuring biological information based on the image of the corrected region of interest contained in the third image.

Images