terminal device

Abstract

To provide a terminal device which can confirm information about a state related to measurement displayed from the terminal device while gazing an index image.SOLUTION: A terminal device (100) comprises: an imaging unit (2); a display unit (4); an information acquisition unit (11) which acquires an image of a measurement object that is imaged as information for measuring biological information of the measurement object; a state specification unit (16) which specifies a state related to measurement; and a display control unit (17) which displays a first index image on a superimposed manner on an image including the measurement object displayed on the display unit (4) so as to make a measurement person continue imaging such that the face position of the measurement object is not changed, and changes the display mode of the first index image according to the state specified by the state specification unit (16).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a terminal device that measures biological information of a subject in a non-contact manner. [Background technology]

[0002] Conventionally, there is known a technique for superimposing and displaying an index image on an image of a person being photographed. For example, Patent Document 1 discloses a video processing system that synthesizes a video of a user photographed with a contour (index image) of a model performer in a model video to produce a video for display, and displays the video for display. Furthermore, Patent Document 2 discloses an image display control device that generates a face display frame (index image) that surrounds the face region of a subject and a human body display frame (index image) that surrounds the human body region, and displays these display frames on a screen together with an image obtained by photographing the subject.

[0003] Furthermore, the above-described technology can be particularly applied to a terminal device that measures the biometric information of a person being measured (photographed person) based on a captured image of the person being measured. In such a terminal device that measures biometric information, it is important to continue capturing the person being measured at a predetermined measurement position on the display screen. Therefore, the terminal device displays an index image (such as a face position guide or a human-shaped outline guide) indicating the measurement position on the display screen, superimposed on an image including the person being measured. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-258613 [Patent Document 2] Japanese Patent Application Laid-Open No. 2012-165174 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the above-mentioned technology, when information about the measurement situation is displayed while measuring the subject in accordance with the index image, the subject may look away from the index image, which may result in a significant shift in face position and a failed measurement.

[0006] One aspect of the present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide a terminal device that allows the user to check information about the measurement situation displayed on the terminal device while keeping an eye on the index image. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, a terminal device according to one aspect of the present invention includes an image capturing unit, a display unit, an information acquiring unit that acquires an image of a person being measured captured by the image capturing unit as information for measuring biological information of the person being measured, a situation identifying unit that identifies a situation related to the measurement, and a display unit that displays a first index image superimposed on an image including the person being measured displayed on the display unit, and a display unit that displays a second index image superimposed on the image including the person being measured. If the situation specified by the situation specifying unit is before the measurement, a circular frame guide in which a face part is contained is displayed as the first indicator image, and if the situation specified by the situation specifying unit is during the measurement, a human-shaped outline guide including a face part and a shoulder part is displayed as the first indicator image. and a display control unit that changes the display form of the first index image. In order to solve the above problem, another aspect of the present invention provides a terminal device comprising: an imaging unit; a display unit; an information acquisition unit that acquires an image of the subject captured by the imaging unit as information for measuring the subject's biometric information; a situation identification unit that identifies a situation related to the measurement; and a display control unit that displays a first index image superimposed on an image including the subject displayed on the display unit and changes the display form of the first index image in accordance with the situation identified by the situation identification unit, wherein the situation identification unit includes a first identification unit that identifies a deviation of the subject's face position from a predetermined detection area during imaging based on a facial image of the subject included in the image being captured by the imaging unit, and the display control unit changes the display form of the first index image based on the identification by the first identification unit so that the person being measured can grasp the magnitude of the deviation, and the display control unit changes the thickness of the first index image in accordance with the magnitude of the deviation. In order to solve the above-mentioned problems, another aspect of the present invention provides a terminal device comprising: an imaging unit; a display unit; an information acquisition unit that acquires an image of the subject captured by the imaging unit as information for measuring the subject's biometric information; a situation identification unit that identifies a situation related to the measurement; and a display control unit that displays a first index image superimposed on an image including the subject displayed on the display unit and changes the display form of the first index image according to the situation identified by the situation identification unit, wherein the situation identification unit includes a first identification unit that identifies a deviation of the subject's face position from a predetermined detection area during image capture based on a facial image of the subject included in the image being captured by the imaging unit, and the display control unit changes the display form of the first index image based on the identification by the first identification unit so that the person taking the image can grasp the magnitude of the deviation, and the display control unit displays, in addition to the first index image located in the detection area, an image similar to the first index image that moves together with the facial image of the subject, like an afterimage.In order to solve the above-mentioned problems, another aspect of the present invention provides a terminal device comprising: an imaging unit; a display unit; an information acquisition unit that acquires an image of the subject captured by the imaging unit as information for measuring the subject's biometric information; a situation identification unit that identifies a situation related to the measurement; and a display control unit that displays a first index image superimposed on an image including the subject displayed on the display unit and changes the display form of the first index image according to the situation identified by the situation identification unit, wherein the situation identification unit includes a first identification unit that identifies a deviation of the subject's face position from a predetermined detection area during capture based on a facial image of the subject included in the image being captured by the imaging unit, and the display control unit changes the display form of the first index image based on the identification by the first identification unit so that the person taking the image can grasp the magnitude of the deviation, and when the deviation of the face position is within a predetermined range, the display control unit displays the first index image in a manner that follows the change in the face position, and when the deviation of the face position exceeds the predetermined range, changes the first index image to a position that would be displayed if there was no deviation, more quickly than the change in the face position, and displays the first index image. In order to solve the above problem, another aspect of the present invention provides a terminal device comprising: an imaging unit; a display unit; an information acquisition unit that acquires an image of the subject captured by the imaging unit as information for measuring the subject's biometric information; a situation identification unit that identifies a situation related to the measurement; and a display control unit that displays a first index image superimposed on an image including the subject displayed on the display unit and changes the display form of the first index image in accordance with the situation identified by the situation identification unit, wherein the situation identification unit includes a second identification unit that identifies the progress of the measurement of the subject's biometric information, and the display control unit changes the display form of the first index image based on the identification by the second identification unit so that the person measuring can grasp the progress, and the display control unit gradually changes the color of the first index image from one end to the other end in accordance with the progress.In order to solve the above problem, another aspect of the present invention provides a terminal device comprising: an imaging unit; a display unit; an information acquisition unit that acquires an image of the subject captured by the imaging unit as information for measuring the subject's biometric information; a situation identification unit that identifies a situation related to the measurement; and a display control unit that displays a first index image superimposed on an image including the subject displayed on the display unit and changes the display form of the first index image according to the situation identified by the situation identification unit, wherein the situation identification unit includes a third identification unit that identifies that the subject's biometric information is being measured, and the display control unit changes the display form of the first index image based on the identification by the third identification unit so that the person being measured can understand that measurement is being performed, and the display control unit regularly changes the thickness or size of the first index image when measurement is being performed. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to provide a terminal device that allows a user to check information about the status of measurement displayed on the terminal device while gazing at an index image. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of a terminal device according to an embodiment of the present invention; [Figure 2] 10A and 10B are diagrams showing examples of images displayed by the terminal device when measuring a subject in a seated or standing position. [Figure 3] FIG. 10 is a diagram showing an example of an image displayed by the terminal device when measuring a subject lying down. [Figure 4]FIG. 10 is a diagram showing an example of an image displayed by the terminal device when measuring a subject whose face is turned to the side. [Figure 5] 10 is a diagram showing an example of an image displayed by the terminal device when the position of the subject's face on the screen of the display unit is shifted within a predetermined range. FIG. [Figure 6] 10A and 10B are diagrams showing other examples of images displayed by the terminal device when the position of the subject's face on the screen of the display unit is shifted within a predetermined range. [Figure 7] 10 is a diagram showing an image displayed by the terminal device when the size of the subject's face on the screen of the display unit becomes smaller than a predetermined threshold value. FIG. [Figure 8] 10 is a diagram showing an example of an image displayed by the terminal device based on the amount of deviation of the subject's face position from a predetermined detection area on the screen of the display unit. FIG. [Figure 9] 10 is a diagram showing another example of an image displayed by the terminal device based on the amount of deviation of the subject's face position from the predetermined detection area on the screen of the display unit. FIG. [Figure 10] FIG. 10 is a diagram showing yet another example of an image displayed by the terminal device based on the amount of deviation of the subject's face position from a predetermined detection area on the screen of the display unit. [Figure 11] FIG. 10 is a diagram showing yet another example of an image displayed by the terminal device based on the amount of deviation of the subject's face position from a predetermined detection area on the screen of the display unit. [Figure 12] 10A and 10B are diagrams showing examples of images displayed by a terminal device based on the progress of measurement of biological information of a subject. [Figure 13] FIG. 10 is a diagram showing an example of an image displayed by a terminal device based on the fact that biological information of a person being measured is currently being measured. [Figure 14] FIG. 10 is a diagram showing another example of an image displayed by the terminal device based on the fact that biological information of the subject is being measured. [Figure 15] FIG. 10 is a diagram showing an example of an image displayed by the terminal device upon completion of measurement of the subject's biological information. [Figure 16]FIG. 10 is a diagram showing a gradual change from a second index image to a first index image. DETAILED DESCRIPTION OF THE INVENTION

[0010] [Embodiment 1] (1. General Configuration of Terminal Device 100) FIG. 1 is a block diagram showing a schematic configuration of a terminal device according to this embodiment. As shown in FIG. 1, the terminal device 100 includes a control unit 1, an image capturing unit 2, an input unit 3, a display unit 4, a communication unit 5, and a storage unit 6. The terminal device 100 is a device used to measure biometric information of a person being measured based on a captured image of the person being measured (i.e., non-contact). Examples of biometric information include, but are not limited to, the person's body temperature, blood pressure, pulse rate, respiratory rate, etc. The terminal device 100 may be any device having an image acquisition function and an image display function, and may be, for example, a smartphone or a tablet terminal.

[0011] The person measuring measures the biological information of the person being measured by using the terminal device 100 to continue photographing the person being measured while checking an image including the person being measured displayed on the screen of the display unit 4. Such a terminal device 100 is used, for example, in a care setting when a caregiver (measurer) measures the biological information of a person being cared for (person being measured).

[0012] The functions of the terminal device 100 described below can also be realized by installing a program on a general-purpose computer. The program can also be called application software. In the following, an example will be described in which the terminal device 100 is a smartphone on which the program is installed.

[0013] The control unit 1 controls each unit of the terminal device 100 in an integrated manner. The photographing unit 2 photographs images. The input unit 3 accepts input operations for the terminal device 100. The display unit 4 has a screen and displays images photographed by the photographing unit 2 on the screen. The communication unit 5 transmits and receives various data between an external communication device and the control unit 1. The memory unit 6 stores various data handled by the terminal device 100. The input unit 3 may be a touch panel, in which case the input unit 3 is configured as an integrated unit with the display unit 4.

[0014] The control unit 1 includes an information acquisition unit 11 , a measurement unit 12 , a tilt detection unit 13 , a direction detection unit 14 , a size detection unit 15 , a situation identification unit 16 , and a display control unit 17 .

[0015] The information acquiring unit 11 acquires an image of the subject captured by the imaging unit 2 (hereinafter referred to as a captured image) as information for measuring the subject's biological information. The captured image may be a moving image of the subject or a still image of the subject.

[0016] The measurement unit 12 measures the biological information of the subject based on the captured image. Note that an external server may measure the biological information of the subject instead of the measurement unit 12. In this case, the captured image acquired by the information acquisition unit 11 is transmitted to the external server via the communication unit 5.

[0017] The tilt detection unit 13 detects the tilt of the subject's face on the screen of the display unit 4 based on the facial image of the subject (hereinafter simply referred to as the facial image) included in the captured image acquired by the information acquisition unit 11. The tilt of the face is calculated from the center line of the face. The center line is, for example, a line connecting the center of the nose and the center of the mouth, or a line connecting the center of the right eye and the center of the left eye. The orientation detection unit 14 detects whether the subject's face is facing sideways on the screen of the display unit 4 based on the facial image. The size detection unit 15 detects the size of the subject's face on the screen of the display unit 4 based on the facial image.

[0018] The "face tilt" of the subject on the screen of the display unit 4 refers to the angle between a predetermined reference direction on the screen and the center line of the facial image, such as the up-down direction parallel to the vertical side of the display screen or the left-right direction parallel to the horizontal side.

[0019] The situation identification unit 16 identifies a situation related to the measurement. The situation identification unit 16 identifies whether the measurement of the biological information has failed, whether the measurement of the biological information has been completed, etc. The situation identification unit 16 also includes a first identification unit 161, a second identification unit 162, and a third identification unit 163.

[0020] The first identifying unit 161 identifies a deviation of the face position of the person being measured from a predetermined detection area during imaging, based on the face image acquired by the information acquiring unit 11. The predetermined detection area may be, for example, the position (predetermined position) of a second index image 52 (see FIGS. 2 and 3) described later that is displayed by the display control unit 17 before the start of measurement, or the position of the face image on the screen of the display unit 4 at the start of measurement.

[0021] The second specifying unit 162 specifies the progress of measurement of the biological information of the subject by the measurement unit 12. The third specifying unit 163 specifies that the measurement unit 12 is currently measuring the biological information of the subject.

[0022] The display control unit 17 displays the captured image on the screen of the display unit 4. Hereinafter, the image including the person being measured that is displayed on the screen will be referred to as the display image. The display control unit 17 displays a first index image 51, which is used to encourage the person being measured to continue taking pictures without changing the position of the person's face, superimposed on the image including the person being measured that is displayed on the display unit 4 (displayed on the screen), and changes the display form of the first index image 51 according to the situation identified by the situation identification unit 16.

[0023] Furthermore, the display control unit 17 may superimpose a first index image 51 that matches the inclination of the face detected by the inclination detection unit 13 on the display image (see FIGS. 2 to 14).

[0024] The first index image 51 may be a human-shaped contour guide including a face and shoulders. Here, the first index image 51 may be generated by rotating a contour guide selected from a plurality of human-shaped contour guides pre-stored in the storage unit 6 based on the face inclination detected by the inclination detection unit 13. Alternatively, the first index image 51 may be generated by extracting the contours of the face and shoulders of the person being measured included in the captured image.

[0025] Furthermore, before measurement by the measurement unit 12 begins, the display control unit 17 may superimpose on the display image a second index image 52 for the person being measured to align the face position of the person being measured to a predetermined position on the screen of the display unit 4 (see FIGS. 2 and 3). In this case, after measurement begins, the display control unit 17 superimposes on the display image a first index image 51, which is a human-shaped outline guide including the face and shoulders.

[0026] The second index image 52 may be a frame-shaped guide that allows the subject's face displayed on the screen to fit inside. The size of the frame-shaped guide is such that, by having the subject photograph the subject's face at approximately the same size as the frame guide, the distance between the subject and the terminal device 100 is an appropriate distance (e.g., 50 to 60 cm) that corresponds to the performance of the photographing unit 2. The subject can measure the subject's biometric information by maintaining an appropriate distance between the subject and the terminal device 100. The second index image 52 may be a figure (e.g., a cross) that indicates the center of the subject's face when the subject's face is aligned with a predetermined position on the screen of the display unit 4, or a figure that indicates the positions of the eyes, nose, mouth, ears, etc.

[0027] The frame guide may have an isotropic shape, including but not limited to a circle, a regular polygon, or a sphere.

[0028] The display control unit 17 may also superimpose on the display image a notification image 61 showing notification information related to the measurement to the subject based on the situation related to the measurement identified by the situation identification unit 16 (see FIGS. 2 and 3). The display control unit 17 may further superimpose on the display image a measurement result image 62 showing the biological information items of the subject measured by the measurement unit 12 and the measurement results (see FIGS. 2 and 3).

[0029] Note that instead of the display control unit 17 displaying the notification image 61, a speaker provided in the terminal device 100 may output notification information related to the measurement to the subject by voice. Similarly, instead of the display control unit 17 displaying the measurement result image 62, a speaker provided in the terminal device 100 may output the items of bioinformation of the measurement target and the measurement result by voice.

[0030] (2. Display Method of First Index Image 51 and Second Index Image 52) 2 to 7, a detailed description will be given below of a display method of the first index image 51 and the second index image 52 by the display control unit 17. In the example shown in Figs. 2 to 7, the second index image 52 is a circular frame guide with an isotropic shape, and the first index image 51 is a human-shaped outline guide.

[0031] 2 to 7, while the display image is being displayed, the display control unit 17 displays a notification image 61 at the top of the screen of the display unit 4 and a measurement result image 62 at the bottom of the screen of the display unit 4. However, the positions and timing at which the display control unit 17 displays the notification image 61 and the measurement result image 62 are not limited to this. For example, the notification image 61 and the measurement result image 62 may be displayed on the left and right sides of the screen of the display unit 4. The measurement result image 62 may be displayed after the measurement is completed. For convenience, the measurement result image 62 is not shown in FIGS. 4 to 7.

[0032] Fig. 2 is a diagram showing an example of an image displayed by the terminal device 100 when measuring a subject in a seated or standing position. Fig. 2 shows an example of display images P201 to P204 displayed by the display control unit 17 to make the subject aware of taking an appropriate photograph of the subject in a seated position.

[0033] Display image P201 indicates an image displayed by display control unit 17 at the start of imaging. Here, display control unit 17 displays second index image 52 at a predetermined position on the screen of display unit 4. The predetermined position is, for example, approximately the center of an area on the screen of display unit 4 excluding notification image 61 and measurement result image 62. In addition, display control unit 17 displays notification information in notification image 61 to indicate that the subject's face should be placed within the frame of second index image 52 (the sentence "Please place your face within the frame").

[0034] The display control unit 17 also displays a list of items of biological information to be measured ("body temperature," "blood pressure," "pulse rate," and "respiration") in the measurement result image 62. The result display field in the measurement result image 62 remains blank until the measurement is complete. The person taking the measurement checks this display image P201 and moves the terminal device 100 to adjust the shooting area so that the face of the person being measured is within the frame of the second index image 52.

[0035] Display image P202 shows a state in which the face of the person being measured has been adjusted by the measurer so that it is within the frame of second index image 52. Here, tilt detection unit 13 detects the tilt of the face of the person being measured (which has entered the frame of second index image 52) in display image P202. Also, measurement unit 12 starts measuring the biological information of the person being measured.

[0036] Display image P203 indicates an image displayed by display control unit 17 after measurement of biological information begins. Here, display control unit 17 displays first index image 51 that matches the inclination of the face detected by inclination detection unit 13. That is, display control unit 17 switches second index image 52 to the above-mentioned first index image 51. When the subject is seated as shown in FIG. 2, the inclination of the subject's face is aligned with the vertical direction on the screen of display unit 4. Therefore, first index image 51 is a left-right symmetrical human-shaped outline guide facing forward, with the shoulders located directly below the face.

[0037] Furthermore, the display control unit 17 displays notification information (the text "Measurement in progress") in the notification image 61 to the effect that measurement of the biological information is underway. Furthermore, the display control unit 17 displays the measurement results in the result display field in the measurement result image 62, one by one, starting with the items of biological information for which measurement has been completed. The display control unit 17 displays the display image P203 until measurement of the biological information is complete. The person measuring the biological information checks this display image P203 and continues to photograph the person being measured without changing the position of their face until measurement of the biological information is complete.

[0038] Display image P204 indicates an image displayed by the display control unit 17 when measurement of all items of biological information of the measurement target is completed. Here, the display control unit 17 ends the display of the first index image 51. In addition, the display control unit 17 displays notification information (the text "Measurement completed") in the notification image 61 to the effect that measurement of all items of biological information has been completed. In addition, the display control unit 17 displays the measurement results of the biological information measured in the result display field in the measurement result image 62. The subject confirms this display image P204 to recognize that measurement of the biological information has been completed.

[0039] As described above, the display control unit 17 can make the subject aware of the need to first start capturing the image so that the subject's face is within the frame of the second index image 52, and then continue capturing the image so that the face position does not shift in line with the first index image 51. This allows the subject to capture images required to measure biological information relatively easily by using the terminal device 100, even if the subject is unfamiliar with devices such as smartphones.

[0040] Furthermore, the display control unit 17 sequentially displays the measurement results for the completed biological information on the measurement result image 62, allowing the subject to check their progress. This allows the subject to feel that the measurement time is shorter than when the measurement results are displayed when the measurement of all biological information is completed, reducing the stress of the subject having to wait until the measurement is complete.

[0041] Furthermore, the display control unit 17 may display only the biological information that can be measured by the measurement unit 12 from the captured image, among the predetermined biological information (biological information of the measurement subject), in the measurement result image 62. For example, to measure the respiratory rate, the subject's shoulders must be visible in the captured image. If the subject's shoulders are not visible in the captured image, the measurement unit 12 will not measure the respiratory rate even if the predetermined biological information includes the respiratory rate.

[0042] In such a case, the situation identification unit 16 identifies the respiratory rate as unmeasurable biological information among the predetermined biological information. The display control unit 17 displays a list of biological information excluding the unmeasurable biological information in the measurement result image 62. This prevents the person being confused by the presence of biological information (biological information items) for which the measurement results are not displayed even after the measurement is completed. Furthermore, reducing the number of displayed biological information items is expected to have the effect of making the person being measured feel that the measurement time is shorter.

[0043] Fig. 3 is a diagram showing an example of an image displayed by the terminal device 100 when measuring a subject who is lying down. Fig. 3 shows an example of display images P301 to P304 that the display control unit 17 displays to make the subject aware of the importance of properly photographing the subject who is lying down.

[0044] Similar to the display images P201, P202, and P204 in FIG. 2, the display control unit 17 displays the display images P301, P302, and P304, respectively.

[0045] Furthermore, in the display image P303, the display control unit 17 displays a first index image 51 corresponding to the person being measured who is lying down. That is, as shown in Fig. 3, when the person being measured who is lying down is photographed from diagonally above while positioned to the right of the person being measured, the inclination of the person's face will be to the left with respect to the up-and-down direction on the screen of the display unit 4. The inclination detection unit 13 detects such an inclination of the person's face, and therefore the first index image 51 becomes an outline guide of a diagonally inclined human figure with the shoulder located at the lower right of the face facing diagonally upward to the right, and the left shoulder located above the right shoulder.

[0046] In this way, the display control unit 17 displays the first index image 51 in accordance with the inclination of the photographed face of the subject so that the person taking the photograph continues taking the photograph without changing the position of the subject's face. In other words, if the face of the subject lying on his / her side is photographed at an angle to the screen, the inclination of the face is detected to be oblique, and the first index image 51 in which the face is inclined obliquely is displayed. This allows the person taking the photograph to intuitively understand that there is no need to rotate the terminal device 100 in accordance with the inclination of the face of the subject lying on his / her side, that there is no need to point the terminal device 100 directly in front of the subject's face when taking the photograph, and that there is no problem in continuing to take the photograph in this state.

[0047] Furthermore, by using the first index image 51 as a human-shaped contour guide including the face and shoulders, the person being measured is conscious of trying to align the lines of the face and shoulders of the image being photographed with the lines of the human-shaped contour guide displayed nearby, and this makes it possible to make the person being measured conscious not to move the face position of the person being measured and not to move the terminal device. This makes it easy to photograph without moving (shifting) the face position.

[0048] Furthermore, by displaying the second index image 52 before starting measurement, the person taking the photograph can start taking the photograph without any confusion about the position of the person's face in the displayed image. After starting the measurement, a human-shaped outline guide including the shoulders is displayed as the first index image 51, which makes the person taking the photograph aware of not moving the person's face and not moving the terminal device, as described above. This makes it easy to take photographs without moving (shifting) the face position.

[0049] Furthermore, depending on the performance of the imaging unit 2, there may be restrictions on the distance between the subject and the terminal device 100. Even in such cases, before starting measurement, a frame-shaped guide such as that shown in Figures 2 and 3 is displayed as the second index image 52, and the subject's face is allowed to fit into the frame-shaped guide, thereby easily adjusting the distance between the subject and the terminal device 100 to within the restrictions. Note that the measurement unit 12 may start measurement after a certain period of time has passed in which the difference between the size of the frame-shaped guide and the size of the face is smaller than a threshold.

[0050] Furthermore, if a directional frame guide such as an ellipse or rectangle is displayed as the frame guide, the person being measured may be conscious of the need to align their face to the direction of the frame guide, which may cause them to feel burdened or start the measurement in an awkward position.In contrast, by using an isotropic frame guide, the person being measured can be made aware that it is sufficient for their face to be within the frame guide.

[0051] Display image P304 shows an image that the display control unit 17 displays when measurement of all items of biological information of the measurement target is completed. Here, the display control unit 17 ends the display of the first index image 51. In addition, the display control unit 17 displays, in the notification image 61, notification information that measurement of all items of biological information has been completed (the text "Measurement completed"). In addition, the display control unit 17 displays the measurement results of the biological information that has been measured in the result display field in the measurement result image 62. The person taking the measurement recognizes that measurement of the biological information has been completed by checking this display image P304.

[0052] Fig. 4 is a diagram showing an image displayed by the terminal device 100 when measuring a subject whose face is turned to the side. Fig. 4 shows an example of a display image P401 that the display control unit 17 displays when the orientation detection unit 14 determines, based on the facial image, that the subject's face is turned to the side on the screen of the display unit 4.

[0053] In the display image P401, the display control unit 17 displays a first index image 51 to prompt the person taking the photograph to keep the face position of the person being measured, who is facing sideways. Here, the human-shaped contour guide, which is the first index image 51, includes characteristic parts facing sideways (for example, a nose facing sideways or the shape of shoulders facing sideways).

[0054] This allows the subject to be aware that there is no problem with capturing a profile image. This makes it possible to avoid the possibility of the subject moving the terminal device 100 in an attempt to capture a frontal image of the face, resulting in a failed measurement.

[0055] Fig. 5 is a diagram showing an example of an image displayed by the terminal device 100 when the face position of the person being measured on the screen of the display unit 4 is displaced within a predetermined range. Fig. 5 shows examples of display images P501 to P503 in which the first index image 51 is displayed at a position where it would be displayed if there was no face position displacement, without following the displacement of the face position of the person being measured on the screen of the display unit 4. Note that when the face position is displaced, the position of the shoulders naturally also displaces, but this will hereinafter be expressed as "displacement of face position."

[0056] Display image P501 indicates an image displayed by display control unit 17 at the start of measurement of biological information. Here, display control unit 17 displays first index image 51 that is aligned with the inclination of the face detected by inclination detection unit 13 and is not misaligned with the position of the face of the subject.

[0057] Display image P502 shows a state in which the face position of the subject has shifted from the face position of the subject (shown in display image P501) at the start of measurement of biological information, for example, due to hand shake by the subject. Here, display control unit 17 superimposes and displays first index image 51 at a position where it would be displayed if there was no shift in face position, so that it does not follow the shift in face position. Therefore, first index image 51 is held in the position at the start of measurement, and in display image P502, first index image 51 is shifted from the face position of the subject.

[0058] Display image P503 shows a state in which the position of the subject's face is further displaced from the position of the subject's face at the start of measurement of the biological information than in display image P502. In display image P503, first index image 51 is further displaced from the position of the subject's face than in display image P502.

[0059] In a configuration in which the first index image 51 follows any deviation in face position, there is a concern that some people taking the photograph may become negligent in trying to keep their face position still, that is, taking a photograph without moving the terminal device 100. On the other hand, as shown in Fig. 5, by displaying the first index image 51 at the position where it would be displayed if there was no deviation in face position, the person taking the photograph can be made more aware of any deviation in face position.

[0060] Fig. 6 is a diagram showing another example of an image displayed by the terminal device 100 when the face position of the person being measured is displaced on the screen of the display unit 4. Fig. 6 shows examples of display images P601 to P603 in which the first index image 51 follows the change caused by the displacement of the face position of the person being measured on the screen of the display unit 4. Note that the display in which the first index image 51 follows the change in face position is performed only when the displacement of the face position is within a predetermined range. The predetermined range is, for example, a range in which predetermined biological information of the person being measured can be measured.

[0061] Display image P601 shows an image displayed by display control unit 17 at the start of measurement of biological information. Here, display control unit 17 displays first index image 51 that is aligned with the inclination of the face detected by inclination detection unit 13 and is not misaligned with the face position of the subject.

[0062] Display image P602 shows a state in which the face position of the subject has shifted from the face position of the subject (shown in display image P601) at the start of measurement of biological information, for example, due to hand shake by the subject. The shift in the face position of the subject in display image P602 is within a predetermined range. Here, when the shift in face position is within the predetermined range, display control unit 17 displays first index image 51 superimposed on the display image, following the change in face position (change in the position of the face image). Therefore, in display image P602, first index image 51 is not shifted from the face position of the subject.

[0063] Display image P603 shows a state in which the face position of the subject is further displaced from the face position of the subject at the start of measurement of biological information than in display image P602. The displacement of the face position of the subject in display image P603 is within a predetermined range. Since first index image 51 is displayed following changes in the face image, first index image 51 does not displace from the subject in display image P603 either.

[0064] In a configuration in which the first index image 51 is displayed at the position where it would be if there was no misalignment of the face position, some subjects may become concerned about the misalignment of the face position and move the terminal device significantly, which may result in failure to acquire information necessary for measuring biometric information. For example, continuous changes are required to measure a pulse, but if the face image changes significantly, continuous changes cannot be acquired. In such cases, by displaying the image in accordance with the change in face position as described above, it is possible to reduce measurement failures caused by moving the terminal device 100. Furthermore, by displaying the image in accordance with the change in face position, the subject can be assured that there are no problems with the image capture and that the measurement is in progress, which can provide peace of mind.

[0065] Fig. 7 is a diagram showing images displayed by the terminal device 100 when the size of the subject's face on the screen of the display unit 4 becomes smaller than a predetermined threshold. Fig. 7 shows examples of display images P701 to P703 in which the size of the first index image 51 is set to a predetermined minimum size when the size of the subject's face on the screen of the display unit 4 is smaller than a predetermined threshold.

[0066] Display image P701 shows an image displayed by display control unit 17 at the start of measurement of biological information. Here, display control unit 17 displays first index image 51 that matches the size of the subject's face detected by size detection unit 15.

[0067] Display image P702 shows a state in which the size of the subject's face has become smaller than the size of the subject's face at the start of measurement of biological information (shown in display image P701). The size of the subject's face in display image P702 is equal to or larger than a predetermined threshold. Here, when the size of the subject's face is equal to or larger than the predetermined threshold, display control unit 17 displays first index image 51 (human-shaped outline guide) so that the size of the first index image 51 matches the size of the subject's face in the display image. Therefore, from display image P701 to display image P702, first index image 51 matches the size of the subject's face.

[0068] Display image P703 shows a state in which the size of the subject's face is even smaller than in display image P702, relative to the size of the subject's face at the start of measurement of biological information. The size of the subject's face in display image P703 is smaller than a predetermined threshold. Here, when the size of the subject's face is smaller than the predetermined threshold, display control unit 17 displays first index image 51 (human-shaped outline guide) at a predetermined minimum size. In display image P703, first index image 51 does not follow the size of the subject's face but maintains the predetermined minimum size.

[0069] If the distance between the terminal device 100 and the subject is far and the size of the subject's face in the captured image is small, there is a possibility that measurement of biological information will fail. Therefore, it is necessary to prevent the size of the subject's face in the captured image from becoming too small. Therefore, when the size of the subject's face is smaller than a predetermined threshold, the display control unit 17 fixes the size of the first index image 51 to a predetermined minimum size and displays the first index image 51 so that it does not become smaller than the predetermined minimum size. This makes it possible to make the subject aware that the size of the subject's face on the screen of the display unit 4 must not be smaller than the predetermined threshold (i.e., to maintain an appropriate distance between the terminal device 100 and the subject).

[0070] (3. Configuration for changing the display form of the first index image 51) Below, we will explain the configuration in which the display control unit 17 superimposes the first index image 51 on the display image to encourage the person being measured to continue taking pictures without changing the position of the person's face, and changes the display form of the first index image 51 depending on the situation identified by the situation identification unit 16.

[0071] 8 to 14, the change in the display form of the first index image 51 by the display control unit 17 will be described below. In the example shown in FIGS. 8 to 14, the first index image 51 is a human-shaped outline guide. Furthermore, the first index image 51 is exemplified as the first index image 51 that matches the inclination of the face of the person being measured as described above, but a configuration in which the display form of a conventional type of index image fixed on the screen is changed may also be used.

[0072] 8 to 14, while the display image is being displayed, the display control unit 17 displays a notification image 61 at the top of the screen of the display unit 4 and a measurement result image 62 at the bottom of the screen of the display unit 4. However, the positions and timing at which the display control unit 17 displays the notification image 61 and the measurement result image 62 are not limited to this. For example, the notification image 61 and the measurement result image 62 may be displayed on the left and right sides of the screen of the display unit 4. The measurement result image 62 may be displayed after the measurement is completed. For convenience, the measurement result image 62 is not shown in FIGS. 8 to 14.

[0073] 8 to 11 are diagrams showing examples of images displayed by the terminal device 100 based on the magnitude of deviation of the subject's face position from the predetermined detection area on the screen of the display unit 4. The first specifying unit 161 specifies the deviation of the subject's face position from the predetermined detection area during shooting, based on the face image of the subject included in the captured image. The display control unit 17 changes the display form of the first index image 51 based on the determination by the first specifying unit 161 so that the subject can grasp the magnitude of the deviation. FIGS. 8 to 11 show such changes in the display form of the first index image 51.

[0074] FIG. 8 shows examples of display images P801 to P803 in which the thickness of the first index image 51 is changed according to the magnitude of the deviation.

[0075] Display image P801 shows an image displayed by display control unit 17 at the start of measurement of biological information. Here, display control unit 17 displays first index image 51 (human-shaped outline guide) that is not misaligned with the face position of the subject.

[0076] Display images P802 and P803 show a state in which the face position of the subject is displaced from the face position of the subject at the start of measurement of biological information (shown in display image P1101). Here, display control unit 17 changes the thickness of first index image 51 according to the amount of displacement of the face position of the subject during imaging from a predetermined detection area.

[0077] 8, the display control unit 17 makes the thickness of the first index image 51 thinner as the deviation increases. When the deviation is small, the first index image 51 is displayed thicker, and as the deviation increases, the first index image 51 becomes thinner, which allows the measurer to intuitively sense that there is deviation. Conversely, as the deviation increases, the first index image 51 is displayed thicker, which also allows the measurer to intuitively sense that there is deviation.

[0078] 9 shows examples of display images P901 to P903 in which the color of the first index image 51 changes depending on the magnitude of the deviation. Display image P901 shows the same state as display image P801.

[0079] Display images P902 and P903 show a state in which the subject's face position has shifted from the face position of the subject at the start of biometric information measurement (shown in display image P901). Here, the display control unit 17 changes the color of the first index image 51 depending on the magnitude of deviation of the subject's face position from a predetermined detection area during image capture. For example, the display control unit 17 changes the color of the first index image 51 from white to red as the deviation increases. Multiple thresholds may be set for the magnitude of deviation, and the color may be changed for each threshold. For example, if the deviation is smaller than a first threshold, the color may be green; if the deviation is greater than the first threshold but less than a second threshold, the color may be yellow; and if the deviation is greater than the second threshold, the color may be red. Using the same colors as traffic lights makes it easier to understand the meaning of the colors.

[0080] 10 shows examples of display images P1001 to P1003 in which an image similar to the first index image is displayed like an afterimage. Display image P1001 shows the same state as display image P801.

[0081] Display images P1002 and P1003 show a state in which the face position of the subject has shifted from the face position of the subject (shown in display image P1001) at the start of biometric information measurement. Here, the display control unit 17 displays, as an afterimage, not only the first index image 51 located in the detection area but also an image similar to the first index image 51 that moves along with the subject's face image. In the example shown in FIG. 10 , the display control unit 17 displays, as afterimages 51a to 51c, images similar to the first index image 51 that move along with the subject's face image from the start of biometric information measurement to the present. Here, the first index image 51 at the face position at the start of measurement is highlighted by being bolded, but the first index image 51 at the current face position may also be highlighted by being bolded. Alternatively, all the first index images may be highlighted by being the same bolded. Any number of similar images may be displayed as afterimages.

[0082] 11 shows examples of display images P1101 to P1104 in which the first index image, which has been following the changes in the facial image, is displayed as if it is quickly returned to its original position at the start of measurement when the facial image shifts beyond a predetermined range. Display image P1101 shows the same state as display image P801.

[0083] Display image P1102 shows a state in which the face position of the subject has shifted from the face position of the subject at the start of measurement of biological information (shown in display image P1101). Note that the shift in the face position of the subject in display image P1102 is within a predetermined range. When the shift in face position is within the predetermined range, display control unit 17 superimposes and displays first index image 51 on the display image in accordance with changes in face position. Therefore, in display image P1102, first index image 51 is not shifted from the face position of the subject.

[0084] Display image P1103 shows a state in which the position of the subject's face is further displaced from the position of the subject's face at the start of measurement of biological information than in display image P1102. Note that the displacement of the subject's face position in display image P1103 is within a predetermined range. In display image P1103, first index image 51 is not displaced from the position of the subject's face.

[0085] Display image P1104 shows a state in which the face position of the subject has shifted beyond a predetermined range relative to the face position of the subject at the start of measurement of the biological information. When the shift in face position exceeds the predetermined range, the display control unit 17 displays the first index image 51 at a position where it would be displayed if there was no shift (toward the direction of the first index image 51 displayed at the start of measurement of the biological information) by changing the face position at a speed faster than the change in face position. In the example shown in Fig. 11, since the face position has shifted upward, the display control unit 17 represents the movement of the first index image as the first index image quickly moving downward.

[0086] 12 is a diagram showing an example of an image displayed by the terminal device 100 based on the progress of measurement of the subject's biological information. The second specifying unit 162 specifies the progress of measurement of the subject's biological information in the measurement unit 12 (hereinafter referred to as measurement progress). The display control unit 17 changes the display form of the first index image 51 based on the specification by the second specifying unit 162 so that the subject can grasp the measurement progress.

[0087] 12 shows examples of display images P1201 to P1204 in which the color of the first index image gradually changes from one end to the other according to the progress of measurement. Display image P1201 shows the same state as display image P801. Note that the display control unit 17 displays the first index image 51 in the first color.

[0088] Display image P1202 shows a state in which the measurement progress has progressed from the state shown in display image P1201. Here, the display control unit 17 gradually changes the color of the first index image 51 from one end to the other end according to the measurement progress. In the example shown in Fig. 12, the display control unit 17 changes the color of the first index image 51 from a first color to a second color from one end (left shoulder) to the other end (right shoulder) of the first index image 51.

[0089] Display image P1203 shows a state in which the measurement progress has progressed further from the state shown in display image P1202. Display image P1204 shows a state in which the measurement progress has progressed further from the state shown in display image P1203, and the measurement has been completed. At this point, the display control unit 17 displays the first index image 51 in the second color from one end to the other end.

[0090] 13 is a diagram showing an example of an image displayed by the terminal device 100 based on the fact that the biological information of the subject is being measured. The third identification unit 163 identifies that the biological information of the subject is being measured by the measurement unit 12. Based on the identification by the third identification unit 163, the display control unit 17 changes the display form of the first index image 51 so that the subject can understand that measurement is being performed.

[0091] 13 shows examples of display images P1301 to P1304 in which the thickness of the first index image is changed regularly. Display image P1301 shows the same state as display image P801.

[0092] When the measurement unit 12 is measuring the biological information of the subject, the display control unit 17 regularly changes the thickness of the first index image 51. In the example shown in Fig. 13, the display control unit 17 displays the first index image 51 so that the thickness thereof alternates between the thickness shown in display images P1301 and P1303 and the thickness shown in display images P1302 and P1304.

[0093] 14 is a diagram showing another example of an image displayed by the terminal device 100 based on the fact that the biological information of the subject is being measured. The third identification unit 163 identifies that the biological information of the subject is being measured by the measurement unit 12. Based on the identification by the third identification unit 163, the display control unit 17 changes the display form of the first index image 51 so that the subject can understand that measurement is being performed.

[0094] 14 shows an example of display images P1401 to P1404 in which the size of the first index image is changed regularly. Display image P1401 shows the same state as display image P801.

[0095] When the measurement unit 12 is measuring the biological information of the subject, the display control unit 17 regularly changes the size of the first index image 51. In the example shown in Fig. 14, the display control unit 17 displays the first index image 51 so that the size thereof alternates between a size that matches the size of the subject's face as shown in display images P1401 and P1403 and a size that is slightly larger than the size of the subject's face as shown in display images P1402 and P1404.

[0096] Alternatively, the display control unit 17 may display the size of the first index image 51 by alternating between a size that matches the size of the subject's face, as shown in display images P1401 and P1403, and a size that is slightly smaller than the size of the subject's face.

[0097] (4. Example of display when measurement is completed) 15 is a diagram showing an image displayed by the terminal device 100 when measurement of the subject's biological information has ended. The situation identification unit 16 identifies that measurement of the subject's biological information has ended in the measurement unit 12. Based on the identification by the situation identification unit 16, the display control unit 17 changes the display format of the screen of the display unit 4 so that the subject can understand that measurement has ended.

[0098] 15 shows examples of display images P1501 and P1502 that are displayed facing forward on the screen of the display unit 4 when measurement of the subject's biological information is completed. Display image P1501 shows the same state as display image P801.

[0099] Display image P1502 shows an image that the display control unit 17 displays after measurement of the subject's biological information has been completed. Here, the display control unit 17 displays the screen of the display unit 4 facing the subject directly, based on the identification by the situation identification unit 16, so that the subject can understand that the measurement has been completed. In other words, the display control unit 17 rotates and displays the display image so that the inclination of the face is aligned with the vertical direction on the screen of the display unit 4. The display control unit 17 may also display the process of the rotation. This allows the terminal device 100 to allow the subject to intuitively understand that it is time to end photographing the subject.

[0100] The display control unit 17 may terminate the display of the first index image 51 so that the subject can understand that the measurement has ended. Furthermore, regardless of whether the subject is facing the screen of the display unit 4 directly or not, including when other display examples are shown, the display control unit 17 may display the biological information for which the measurement has been completed in a larger size.

[0101] Furthermore, regardless of whether the screen of the display unit 4 is facing the subject directly, including the cases shown in other display examples, if a video is being displayed during measurement, the display control unit 17 may switch the display format of the screen of the display unit 4 from a video to a still image after the measurement is completed.

[0102] 16 is a diagram showing a gradual change from the second index image 52 to the first index image 51. As shown in FIG. 16, when switching the second index image 52 to the first index image 51, the display control unit 17 may gradually change the second index image 52 (figure m1), which is a circular frame-shaped guide, to the first index image 51 (figure m8), which is a human-shaped outline guide. Then, a portion of the human-shaped outline guide whose size corresponds to the captured image may be displayed on the screen of the display unit 4.

[0103] In this way, by smoothly changing from the second index image 52 to the first index image 51 that matches the inclination of the face of the person being measured, the person being measured can be made to recognize that appropriate photography is possible in the current state.

[0104] (5. Effects) As described above, the display control unit 17 displays the first index image, which is for urging the subject to continue taking pictures without changing the face position, superimposed on the image including the subject. Therefore, the terminal device 100 can make the subject aware of not changing the face position, and can easily take pictures without changing (shifting) the face position.

[0105] Moreover, the display control unit 17 changes the display form of the first index image according to the measurement-related situation identified by the situation identification unit 16. This makes it possible to communicate the identified situation to the subject gazing at the portion where the first index image is displayed without making the subject look away. This makes it possible to provide a terminal device that allows the subject to check information about the measurement-related situation displayed on the terminal device while gazing at the index image.

[0106] Furthermore, by using a human-shaped outline guide as the first index image 51, the person taking the measurement can be made aware of not moving the face of the person being measured and not moving the terminal device. This makes it easy to take a photograph without moving (shifting) the face position.

[0107] 8 to 11, the display control unit 17 can change the display form of the first index image 51 based on the amount of displacement of the face position. Therefore, the terminal device 100 can allow the subject to intuitively grasp the amount of displacement of the face position using the first index image 51 without having to avert their eyes.

[0108] 10, the display control unit 17 can display afterimages 51a to 51c based on the amount of displacement of the face position. Therefore, the interval between the displayed similar image and the original first index image narrows or widens depending on the amount of displacement of the face position, so the terminal device 100 allows the person to intuitively understand that the face position has shifted and the amount of the shift.

[0109] 11, when the amount of deviation of the face position exceeds a predetermined range, the display control unit 17 displays the first index image 51 with a movement that quickly returns to the position where it would be displayed if there was no deviation. In other words, the terminal device 100 does not simply make the first index image 51 follow the face position, but makes it change faster than the change in the face position and makes it bounce, thereby making the person being measured more aware that the face position has shifted.

[0110] 12, the display control unit 17 changes the display form of the first index image 51 depending on the measurement progress. Therefore, the terminal device 100 can allow the subject to intuitively grasp the measurement progress using the first index image 51 without having to avert their eyes. In particular, the display control unit 17 displays the measurement progress like an indicator depending on the measurement progress, allowing the subject to intuitively grasp the measurement progress.

[0111] 13 and 14, when the biological information of the subject is being measured, the display control unit 17 changes the display form of the first index image 51. For example, the thickness or size of the first index image is changed regularly. Therefore, the terminal device 100 can use the first index image 51 to let the subject know that measurement is underway without having to look away.

[0112] A speaker provided in the terminal device 100 may be used to enable the subject to grasp the magnitude of the deviation. That is, the control unit 1, which controls the speaker output, changes the speaker output mode depending on the magnitude of deviation of the subject's face position from a predetermined detection area during imaging. This allows the subject, who is gazing at the area where the first index image is displayed, to be informed of the specified situation by the sound output from the speaker without having to avert their gaze. For example, the sound may be output at intervals according to the magnitude of the deviation, or only when the deviation becomes larger than a threshold, or only when the deviation becomes smaller than a threshold. This allows the subject to be informed of the deviation situation.

[0113] As described above, the terminal device 100 according to this embodiment can contribute to the achievement of Goal 3 of the Sustainable Development Goals (SDGs), which is "Good health and well-being for all." This can contribute to the achievement of the Sustainable Development Goals (SDGs).

[0114] [Software implementation example] The functions of the terminal device 100 (hereinafter referred to as the "device") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (particularly each part included in the control unit 1).

[0115] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0116] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0117] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0118] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0119] 〔summary〕 A terminal device according to aspect 1 of the present invention comprises an imaging unit, a display unit, an information acquisition unit that acquires an image of the subject captured by the imaging unit as information for measuring the subject's biometric information, a situation identification unit that identifies a situation related to the measurement, and a display control unit that displays a first index image superimposed on an image including the subject displayed on the display unit and changes the display form of the first index image according to the situation identified by the situation identification unit.

[0120] In the terminal device according to a second aspect of the present invention, in the above-mentioned first aspect, the first index image is a human-shaped outline guide including a face and shoulders.

[0121] In the terminal device of aspect 3 of the present invention, in aspect 1 or 2 above, the situation identification unit includes a first identification unit that identifies a deviation of the face position of the person being measured during imaging from a predetermined detection area based on the face image of the person being measured contained in the image being captured by the imaging unit, and the display control unit changes the display form of the first indicator image based on the identification by the first identification unit so that the person being measured can grasp the magnitude of the deviation.

[0122] A fourth aspect of the present invention is directed to the terminal device of the third aspect, wherein the display control unit changes the thickness of the first index image in accordance with the magnitude of the deviation.

[0123] A fifth aspect of the present invention is directed to the terminal device of the third aspect, wherein the display control unit changes the color of the first indicator image in accordance with the magnitude of the deviation.

[0124] In the terminal device according to aspect 6 of the present invention, in addition to the first index image located in the detection area, in the above-mentioned aspect 3, the display control unit displays an image similar to the first index image that moves together with the facial image of the subject, like an afterimage.

[0125] In a terminal device according to aspect 7 of the present invention, in the above-mentioned aspect 3, the display control unit displays the first indicator image so as to follow the change in the face position when the deviation in the face position is within a predetermined range, and when the deviation in the face position exceeds the predetermined range, displays the first indicator image at a position that would be displayed if there was no deviation, changing it faster than the change in the face position.

[0126] In the terminal device of aspect 8 of the present invention, in any of aspects 1 to 7 above, the situation identification unit includes a second identification unit that identifies the progress of the measurement of the subject's biological information, and the display control unit changes the display form of the first indicator image based on the identification by the second identification unit so that the subject can grasp the progress.

[0127] A ninth aspect of the present invention is directed to the terminal device of the eighth aspect, wherein the display control unit gradually changes the color of the first indicator image from one end to the other end in accordance with the progress status.

[0128] In the terminal device of aspect 10 of the present invention, in any of aspects 1 to 9 above, the situation identification unit includes a third identification unit that identifies that the biometric information of the person being measured is being measured, and the display control unit changes the display form of the first indicator image based on the identification by the third identification unit so that the person being measured can understand that measurement is being performed.

[0129] A terminal device according to an eleventh aspect of the present invention is based on the tenth aspect, wherein the display control unit regularly changes the thickness or size of the first index image during measurement.

[0130] The terminal device according to each aspect of the present invention may be realized by a computer. In this case, the control program of the terminal device that realizes the terminal device on a computer by causing the computer to operate as each part (software element) of the terminal device, and the computer-readable recording medium on which the program is recorded, also fall within the scope of the present invention.

[0131] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment. [Explanation of symbols]

[0132] 1. Control section 2. Filming Department 4 Display 11 Information acquisition department 13 Tilt detection unit 14 Orientation detection unit 15 Size detection unit 16 Situation Identification Department 17 Display control unit 51 First index image 52 Second index image 100 Terminal Device 161 1st Specific Part 162 Second Specific Part 163 Third Specific Part

Claims

1. The photography department and A display unit; an information acquisition unit that acquires an image of the subject taken by the imaging unit as information for measuring biological information of the subject; a situation identification unit that identifies a situation related to the measurement; a display control unit that displays a first indicator image superimposed on an image including the subject displayed on the display unit, and that displays a circular frame-shaped guide that fits into the face as the first indicator image when the situation identified by the situation identification unit is before the measurement, and changes the display form of the first indicator image to a human-shaped outline guide that includes the face and shoulders when the situation identified by the situation identification unit is during the measurement.

2. A tilt detection unit is provided that detects the tilt of the subject's face on the screen of the display unit based on the image acquired by the information acquisition unit, The terminal device according to claim 1 , wherein the display control unit rotates the outline guide based on the inclination of the face detected by the inclination detection unit.

3. A photographing unit, A display unit; an information acquisition unit that acquires an image of the subject taken by the imaging unit as information for measuring biological information of the subject; a situation identification unit that identifies a situation related to the measurement; a display control unit that displays a first indicator image superimposed on an image including the subject displayed on the display unit, and changes a display form of the first indicator image according to the situation identified by the situation identifying unit, the situation specifying unit includes a first specifying unit that specifies a deviation of a face position of the subject during photographing from a predetermined detection area based on a face image of the subject included in the image being photographed by the photographing unit, the display control unit changes a display form of the first index image based on the identification by the first identification unit so that the measurer can grasp the magnitude of the deviation; The display control unit changes the thickness of the first indicator image according to the magnitude of the deviation.

4. A photographing unit, A display unit; an information acquisition unit that acquires an image of the subject taken by the imaging unit as information for measuring biological information of the subject; a situation identification unit that identifies a situation related to the measurement; a display control unit that displays a first indicator image superimposed on an image including the subject displayed on the display unit, and changes a display form of the first indicator image according to the situation identified by the situation identifying unit, the situation specifying unit includes a first specifying unit that specifies a deviation of a face position of the subject during photographing from a predetermined detection area based on a face image of the subject included in the image being photographed by the photographing unit, the display control unit changes a display form of the first index image based on the identification by the first identification unit so that the measurer can grasp the magnitude of the deviation; The display control unit is a terminal device that displays, in addition to the first index image located in the detection area, an image similar to the first index image that moves together with the facial image of the subject, like an afterimage.

5. A photographing unit, A display unit; an information acquisition unit that acquires an image of the subject taken by the imaging unit as information for measuring biological information of the subject; a situation identification unit that identifies a situation related to the measurement; a display control unit that displays a first indicator image superimposed on an image including the subject displayed on the display unit, and changes a display form of the first indicator image according to the situation identified by the situation identifying unit, the situation specifying unit includes a first specifying unit that specifies a deviation of a face position of the subject during photographing from a predetermined detection area based on a face image of the subject included in the image being photographed by the photographing unit, the display control unit changes a display form of the first index image based on the identification by the first identification unit so that the measurer can grasp the magnitude of the deviation; The display control unit displays the first index image in accordance with changes in the face position when the face position deviation is within a predetermined range, and when the face position deviation exceeds the predetermined range, the display control unit changes the first index image to a position that would be displayed if there was no deviation, at a faster rate than the change in the face position.

6. The situation identification unit includes a first identification unit that identifies a deviation of the face position of the subject during shooting from a predetermined detection area based on a facial image of the subject included in the image being shot by the shooting unit, The terminal device according to claim 1 , wherein the display control unit changes a display form of the first index image based on the identification by the first identification unit so that the measurer can grasp the magnitude of the deviation.

7. A terminal device described in any one of claims 3 to 6, wherein the display control unit changes the color of the first indicator image depending on the magnitude of the deviation.

8. A photographing unit, A display unit; an information acquisition unit that acquires an image of the subject taken by the imaging unit as information for measuring biological information of the subject; a situation identification unit that identifies a situation related to the measurement; a display control unit that displays a first indicator image superimposed on an image including the subject displayed on the display unit, and changes a display form of the first indicator image according to the situation identified by the situation identifying unit, the status specifying unit includes a second specifying unit that specifies a progress status of measurement of the biological information of the subject, the display control unit changes a display form of the first indicator image based on the identification by the second identification unit so that the measurer can grasp the progress status; and The display control unit is a terminal device that gradually changes the color of the first indicator image from one end to the other end according to the progress status.

9. A photographing unit, A display unit; an information acquisition unit that acquires an image of the subject taken by the imaging unit as information for measuring biological information of the subject; a situation identification unit that identifies a situation related to the measurement; a display control unit that displays a first indicator image superimposed on an image including the subject displayed on the display unit, and changes a display form of the first indicator image according to the situation identified by the situation identifying unit, the situation specifying unit includes a third specifying unit that specifies that biological information of the subject is being measured, the display control unit changes a display form of the first indicator image based on the identification by the third identification unit so that the measurer can understand that measurement is in progress; and The display control unit is a terminal device that regularly changes the thickness or size of the first indicator image during measurement.

10. A terminal device described in any one of claims 3 to 9, wherein the first indicator image is a human-shaped outline guide including a face and shoulders.

11. The situation identification unit includes a third identification unit that identifies that the biological information of the subject is being measured, The terminal device according to any one of claims 3 to 8, wherein the display control unit changes the display form of the first indicator image based on the identification by the third identification unit so that the person taking the measurement can understand that measurement is in progress.

12. 12. A program for causing a computer to function as the terminal device according to claim 1, the program causing the computer to function as at least the information acquisition unit, the situation identification unit, and the display control unit.

Images