Terminal

Abstract

To achieve highly convenient terminal while securing proper handling of individual information.SOLUTION: A terminal (100) includes an identification processing unit (12) that extracts a face image for identifying a person to be measured from a captured image, a communication control unit (13) that transmits information indicating the face image to a server (200) and receives measuring history information from the server, and a display control unit (15) that superimposes and displays measuring state information on the captured image based on the measuring history information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a terminal device that measures biological information of a subject in a non-contact manner. [Background technology]

[0002] Conventionally, there is known a technique for superimposing information for identifying a person on an image of the person. For example, Patent Document 1 discloses a technique for displaying a name of a person recognized from a facial image of the person in association with the person. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-212867 Summary of the Invention [Problem to be solved by the invention]

[0004] Furthermore, there is known a technology (non-contact vital sensing) that measures the biological information of a person being measured based on an image of the person being measured taken by a terminal device. In this technology, from the viewpoint of protecting personal information, information such as measurement results is stored on a server with access restrictions, and is managed so that as little information as possible is left on the terminal device.

[0005] Therefore, the technology disclosed in Patent Document 1 cannot be applied to the above-mentioned non-contact vital sensing.

[0006] Furthermore, when the person taking the measurement checks the past measurement status of the person being measured (for example, whether the person's biological information has already been measured on that day), it is necessary to access the server from a terminal other than the terminal device that measures the biological information, which may be inconvenient for the person being measured.

[0007] One aspect of the present invention has been made in consideration of the above problems, and its purpose is to realize a highly convenient terminal device while ensuring the proper handling of personal information. [Means for solving the problem]

[0008] In order to solve the above problem, a terminal device according to one embodiment of the present invention comprises an image processing unit that extracts a facial image for authenticating the person being measured from an image captured by an imaging device, a communication control unit that transmits information indicating the facial image to a server and receives from the server measurement history information regarding the measurement history of the person's biometric information identified from the information indicating the facial image, and a display control unit that displays measurement status information indicating the person's past measurement status based on the measurement history information, superimposed on the captured image. [Effects of the Invention]

[0009] According to one aspect of the present invention, it is possible to realize a highly convenient terminal device while ensuring the proper handling of personal information. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a schematic configuration of a terminal device according to a first embodiment. [Figure 2] 10A and 10B are diagrams illustrating switching of the display screen of the terminal device. [Figure 3] 10 is a flowchart showing switching of the control mode of the terminal device. [Figure 4] FIG. 10 is a diagram showing an example of a captured image before authentication displayed by the terminal device. [Figure 5] FIG. 10 is a diagram showing an example of a captured image displayed by the terminal device after authentication. [Figure 6] FIG. 10 is a diagram showing a sequence of basic operations of an authentication operation performed using the terminal device. [Figure 7] FIG. 10 is a diagram showing another example of a captured image after authentication. [Figure 8] FIG. 10 is a diagram showing yet another example of a captured image after authentication. [Figure 9] FIG. 10 is a diagram showing yet another example of a captured image after authentication. [Figure 10] FIG. 10 is a diagram showing yet another example of a captured image after authentication. [Figure 11] FIG. 10 is a diagram showing yet another example of a captured image after authentication. [Figure 12] FIG. 10 is a diagram showing yet another example of a captured image after authentication. [Figure 13] FIG. 10 is a diagram showing yet another example of a captured image after authentication. [Figure 14] FIG. 10 is a diagram showing switching of the display screen of the terminal device according to the second embodiment. [Figure 15] 10 is a flowchart showing switching of control modes of a terminal device according to a second embodiment. [Figure 16] FIG. 11 is a diagram showing a sequence of basic operations of an authentication operation performed using a terminal device according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] [Embodiment 1] (1. General configuration of terminal device) Fig. 1 is a block diagram showing a schematic configuration of a terminal device 100 according to embodiment 1. As shown in Fig. 1, the terminal device 100 includes a control unit 1, an image capturing device 2, an input device 3, a display device 4, a communication device 5, a storage device 6, a temperature measuring device 7, a microphone 8, and a speaker 9. Hereinafter, a subject of the measurement process and authentication process executed by the terminal device 100 will be referred to as a "subject," and an operator who performs the measurement operation and authentication operation of the terminal device 100 will be referred to as a "measurer."

[0012] The terminal device 100 is a device used to measure the biological information of the subject based on a captured image of the subject (i.e., without contact). Examples of biological information include, but are not limited to, the subject's temperature, blood vessel information, and heart rate information. The terminal device 100 may be any device that has an image acquisition function and an image display function, and may be, for example, a smartphone or a tablet terminal.

[0013] The person taking the measurement measures the biological information of the person being measured by using the terminal device 100 to photograph the person being measured while checking an image including the person being measured displayed on the screen of the display device. Such a terminal device 100 is used, for example, in a care setting when a caregiver (measurer) measures the biological information of a person being cared for (person being measured).

[0014] The functions of the terminal device 100 described below can also be realized by installing a program on a general-purpose computer. The program can also be called application software. In the following, an example will be described in which the terminal device 100 is a smartphone on which the program is installed.

[0015] The control unit 1 controls each unit of the terminal device 100 in an integrated manner. The photographing device 2 photographs images. The input device 3 accepts input operations to the terminal device 100. The display device 4 displays images photographed by the photographing device 2. The input device 3 may be a touch panel, in which case the input device 3 is configured integrally with the display device 4.

[0016] The communication device 5 transmits and receives various data between an external communication device and the control unit 1. The communication device 5 connects the terminal device 100 and the server 200 via the network N, enabling data exchange between them. The storage device 6 stores various data handled by the terminal device 100. The temperature measurement device 7 detects the temperature of the person being measured in a non-contact manner.

[0017] The server 200 stores personal information such as biometric information of the subject obtained by the terminal device 100 on behalf of the terminal device 100. The server 200 accumulates the measurement results of the subject's biometric information received from the terminal device 100 as a measurement history, together with the time of measurement and identification information for identifying the subject (the subject's name or ID). The server 200 also performs face authentication based on information indicating the subject's facial image received from the terminal device 100. The server 200 transmits the face authentication result to the terminal device 100. That is, the server 200 transmits identification information for identifying the subject specified from information related to the subject's facial image to the terminal device 100.

[0018] The input terminal 300 is a terminal device for accessing the server 200. The person making the measurement can access the server 200 via the input terminal 300.

[0019] (2. Control Unit) The control unit 1 includes a measurement processing unit 11, an authentication processing unit 12 (image processing unit), a communication control unit 13, a determination unit 14, and a display control unit 15.

[0020] The measurement processing unit 11 measures the biological information of the subject based on a captured image of the subject. Specifically, the measurement processing unit 11 measures the biological information of the subject based on an image of the subject captured by the imaging device 2 (hereinafter referred to as a captured image). The measurement processing unit 11 measures the temperature of the subject, which is biological information of the subject, based on the captured image captured by the temperature measuring device 7. The captured image may be a video of the subject or a still image of the subject.

[0021] The authentication processing unit 12 extracts a facial image for authenticating the subject from the image captured by the photographing device 2. The authentication processing unit 12 may analyze the facial image and calculate facial feature data (feature amounts used to identify a person).

[0022] The communication control unit 13 transmits the measurement results of the biometric information obtained by the measurement processing unit 11 and information about the facial image obtained by the authentication processing unit 12 to the server 200 via the communication device 5. The communication control unit 13 may transmit the facial image of the person being measured itself to the server 200 as information about the facial image, or may transmit facial feature data obtained from the facial image of the person being measured to the server 200. Hereinafter, the measurement results of the biometric information may be simply referred to as the measurement results.

[0023] The communication control unit 13 also receives an authentication result for information relating to the face image of the person being measured from the server 200. That is, the communication control unit 13 acquires identification information for identifying the person being measured identified from the information relating to the face image of the person being measured in the server 200. The communication control unit 13 further receives measurement history information relating to the measurement history of the biological information of the person being measured identified from the information indicating the face image in the server 200 from the server 200. Hereinafter, the measurement history of the biological information may be simply referred to as the measurement history.

[0024] When the communication control unit 13 receives the measurement history itself as measurement history information, the determination unit 14 determines the past measurement status of the person being measured based on the measurement history. Specifically, the determination unit 14 determines whether the person's biological information has been measured within a predetermined period based on the measurement history. Alternatively, the determination unit 14 calculates the elapsed time from the most recent measurement time in the person's measurement history to the present based on the measurement history. This makes it possible to determine the person's past measurement status based on the rule settings held by each terminal device 100.

[0025] Note that the server 200 may determine the past measurement status of the subject instead of the determination unit 14 of the terminal device 100. In this case, the communication control unit 13 receives the determination result of the past measurement status of the subject as measurement history information. This eliminates the need to provide the determination unit 14 in the terminal device 100, and can reduce the processing load on the terminal device 100.

[0026] The display control unit 15 controls the display screens displayed by the display device 4. Specifically, the display control unit 15 displays, on the display device 4, a measurement standby screen D1 indicating that authentication is in progress, a measurement in progress screen D2 indicating that measurement is in progress, and a measurement result screen D3 displaying the authentication result and the measurement result (see FIG. 2).

[0027] Furthermore, the display control unit 15 displays measurement status information indicating the subject's past measurement status superimposed on the captured image included in the measurement standby screen D1 based on the measurement history information. That is, the display control unit 15 displays the measurement status information indicating the subject's past measurement status superimposed on the captured image so that the subject can view the display screen and understand the subject's past measurement status. Specifically, the display control unit 15 displays information indicating whether the subject's biological information has been measured within a predetermined period based on the determination result of the server 200 or the determination unit 14. The display control unit 15 may also display information indicating the elapsed time from the most recent measurement time in the subject's measurement history to the present based on the determination result of the server 200 or the determination unit 14. When the display control unit 15 receives a signal from the server 200 indicating that the subject is unregistered, it may display information indicating that the subject is unregistered.

[0028] (3. Switching control modes) 2 is a diagram showing switching of the display screen of the terminal device 100. The terminal device 100 switches, as a control mode, between an authentication mode in which authentication processing is performed and a measurement mode in which measurement processing is performed. In this embodiment, the terminal device 100 performs processing based on the authentication mode by default, and transitions from the authentication mode to the measurement mode when a predetermined condition is satisfied.

[0029] <Measurement standby screen> As indicated by reference numerals 1021 and 1022 in Fig. 2, the display control unit 15 displays a measurement standby screen D1 (authentication screen) in the authentication mode. As shown in Fig. 2, the measurement standby screen D1 displays a captured image 101, a first index image 102, a second index image 103 (a predetermined index image), an instruction message 104, a staff member name 105, a staff member change button 106, and the like.

[0030] The photographed image 101 is an image photographed by the photographing device 2. The staff name 105 is the name of the logged-in operator who performs the measurement using the terminal device 100. The staff change button 106 accepts a change of operator.

[0031] The first index image 102 is an index image that allows the measurer to recognize the area where the authentication process and measurement process are performed in the captured image 101. The first index image 102 is, for example, a circular frame image.

[0032] The second index image 103 is an index image that allows the subject to recognize that the face of the subject has been detected by the authentication processing unit 12 and that authentication processing is being performed on the subject. The second index image 103 is displayed at the position of the face image of the subject in the captured image 101. The second index image 103 is, for example, a rectangular frame image that surrounds the face of the subject.

[0033] The person taking the measurement operates the terminal device 100 so that the face image of the person being measured displayed on the screen is inside the first index image 102. This causes authentication processing to be performed on the person being measured. As shown by reference numeral 1021 in FIG. 2, if the faces of multiple people being measured are shown in the first index image 102, authentication processing is performed on those multiple people being measured. The authentication processing unit 12 transmits the face images in the first index image 102 to the server 200 and receives the authentication result transmitted from the server 200.

[0034] Here, when the authentication processing unit 12 determines that a face image of a predetermined size or larger exists in the captured image 101, the terminal device 100 executes control to measure the biometric information of the person being measured corresponding to the face image. That is, when the person measuring brings the terminal device 100 close to the person being measured and the size of the face image of the specific person being measured in the captured image 101 becomes equal to or larger than a predetermined first dimension, the terminal device 100 switches the control mode from the default authentication mode to the measurement mode.

[0035] <Measurement screen> 2, when the authentication mode is switched to the measurement mode, the display control unit 15 switches the display from the measurement standby screen D1 to a measurement in progress screen D2. The measurement in progress screen D2 displays the captured image 101, a message 107 indicating that measurement is in progress, information 108 about the time remaining until the end of measurement, etc.

[0036] Furthermore, if the face image of the person being measured deviates from the range of the first index image 102 during measurement, the measurement processing unit 11 determines that the face has not been detected and stops the measurement, and the display control unit 15 switches the display from the measurement-in-progress screen D2 to the measurement standby screen D1. When the face image enters the first index image 102 on the switched measurement standby screen D1, the measurement processing unit 11 starts the measurement process again.

[0037] <Measurement result screen> 2, when the measurement processing unit 11 completes the measurement, the display control unit 15 switches the display from the measurement in progress screen D2 to a measurement result screen D3. The measurement result screen D3 displays authentication result information 111 received from the server 200, a facial image 112 of the subject, comparison result information 113 received from the server 200, measurement value information 114, a button 115 to return to the measurement screen, etc.

[0038] The authentication result information 111 contains information such as the name and ID of the person being measured if authentication is successful, and contains information informing the user that authentication was not successful if authentication is not successful. The facial image 112 of the person being measured is the facial image of the person being measured that was sent to the server 200. The measurement value information 114 is information on the measurement value obtained in this measurement. Buttons such as "To measurement screen" 115 accept an operation to display the measurement standby screen D1.

[0039] The comparison result information 113 is the result of the server 200 comparing the current measurement result with the past measurement result. In the example shown by reference numeral 1024 in Fig. 2, "Active level: similar level to previous" is displayed as the comparison result information 113, indicating that the level of the current measurement value is the same as the level of the previous measurement value. If the level of the current measurement value has changed from the level of the previous measurement value, "Active level: changed from previous level" is displayed as the comparison result information 113.

[0040] Furthermore, the authentication result information 111 on the measurement result screen D3 may display the result of identity verification, such as "Identity verified and recorded." The authentication result information 111 on the measurement result screen D3 may also display information on the date and time when the measurement operation was performed.

[0041] <Switching display screens> The display control unit 15 displays a measurement standby screen D1 by default. When the size of the facial image of a specific person being measured becomes equal to or larger than a predetermined first dimension while the measurement standby screen D1 is displayed, the display control unit 15 switches the display from the measurement standby screen D1 to a measurement in progress screen D2. When the size of the facial image of the person being measured becomes smaller than the predetermined first dimension while the measurement in progress screen D2 is displayed, or when the facial image of the person being measured deviates beyond the range of the first index image 102, the display control unit 15 switches the display from the measurement in progress screen D2 to the measurement standby screen D1. When the measurement process for the person being measured is completed while the measurement in progress screen D2 is displayed, the display control unit 15 switches the display from the measurement in progress screen D2 to the measurement result screen D3. When the measurement screen button 115 is tapped while the measurement result screen D3 is displayed, or when the device enters a sleep state and then the sleep state is released, or when the app is restarted, the display control unit 15 displays the default measurement standby screen D1.

[0042] <Switching control modes> Fig. 3 is a flowchart showing switching of the control mode of the terminal device 100. As shown in Fig. 3, the terminal device 100 executes processing based on the authentication mode by default (S1). That is, when a person is captured in a captured image, the terminal device 100 recognizes the person as the person to be measured and executes authentication processing, completing processing prior to measurement processing. Then, the terminal device 100 waits until a predetermined condition for transitioning to the measurement mode is met.

[0043] The terminal device 100 determines whether or not a person is captured in the captured image (S2). If a person is not captured in the captured image (NO in S2), the terminal device 100 continues the authentication mode. If a person is captured in the captured image (YES in S2), the terminal device 100 recognizes the person as the subject and performs authentication processing.

[0044] After performing authentication processing on the person being measured, the terminal device 100 determines whether the size of the face image of the person being measured is equal to or larger than a predetermined first dimension (S3). If the size of the face image of the person being measured is not equal to or larger than the predetermined first dimension (NO in S3), the terminal device 100 continues the authentication mode. If the size of the face image of the person being measured is equal to or larger than the predetermined first dimension (YES in S3), the terminal device 100 switches the control mode from the default authentication mode to the measurement mode (S4).

[0045] (4. Measurement standby screen details) Fig. 4 is a diagram showing an example of the captured image 101 before authentication. Fig. 5 is a diagram showing an example of the captured image 101 after authentication. Details of the captured image 101 displayed on the measurement standby screen D1 will be described below with reference to Figs. 4 and 5. Note that Figs. 4 and 5 show the captured image 101 in which the faces of multiple subjects are captured and authentication processing is performed on the multiple subjects.

[0046] During the period from when the authentication processing unit 12 detects the face of the person being measured in the captured image until the communication control unit 13 receives the measurement history information of the person being measured from the server 200, the display control unit 15 displays a second index image 103 at the position of the face image of the person being measured, as shown in Fig. 4. The second index image 103 is a frame image that surrounds the face of the person being measured detected by the authentication processing unit 12.

[0047] On the other hand, when the communication control unit 13 receives the measurement history information of the subject from the server 200, the display control unit 15 displays measurement status information 121 based on the measurement history information of the subject, in addition to the second index image 103, superimposed on the photographed image 101, as shown in Fig. 5. The display control unit 15 displays the measurement status information 121 at a position in the photographed image 101 that corresponds to the position of the face image of the subject.

[0048] 5, the display control unit 15 displays the word "Completed" on the face image of the subject whose biological information has been measured during a predetermined period. The predetermined period is, for example, the period from midnight on the current day to the present.

[0049] As a result, the person taking the measurement can see, by simply looking at the captured image 101, that the measurement process for the predetermined period has been completed for the person being measured for whom the word "Completed" is displayed. In other words, the person taking the measurement can see that the measurement process for the predetermined period has not yet been completed for the person being measured for whom the word "Completed" is not displayed. By taking photographs of multiple people being measured and performing face authentication, the person taking the photographs of multiple people being measured can identify those people for whom the measurement process for the predetermined period has not yet been completed (i.e., those people for whom measurement of biological information is required).

[0050] (5. Sequence) Fig. 6 is a diagram showing the sequence of basic operations of an authentication operation performed using the terminal device 100. As shown in Fig. 6, the subject performs the authentication operation using the terminal device 100 (S11). That is, the subject operates the terminal device 100 so that the facial image of the subject displayed on the screen is positioned inside the first index image 102. This executes the authentication process for the subject.

[0051] The terminal device 100 extracts a facial image for authenticating the subject from the captured image (S12). If multiple subjects appear in the captured image, the terminal device 100 extracts multiple facial images corresponding to the multiple subjects. The terminal device 100 transmits the extracted facial images to the server 200 (S13).

[0052] The server 200 performs face authentication based on the received face image (S14). The server 200 also determines the past measurement status of the person being measured based on the measurement history information of the person being measured identified from the received face image (S15). The server 200 transmits the face authentication result (identification information for identifying the person being measured) and the past measurement status of the person being measured to the terminal device 100 (S16).

[0053] The terminal device 100 displays measurement status information 121 superimposed on the captured image 101 based on the received past measurement status of the person being measured (S17). The terminal device 100 may also display the identification information of the person being measured (such as name, age, sex, and date of birth) superimposed on the captured image 101 based on the received face authentication result. The person measuring the person looks at the captured image 101 including the measurement status information 121 to understand the past measurement status of the person being measured (for example, whether or not biometric information has been measured within a predetermined period of time). The person measuring the person performs a measurement operation based on the past measurement status of the person being measured. In other words, the person measuring the person operates the terminal device 100 so that the size of the face image of the person whose biometric information needs to be measured in the captured image 101 is equal to or larger than a predetermined first dimension.

[0054] The terminal device 100 may analyze the facial image extracted in S12 and calculate facial feature data (feature amounts used to identify a person). The terminal device 100 may transmit the facial feature data instead of the facial image in S13. The server 200 identifies the corresponding person by comparing the transmitted facial feature data with stored facial feature data. This reduces the amount of data transmitted compared to transmitting facial image data to the server 200.

[0055] Furthermore, instead of the server 200 determining the past measurement status of the subject, the terminal device 100 may determine the past measurement status of the subject based on the measurement history information of the subject received from the server 200.

[0056] (6. Action and Effects) According to the above configuration, the display control unit 15 displays the measurement status information 121 of the subject based on the measurement history of the subject stored in the server 200, superimposed on the captured image. Therefore, the subject can understand the subject's past measurement status simply by looking at the captured image 101 displayed on the display screen of the terminal device 100. In other words, the subject does not need to access the server from a separate terminal (input terminal 300) to understand the subject's past measurement status. This reduces the burden on the subject.

[0057] In addition, since face authentication is performed by the server 200, the terminal device 100 does not need to store personal information (such as a face image) relating to the subject. Furthermore, since the terminal device 100 receives measurement history information together with the face authentication result from the server 200, it is possible to ensure the appropriate handling of personal information without impairing the convenience of being able to display the measurement status information 121 superimposed on the captured image 101.

[0058] Furthermore, it is not a problem if the captured image used in the authentication process has a lower resolution than the captured image used in the measurement process. Under these circumstances, the terminal device 100 may perform authentication processing on multiple subjects in the captured image all at once. When capturing images of multiple subjects and performing face authentication, the terminal device 100 displays measurement status information 121 corresponding to the multiple subjects superimposed on the captured image. This allows the person taking the measurement to properly identify the subjects whose biometric information needs to be measured among the multiple subjects.

[0059] 2 and 3, when the authentication processing unit 12 determines that a face image of a predetermined size or larger is present in the captured image 101, the terminal device 100 executes control to measure the biometric information of the subject corresponding to the face image. As a result, when a face image of a predetermined size or larger is present in the captured image 101, the terminal device 100 can automatically execute control related to the measurement.

[0060] 5, the display control unit 15 displays information indicating whether or not the biological information of the subject has been measured within a predetermined period as measurement status information 121. This allows the person taking the measure to quickly determine whether or not it is necessary to measure the biological information of the subject at present, based on the information indicating whether or not the biological information of the subject has been measured within the predetermined period.

[0061] (7. Variation of the measurement standby screen) 7 to 13 are diagrams showing modified examples of the captured image 101 after authentication that is displayed on the measurement standby screen D1. The display control unit 15 may display the captured image 101 shown in FIGS. 7 to 13 instead of the captured image 101 shown in FIG. 5 as the captured image 101 after authentication. In FIGS. 7 to 11, the display control unit 15 displays information indicating whether or not the biometric information of the person being measured has been measured within a predetermined period as measurement status information 121, superimposed on the captured image 101. In FIGS. 12 and 13, the display control unit 15 displays information indicating the elapsed time from the most recent measurement time in the measurement history of the person being measured to the present as measurement status information 121, superimposed on the captured image 101.

[0062] 7, when the biological information of a person being measured has been measured during a predetermined period, the display control unit 15 may display the measurement results of the biological information of the person being measured during the predetermined period (in the example shown in FIG. 7, the temperature, blood vessel information, heart rate information, etc. of the person being measured) as measurement status information 121. For example, the display control unit 15 displays the most recent measurement results of the person being measured over a facial image of the person whose biological information has been measured during the predetermined period. By looking at such a captured image 101, the person being measured can see that the measurement process for the predetermined period has been completed for the person being measured whose measurement results are displayed, and can also understand the measurement results of the person being measured during the predetermined period.

[0063] 8 to 10, when the biological information of the subject has been measured during a predetermined period, the display control unit 15 may display information related to the measurement time together with the measurement results of the subject during the predetermined period as measurement status information 121. In the example shown in FIG. 8, the display control unit 15 displays the most recent measurement date and time (e.g., the text "2024 / 6 / 3 11:03") as information related to the measurement time as measurement status information 121. In the example shown in FIG. 9, the display control unit 15 displays whether the most recent measurement date and time was morning, afternoon, or evening of the day as measurement status information 121. By looking at such a captured image 101, the subject can understand that the measurement results displayed in the captured image 101 do not represent the subject's current biological information but represent the biological information of the subject measured during the predetermined period.

[0064] 9, the display control unit 15 may change the display form of the second index image 103 depending on whether the most recent measurement date and time was in the morning, afternoon, or evening of the current day. For example, the display control unit 15 changes the color of the second index image 103 displayed on the face image of a subject whose biological information was not measured on the current day to yellow. Furthermore, the display control unit 15 changes the colors of the second index image 103 displayed on the face image of a subject whose most recent measurement date and time was in the morning, afternoon, or evening of the current day to blue, orange, and black, respectively. By looking at such a captured image 101, the subject can intuitively understand the subject's past measurement status.

[0065] 10, when measurements are performed in multiple periods of the morning, afternoon, and night on the same day, the display control unit 15 may display measurement status information 121 indicating the multiple periods for the corresponding person being measured. Furthermore, when no biological information is measured on the same day, the display control unit 15 may display measurement status information 121 indicating that no biological information is measured on the same day (the text "No measurement today") for the corresponding person being measured. When measurements need to be performed multiple times in one day, the person being measured can see that multiple measurement processes have been performed on the person being measured on the same day by looking at such captured image 101.

[0066] 11, the display control unit 15 may display the second index image 103 and the measurement status information 121 (the text "No Measurement") only on the face image of the subject whose biological information has not been measured within a predetermined period. By looking at such a captured image 101, the person measuring can quickly determine which subjects need to have their biological information measured.

[0067] 12, the display control unit 15 may display information indicating the elapsed time from the most recent measurement to the present as measurement status information 121. In the example shown in FIG. 12, the display control unit 15 displays the words "One day has passed since the previous measurement" above the face image of the person being measured for whom the elapsed time from the most recent measurement to the present is about one day. This allows the person being measured to appropriately determine whether or not it is necessary to measure the person's biological information at the present time, based on the elapsed time from the most recent measurement to the present.

[0068] 13, the display control unit 15 may indicate the elapsed time by the display format of the second index image 103. In the example shown in FIG. 13, the display control unit 15 changes the color of the second index image 103 displayed on the face image of the subject whose biological information was measured on the day to green. The display control unit 15 also changes the color of the second index image 103 displayed on the face image of the subject whose most recent measurement date and time was one to three days ago to yellow. The display control unit 15 also changes the color of the second index image 103 displayed on the face image of the subject whose most recent measurement date and time was three days or more ago to red. By looking at such a captured image 101, the subject can intuitively understand the measurement status of the subject.

[0069] The display control unit 15 may superimpose the identification information of the subject (such as name, age, sex, and date of birth) on the captured image 101 based on the face authentication result received from the server 200. For a subject who had an error in a past measurement (e.g., the most recent measurement), the display control unit 15 may display information indicating that an error occurred in the past measurement (such as the text "Error Occurred"). For a subject whose most recent measurement result was significantly different from the previous measurement results, the display control unit 15 may display information indicating that the most recent measurement result was significantly different from the previous measurement results (such as the text "Unusual Value"). For a subject whose face image is not registered (a subject not registered in the server 200), the display control unit 15 may display information indicating that the face image is not registered (such as the text "Face Not Registered").

[0070] [Embodiment 2] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0071] Fig. 14 is a diagram showing switching of a display screen of the terminal device 100 according to embodiment 2. Fig. 15 is a flowchart showing switching of a control mode of the terminal device 100 according to embodiment 2. The terminal device 100 according to embodiment 2 switches the control mode as described below with reference to Figs. 14 and 15, instead of the switching of the control mode described in embodiment 1 (see Figs. 2 and 3).

[0072] The terminal device 100 switches the control mode among an authentication mode in which authentication processing is performed, a measurement mode in which measurement processing is performed, and a standby mode in which the terminal device 100 waits until a predetermined condition for transitioning to the authentication mode or the measurement mode is met.

[0073] In this embodiment, the terminal device 100 executes processing based on the standby mode by default, and transitions from the standby mode to the authentication mode when a predetermined first condition is satisfied, and transitions from the standby mode to the measurement mode when a predetermined second condition is satisfied. The predetermined second condition is the same as the condition for transitioning from the authentication mode to the measurement mode described in the first embodiment (see FIG. 3).

[0074] <Authentication screen> 14, the display control unit 15 displays an authentication screen D11 in the authentication mode. The authentication screen D11 is the same as the measurement standby screen D1 according to the first embodiment.

[0075] <Standby screen> 14, the display control unit 15 displays a standby screen D12 in the standby mode. The standby screen D12 differs from the authentication screen D11 in that the second index image 103 is not displayed.

[0076] Here, when the authentication processing unit 12 determines that there is no face image of a predetermined size or larger in the captured image 101, or when it determines that there are multiple face images in the captured image 101, it executes control to authenticate the person being measured corresponding to the face image. That is, when the person being measured moves the terminal device 100 away from the person being measured and the size of the face image of a specific person being measured in the captured image 101 becomes smaller than a predetermined second dimension (smaller than a predetermined first dimension), the terminal device 100 switches the control mode from the default standby mode to the authentication mode. Alternatively, when the person being measured moves the terminal device 100 away from the person being measured and multiple face images enter the angle of view of the captured image 101, the terminal device 100 switches the control mode from the default standby mode to the authentication mode. When the terminal device 100 receives a predetermined button operation by the person being measured, the terminal device 100 may switch the control mode from the default standby mode to the authentication mode.

[0077] On the other hand, when the size of the face image of the specific subject in the captured image 101 becomes equal to or larger than a predetermined second dimension, or when there is only one face image in the captured image 101, the terminal device 100 returns the control mode from the authentication mode to the default standby mode. When the terminal device 100 receives a predetermined button operation by the subject, the terminal device 100 may return the control mode from the authentication mode to the default standby mode.

[0078] <Measurement screen> As indicated by reference numeral 1143 in FIG. 14, when the standby mode is switched to the measurement mode, the display control unit 15 switches the display from the standby screen D12 to the above-described measurement-in-progress screen D2.

[0079] <Measurement result screen> As indicated by reference numeral 1144 in FIG. 14, when the measurement by the measurement processing unit 11 is completed, the display control unit 15 switches the display from the measurement-in-progress screen D2 to the measurement result screen D3 described above.

[0080] <Switching control modes> Fig. 15 is a flowchart showing switching of the control mode of the terminal device 100 according to embodiment 2. As shown in Fig. 15, the terminal device 100 executes processing based on the standby mode by default (S21). That is, the terminal device 100 waits until a predetermined condition for transitioning to the authentication mode or the measurement mode is met.

[0081] The terminal device 100 determines whether or not a predetermined button operation by the subject has been detected (S22). If a predetermined button operation has been detected (YES in S22), the terminal device 100 switches the control mode from the default standby mode to the authentication mode (S26). If a predetermined button operation has not been detected (NO in S22), the process proceeds to S23.

[0082] The terminal device 100 determines whether or not a person is captured in the captured image (S23). If no person is captured in the captured image (NO in S23), the terminal device 100 continues in the standby mode.

[0083] If a person is captured in the captured image (YES in S23), the terminal device 100 recognizes the person as the person being measured. Then, the terminal device 100 determines whether the size of the face image of the person being measured is less than a predetermined second dimension (S24). If the size of the face image of the person being measured is less than the predetermined second dimension (YES in S24), the terminal device 100 switches the control mode from the default standby mode to the authentication mode (S26).

[0084] If the size of the face image of the subject is equal to or larger than the predetermined second dimension (NO in S24), the terminal device 100 determines whether or not there are multiple face images (or a predetermined number or more of face images) in the captured image 101 (S25). If there are multiple face images in the captured image 101 (YES in S25), the terminal device 100 switches the control mode from the default standby mode to the authentication mode (S26). If there are not multiple face images in the captured image 101 (NO in S25), the terminal device 100 continues in the standby mode.

[0085] That is, when any of the conditions S22, S24, and S25 is satisfied, the terminal device 100 switches the control mode from the default standby mode to the authentication mode. Note that the order in which the determinations for the conditions S24 and S25 are made may be reversed. Furthermore, the terminal device 100 only needs to make a determination for at least one of the conditions S22, S24, and S25.

[0086] [Embodiment 3] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0087] 16 is a diagram showing a sequence of basic operations of an authentication operation performed using the terminal device 100 according to embodiment 3. The authentication operation according to embodiment 3 differs from the authentication operation described in embodiment 1 (see FIG. 6) in that face authentication from the second time onwards is performed by the terminal device 100.

[0088] As shown in Fig. 16, in the sequence of the authentication operation according to the third embodiment, steps S11 to S17 are executed in the same manner as in the authentication operation according to the first embodiment shown in Fig. 6. The terminal device 100 displays the measurement status information 121 superimposed on the captured image 101, and then stores the face image captured in S11 and information that face authentication has been performed by the server 200 using the face image in the storage device 6 (S31). Here, the terminal device 100 stores the recognition information of the person being measured corresponding to the face image and the measurement status received from the server 200, along with the face image, in the storage device 6. The face image and the like stored in the storage device 6 are deleted when the device logs out or enters sleep mode.

[0089] Next, consider a situation in which the subject continues to perform authentication operations using the terminal device 100 without logging out or putting the terminal device 100 into sleep mode (S32). In this case, the terminal device 100 does not transmit the captured face image to the server 200, but instead performs simple face authentication based on the face image stored in the storage device 6 (S33). That is, the terminal device 100 extracts a face image for authenticating the subject from the captured image and compares the extracted face image with the face image stored in the storage device 6. If it is determined that the extracted face image matches the face image stored in the storage device 6, the terminal device 100 displays measurement status information 121 superimposed on the captured image 101 based on the measurement status corresponding to the extracted face image (S34).

[0090] According to the above configuration, when the subject performs authentication operations consecutively, the terminal device 100 does not need to send a facial image to the server 200 each time, thereby reducing the number of communications between the terminal device 100 and the server 200. Therefore, the processing load on the terminal device 100 and the server 200 can be reduced.

[0091] The terminal device 100 may or may not send the face authentication result of the face authentication performed in S33 to the server 200. Furthermore, the user may set in the terminal device 100 whether or not to send the face authentication result to the server 200.

[0092] [Embodiment 4] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0093] Rules relating to the authentication process and measurement process of the terminal device 100, such as which of the captured images 101 shown in FIGS. 5 and 7 to 13 should be displayed, may be set as follows.

[0094] The terminal device 100 displays a rule setting screen for determining the rule setting. The measurer inputs an operation to determine the rule setting to the terminal device 100 on the rule setting screen. Here, if the rule settings of the terminal device 100 are managed by the server 200, the terminal device 100 transmits the rule setting determined by the measurer to the server 200. The server 200 updates the rule setting based on the received rule setting. The server 200 transmits a notification to the terminal device 100 that the rule setting has been updated. The terminal device 100 executes authentication processing and measurement processing based on the updated rule setting. Furthermore, the terminal device 100 displays the updated rule setting on the rule setting screen.

[0095] The terminal device 100 may store the rule settings. In this case, the terminal device 100 updates the rule settings based on the rule settings determined by the subject. The subject may also input an operation to determine the rule settings to the server 200 via the input terminal 300. In other words, the subject may input an operation to determine the rule settings directly to the server 200 without going through the terminal device 100. The server 200 may also automatically change the rule settings based on the measurement history. For example, a rule may be set so that an alert is displayed on the terminal device 100 when there are no measurement results for a specific subject within a predetermined period.

[0096] [Software implementation example] The functions of the terminal device 100 (hereinafter referred to as the "device") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (particularly each part included in the control unit 1).

[0097] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0098] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0099] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0100] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0101] 〔summary〕 The terminal device according to aspect 1 of the present invention comprises an image processing unit that extracts a facial image for authenticating the person being measured from an image captured by an imaging device, a communication control unit that transmits information indicating the facial image to a server and receives from the server measurement history information relating to the measurement history of the person being measured's biometric information identified from the information indicating the facial image, and a display control unit that displays measurement status information indicating the person's past measurement status, superimposed on the captured image, based on the measurement history information.

[0102] According to the above configuration, the display control unit displays the measurement status information of the subject based on the subject's measurement history stored in the server by superimposing it on the captured image. Therefore, the subject can understand the subject's past measurement status simply by looking at the captured image displayed on the display screen of the terminal device. In other words, the subject does not need to access the server from a separate terminal to understand the subject's past measurement status. This reduces the burden on the subject.

[0103] Furthermore, since the terminal device receives measurement history information from the server, the convenience of being able to display measurement status information superimposed on the captured image is not compromised, and proper handling of personal information can be ensured.

[0104] Furthermore, it is not a problem if the captured image used in the authentication process has a lower resolution than the captured image used in the measurement process. Under these circumstances, the terminal device may perform authentication processing on multiple subjects in the captured image all at once. When capturing images of multiple subjects and performing face authentication, the terminal device displays measurement status information corresponding to the multiple subjects superimposed on the captured image. This allows the person making the measurement to properly identify the subjects among the multiple subjects whose biometric information needs to be measured.

[0105] In the terminal device according to aspect 2 of the present invention, in the above aspect 1, the communication control unit may further include a determination unit that receives a measurement history of the subject's biological information from the server as the measurement history information, and determines the subject's past measurement status based on the measurement history.

[0106] According to the above configuration, the past measurement conditions of the subject can be determined based on the rule settings held by each terminal device.

[0107] In the terminal device according to aspect 3 of the present invention, in the above aspect 1, the communication control unit may receive, from the server, a determination result of a past measurement situation of the subject as the measurement history information.

[0108] According to the above configuration, the terminal device 100 does not need to determine the past measurement conditions of the subject, and the processing load on the terminal device 100 can be reduced.

[0109] In the terminal device according to aspect 4 of the present invention, in the above aspects 1 to 3, the display control unit may display the measurement status information at a position corresponding to the position of the face image of the person being measured in the captured image.

[0110] According to the above configuration, the display control unit displays the measurement status information of the person being measured in association with the face image of the person being measured. Therefore, even if multiple people are included in the captured image, the person being measured can properly understand the measurement status information corresponding to the multiple people being measured.

[0111] In a terminal device according to aspect 5 of the present invention, in aspects 1 to 4 above, the display control unit may display information indicating whether or not the subject's biometric information has been measured within a predetermined period as the measurement status information.

[0112] According to the above configuration, the person being measured can quickly determine whether or not it is currently necessary to measure the subject's biological information based on information indicating whether or not the subject's biological information has been measured within a specified period of time.

[0113] In a terminal device according to aspect 6 of the present invention, in aspect 5 above, the display control unit may display the measurement results of the biometric information of the subject during the specified period if the biometric information of the subject has been measured during the specified period.

[0114] According to the above configuration, the measurer can determine that the biological information of the person being measured, whose measurement values ​​are being displayed, has been measured during a predetermined period.

[0115] In a terminal device according to aspect 7 of the present invention, in aspect 6 above, the display control unit may, if the biometric information of the subject has been measured during the specified period, display information regarding the measurement time at which the biometric information was measured together with the measurement results of the biometric information of the subject.

[0116] According to the above configuration, the person being measured can understand that the measurement results displayed in the captured image do not represent the current biometric information of the person being measured, but rather represent the biometric information of the person being measured over a specified period of time.

[0117] In a terminal device according to aspect 8 of the present invention, in aspects 1 to 4 above, the display control unit may display information indicating the elapsed time from the most recent measurement time in the measurement history of the subject to the present as the measurement status information.

[0118] According to the above configuration, the measurer can appropriately determine whether or not it is necessary to measure the biological information of the subject at present, based on the time elapsed from the most recent measurement time to the present.

[0119] In the terminal device according to aspect 9 of the present invention, in aspect 8 above, the display control unit may display a predetermined index image at the position of the facial image of the subject in the captured image, and the information indicating the elapsed time may be displayed in the display format of the index image.

[0120] According to the above configuration, the person being measured can intuitively grasp the measurement status of the person being measured by looking at the index image displayed at the position of the face image of the person being measured.

[0121] In the terminal device according to aspect 10 of the present invention, in aspects 1 to 9 above, when the image processing unit determines that a facial image of a predetermined size or larger is present in the captured image, control may be executed to measure the biometric information of the subject corresponding to the facial image.

[0122] According to the above configuration, when a face image of a predetermined size or larger is present in the captured image, the terminal device can automatically execute control relating to the measurement.

[0123] In the terminal device according to aspect 11 of the present invention, in the above aspects 1 to 9, when the image processing unit determines that there is no facial image of a predetermined size or larger in the captured image, or when it determines that there are multiple facial images in the captured image, it may execute control to authenticate the person being measured corresponding to the facial image.

[0124] According to the above configuration, if there is no face image of a predetermined size or larger, or if there are multiple face images, the terminal device can automatically execute control related to authentication.

[0125] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment. [Explanation of symbols]

[0126] 1. Control section 2. Imaging equipment 4 Display device 5. Communications equipment 11 Measurement processing section 12 Authentication processing section (image processing section) 13 Communication control section 14 Judgment section 15 Display control unit 100 Terminal Device 101 images 103 Second index image (predetermined index image) 121 Measurement status information 200 servers 300 Input terminal

Claims

1. an image processing unit that extracts a facial image for authenticating the subject from the image captured by the imaging device; a communication control unit that transmits information indicating the facial image to a server and receives, from the server, measurement history information regarding a measurement history of the subject's biological information identified from the information indicating the facial image; and a display control unit that displays measurement status information indicating the subject's past measurement status superimposed on the captured image based on the measurement history information.

2. the communication control unit receives a measurement history of the biological information of the subject from the server as the measurement history information; The terminal device according to claim 1 , further comprising a determination unit that determines past measurement conditions of the subject based on the measurement history.

3. The terminal device according to claim 1 , wherein the communication control unit receives, as the measurement history information, a determination result of a past measurement situation of the subject from the server.

4. The terminal device according to claim 1 , wherein the display control unit displays the measurement status information at a position corresponding to a position of a face image of the subject in the captured image.

5. The terminal device according to claim 1 , wherein the display control unit displays, as the measurement status information, information indicating whether or not the biological information of the subject has been measured within a predetermined period.

6. The terminal device according to claim 5, wherein the display control unit displays the measurement results of the subject's biological information during the predetermined period if the subject's biological information has been measured during the predetermined period.

7. The terminal device according to claim 6, wherein the display control unit, when measuring the biometric information of the subject during the specified period, displays information regarding the measurement time when the biometric information was measured together with the measurement results of the biometric information of the subject.

8. The terminal device according to claim 1 , wherein the display control unit displays, as the measurement status information, information indicating the elapsed time from the most recent measurement time in the measurement history of the subject to the present time.

9. 9. The terminal device according to claim 8, wherein the display control unit displays a predetermined index image at a position of a face image of the subject in the captured image, and represents the information indicating the elapsed time by a display form of the index image.

10. A terminal device according to any one of claims 1 to 4, wherein when the image processing unit determines that a facial image of a predetermined size or larger exists in the captured image, control is executed to measure the biometric information of the subject corresponding to the facial image.

11. A terminal device described in any one of claims 1 to 4, wherein when the image processing unit determines that there is no facial image of a predetermined size or larger in the captured image, or when it determines that there are multiple facial images in the captured image, it executes control to authenticate the person being measured corresponding to the facial image.

Images