Biological information processing devices and methods

Abstract

The biological information processing device includes: a display instruction unit that generates an instruction signal for causing a display unit to display specified information in a specified display format; a biological information measurement unit that measures the biological information of the user based on sensor information indicating a change in the state of the user's biological body acquired by a sensor unit after the display instruction unit generates the instruction signal; a relative speed calculation unit that calculates the relative speed between the display unit and the user during a period in which the biological information measurement unit measures the biological information; and a judgment unit that judges whether the relative speed calculated by the relative speed calculation unit is within a specified range.

Description

Technical Field

[0001] One aspect of the present disclosure relates to a biological information processing apparatus and a biological information processing method. Background Art

[0002] Japanese Patent Application Laid-Open No. 2017-93760 discloses a measuring device for measuring the pulse rate of a subject based on a camera image of the subject captured by a camera.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2017-93760 Summary of the invention

[0006] According to the measurement device of Japanese Patent Publication No. 2017-93760, there is a situation where the subject moves his face quickly, etc., and the subject moves at a relatively faster speed relative to the camera photographing the subject. In this case, the accuracy of the pulse rate measured based on the camera image obtained by photographing the subject with the camera becomes low. One aspect of the present disclosure is to obtain the user's biological information with high accuracy by suppressing the speed of the user's movement.

[0007] One aspect of the present disclosure relates to a living body information processing device including: a display instruction unit that generates an instruction signal for causing a display unit to display specified information in a specified display format; a living body information measuring unit that measures the living body information of the user based on sensor information indicating a change in a state of the living body of the user acquired by a sensor unit after the display instruction unit generates the instruction signal; a relative speed calculation unit that calculates a relative speed between the display unit and the user during a period in which the living body information measurement unit measures the living body information; and a judgment unit that judges whether the relative speed calculated by the relative speed calculation unit is within a specified range.

[0008] One aspect of the present disclosure relates to a biological information processing method including: a step of generating an indication signal, wherein the indication signal is used to cause a display unit to display specified information in a specified display format; a step of measuring the biological information of the user based on sensor information of a biological state change of the user acquired by a sensor unit after the indication signal is generated; a step of calculating a relative speed between the sensor unit and the user during the period of measuring the biological information; and a step of determining whether the relative speed is within a specified range. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1This is a front view showing the portable terminal according to the first embodiment.

[0010] Figure 2 This is a functional block diagram of the biological information processing system according to the first embodiment.

[0011] Figure 3 The diagram shows a display image of a health management application that is displayed on a display unit by a display instruction unit of the portable terminal according to the first embodiment.

[0012] Figure 4 This is a diagram showing an example of a moving image captured by the imaging unit of the first embodiment, immediately before the user moves.

[0013] Figure 5 This is a diagram showing an example of a moving image captured by the imaging unit according to the first embodiment after the user has moved.

[0014] Figure 6 This is a diagram showing an example of the waveform of the pulse wave measured by the biological information measurement unit based on the moving image captured by the imaging unit of the first embodiment.

[0015] Figure 7 It is a diagram showing a change state of biological information monitored by the change amount monitoring unit according to the first embodiment.

[0016] Figure 8 This is a flowchart showing an example of the operation of the control unit of the portable terminal according to the first embodiment.

[0017] Fig. 9 This is a functional block diagram of a biological information processing system including a portable terminal according to a second embodiment.

[0018] Fig.10 This is a flowchart showing an example of the operation of the control unit of the portable terminal according to the second embodiment. DETAILED DESCRIPTION

[0019] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In addition, in the drawings, the same or equivalent elements are given the same reference numerals, and repeated descriptions are omitted.

[0020] [First embodiment]

[0021] Figure 1 1 is a front view of the portable terminal 1 according to the first embodiment. Figure 1 2 shows a top view of the portable terminal 1. The portable terminal 1 is an example of a biological information processing device that measures biological information of a person in a contactless manner.

[0022] Biometric information refers to various information related to a biological body that can be measured from a person's body surface, and can include, for example, pulse wave, blood pressure value, pulse rate, stress level, heart rate, body temperature, respiratory rate, blood oxygen saturation, and information indicating the health status of blood vessels. The portable terminal 1 obtains information including at least one of blood pressure value, pulse rate, heart rate, respiratory rate, body temperature, and blood oxygen saturation as biometric information based on information (sensor information) indicating changes in the state of the body surface, such as a dynamic image reflecting the body surface of the user (person).

[0023] An example of information indicating changes in the state of the body surface is a dynamic image reflecting the body surface. The dynamic image reflecting the body surface shows changes in the color of the body surface based on the flow of the user's blood, changes in the body surface temperature, and changes in the movement of the body surface accompanied by breathing, etc. as changes in the state of the body surface in each frame. In addition, as an example, the information (sensor information) indicating changes in the state of the body surface is described below as information mainly reflecting a dynamic image of the body surface, but the information (sensor information) indicating changes in the state of the body surface may also be, for example, information indicating body movements based on the user's heartbeat obtained by a Doppler sensor, etc., and is not limited to dynamic images reflecting the body surface.

[0024] In this embodiment, for example, when the portable terminal 1 receives an instruction from the user to start a specific application (software), the portable terminal 1 starts the specific application to display images such as dynamic images and still images containing specified information in a specified display format described later. In addition, for example, when the portable terminal 1 starts the specific application, it simultaneously starts measuring the user's biological information.

[0025] The specific application is, for example, an application including information on the content that the user is interested in. The information on the content that the user is interested in is, for example, information for managing daily changes in the user's biological information and information related to daily news.

[0026] Thus, when measuring the biological information of the user, the portable terminal 1 suppresses the relative speed between the portable terminal 1 and the user by displaying images such as dynamic images and still images containing prescribed information in a prescribed display format. Thus, the measurement accuracy of the biological information of the user can be improved. In addition, the detailed description will be given later.

[0027] In this embodiment, the portable terminal 1 is described by taking a smartphone as an example. The portable terminal 1 has a size and weight that can be carried by a person. The portable terminal 1 is, for example, a size that can be held in one hand. In addition, the portable terminal 1 is not limited to a smartphone, and may also be, for example, a tablet computer, a laptop computer, etc.

[0028] The portable terminal 1 includes, for example, a housing 2, a display unit 3, an input unit 4, a camera (sensor unit) 5, a speaker (sound output unit) 6, and a microphone 7. In this embodiment, the display unit 3, the input unit 4, the camera 5, the speaker 6, and the microphone 7 are provided in the same housing 2, for example.

[0029] The display unit 3 is a display for displaying images such as moving images and still images. The display unit 3 has a display area as an area for displaying images such as moving images and still images. The display unit 3 can be a display such as a liquid crystal display, an organic EL (electro-luminescence) display, etc. that can display images such as moving images and still images.

[0030] The input unit 4 is an input device that accepts instructions, i.e., inputs, from a user and generates an input signal based on the accepted inputs. The input unit 4 is, for example, a transparent touch sensor, and is arranged to overlap with the display area of ​​the display unit 3. That is, for example, the display unit 3 and the input unit 4 are integrated to form a touch panel. Then, for example, the contact of a user's finger or pen is detected, and an input signal corresponding to the detected position is generated. In addition, the input unit 4 is not limited to a touch sensor, and may be, for example, a mechanical switch that accepts inputs from a user by being pressed by the user, a sensor that senses the distance or movement of a user's finger or pen, etc., and accepts inputs in a non-contact manner, etc.

[0031] The photographing unit 5 is an example of a sensor unit that obtains information (sensor information) indicating a change in the state of the user's biological body in a non-contact manner. The photographing unit 5 is a camera (photographing device) for photographing images such as dynamic images and static images. The photographing unit 5 photographs dynamic images at a predetermined frame rate (e.g., 10 to 300 fps (frame per second)) to thereby obtain sensor information. For example, the photographing unit 5 photographs a dynamic image of the user's body surface in order to measure the user's biological information. In addition, for example, the photographing unit 5 also photographs a dynamic image of the user's body surface in order to measure the relative speed between the portable terminal 1 and the user.

[0032] In addition, the sensor unit involved in the present disclosure is not limited to the shooting unit 5 for shooting dynamic images, and may be various sensors such as Doppler sensors such as microwave radars or millimeter wave radars that can obtain information (sensor information) indicating changes in the state of the user's biological body in a non-contact manner. For example, even if a Doppler sensor is used, sensor information indicating changes in the state of the user's biological body, such as sensor information indicating changes in the user's body movement based on the user's heartbeat, can be obtained.

[0033] In addition, the sensor unit according to the present disclosure may be a combination of a plurality of sensors such as the imaging unit 5 and a Doppler sensor, and sensor information may be obtained by a combination of the plurality of sensors.

[0034] The sensor unit according to the present disclosure obtains sensor information in order to measure biological information of the user. In addition, for example, the sensor unit according to the present disclosure also obtains sensor information in order to measure the relative speed between the portable terminal 1 and the user.

[0035] The speaker 6 is a sound output device that generates sound based on a sound signal. The microphone 7 is a sound input device that receives sound from outside the portable terminal 1, such as a user's voice, as input and converts it into a sound signal.

[0036] Figure 2 1 is a functional block diagram of a biological information processing system 100 including the portable terminal 1 according to the first embodiment. Figure 2 As shown, the biological information processing system 100 includes, for example, a portable terminal 1 and a server 200. For example, the portable terminal 1 is configured to be able to communicate with the server 200 by wire or wireless. In this embodiment, the portable terminal 1 is configured to be able to communicate with the server 200 by wireless via the Internet 300. The portable terminal 1 is provided in a housing 2 (see Figure 1 ) also includes a control unit 10, a storage unit 30 and a communication unit 40 as a computer.

[0037] The server 200 includes a communication unit 210, a control unit 220, and a storage unit 230. The communication unit 210 in the server 200 and the communication unit 40 in the portable terminal 1 are configured to communicate via the Internet 300, for example. The control unit 220 is configured by a processor such as a CPU (Central Processing Unit) and centrally controls various functions of the server 200. The storage unit 230 can be implemented by a hard disk, an SSD (Solid State Drive), or a semiconductor memory, for example.

[0038] The control unit 10 includes a display instruction unit 11, a relative speed calculation unit 12, a biological information measurement unit 13, a timing unit 14, a determination unit 15, and a variation monitoring unit 16. The control unit 10 is composed of a processor such as a CPU (Central Processing Unit) and controls various functions of the portable terminal 1 in a unified manner.

[0039] The storage unit 30 is a storage medium that can store any information. The storage unit 30 can be implemented by, for example, a hard disk, an SSD (Solid State Drive) or a semiconductor memory. As a part of the portable terminal 1, the storage unit 30 can be a storage medium fixed inside the portable terminal 1, or a storage medium that can be taken out of the portable terminal 1 such as a SIM card. The storage unit 30 stores a biological information processing program that is pre-installed or installed from a server (for example, the server 200) that can communicate with the portable terminal 1. The biological information processing program enables the control unit 10 as a computer to function as a display indication unit 11, a relative speed calculation unit 12, a biological information measurement unit 13, a judgment unit 15, a timing unit 14, and a variation monitoring unit 16.

[0040] In addition, for example, a specific program for starting a specific application that is preinstalled or installed from a server (e.g., server 200) that can communicate with the portable terminal 1 and includes images such as moving images and still images of predetermined information in a predetermined display format as information is stored in the storage unit 30. Then, for example, when an input signal indicating the start of a specific application is received from the input unit 4 as a user's instruction, the control unit 10 refers to the storage unit 30 and executes the specific program while executing the biological information processing program.

[0041] In the present embodiment, the control unit 10 is provided in the portable terminal 1. However, the control unit 10 may be provided not in the portable terminal 1 but in a server (for example, the server 200) communicably connected to the portable terminal 1 by wire or wireless.

[0042] When receiving an input signal from the input unit 4 as an instruction from the user, the display instruction unit 11 causes the display unit 3 to display various images such as dynamic images and still images based on the input signal. For example, the display instruction unit 11 generates an instruction signal based on the input signal from the input unit 4, and the instruction signal is used to cause the display unit 3 to display specified information in a specified display format. Then, the display instruction unit 11 outputs the generated instruction signal to the display unit 3. As a result, the display instruction unit 11 causes the display unit 3 to display an image containing the specified information in a specified display format.

[0043] The prescribed display format refers to a format that includes small-sized characters, increases the number of characters, and is difficult to recognize if the user does not keep looking at the image displayed on the display unit 3 to a certain extent (hereinafter referred to as " Figure 3 In this way, the display instruction unit 11 can suppress the relative speed between the user and the portable terminal 1 by causing the display unit 3 to display an image including predetermined information in a predetermined display format.

[0044] In addition, when an instruction signal for causing the display unit 3 to display the specified information in a specified display format is generated, the display instruction unit 11 outputs a shooting instruction signal for starting the shooting of a dynamic image (i.e., a sensor information acquisition instruction signal for starting to acquire sensor information) to the shooting unit 5. As a result, the shooting unit 5 starts shooting a dynamic image including the user's body surface at a predetermined frame rate (starts to acquire sensor information). Then, the shooting unit 5 outputs the captured dynamic image as a dynamic image signal to the relative speed calculation unit 12 and the biological information measurement unit 13.

[0045] When the display instruction unit 11 generates an instruction signal for causing the display unit 3 to display prescribed information in a prescribed display format, the relative speed calculation unit 12 calculates the relative speed between the camera unit 5 and the user based on the dynamic image (sensor information) acquired from the camera unit 5. For example, the relative speed calculation unit 12 calculates the relative speed between the camera unit 5 and the user by performing image processing on the dynamic image acquired from the camera unit 5. Specifically, for example, the relative speed calculation unit 12 determines the feature points (e.g., eyes, nose, or mouth) of the user's face contained in the dynamic image acquired from the camera unit 5, and calculates the direction and amount of movement of the determined feature points in each frame of the dynamic image, thereby calculating the relative speed between the camera unit 5 and the user (hereinafter referred to as "relative speed"). Figure 4 as well as Figure 5 for explanation).

[0046] Then, the relative speed calculation unit 12 outputs information indicating the calculated relative speed between the imaging unit 5 and the user to the determination unit 15 .

[0047] When the display instruction unit 11 generates an instruction signal for causing the display unit 3 to display the prescribed information in a prescribed display format, the biological information measuring unit 13 measures the biological information of the user based on the dynamic image (sensor information) acquired from the imaging unit 5. For example, the biological information measuring unit 13 measures the biological information of the user by performing image processing on the dynamic image acquired from the imaging unit 5. Specifically, for example, the biological information measuring unit 13 determines a local area of ​​the user's face (for example, a local area of ​​the forehead, cheek, or chin) included in the dynamic image acquired from the imaging unit 5, and measures the pulse wave (hereinafter referred to as "pulse wave") based on the state change of each frame in the dynamic image in the determined local area. Figures 4 to 6 for explanation).

[0048] Then, the biological information measuring unit 13 measures, for example, blood pressure value, pulse rate, stress level, heart rate, blood oxygen saturation, and information indicating the health status of blood vessels based on the measured pulse wave, thereby obtaining biological information. Then, the biological information measuring unit 13 outputs information indicating the biological information of the user to the determining unit 15.

[0049] Here, as described above, the biological information measuring unit 13 measures the biological information of the user by performing image processing on, for example, a dynamic image acquired from the imaging unit 5. Therefore, when the user moves the face, for example, quickly relative to the imaging unit 5 that is imaging the user, the accuracy of the biological information of the user measured based on the dynamic image captured by the imaging unit 5 may be reduced due to image shaking of the user's face captured by the imaging unit 5.

[0050] Therefore, the biological information measuring unit 13 measures the biological information of the user during a period in which the relative speed calculating unit 12 calculates the relative speed between the imaging unit 5 and the user based on the dynamic image acquired from the imaging unit 5. In other words, the relative speed calculating unit 12 measures the relative speed between the user and the imaging unit 5 during a period in which the biological information measuring unit 13 calculates the biological information of the user based on the dynamic image acquired from the imaging unit 5.

[0051] Thus, it is possible to determine whether the accuracy of the biological information measured by the biological information measurement unit 13 is high or low based on the relative speed calculated by the relative speed calculation unit 12. That is, when the relative speed calculated by the relative speed calculation unit 12 based on the dynamic image acquired from the imaging unit 5 is slow, it can be determined that the accuracy of the biological information measured by the biological information measurement unit 13 based on the dynamic image at the same time (in other words, the frame at the same time) is high. On the other hand, when the relative speed calculated by the relative speed calculation unit 12 based on the dynamic image acquired from the imaging unit 5 is fast, it can be determined that the accuracy of the biological information measured by the biological information measurement unit 13 based on the dynamic image at the same time (in other words, the frame at the same time) is low.

[0052] In addition, the relative speed calculation unit 12 may calculate the relative speed at each time interval of the input of the dynamic image signal from the shooting unit 5, at each time interval of multiple inputs of dynamic image signals, or only once during the period when the biological information measurement unit 13 measures the biological information.

[0053] In addition, the timing at which the relative speed calculation unit 12 starts the calculation of the relative speed and the timing at which the biological information measurement unit 13 starts the measurement of the biological information may be performed first.

[0054] In addition, when the sensor unit involved in the present invention is not the shooting unit 5 for shooting dynamic images, but various sensors such as a Doppler sensor that can obtain sensor information representing changes in the state of the user's biological body in a non-contact manner, based on the sensor information, the relative speed calculation unit 12 calculates the relative speed, and the biological information measurement unit 13 measures the biological information.

[0055] In addition, when the sensor unit involved in the present invention is composed of a combination of multiple sensors such as the imaging unit 5 and a Doppler sensor, the relative speed calculation unit 12 can calculate the relative speed based on the sensor information obtained by the combination of the multiple sensors, and the biological information measurement unit 13 can also measure the biological information.

[0056] The timekeeping unit 14 generates time information indicating the current time, and sequentially outputs the time information to the determination unit 15. The determination unit 15 sequentially acquires the time information indicating the current time from the timekeeping unit 14.

[0057] In addition, the determination unit 15 determines whether the relative speed calculated by the relative speed calculation unit 12 is below a predetermined range. If the relative speed calculated by the relative speed calculation unit 12 is below a predetermined range, it can be determined that the measurement accuracy of the biological information at this time is high. Therefore, the determination unit 15 associates the time information when the biological information measurement unit 13 acquires the biological information with the biological information acquired from the biological information measurement unit 13 as the time information indicating the time when the biological information measurement unit 13 measures the biological information. Then, the determination unit 15 stores the biological information associated with the time information in the storage unit 230 of the server 200 via the communication unit 40. In addition, the determination unit 15 outputs the biological information associated with the time information to the variation monitoring unit 16.

[0058] In addition, if the relative speed calculated by the relative speed calculation unit 12 exceeds the specified range, it can be determined that the measurement accuracy of the biological information at this time is low. Therefore, for example, the judgment unit 15 does not store the biological information acquired from the biological information measurement unit 13 in the storage unit 230 of the server 200 and deletes it.

[0059] Furthermore, the determination unit 15 may store the biometric information associated with the time information in the storage unit 30 by outputting the biometric information to the storage unit 30 of the portable terminal 1 instead of the storage unit 230 of the server 200 .

[0060] The variation monitoring unit 16 acquires the biometric information associated with the time information via the determination unit 15 from the storage unit 230 of the server 200 via the communication unit 40, and monitors whether the variation of the acquired biometric information is within a prescribed range. For example, if the variation of each moment of the biometric information sequentially associated with the time information exceeds a prescribed range, the variation monitoring unit 16 performs, for example, a notification process.

[0061] That is, the variation monitoring unit 16 outputs notification information for notifying the user to the display instruction unit 11. As a result, the display instruction unit 11 causes the display unit 3 to display an image indicating that the variation of the biological information exceeds the prescribed range. As a result, the user can be visually notified that the variation of the biological information exceeds the prescribed range. Alternatively, the variation monitoring unit 16 may generate notification information including sound information, and notify the user of the situation that the variation of the biological information exceeds the prescribed range as sound via the speaker 6.

[0062] For example, the variation monitoring unit 16 may monitor the variation of the biometric information at each moment in the same day, or monitor the variation of the biometric information at the same moment in other days. If the variation of the biometric information is small, it can be considered that the health status of the user has not changed. On the other hand, if the variation is large and exceeds the prescribed range, it is considered that the health status of the user has changed a little. In this way, when the variation exceeds the prescribed range, the variation monitoring unit 16 performs notification processing, so that the user can start investigating the cause.

[0063] In addition, the variation monitoring unit 16 may not obtain the biometric information associated with the time information from the storage unit 230 of the server 200, but may obtain it from the judgment unit 15 in sequence, or may obtain it from the storage unit 30. In addition, the variation of the biometric information in the past time band may be monitored by reproducing the dynamic image captured by the capturing unit 5 and stored in the storage unit 230. In this case, when the content of the health management application described later is updated at the moment after the time band in which the dynamic image is captured, the biometric information, the relative speed, and the variation of the biometric information can be calculated based on the updated reference.

[0064] use Figure 3 , an example of a display image of a specific application displayed on the display unit 3 by the display instruction unit 11 is described. Figure 3 The figure shows a display image of the health management application displayed on the display unit 3 by the display instruction unit 11 of the portable terminal 1. The health management application is an example of a specific application including predetermined information in a predetermined display format displayed on the display unit 3 by the display instruction unit 11 of the control unit 10.

[0065] like Figure 3As shown, in the display image 60 of the biological information management application, for example, a plurality of columns 61 are provided as prescribed information of a prescribed display format, and the plurality of columns 61 are displayed in a format that is difficult for the user to recognize without keeping a certain degree of viewing of the display image 60. In each column 61, content that the user is interested in, such as daily news and information related to health, is displayed so that the user starts the health management application every day.

[0066] Here, in the browser of a smartphone or PC (personal computer), a font size of 16px (12pt) is used as a font size that is easy to read. In addition, in this browser, 1px (0.75pt) is used as the smallest font size.

[0067] Therefore, as the prescribed information of the prescribed display format, it is preferred that each column 61 displayed on the display image 60 displays characters with a font size of 1 px or more and 16 px or less. In other words, as the prescribed information of the prescribed display format, it is preferred that each column 61 displayed on the display image 60 displays characters with a font size of 0.75 pt or more and 12 pt or less.

[0068] Thus, when the user reads the characters included in each column 61 , it is difficult to recognize the characters without keeping looking at the display image 60 , so the relative speed between the user and the imaging unit 5 can be suppressed.

[0069] In addition, when a plurality of lines of character strings are displayed as images in each column 61 of the display image 60, it is preferred that the line spacing between characters in the plurality of lines is 1 em or more and 1.5 em or less as the prescribed information of the prescribed display format. Thus, since the spacing between characters in the plurality of lines is relatively narrow, it is difficult for the user to recognize the characters in each line contained in each column 61 without keeping looking at the display image 60. Therefore, the relative speed between the user and the imaging unit 5 can be suppressed.

[0070] use Figure 4 as well as Figure 5 , an example of a case where the relative speed calculation unit 12 calculates the relative speed between the user and the imaging unit 5 is described. Figure 4 1 is a diagram showing an example of a moving image captured by the imaging unit 5 immediately before the user moves. Figure 5 1 is a diagram showing an example of a moving image captured by the imaging unit 5 after the user has moved.

[0071] like Figure 4As shown, the relative speed calculation unit 12 identifies a predetermined feature point in the face U of the user shown in the dynamic image 63 acquired from the imaging unit 5, and defines a region R1 in the dynamic image 63 including the identified feature point. Figure 4 As an example, the relative speed calculation unit 12 specifies the nose as a preset feature point in the face U of the user included in the moving image 63 , and defines a region R1 in the moving image 63 including the specified nose.

[0072] Then, if Figure 5 As shown in , in the dynamic image 63 , when the user's face U moves, the predetermined region R1 also moves following the movement of the user's face U as indicated by the arrow X1 .

[0073] Here, the camera unit 5 captures the dynamic image 63 at a predetermined frame rate (10 to 300 fps). Therefore, the relative speed calculation unit 12 calculates the moving speed of the region R1 in the dynamic image 63, that is, the relative speed between the camera unit 5 and the user, based on the predetermined frame rate at which the camera unit 5 captures the dynamic image 63 and the moving direction and amount of the region R1 between frames. In addition, when the moving speed of the user's face U is too fast and the region R1 cannot follow, the relative speed calculation unit 12 determines that the relative speed exceeds the predetermined range.

[0074] The imaging unit 5 causes the display unit 3 to display an image (eg, Figure 3 When the display image 60 is shown, the relative speed calculation unit 12 captures a dynamic image 63 used for calculating the relative speed.

[0075] The display instruction unit 11 may also display a display image 60 (see FIG. 1 ) including predetermined information in a predetermined display format on the display unit 3. Figure 3 ) and at the same time, the dynamic image 63 captured by the camera 5 is displayed (see Figure 4 as well as Figure 5 ).

[0076] Alternatively, the display instruction unit 11 may cause the display unit 3 to display only the display image 60 , out of the display image 60 including the predetermined information in a predetermined display format and the moving image 63 , in the display area, and not display the moving image 63 in the display area.

[0077] use Figures 4 to 6 , an example of a case where the biological information measuring unit 13 measures the biological information of the user is described.

[0078] like Figure 4 and Figure 5As shown, the biological information measuring unit 13 identifies the local area R2 in the user's face U projected in the dynamic image 63 acquired by the imaging unit 5. Figure 4 and Figure 5 In the illustrated example, the biological information measurement unit 13 specifies the forehead of the user's face U projected on the dynamic image 63 as the region R2 .

[0079] Then, the biological information measuring unit 13 measures the pulse wave based on the change in state of the region R2 for each frame of the moving image 63 captured by the imaging unit 5 (for example, the change in the color of the body surface for each frame).

[0080] Figure 6 63 captured by the imaging unit 5. Figure 6 When the pulse wave is shown, Figure 6 The pulse wave shown measures, for example, blood pressure value, pulse rate, stress level, heart rate, blood oxygen saturation, and information indicating the health status of blood vessels to obtain biological information.

[0081] use Figure 7 , describing the changes in the biological information monitored by the change amount monitoring unit 16. Figure 7 2 is a diagram showing the state of the change in the biometric information monitored by the change amount monitoring unit 16. The change amount monitoring unit 16 monitors whether the change amount of the biometric information corresponding to the time information is within a predetermined range from the storage unit 230 of the server 200 via the communication unit 40, for example.

[0082] Furthermore, the variation monitoring unit 16 outputs information showing the variation of the biological information (e.g., blood pressure) corresponding to the time information on a daily basis or at each time to the display instruction unit 11 from the storage unit 230 of the server 200, and the display instruction unit 11 may cause the display unit 3 to display an image of a graph showing the variation of the biological information (e.g., blood pressure) on a daily basis or at each time. In this case, the display instruction unit 11 may display an image of a graph showing the variation of the biological information (e.g., blood pressure) on a daily basis or at each time as the display image 60 (see Figure 3 ) and the specified information in the specified display format (such as Figure 3 The characters in each column 61 shown are displayed together on the display unit 3.

[0083] Figure 8 1 is a flowchart showing an example of the operation of the control unit 10 of the portable terminal 1 according to the first embodiment. Figure 8As shown, in step S11, the portable terminal 1 causes the display unit 3 to display predetermined information in a predetermined display format based on an input signal from the input unit 4 based on the user's operation. Next, in step S12, the relative speed calculation unit 12 calculates the relative speed between the imaging unit 5 and the user based on the dynamic image acquired from the imaging unit 5. Then, in step S13, the biological information measurement unit 13 measures the biological information of the user based on the dynamic image acquired from the imaging unit 5.

[0084] Then, in step S14 , the determination unit 15 determines whether the relative speed calculated by the relative speed calculation unit 12 is within a predetermined range.

[0085] In step S14, when the determination unit 15 determines that the relative speed calculated by the relative speed calculation unit 12 is below the predetermined range (in the case of "yes" in step S14), the determination unit 15 sequentially associates the biological information measured by the biological information measurement unit 13 with the time information when the biological information was measured by the biological information measurement unit 13. Then, the determination unit 15 sequentially stores the biological information associated with the time information in the storage unit 230 of the server 200 via the communication unit 40, for example (step S15).

[0086] In step S14 , when the determination unit 15 determines that the relative speed calculated by the relative speed calculation unit 12 exceeds the predetermined range (“No” in step S14 ), the process returns to step S12 .

[0087] Next, in step S16, the variation monitoring unit 16 monitors via the communication unit 40 whether the variation of the biometric information sequentially corresponding to the time information stored in the storage unit 230 of the server 200 is below the prescribed range. In step S16, if the variation monitoring unit 16 determines that the variation of the biometric information sequentially corresponding to the time information is below the prescribed range (in the case of "Yes" in step S16), the process proceeds to the next step S18. In addition, in step S16, if the variation monitoring unit 16 determines that the variation of the biometric information sequentially corresponding to the time information exceeds the prescribed range (in the case of "No" in step S16), the notification process is performed (step S17), and the process proceeds to the next step S18.

[0088] In step S18, the control unit 10 ends the measurement of the biological information after acquiring the instruction signal for ending the measurement of the biological information from the input unit 4 (in the case of "Yes" in step S18). In addition, in step S18, if the control unit 10 does not acquire the instruction signal for ending the measurement of the biological information from the input unit 4 (in the case of "No" in step S18), the process returns to step S12.

[0089] Thus, the control unit 10 in the portable terminal 1 includes a display instruction unit 11, a biological information measuring unit 13, a relative speed calculation unit 12, and a determination unit 15. The display instruction unit 11 generates an instruction signal for causing the display unit 3 to display the prescribed information in a prescribed display format, and outputs the instruction signal to the display unit 3. As a result, the display unit 3 displays the prescribed information in the prescribed display format in the display area. As a result, the user can visually recognize the prescribed information in the prescribed display format displayed in the display area to a certain extent. As a result, the user hardly moves his face, etc., and the relative speed between the user and the display unit 3 can be slowed down.

[0090] And, when the display instruction unit 11 generates an instruction signal, the biological information measurement unit 13 measures the biological information of the user. In addition, the relative speed calculation unit 12 calculates the relative speed between the display unit 3 and the user during the period when the biological information measurement unit 13 measures the biological information. Then, the judgment unit 15 judges whether the relative speed calculated by the relative speed calculation unit 12 is within a prescribed range. Thus, the control unit 10 in the portable terminal 1 can obtain the biological information when the relative speed is within the prescribed range. That is, the control unit 10 in the portable terminal 1 can obtain the biological information of the user measured with high accuracy.

[0091] The prescribed information in the prescribed display format displayed on the display unit 3 preferably includes information of content that the user is interested in, for example, information for managing the daily changes in the user's biological information and information related to daily news. Thus, the portable terminal 1 can measure the user's biological information while the user is visually recognizing the information of the content that the user is interested in displayed on the display unit 3. Thus, the burden of measuring the user's biological information can be reduced, so that the user can easily obtain a large amount of biological information measurement. As a result, the portable terminal 1 can more accurately grasp the user's health status.

[0092] In addition, in the portable terminal 1 involved in the present embodiment, the display unit 3 for displaying specified information in a specified display format and the photographing unit 5 for photographing the user are arranged in the same housing 2, thereby being constituted as the same device (portable terminal 1), but the display unit 3 and the photographing unit 5 can also be constituted as different devices (separate devices) that can be communicatively connected via wired or wireless means.

[0093] [Second embodiment]

[0094] A second embodiment of the present disclosure will be described below. For ease of description, components having the same functions as those described in the first embodiment are denoted by the same reference numerals, and description thereof will not be repeated.

[0095] Fig. 91 is a functional block diagram of a biological information processing system 100 including a portable terminal 1 according to the second embodiment. Fig. 9 As shown in the second embodiment, the control unit 10 includes a value information generating unit 17, and the storage unit 230 of the server 200 stores user identification information 231, score information 234, and health management information 233. The functional blocks of the other biological information processing system 100 are the same as those of the biological information processing system 100 of the first embodiment (see Figure 2 ) is the same. In addition, the user identification information 231, the point information 234 and the health management information 233 may also be stored in the storage unit 30 of the portable terminal 1 instead of being stored in the storage unit 230 of the server 200.

[0096] In this embodiment, when the variation of the biological information acquired from the storage unit 230 of the server 200 exceeds a prescribed range, the variation monitoring unit 16 outputs information indicating whether the variation is below the prescribed range or exceeds the prescribed range to the value information generating unit 17 instead of performing notification processing.

[0097] The value information generating unit 17 generates value information 232 which is information corresponding to the amount of change in the biological information monitored by the amount of change monitoring unit 16 .

[0098] The user identification information 231 is identification information assigned to each user in order to identify the user.

[0099] The value information 232 is information including at least one of the health management information 233 and the point information 234 .

[0100] The health management information 233 is, for example, visual information and / or audio information corresponding to biological information (e.g., pulse wave, blood pressure value, pulse rate, stress level, heart rate, respiratory rate, body temperature, blood oxygen saturation, and information indicating the health status of blood vessels, etc.). Specifically, for example, the health management information 233 is news, advice, etc. corresponding to the biological information, output as an image, and / or output as an audio.

[0101] When the health management information 233 is visual information, the variation monitoring unit 16 outputs the generated health management information 233 to the display indication unit 11, and displays the generated health management information 233 as an image on the display unit 3 via the display indication unit 11. In addition, when the health management information 233 is sound information, the value information generating unit 17 outputs the generated value information to the speaker 6, so that the generated value information is output to the speaker 6 as sound.

[0102] Thereby, the portable terminal 1 can present useful information related to the health status of the user himself to the user.

[0103] When the health management information 233 is visual information, the value information generating unit 17 may also display the health management information 233 as specified information in a specified display format on a portion of a display image of a specific application such as a health management application. Thus, the relative speed between the display unit 3 and the user can be reduced, and as a result, the biological information measuring unit 13 can measure the biological information with high accuracy.

[0104] The point information 234 is visual information indicating the point corresponding to the user specified information 231 .

[0105] The value information generating unit 17 changes the integral indicated by the integral information 234 based on the result of the determination by the variation monitoring unit 16 whether the variation of the biometric information is below the prescribed range or exceeds the prescribed range. For example, if the variation monitoring unit 16 determines that the variation of the biometric information is below the prescribed range, the value information generating unit 17 increases and updates the integral of the integral information 234 stored in the storage unit 230. In addition, for example, if the variation monitoring unit 16 determines that the variation of the biometric information exceeds the prescribed range, the value information generating unit 17 does not change the integral of the integral information 234 stored in the storage unit 230 or decreases and updates the integral.

[0106] The value information generating unit 17 displays the points based on the points information 234 on a part of the display image of a specific application such as a health management application in the storage unit 230. The value information generating unit 17 may also display the points based on the points information 234 on a part of the display image of a specific application as prescribed information in a prescribed display format.

[0107] Then, for example, the user can obtain various prizes and other benefits according to the points indicated by the point information 234. Thus, the user can feel the advantage of measuring biological information and increase the chance of measuring biological information. As a result, the portable terminal 1 can more accurately grasp the health status of the user.

[0108] Fig.10 1 is a flowchart showing an example of the operation of the control unit 10 of the portable terminal 1 according to the second embodiment. The processing up to steps S11 to S16 is similar to that of using Figure 8 The operation of the control unit 10 of the portable terminal 1 according to the first embodiment described above is the same.

[0109] Fig.10 As shown, the control unit 10 performs the processing of step S11 to step S16.

[0110] In step S16, when the change amount monitoring unit 16 determines that the change amount of the biological information sequentially corresponding to the time information stored in the storage unit 230 of the server 200 is below the prescribed range (the case of "yes" in step S16), the value information generating unit 17 generates and updates the value information 232 with reference to the storage unit 230 in step S21. For example, in step S21, the value information generating unit 17 generates and updates the health management information 233 of the content that the user is interested in with reference to the storage unit 230. In addition, for example, in step S21, the value information generating unit 17 generates and updates the points information 234 that increases the points represented by the points information 234 with reference to the storage unit 230.

[0111] In addition, in step S16, when the change amount monitoring unit 16 determines that the change amount of the biological information sequentially corresponding to the time information stored in the storage unit 230 of the server 200 exceeds the prescribed range (the case of "No" in step S16), the value information generating unit 17 refers to the storage unit 230, generates and updates the value information 232 in step S22. For example, in step S22, the value information generating unit 17 refers to the storage unit 230, generates and updates the health management information 233 corresponding to the content of the biological information (for example, pulse wave, blood pressure value, pulse rate, stress level, heart rate, respiratory rate, body temperature, blood oxygen saturation, and information indicating the health status of blood vessels) and the content for maintaining health. In addition, for example, in step S22, the value information generating unit 17 refers to the storage unit 230, generates and updates the score information 234 that maintains or reduces the score indicated by the score information 234.

[0112] Next, in step S22, the value information generating unit 17 outputs the content of the value information 232 generated in step S21 or step S22 to the display unit 3 and / or the speaker 6. Then, the process proceeds to step S18.

[0113] Thus, according to the portable terminal 1 involved in this embodiment, the content of the useful value information 232 can be presented to the user according to whether the variation of the biological information is below the prescribed range. Thus, the user can feel the advantage of measuring the biological information and increase the chance of measuring the biological information. As a result, the portable terminal 1 can more accurately grasp the health status of the user.

[0114] [Variation]

[0115] The portable terminal 1 or the main body of the biological information processing method in the present disclosure has a control unit 10 as a computer. The function of the portable terminal 1 or the main body of the biological information processing method in the present disclosure is realized by executing the biological information processing program by the computer. The computer has a processor that acts according to the biological information processing program as the main hardware structure. As long as the processor can realize the function by executing the biological information processing program, its type does not matter. The processor includes a semiconductor integrated circuit (IC) or one or more electronic circuits including LSI (Large Scale Integration). Multiple electronic circuits can be integrated into one chip or arranged in multiple chips. Multiple chips can be integrated into one device or arranged in multiple devices. The biological information processing program is recorded in a non-temporary storage medium such as a computer-readable ROM, an optical disk, a hard disk drive, etc. The biological information processing program can be pre-stored in a storage medium or provided to a storage medium via an electrical communication line including the Internet.

[0116] The present invention is not limited to the above-described embodiments, and may be replaced with a configuration that is substantially the same as the configuration shown in the above-described embodiments, a configuration that achieves the same function and effect, or a configuration that can achieve the same purpose.

Claims

1. A biological information processing device, characterized in that: include: a display instruction unit that generates an instruction signal for causing the display unit to display prescribed information in a prescribed display format; a biological information measuring unit that measures biological information of the user based on sensor information indicating a change in a biological state of the user acquired by the sensor unit after the display instructing unit generates the instruction signal; a relative speed calculation unit that calculates a relative speed between the sensor unit and the user during a period in which the biological information measurement unit measures the biological information; and a determination unit configured to determine whether the relative speed calculated by the relative speed calculation unit is within a predetermined range; a timing unit that generates time information; a variation monitoring unit that monitors whether the variation of the biometric information associated with the time information is within a predetermined range; a value information generating unit configured to generate value information corresponding to the variation monitored by the variation monitoring unit, The determination unit sequentially associates the biological information when the relative speed is within the predetermined range with the time information when the biological information is measured, The value information includes point information indicating points associated with user identification information identifying the user, If the amount of change in the biometric information is below the prescribed range, the value information generating unit increases the points, and if the amount of change in the biometric information exceeds the prescribed range, the value information generating unit decreases the points or does not change them. The prescribed information includes the points, The display instruction unit causes the display unit to display an image of a graph indicating a change in the biological information together with the predetermined information.

2. The biological information processing device according to claim 1, wherein: comprising the sensor portion, The sensor unit acquires the sensor information by capturing a moving image of the user.

3. The biological information processing device according to claim 2, wherein: The relative speed calculation unit calculates the relative speed based on the moving image.

4. The biological information processing device according to claim 1, wherein: The biological information includes information indicating at least one of a blood pressure value, a pulse rate, a heart rate, a respiratory rate, a body temperature, and a blood oxygen saturation level.

5. The biological information processing device according to claim 1, wherein: The predetermined information is an image including characters having a font size of not less than 1 px and not more than 16 px, or not less than 0.75 pt and not more than 12 pt.

6. The biological information processing device according to claim 1, wherein: The predetermined information is an image in which the line spacing between characters in a plurality of lines is not less than 1 em and not more than 1.5 em.

7. The biological information processing device according to claim 1, wherein: comprising the display unit, The value information includes visual information corresponding to the content of the biological information, The value information generating unit causes the display unit to display an image based on the visual information.

8. The biological information processing device according to claim 1 or 7, characterized in that: including a sound output portion for outputting sound, The value information includes sound information corresponding to the content of the biological information, The value information generating unit causes the sound output unit to output sound based on the sound information.

9. A biological information processing method, characterized in that: Include: a step of generating an instruction signal for causing a display unit to display prescribed information in a prescribed display format; After generating the instruction signal, a step of measuring biological information of the user based on sensor information indicating a change in a biological state of the user acquired by a sensor unit; a step of calculating a relative speed between the sensor unit and the user during a period of measuring the biological information; as well as A step of determining whether the relative speed is within a specified range; Steps for generating moment information; A step of sequentially associating the biological information when the relative speed is within the prescribed range with the time information when the biological information is measured; A step of monitoring whether a change amount of the biometric information associated with the time information is within a prescribed range; a step of generating value information, wherein the value information is information corresponding to the change amount, The value information includes point information indicating points associated with user identification information identifying the user, If the amount of change in the biometric information is below the prescribed range, the integral is increased, and if the amount of change in the biometric information exceeds the prescribed range, the integral is decreased or not changed. The prescribed information includes the points, An image of a graph indicating a change in the biological information is displayed on the display unit together with the predetermined information.

Images