Biological information measurement device, biological information measurement method, and biological information measurement program

Abstract

[Problem] To provide a biological information measurement device, a biological information measurement method, and a biological information measurement program, each of which can measure biological information while reducing the burden on a user. [Solution] A biological information measurement device according to the present technology is provided with a reflection spectrum generation unit and an analysis processing unit. The reflection spectrum generation unit forms an image to be displayed, and generates a reflection spectrum, which is a wavelength spectrum of reflected light of the irradiation light emitted onto a measurement range, which is a specific range in the body of a user in which the irradiation light is reflected. The analysis processing unit calculates biological information about the user on the basis of an irradiation spectrum, which is a wavelength spectrum of the irradiation light, and the reflection spectrum.

Description

Technical Field

[0001] The present technology relates to a biological information measuring device, a biological information measuring method, and a biological information measuring program for measuring biological information about a user using light. Background Art

[0002] The measurement of biological information such as pulse rate, arterial oxygen saturation, and body surface temperature is called biosensing. Advantageously, biosensing that does not impose a heavy burden on the user is used. In particular, for example, when measuring biological information for a long time at home, the burden level may have a great influence on the measurement of the biological information. In this regard, biosensing using light does not impose a heavy burden on the user, but is easily affected by external light. Therefore, it is necessary to take measures against this problem.

[0003] For example, Patent Document 1 discloses a device included in a head-mounted display. The device causes a display panel to emit light according to a specified rule, images reflected light reflected from the user's skin or eyeball, and checks image information against registered information. Thus, the device performs individual identification on the user. Use of a head-mounted display makes it possible to block external light, and thus perform individual identification with high accuracy.

[0004] Reference List

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-18729 Summary of the invention

[0007] Technical issues

[0008] However, when such a method disclosed in Patent Document 1 is applied, it is necessary to make the display panel emit light according to a specified rule. Emitting light according to the specified rule includes: making the display panel emit light having a specific color, and performing color switching in a sequential manner; and making light of different colors emitted for corresponding partitions of the display panel. The light has no special meaning for the user, and the user needs to wait until the recognition process is completed. Therefore, there is still a burden on the user.

[0009] In view of the above circumstances, an object of the present technology is to provide a biological information measurement device, a biological information measurement method, and a biological information measurement program, which make it possible to measure biological information while reducing the burden on the user.

[0010] Solution to the problem

[0011] In order to achieve the above-mentioned object, a biological information measurement device according to an embodiment of the present technology includes a reflection spectrum generation unit and an analysis processing unit.

[0012] The reflection spectrum generating unit generates a reflection spectrum which is a wavelength spectrum of reflected light obtained by irradiation light irradiated onto a measurement range constituting a display video being reflected from the measurement range, the measurement range being a specific range on a user's body.

[0013] The analysis processing unit calculates biological information about the user based on the illumination spectrum and the reflection spectrum, the illumination spectrum being a wavelength spectrum of the illumination light.

[0014] The reflection spectrum generation unit may generate a reflection spectrum according to the reflection light reception data, which is data generated by imaging the reflection light.

[0015] The biological information measuring apparatus may further include an irradiation spectrum calculation unit which calculates the irradiation spectrum based on video data of the display video and based on device characteristics of the biological information measuring apparatus, and the spectrum acquisition unit may acquire the irradiation spectrum from the irradiation spectrum calculation unit.

[0016] The biological information measuring apparatus may further include an illumination spectrum generating unit which generates an illumination spectrum from illumination light reception data which is data generated by imaging the illumination light, and the spectrum acquiring unit may acquire the illumination spectrum from the illumination spectrum calculating unit.

[0017] The biological information measuring apparatus may further include a display unit on which a display video is displayed, and a reflected light imaging unit which images the reflected light to generate reflected light reception data.

[0018] The biological information measuring apparatus may further include an illumination light imaging unit that images the illumination light to generate illumination light reception data.

[0019] The bio-information measuring device may be a head mounted display which is worn on the user's head and whose display unit is arranged in front of the user's eyes.

[0020] The reflected light imaging unit may be arranged to face the measurement range.

[0021] The biological information measuring apparatus may further include an illumination light imaging unit which images the illumination light to generate illumination light reception data, and the illumination light imaging unit may be arranged to face the display unit.

[0022] The display unit and the reflected light imaging unit may each be spaced apart from the user, and the biological information measuring apparatus may further include a measurement range recognition unit that recognizes the measurement range based on an image obtained by imaging performed by the reflected light imaging unit.

[0023] The analysis processing unit may determine the biological information about the user using a biological information calculation model obtained by performing modeling on the irradiation spectrum, the reflection spectrum, and the biological information in accordance with each type of the biological information.

[0024] The bioinformatic computational model can be a predictive model generated by machine learning.

[0025] The biological information may be pulse rate, arterial oxygen saturation, skin moisture, body surface temperature, respiratory rate or blood pressure.

[0026] The biometric information may be a vein pattern or an iris pattern, and the analysis processing unit may perform biometric authentication using the biometric information.

[0027] In order to achieve the above-mentioned object, a biological information measuring program according to an embodiment of the present technology causes an information processing apparatus to operate as a reflection spectrum generating unit and an analysis processing unit.

[0028] The reflection spectrum generating unit generates a reflection spectrum which is a wavelength spectrum of reflected light obtained by irradiation light irradiated onto a measurement range constituting a display video being reflected from the measurement range, the measurement range being a specific range on a user's body.

[0029] The analysis processing unit calculates biological information about the user based on the illumination spectrum and the reflection spectrum, the illumination spectrum being a wavelength spectrum of the illumination light.

[0030] In order to achieve the above-mentioned purpose, a biological information measurement method according to an embodiment of the present technology includes generating a reflection spectrum, which is a wavelength spectrum of reflected light obtained by irradiating light onto a measurement range and being reflected from the measurement range, the irradiating light constituting a display video, and the measurement range is a specific range on the user's body.

[0031] The biological information about the user is calculated based on the illumination spectrum and the reflection spectrum, the illumination spectrum being the wavelength spectrum of the illumination light. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A biological information detection apparatus according to a first embodiment of the present technology is schematically shown.

[0033] Figure 2 The measuring range is schematically shown.

[0034] Figure 3 is a block diagram showing the functional configuration of the biological information detection apparatus.

[0035] Figure 4 An example of an irradiation spectrum calculated by an irradiation spectrum calculation unit included in the biological information detection apparatus is shown.

[0036] Figure 5 An example of a reflection spectrum calculated by a reflection spectrum generation unit included in the biological information detection apparatus is shown.

[0037] Figure 6 is a flowchart showing the operation of the biological information detection apparatus.

[0038] Figure 7 The diagram schematically shows how to handle the offset of the biological information detection device.

[0039] Figure 8 The diagram schematically shows how to handle the offset of the biological information detection device.

[0040] Fig. 9 A biological information detection apparatus having another configuration is schematically shown.

[0041] Fig.10 is a block diagram showing a functional configuration of a biological information detection apparatus having another configuration.

[0042] Fig.11 A biological information detection apparatus according to a second embodiment of the present technology is schematically shown.

[0043] Fig.12 is a block diagram showing the functional configuration of the biological information detection apparatus.

[0044] Fig.13 is a flowchart showing the operation of the biological information detection apparatus.

[0045] Fig.14 It is a block diagram showing the hardware configuration of each of the biological information detection apparatus according to the first embodiment of the present technology and the biological information detection apparatus according to the second embodiment of the present technology. DETAILED DESCRIPTION

[0046] (First Embodiment)

[0047] The biological information detection apparatus according to the first embodiment of the present technology has been described.

[0048] [Configuration of biological information detection device]

[0049] Figure 1 The biological information measuring device 100 according to the present embodiment is schematically shown. As shown in the figure, the biological information measuring device 100 is a head mounted display worn on the head H of the user U, and includes a housing 111 , a display unit 112 , and a reflected light imaging unit 113 .

[0050] The housing 111 is worn on the head H and supports the display unit 112 and the reflected light imaging unit 113. A partition 111a that separates the display unit 112 from the reflected light imaging unit 113 is provided to the housing 111. The specific configuration of the housing 111 is not particularly limited. It is sufficient if the housing 111 can maintain the positions of the display unit 112 and the reflected light imaging unit 113 relative to the head H, and advantageously, the housing 111 excellently blocks external light.

[0051] The display unit 112 is arranged in front of the eyes of the user U, and displays a video thereon. The video displayed on the display unit 112 is hereinafter referred to as "display video", and the light constituting the display video is hereinafter referred to as "illumination light". Figure 1 1 and 2 show the illumination light L1 as the illumination light. The illumination light L1 is light incident on the face surface of the user U, and includes the emission light emitted from the display unit 112 and the light obtained by the emission light being reflected inside the housing 111. Figure 2 As shown, the user U can view the display video by using the illumination light L1 incident on the eyes of the user U. In addition, a part of the illumination light L1 is reflected from the face surface of the user U to become reflected light. Figure 1 The reflected light L2 is shown as the reflected light.

[0052] A device that emits light with an amount of light greater than a specified amount of light can be used as the display unit 112, and for example, a liquid crystal display can be used as the display unit 112. A single display unit 112 may be provided, or two display units 112 for the left eye and for the right eye may be provided. Note that there is a head-mounted display to which another display device (e.g., a smartphone) can be attached, and another display device can also be used as the display unit 112. In addition, an optical element such as a lens may be arranged between the display unit 112 and the eyes of the user U.

[0053] The reflected light imaging unit 113 images the reflected light L2 to generate reflected light reception data as reception data of the reflected light L2. The reflected light imaging unit 113 performs imaging on the "measurement area" on the facial surface of the user U. The measurement range refers to a specific range on the facial surface of the user U. Figure 1 A measuring range S is shown as the measuring range. Figure 2An example of a measurement range S is schematically shown. The figure shows a range S1 corresponding to the area between the eyes, a range S2 corresponding to the temples, and a range S3 corresponding to the area under the eyes. The measurement range S may include at least one of these ranges. In addition, the measurement range S may be a range other than the range S1 to the range S3 on the facial surface of the user U. The reflected light imaging unit 113 faces the measurement range S, and images the reflected light L2 reflected from the measurement range S. Note that an optical element such as a lens may be arranged between the reflected light imaging unit 113 and the facial surface of the user U.

[0054] The reflected light imaging unit 113 may be a common imaging device, or may be an imaging device included in a multi-spectral camera (MSC). Common camera devices use light receiving elements provided with corresponding color filters of red, green and blue to perform image capture on three channels of red, green and blue. On the other hand, MSC is a camera device capable of performing image capture on a larger number of channels such as 9 channels.

[0055] Figure 3 1 is a block diagram showing the functional configuration of the biological information measuring device 100. As shown in the figure, the biological information measuring device 100 includes a video acquisition unit 121, a display controller 122, an irradiation spectrum calculation unit 123, a reflection spectrum generation unit 124, and an analysis processing unit 125. The functional configuration is realized by the cooperation of software and hardware. The functional configuration may be included in the housing 111 or included in a device outside the housing 111.

[0056] The video acquisition unit 121 acquires video data V, and generates video data V(t) constituting a display video displayed on the display unit 112. The video acquisition unit 121 may acquire the video data V from, for example, a storage device included in the biological information measuring device 100, an external device, or a network. The details of the video data V are not particularly limited, and the video data V may be a moving image such as a movie or game video, or may be a still image. The video data V(t) is expressed as follows.

[0057] V(t)=[V r (t); V g (t); V b (t)] (Formula 1)

[0058] In (Equation 1), V r (t) represents the intensity of the red component in the displayed video, V g (t) represents the intensity of the green component in the displayed video, and V b(t) represents the intensity of the blue component in the display video. The video acquisition unit 121 supplies the generated video data V(t) to the display controller 122 and the irradiation spectrum calculation unit 123.

[0059] The display controller 122 controls the display unit 112 (see Figure 1 ), so that the video data V(t) provided by the video acquisition unit 121 will be displayed on the display unit 112. Therefore, the illumination light L1 constituting the display video is illuminated onto the face surface of the user.

[0060] The irradiation spectrum calculation unit 123 calculates an irradiation spectrum based on the video data V(t) and the device characteristics F of the biological information measuring apparatus 100. The irradiation spectrum is a wavelength spectrum of the irradiation light L1, and Figure 4 An example of an illumination spectrum is shown.

[0061] The device characteristic F is a characteristic that represents the relationship between the emission light emitted from the display unit 112 and the illumination light L1 incident on the user's facial surface. The emission light emitted from the display unit 112 is affected by, for example, the display characteristics of the display unit 112, the characteristics of the optical element, the reflection on the inner circumferential surface of the housing 111, and the position of the measurement range S, and the device characteristic F includes these influences. The device characteristic F can be obtained by performing measurement in advance. Specifically, an imaging device whose characteristics are known is arranged for the measurement range S, and the light incident from the display unit 112 is received by the imaging device. Therefore, the device characteristic F can be obtained. The device characteristic F is expressed as follows.

[0062]

[0063] In (Formula 2), N represents the number of divisions of the spectrum (the number of binary bits), and is, for example, 512. This means that the wavelength band of the illumination spectrum is divided into 512 divisions to perform calculations. When the imaging device for measuring the device characteristic F is arranged for the measurement range S, the light receiving element with a red filter is referred to as a red light receiving element, when the imaging device is arranged for the measurement range S, the light receiving element with a green filter is referred to as a green light receiving element, and when the imaging device is arranged for the measurement range S, the light receiving element with a blue filter is referred to as a blue light receiving element. In (Formula 2), f r1 represents the rate of change of the intensity of the light received by the red light receiving element in the first sub-area, f r2 represents the rate of change of the intensity of light received by the red light receiving element in the second sub-area, and f rN represents the rate of change of the intensity of light received by the red light receiving element in the Nth partition. The same applies to the green light receiving element (f g ) and blue light receiving element (f b ).

[0064] It is assumed that the device characteristic F depends on the number of channels of the reflected light imaging unit 113. In (Formula 2), the device characteristic F refers to the display characteristic when the reflected light imaging unit 113 includes three channels of red, green and blue. It is assumed that when the reflected light imaging unit 113 is a MSC, the device characteristic F depends on the number of channels of the MSC.

[0065] The irradiation spectrum calculation unit 123 multiplies the video data V(t) by the device characteristic F. This makes it possible to calculate the irradiation spectrum. The irradiation spectrum calculated by the irradiation spectrum calculation unit 123 is hereinafter referred to as the irradiation spectrum R(t). The irradiation spectrum R(t) is as follows.

[0066] R(t)=V(t)F=[r1(t), r2(t),…r N (t)] (Equation 3)

[0067] In (Equation 3), r1 represents the intensity in the first partition, r2 represents the intensity in the second partition, and r N represents the intensity in the Nth partition. As described above, the irradiation spectrum calculation unit 123 reflects the display characteristics of the display unit 112 and the reflection characteristics of the housing 111 in the emission light emitted from the display unit 112 to calculate the wavelength spectrum of the irradiation light L1, that is, the irradiation spectrum. Note that the irradiation spectrum calculation unit 123 averages the pixels of the video data V(t) to make the resolution lower. This also makes it possible to reduce the amount of calculation. The irradiation spectrum calculation unit 123 provides the calculated irradiation spectrum R(t) to the analysis processing unit 125.

[0068] The reflection spectrum generation unit 124 acquires the reflection light reception data from the reflection light imaging unit 113, and generates a reflection spectrum A(t) from the reflection light reception data. The reflection spectrum A(t) is a wavelength spectrum of the reflection light L2, and is expressed as follows.

[0069] A(t)=[a1(t), a2(t), …a M (t)] (Equation 4)

[0070] In (Formula 4), a iRepresents the amount of light output by the light receiving element of channel i of the reflected light imaging unit 113. In addition, M represents the number of channels of the reflected light imaging unit 113. The reflected spectrum generating unit 124 can average the amounts of light respectively output by all the light receiving elements of the reflected light imaging unit 113 to obtain a scalar, or can process the amounts of light respectively output by the light receiving elements separately. In addition, the reflected spectrum generating unit 124 can average the amounts of light respectively output by a plurality of light receiving elements. When there are a large number of processing target light receiving elements, the bioinformation calculation model described later will become complicated. The reflected spectrum generating unit 124 provides the generated reflected spectrum A(t) to the analysis processing unit 125.

[0071] The analysis processing unit 125 calculates biological information about the user based on the irradiation spectrum R(t) and the reflection spectrum A(t). Figure 5 is a graph in which examples of the irradiation spectrum R(t) and the reflection spectrum A(t) are given. As shown in the figure, there is a difference between the irradiation spectrum R(t) and the reflection spectrum A(t). The biological information in the measurement range S is reflected in the characteristics of the difference, that is, reflected in the wavelength band where the difference exists and in the intensity change amount where the difference exists. Therefore, the analysis processing unit 125 can calculate the biological information about the user based on the irradiation spectrum R(t) and the reflection spectrum A(t).

[0072] Specifically, the analysis processing unit 125 can obtain the biological information calculation model N from the wavelength spectrum difference database 126. The biological information calculation model N is obtained by performing modeling on the irradiation spectrum R(t), the reflection spectrum A(t), and the biological information for each type of biological information. As shown below (Formula 5), ​​the analysis processing unit 125 can calculate the biological information (index) by applying the irradiation spectrum R(t) and the reflection spectrum A(t) to the biological information calculation model N.

[0073] N(R(t), A(t)) = index (Formula 5)

[0074] Examples of biological information that can be calculated using the biological information calculation model N include pulse rate, arterial oxygen saturation (SpO2), skin moisture, body surface temperature, respiratory rate, blood pressure, vein pattern, and iris pattern, and the biological information calculation model N is provided for each type of biological information. The analysis processing unit 125 selects the biological information calculation model N according to the type of biological information to be calculated, and can calculate the biological information using the selected biological information calculation model N.

[0075] The bio-information calculation model N may be a prediction model generated by a machine learning method, in which the irradiation spectrum R(t) and the reflection spectrum A(t) are used as inputs, and, for example, bio-information measured by a wearable device is used as training data. In addition, the bio-information calculation model N is not limited to a model generated by machine learning. A specific example of bio-information calculated by the analysis processing unit 125 using the bio-information calculation model N is described below.

[0076] The analysis processing unit 125 may perform filtering using a green wavelength band filter, perform fast Fourier transform (FFT), perform filtering using a band pass filter (BPF) between 1 Hz and 2.5 Hz (including end values), and detect a peak value to set the value of the peak value as the pulse rate. In addition, the analysis processing unit 125 may perform filtering using a red wavelength band or near infrared wavelength band filter with respect to each of the illumination spectrum R (t) and the reflection spectrum A (t) to calculate SpO2 based on the ratio of the corresponding results of the filtering.

[0077] In addition, the analysis processing unit 125 can calculate the body surface temperature based on the intensity of the irradiation spectrum R(t) and the reflection spectrum A(t) in the far-infrared wavelength band and based on the value obtained by the temperature sensor. The reason is that a part of the far-infrared wavelength is related to a temperature close to the body temperature. In addition, the analysis processing unit 125 can perform filtering using a near-infrared wavelength band filter with respect to each of the irradiation spectrum R(t) and the reflection spectrum A(t) to calculate the skin moisture amount based on the ratio of the corresponding results of the filtering. Veins absorb light of the near-infrared wavelength band different from that of arteries. Therefore, a wavelength band different from the green wavelength band used for the pulse rate is used.

[0078] In addition, the analysis processing unit 125 can perform filtering using a green wavelength band filter, filtering using a BPF between 1 Hz and 2.5 Hz (including the end value) and detecting the envelope with respect to each of the irradiation spectrum R(t) and the reflection spectrum A(t) to calculate the respiratory rate. In addition, the analysis processing unit 125 can perform filtering using a green wavelength band filter, filtering using a BPF between 1 Hz and 2.5 Hz (including the end value) and detecting the peak value with respect to each different measurement range S on the facial surface to calculate the blood pressure based on the propagation speed at the peak value. The blood wave transported from the heart can be observed as a pulse wave. Therefore, the pulse wave is measured at two points in the same path, and the speed of the pulse wave propagation is obtained. Therefore, the relative blood pressure (cuffless blood pressure estimation performed by the pulse-transit-time (PPT) method) can be estimated.

[0079] In addition, the analysis processing unit 125 can recognize the vein pattern of the face based on the two-dimensional information obtained by performing filtering using a green wavelength band filter with respect to each of the illumination spectrum R(t) and the reflection spectrum A(t), and can perform biometric authentication on the user. In addition, the analysis processing unit 125 can recognize the iris pattern of the eye based on the two-dimensional information obtained by performing filtering using an infrared wavelength band filter with respect to each of the illumination spectrum R(t) and the reflection spectrum A(t), and can perform biometric authentication on the user.

[0080] The biological information calculated by the analysis processing unit 125 is not limited to the above-described example, and any biological information that can be calculated based on the irradiation spectrum R(t) and the reflection spectrum A(t) may be employed.

[0081] [Operation of biological information detection device]

[0082] The operation of the biological information measuring apparatus 100 is described. Figure 6 is a flowchart showing the operation of the biological information measurement device 100 .

[0083] When the biological information measuring device 100 starts operating, the analysis processing unit 125 acquires the biological information calculation model N for calculating the target biological information from the wavelength spectrum difference database 126 (St101). Subsequently, the video acquisition unit 121 acquires the video data V (St102), and the display controller 122 supplies the video data V (t) to the display unit 112 so that the display video will be displayed on the display unit 112 (St103).

[0084] Subsequently, the irradiation spectrum calculation unit 123 calculates the irradiation spectrum R(t) based on the video data V(t) and the device characteristics F (St104). Subsequently, the reflection spectrum generation unit 124 acquires the reflection light reception data from the reflection light imaging unit 113, and generates the reflection spectrum A(t) according to the reflection light reception data (St105).

[0085] Subsequently, the analysis processing unit 125 applies the irradiation spectrum R(t) and the reflection spectrum A(t) to the biological information calculation model N, and calculates the biological information according to N(R(t), A(t)) (St106). Subsequently, the analysis processing unit 125 changes the time from time t to time t+1 (St107), and repeatedly performs the process of acquiring the biological information calculation model N (St101) and the process after acquiring the biological information calculation model N. Note that the time t corresponds to the frame of the video data V(t), and the time t+1 means an increase of one frame.

[0086] The biological information measuring device 100 measures biological information as described above. Note that the reflected light reception data may be stored when the reflected spectrum A(t) is generated (St105), and a processing device such as a server with high computing performance may later use the reflected light reception data together with the video data V(t) and the device characteristic F to calculate N(R(t), A(t)).

[0087] [Effects provided by the biological information detection device]

[0088] As described above, the bio-information measuring device 100 can measure bio-information based on the difference between the light irradiated onto the user's facial surface and the light reflected from the user's facial surface. The light irradiated onto the user's facial surface is the light constituting the display video displayed on the display unit 112, and the details of the display video are not particularly limited. Therefore, the bio-information can be measured while the user is watching any video such as a movie or a game video. This leads to a reduction in the burden on the user when measuring the bio-information. Therefore, the bio-information measuring device 100 is also suitable for measuring bio-information every day or for a long time at home.

[0089] [How to deal with the offset of the head mounted display]

[0090] When the head mounted display as the biological information measuring device 100 is offset relative to the head H of the user U, the biological information measuring device 100 can process the offset as indicated below. Figure 7 and Figure 8 Examples of captured images each obtained using the reflected light imaging unit 113 are shown.

[0091] For example, the analysis processing unit 125 detects the approach of the face by using a proximity sensor using infrared light, and sets the time of the detection as the initial time t0. all (t0), the reflection spectrum generation unit 124 averages the amount of light received by the light receiving element located in the center part of all the light receiving elements, and sets the obtained average value as A(t0). The reflection spectrum generation unit 124 uses A(t0) to calculate the biological information. Here, the analysis processing unit 125 stores therein the amount of light A received by all the light receiving elements. all (t0).

[0092] When the next frame t1 arrives after a certain period of time, calculate A all (t1) and A all (t0) to calculate the offset (x, y) of the entire captured image. allWhen A(t) is extracted, the analysis processing unit 125 performs updating by setting the position obtained by subtracting the calculated offset (x, y) from the center position of the captured image as the new center of the captured image. Figure 7 , Figure 8 In the example shown, it is assumed that the center of the captured image at time t0 is represented by Figure 7 The point P0 is shown, and the center of the captured image at time t1 is represented by Figure 8 Point P1 is shown. Thereafter, the analysis processing unit 125 updates the center of the captured image over time to account for the offset of the head mounted display.

[0093] [Another Configuration of the Biological Information Measuring Device]

[0094] An example in which the biological information measuring device 100 is a head mounted display has been described above. However, the biological information measuring device 100 may also have another configuration. Fig. 9 A bio-information measuring device 100 having another configuration is schematically shown. As shown in the figure, the display unit 112 and the reflected light imaging unit 113 are each spaced apart from the user U. The display unit 112 is a display device that is not a head mounted display, and is, for example, a television device or a screen in a movie theater. The reflected light imaging unit 113 is an imaging device that is arranged to face the user U and to image the user U.

[0095] Fig.10 1 is a block diagram showing a functional configuration of the biological information measurement device 100 having another configuration. As shown in the figure, the biological information measurement device 100 may further include a measurement range recognition unit 127. The measurement range recognition unit 127 performs a face recognition process or a face part recognition process with respect to an image obtained by performing imaging by the reflected light imaging unit 113, and recognizes the measurement range S. In addition, the measurement range recognition unit 127 extracts a reflection spectrum A'(t) in the measurement range S from the reflection spectrum A(t) generated by the reflection spectrum generation unit 124, and provides the extracted reflection spectrum A'(t) to the analysis processing unit 125. The analysis processing unit 125 can apply the irradiation spectrum R(t) and the reflection spectrum A'(t) to the biological information calculation model N, and can calculate the biological information according to N(R(t), A'(t)).

[0096] When there are a plurality of users U in the field of view of the reflected light imaging unit 113, the measurement range identification unit 127 can identify the measurement range S for each of the plurality of users U. In this case, the biological information measuring device 100 can simultaneously measure biological information about the plurality of users U. The measurement range S is not limited to the facial surface of the user U, and it is sufficient if the measurement range S is any range on the body surface of the user U (for example, an earlobe or a hand whose image can be captured).

[0097] Note that although it is not necessary to eliminate the influence of external light in a dark situation where there is no external light in the viewing environment of user U (such as in a movie theater), it is necessary to eliminate the influence of external light when there is external light such as in a normal room. Specifically, the measurement range recognition unit 127 calculates the distance from the user U to the reflected light imaging unit 113, and specifies the lighting condition for the position of the user U. The analysis processing unit 125 determines whether the specified lighting condition is similar to the lighting condition when constructing the bio-information calculation model N. When it has been determined that the two lighting conditions are similar to each other, the analysis processing unit 125 can use the reflection spectrum A'(t) to calculate the bio-information, and when it has been determined that the two lighting conditions are different from each other, the analysis processing unit 125 can correct the reflection spectrum A'(t) according to the difference and calculate the bio-information using the corrected reflection spectrum.

[0098] (Second Embodiment)

[0099] A biological information detection apparatus according to a second embodiment of the present technology has been described.

[0100] [Configuration of biological information detection device]

[0101] Fig.11 A biological information measuring device 150 according to the present embodiment is schematically shown. As shown in the figure, the biological information measuring device 150 is a head mounted display worn on the head H of the user U, and includes a housing 111, a display unit 112, a reflected light imaging unit 113, and an irradiated light imaging unit 151. Note that the structural elements other than the irradiated light imaging unit 151 are similar to those of the biological information measuring device 100 according to the first embodiment. Therefore, the description thereof is omitted.

[0102] The illumination light imaging unit 151 images the illumination light L1 to generate illumination light reception data as reception data of the illumination light L1. Fig.11 As shown, the illumination light imaging unit 151 is arranged near the facial surface of the user U and faces the display unit 112. As described above, the emission light emitted from the display unit 112 is affected by, for example, the display characteristics of the display unit 112, and is irradiated onto the facial surface of the user U in the form of illumination light L1. The illumination light imaging unit 151 images the illumination light L1 to generate illumination light reception data.

[0103] Fig.121 is a block diagram showing the functional configuration of the biological information measuring device 150 according to the present embodiment. As shown in the figure, the biological information measuring device 150 includes a video acquisition unit 161, a display controller 162, an irradiation spectrum generation unit 163, a reflection spectrum generation unit 164, and an analysis processing unit 165. The functional configuration is realized by the cooperation of software and hardware. The functional configuration may be included in the housing 111 or included in a device outside the housing 111.

[0104] As in the case of the first embodiment, the video acquisition unit 161 acquires video data V, and generates video data V(t) constituting a display video displayed on the display unit 112. The display controller 162 controls the display unit 112 (see Fig.11 ), so that the video data V(t) provided by the video acquisition unit 161 will be displayed on the display unit 112. Therefore, the illumination light L1 constituting the display video is illuminated onto the face surface of the user.

[0105] The illumination spectrum generation unit 163 acquires the illumination light reception data from the illumination light imaging unit 151 and generates an illumination spectrum B(t) from the illumination light reception data. The illumination spectrum B(t) is a wavelength spectrum of the illumination light L1 and is represented as shown below.

[0106] B(t)=[b1(t), b2(t), ...b Q (t)] (Equation 6)

[0107] In (Equation 6), b i Represents the amount of light output by the light receiving element of channel i of the illumination light imaging unit 151. In addition, Q represents the number of channels of the illumination light imaging unit 151. The illumination spectrum generation unit 163 can average the amounts of light respectively output by all the light receiving elements of the illumination light imaging unit 151 to obtain a scalar, or can process the amounts of light respectively output by the light receiving elements separately. In addition, the illumination spectrum generation unit 163 can average the amounts of light respectively output by a plurality of light receiving elements. In the first embodiment, the illumination spectrum calculation unit 123 calculates the illumination spectrum R(t), while in the present embodiment, the illumination spectrum B(t) generated according to the illumination light receiving data is used instead of calculating the illumination spectrum R(t). The illumination spectrum generation unit 163 provides the generated illumination spectrum B(t) to the analysis processing unit 165.

[0108] The reflection spectrum generation unit 164 acquires the reflection light reception data from the reflection light imaging unit 113, and generates a reflection spectrum A(t) based on the reflection light reception data. As in the case of the first embodiment, the reflection spectrum A(t) is the wavelength spectrum of the reflection light L2. The reflection spectrum generation unit 164 provides the generated reflection spectrum A(t) to the analysis processing unit 165.

[0109] The analysis processing unit 165 calculates the biological information about the user based on the irradiation spectrum B(t) and the reflection spectrum A(t). As in the case of the first embodiment, there is a difference between the irradiation spectrum B(t) and the reflection spectrum A(t) (see Figure 5 ). The biological information in the measurement range S is reflected in the characteristics of the difference, that is, reflected in the wavelength band where the difference exists and in the intensity change amount where the difference exists. Therefore, the analysis processing unit 165 can calculate the biological information about the user based on the illumination spectrum B(t) and the reflection spectrum A(t).

[0110] Specifically, the analysis processing unit 165 can obtain the biological information calculation model N from the wavelength spectrum difference database 166. The biological information calculation model N is obtained by performing modeling on the irradiation spectrum B(t), the reflection spectrum A(t), and the biological information for each type of biological information. As shown below (Formula 7), the analysis processing unit 165 can calculate the biological information (index) by applying the irradiation spectrum B(t) and the reflection spectrum A(t) to the biological information calculation model N.

[0111] N(B(t), A(t)) = index (Formula 7)

[0112] As in the case of the first embodiment, examples of biological information that can be calculated using the biological information calculation model N include pulse rate, arterial oxygen saturation (SpO2), skin moisture content, body surface temperature, respiratory rate, blood pressure, vein pattern, and iris pattern. The analysis processing unit 165 selects the biological information calculation model N according to the type of biological information to be calculated, and can calculate the biological information using the selected biological information calculation model N.

[0113] The biometric information calculation model N may be a prediction model generated by a machine learning method, in which the illumination spectrum B(t) and the reflection spectrum A(t) are used as inputs, and, for example, biometric information measured using a wearable device is used as training data. In addition, the biometric information calculation model N is not limited to a model generated by machine learning. The specific example of biometric information calculated by the analysis processing unit 165 using the biometric information calculation model N is similar to the specific example in the first embodiment, and therefore, if the illumination spectrum R(t) in the description of the first embodiment is used, it can be replaced with the illumination spectrum B(t). The analysis processing unit 165 can also perform biometric authentication based on a measured vein pattern or iris pattern.

[0114] [Operation of biological information detection device]

[0115] The operation of the biological information measuring device 150 is described. Fig.13is a flowchart showing the operation of the biological information measuring device 150 .

[0116] When the biological information measuring device 150 starts operating, the analysis processing unit 165 acquires the biological information calculation model N for calculating the target biological information from the wavelength spectrum difference database 166 (St151). Subsequently, the video acquisition unit 161 acquires the video data V (St152), and the display controller 162 supplies the video data V (t) to the display unit 112 so that the display video will be displayed on the display unit 112 (St153).

[0117] Subsequently, the illumination spectrum generation unit 163 acquires the illumination light reception data from the illumination light imaging unit 151, and generates the illumination spectrum B(t) (St154) based on the illumination light reception data. In addition, the reflection spectrum generation unit 164 acquires the reflection light reception data from the reflection light imaging unit 113, and generates the reflection spectrum A(t) (St154) based on the reflection light reception data.

[0118] Subsequently, the analysis processing unit 165 applies the irradiation spectrum B(t) and the reflection spectrum A(t) to the biological information calculation model N, and calculates the biological information according to N(B(t), A(t)) (St155). Subsequently, the analysis processing unit 165 changes the time from time t to time t+1 (St156), and repeatedly performs the process of acquiring the biological information calculation model N (St151) and the process after acquiring the biological information calculation model N. Note that the time t corresponds to the frame of the video data V(t), and the time t+1 means an increase of one frame.

[0119] The biological information measuring device 150 measures the biological information as described above. Note that the illumination light imaging data and the reflection light imaging data may be stored when the illumination spectrum B(t) and the reflection spectrum A(t) are generated (St154), and a processing device such as a server with high computing performance may later use the illumination light imaging data and the reflection light imaging data together with the video data V(t) to calculate N(B(t), A(t)).

[0120] [Effects provided by the biological information detection device]

[0121] As in the case of the biological information measuring device 100 according to the first embodiment, the biological information measuring device 150 can measure biological information based on the difference between the light irradiated onto the user's facial surface and the light reflected from the user's facial surface. This leads to a reduction in the burden on the user when measuring biological information. Compared with the biological information measuring device 100, it is necessary to provide the biological information measuring device 150 with the irradiation light imaging unit 151, and it is not necessary to measure the device characteristic F in advance, nor is it necessary to calculate the irradiation spectrum R (t) The process.

[0122] [other]

[0123] Using a method similar to that applied in the first embodiment, the biological information measuring device 150 can handle the offset of the head mounted display as the biological information measuring device 150 (see Figure 7 and Figure 8 ). In addition, as in the case of the first embodiment, the biological information measuring device 150 may also be a device other than a head mounted display (see Fig. 9 and Fig.10 ).

[0124] [Hardware configuration of the bio-information measurement device]

[0125] The hardware configuration that makes it possible to realize the functional configuration of each of the biological information measurement apparatus 100 according to the first embodiment and the biological information measurement apparatus 150 according to the second embodiment is described. Fig.14 The hardware configuration is schematically shown.

[0126] As shown in the figure, each of the biological information measuring device 100 and the biological information measuring device 150 includes a central processing unit (CPU) 1001 and a graphics processing unit (GPU) 1002. An input / output interface 1006 is connected to the CPU 1001 and the GPU 1002 via a bus 1005. A read-only memory (ROM) 1003 and a random access memory (RAM) 1004 are connected to the bus 1005.

[0127] An input unit 1007, an output unit 1008, a storage device 1009, and a communication unit 1010 are connected to the input / output interface 1006. The input unit 1007 includes an input device, such as a keyboard and a mouse, for a user to input an operation command. The output unit 1008 outputs an image of a processing operation screen and a processing result to a display device. For example, the storage device 1009 includes a hard disk drive in which programs and various data are stored. For example, the communication unit 1010 includes a local area network (LAN) adapter, and performs communication processing through a network represented by the Internet. In addition, a drive 1011 is connected to the input / output interface 1006. The drive 1011 reads data from a removable storage medium 1012 such as a disk, an optical disk, a magneto-optical disk, or a semiconductor memory and writes data therein.

[0128] The CPU 1001 executes various processes according to a program stored in the ROM 1003, or according to a program read from a removable storage medium 1012 (e.g., a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory) to be installed on the storage device 1009 and loaded from the storage device 1009 into the RAM 1004. As necessary, data required for the CPU 1001 to execute various processes is also stored in the RAM 1004. The GPU 1002 executes calculation processing required for drawing an image under the control of the CPU 1001.

[0129] In the biological information measuring apparatus 100 and the biological information measuring apparatus 150 each having the above configuration, the above series of processing is performed, for example, by the CPU 1001 loading a program stored in the storage device 1009 into the RAM 1004 and executing the program via the input / output interface 1006 and the bus 1005 .

[0130] For example, the program executed by each of the biological information measuring apparatus 100 and the biological information measuring apparatus 150 can be provided by being recorded in the removable storage medium 1012 serving as, for example, a package medium. In addition, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0131] In each of the biological information measuring device 100 and the biological information measuring device 150, the program can be installed on the storage device 1009 via the input / output interface 1006 by the removable storage medium 1012 mounted on the drive 1011. In addition, the program can be received by the communication section 1010 via a wired or wireless transmission medium mounted on the storage device 1009. In addition, the program can be pre-installed on the ROM 1003 or the storage device 1009.

[0132] Note that the program executed by each of the biological information measuring device 100 and the biological information measuring device 150 may be a program in which the processes are executed in time series in the order of the description in the present disclosure, or may be a program in which the processes are executed in parallel or at necessary timings such as the timing of calling. In addition, all of the hardware configurations of each of the biological information measuring device 100 and the biological information measuring device 150 do not have to be included in a single device, and each of the biological information measuring device 100 and the biological information measuring device 150 may include a plurality of devices. In addition, a part or all of the hardware configurations of each of the biological information measuring device 100 and the biological information measuring device 150 may be included in a plurality of devices connected to each other via a network.

[0133] (About this disclosure)

[0134] The effects described in the present disclosure are not restrictive but merely illustrative, and other effects may be provided. The above description of multiple effects does not necessarily mean that multiple effects are provided simultaneously. The above description means that at least one of the above effects is provided according to, for example, conditions. There is a possibility that an effect not described in the present disclosure will be provided. In addition, at least two of the features described in the present disclosure may also be used in combination.

[0135] Note that the present technology can also take the following configurations.

[0136] (1) A biological information measuring device comprising:

[0137] a reflection spectrum generating unit that generates a reflection spectrum that is a wavelength spectrum of reflected light obtained by irradiation light irradiated onto a measurement range that is a specific range on a user's body being reflected from the measurement range, the irradiation light constituting a display video; and

[0138] An analysis processing unit calculates biological information about the user based on an illumination spectrum and the reflection spectrum, the illumination spectrum being a wavelength spectrum of the illumination light.

[0139] (2) The biological information measuring device according to (1), wherein

[0140] The reflection spectrum generating unit generates the reflection spectrum according to reflection light reception data, and the reflection light reception data is data generated by imaging the reflection light.

[0141] (3) The biological information measuring device according to (1) or (2), further comprising:

[0142] an irradiation spectrum calculation unit that calculates the irradiation spectrum based on the video data of the display video and based on the device characteristics of the biological information measuring device, wherein

[0143] The spectrum acquisition unit acquires the illumination spectrum from the illumination spectrum calculation unit.

[0144] (4) The biological information measuring device according to (1) or (2), further comprising:

[0145] an illumination spectrum generating unit, the illumination spectrum generating unit generating the illumination spectrum according to illumination light receiving data, the illumination light receiving data being data generated by imaging the illumination light, wherein

[0146] The spectrum acquisition unit acquires the illumination spectrum from the illumination spectrum calculation unit.

[0147] (5) The biological information measuring device according to (1), further comprising:

[0148] a display unit, on which the display video is displayed; and

[0149] A reflected light imaging unit is used to image the reflected light to generate reflected light receiving data.

[0150] (6) The biological information measuring device according to (5), further comprising:

[0151] An illumination light imaging unit images the illumination light to generate illumination light reception data.

[0152] (7) The biological information measuring device according to (5), wherein

[0153] The biological information measuring device is a head-mounted display which is worn on a head of a user, and the display unit of the head-mounted display is arranged in front of eyes of the user.

[0154] (8) The biological information measuring device according to (7), wherein

[0155] The reflected light imaging unit is arranged to face the measurement range.

[0156] (9) The biological information measuring device according to (8), further comprising:

[0157] an illumination light imaging unit, wherein the illumination light imaging unit images the illumination light to generate illumination light reception data, wherein

[0158] The illumination light imaging unit is arranged to face the display unit.

[0159] (10) The biological information measuring device according to (5), wherein

[0160] The display unit and the reflected light imaging unit are each spaced apart from the user, and

[0161] The biological information measuring apparatus further includes a measurement range identifying unit that identifies the measurement range based on an image obtained by imaging performed by the reflected light imaging unit.

[0162] (11) The biological information measuring device according to any one of (1) to (10), wherein

[0163] The analysis processing unit determines the biological information about the user using a biological information calculation model obtained by performing modeling on the irradiation spectrum, the reflection spectrum, and biological information in each type of biological information.

[0164] (12) The biological information measuring device according to (11), wherein

[0165] The bioinformatics computing model is a prediction model generated by machine learning.

[0166] (13) The biological information measuring device according to any one of (1) to (12), wherein

[0167] The biological information is pulse rate, arterial blood oxygen saturation, skin moisture, body surface temperature, respiratory rate or blood pressure.

[0168] (14) The biological information measuring device according to any one of (1) to (12), wherein

[0169] The biometric information is a vein pattern or an iris pattern, and

[0170] The analysis processing unit performs biometric authentication using the biometric information.

[0171] (15) A biological information measuring program that causes an information processing device to operate as a reflection spectrum generating unit and an analysis processing unit,

[0172] the reflection spectrum generating unit generates a reflection spectrum which is a wavelength spectrum of reflected light obtained by irradiation light irradiated onto a measurement range being reflected from the measurement range, the irradiation light constituting a display video, the measurement range being a specific range on a body of a user,

[0173] The analysis processing unit calculates biological information about the user based on an illumination spectrum and the reflection spectrum, the illumination spectrum being a wavelength spectrum of the illumination light.

[0174] (16) A method for measuring biological information, comprising:

[0175] generating a reflection spectrum which is a wavelength spectrum of reflected light obtained by irradiation light irradiated onto a measurement range, which constitutes a display video, being reflected from the measurement range, the measurement range being a specific range on a user's body; and

[0176] Biometric information about the user is calculated based on an illumination spectrum and the reflection spectrum, the illumination spectrum being a wavelength spectrum of the illumination light.

[0177] Reference numerals list

[0178] 100, 150…Biological information measurement device

[0179] 111…Housing

[0180] 112…Display unit

[0181] 113…Reflected light imaging unit

[0182] 121, 161…Video acquisition unit

[0183] 122, 162…Display controller

[0184] 123…Irradiation spectrum calculation unit

[0185] 124, 164…Reflection spectrum generation unit

[0186] 125, 165…Analysis and processing unit

[0187] 126, 166…Wavelength spectrum difference database

[0188] 127…Measuring range identification unit

[0189] 151…Illumination light imaging unit

[0190] 163…Irradiation spectrum generation unit

Claims

1. A biological information measuring device, comprising: a reflection spectrum generating unit that generates a reflection spectrum that is a wavelength spectrum of reflected light obtained by irradiation light irradiated onto a measurement range being reflected from the measurement range, the irradiation light constituting a display video, the measurement range being a specific range on a user's body; as well as An analysis processing unit calculates biological information about a user based on an illumination spectrum and the reflection spectrum, the illumination spectrum being a wavelength spectrum of the illumination light.

2. The biological information measuring device according to claim 1, wherein: The reflection spectrum generating unit generates the reflection spectrum according to reflection light reception data, and the reflection light reception data is data generated by imaging the reflection light.

3. The biological information measuring device according to claim 1, further comprising: an irradiation spectrum calculation unit that calculates the irradiation spectrum based on the video data of the display video and the device characteristics of the biological information measuring apparatus, wherein A spectrum acquisition unit acquires the illumination spectrum from the illumination spectrum calculation unit.

4. The biological information measuring device according to claim 1, further comprising: an illumination spectrum generating unit, the illumination spectrum generating unit generating the illumination spectrum according to illumination light receiving data, the illumination light receiving data being data generated by imaging the illumination light, wherein: A spectrum acquisition unit acquires the illumination spectrum from the illumination spectrum calculation unit.

5. The biological information measuring device according to claim 1, further comprising: a display unit, on which the display video is displayed; as well as A reflected light imaging unit is used to image the reflected light to generate reflected light receiving data.

6. The biological information measuring device according to claim 5, further comprising: An illumination light imaging unit images the illumination light to generate illumination light reception data.

7. The biological information measuring device according to claim 5, wherein: The biological information measuring device is a head-mounted display which is worn on a head of a user and a display unit of the head-mounted display is arranged in front of eyes of the user.

8. The biological information measuring device according to claim 7, wherein: The reflected light imaging unit is arranged to face the measurement range.

9. The biological information measuring device according to claim 8, further comprising: an illumination light imaging unit, wherein the illumination light imaging unit images the illumination light to generate illumination light reception data, wherein: The illumination light imaging unit is arranged to face the display unit.

10. The biological information measuring device according to claim 5, wherein: The display unit and the reflected light imaging unit are each spaced apart from a user, and The biological information measuring apparatus further includes a measurement range identifying unit that identifies the measurement range based on an image obtained by imaging performed by the reflected light imaging unit.

11. The biological information measuring device according to claim 1, wherein: The analysis processing unit determines the biological information about the user using a biological information calculation model obtained by performing modeling on the irradiation spectrum, the reflection spectrum, and biological information in each type of biological information.

12. The biological information measuring device according to claim 11, wherein: The bioinformatics computing model is a prediction model generated by machine learning.

13. The biological information measuring device according to claim 1, wherein: The biological information is pulse rate, arterial blood oxygen saturation, skin moisture, body surface temperature, respiratory rate or blood pressure.

14. The biological information measuring device according to claim 1, wherein: The biometric information is a vein pattern or an iris pattern, and The analysis processing unit performs biometric authentication using the biometric information.

15. A biological information measuring program causing an information processing device to operate as a reflection spectrum generating unit and an analysis processing unit, the reflection spectrum generating unit generates a reflection spectrum which is a wavelength spectrum of reflected light obtained by irradiation light irradiated onto a measurement range being reflected from the measurement range, the irradiation light constituting a display video, the measurement range being a specific range on a body of a user, The analysis processing unit calculates biological information about the user based on an illumination spectrum and the reflection spectrum, the illumination spectrum being a wavelength spectrum of the illumination light.

16. A method for measuring biological information, comprising: generating a reflection spectrum which is a wavelength spectrum of reflected light obtained by irradiation light irradiated onto a measurement range, which constitutes a display video, being reflected from the measurement range, the measurement range being a specific range on a user's body; and Biometric information about the user is calculated based on an illumination spectrum and the reflection spectrum, the illumination spectrum being a wavelength spectrum of the illumination light.

Images