Biometric measurement device and biometric measurement method

Abstract

To provide a biomedical measuring device for measuring a state of an object by an inexpensive method compared to a conventional method.SOLUTION: A biomedical measuring device includes a light source for emitting light onto an object part of a living body, a light detection cell for receiving reflection light returning from the object part attributed to the emission of the light and outputting a signal, and a signal processing circuit. The signal processing circuit determines whether or not a first value exceeds a threshold on the basis of a brightness value of the object part based on the signal, or the first value corresponding to a value of biomedical measurement data on the living body based on the signal, and stops outputting the biomedical measurement data during a first period in which the first value is determined to exceed the threshold, and in a second period, which is a predetermined period after the end of the first period.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] The present disclosure relates to a biometric measurement device and a biometric measurement method. [Background technology]

[0002] In the field of biomeasurement, a method is used in which light is emitted toward an object and internal information of the object is obtained from the light that passes through the object. In this method, surface reflection components reflected from the surface of the object can become noise. By removing the noise caused by the surface reflection components, it is possible to accurately obtain the desired internal information.

[0003] Patent Document 1 discloses an imaging device that measures internal information of an object in a non-contact manner while suppressing noise due to surface reflection components. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2017-202328 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the above-mentioned conventional imaging device, if the object moves during measurement, the correction requires a high calculation cost.

[0006] The present disclosure provides a bioinstrumentation device that measures the state of an object using a method that is less expensive than conventional methods. [Means for solving the problem]

[0007] A biometric measurement device according to one aspect of the present disclosure includes a light source for irradiating a user's head with light, an image sensor, a control circuit for controlling the light source and the image sensor, and a signal processing circuit. The control circuit causes the light source to emit the light and causes the image sensor to detect at least a portion of the light reflected from the head due to the light irradiation and output an image signal. The signal processing circuit generates brain activity data indicating the state of the user's brain based on the image signal, and stops outputting the brain activity data based on at least one signal selected from the group consisting of the image signal and a sensor signal output from a sensor that detects changes in the user's surrounding environment that affect the brain activity data. [Effects of the Invention]

[0008] According to one aspect of the present disclosure, a bioinstrumentation device can measure internal information of an object using an inexpensive method even if the object moves during measurement or if the surrounding environment of the object changes. [Brief explanation of the drawings]

[0009] [Figure 1A] FIG. 1A is a diagram schematically illustrating an example of a bioinstrumentation device according to this embodiment. [Figure 1B] FIG. 1B is a diagram showing a schematic configuration example of one pixel of an image sensor. [Figure 1C] FIG. 1C is a diagram illustrating an example of the configuration of an image sensor. [Figure 1D] FIG. 1D is a diagram schematically illustrating an example of an operation within one frame in this embodiment. [Figure 1E] FIG. 1E is a flowchart showing an outline of the operation of the control circuit regarding the light source and the image sensor. [Figure 1F] FIG. 1F is a diagram schematically illustrating an optical signal that reaches an image sensor when rectangular pulse light emitted from a light source returns from a user. [Figure 1G] FIG. 1G is a diagram schematically illustrating an example of a timing chart when detecting a surface reflection component. [Figure 1H] FIG. 1H is a diagram schematically illustrating an example of a timing chart when detecting an internal scattering component. [Figure 2] FIG. 2 is a flowchart showing an example of a process for measuring biometric data of a user in this embodiment. [Figure 3A] FIG. 3A is a diagram illustrating an example of the operation of the process of measuring biometric data of a user in this embodiment. [Figure 3B] FIG. 3B is a diagram schematically showing the relationship between the change in the amount of body movement and the invalid period. [Figure 3C] FIG. 3C is a diagram schematically showing the relationship between the moving speed of the head and the invalid period. [Figure 3D] FIG. 3D is a diagram illustrating an example of the operation of the process of measuring biometric data of a user in this embodiment. [Figure 3E] FIG. 3E is a diagram schematically illustrating an example of a bioinstrumentation device according to this embodiment. [Figure 4] FIG. 4 is a diagram schematically illustrating an example of a bioinstrumentation device according to this embodiment. [Figure 5A] FIG. 5A is a flowchart showing an example of a process for measuring biometric data of a user in this embodiment. [Figure 5B] FIG. 5B is a diagram illustrating an example of the relationship between the difference value and the reliability. [Figure 6] FIG. 6 is a diagram illustrating an example of the operation of the process of measuring biometric data of a user in this embodiment. [Figure 7A] FIG. 7A is a diagram schematically illustrating an example of a bioinstrumentation device according to this embodiment. [Figure 7B] FIG. 7B is a diagram schematically showing an example of the arrangement of the components when the bioinstrumentation device is installed inside an automobile. [Figure 7C] FIG. 7C is a diagram schematically showing an example of the arrangement of each part when the biometric device is attached to a game machine or an attraction device and installed. [Figure 8]FIG. 8 is a flowchart showing an example of a process for measuring biometric data of a user in this embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of the operation of the process of measuring biometric data of a user in this embodiment. [Figure 10A] FIG. 10A is a diagram showing the relationship between time and the moving distance of the head. [Figure 10B] FIG. 10B is a graph showing the relationship between time and oxygenated hemoglobin concentration. DETAILED DESCRIPTION OF THE INVENTION

[0010] (Background to one aspect of the present disclosure) In the field of biometrics, it has long been known that changes in the user's state can affect measurement results. For example, when measuring brain activity information using near-infrared spectroscopy (NIRS), movement of the user's head is called body movement. Body movement has been thought to cause irregular fluctuations in signals indicating changes in cerebral blood flow.

[0011] When measuring brain activity information for research purposes, it is possible for the operator of the NIRS device to determine the irregularity of the signal, notify the user that "this measurement was an error," and prompt the user to measure again. However, it is not easy to use such a sequence for measuring a user's brain activity information on a daily basis.

[0012] Furthermore, when a measurement device measures brain activity information from the user's head in a non-contact manner, head movement changes the relative positional relationship between the measurement device and the user, which further increases the impact of changes in cerebral blood flow on the detection signal.

[0013] Patent Document 1 discloses a method for correcting positional deviations due to movement by pattern matching of cerebral blood flow distribution between frames. The pattern of cerebral blood flow distribution has a low spatial frequency. In other words, the cerebral blood flow distribution shows a spatially gentle distribution. For this reason, it is not necessarily easy to detect the feature points required for matching. In addition, it is computationally expensive to restore the cerebral blood flow distribution in all frames as if there was no head movement.

[0014] Based on the above investigations, the present inventors have come up with the bioinstrumentation device described in the following items.

[0015] [Item 1] A biometric measurement device according to a first aspect of the present invention includes a light source for irradiating light onto a user's head, an image sensor, a control circuit for controlling the light source and the image sensor, and a signal processing circuit. The control circuit causes the light source to emit the light and causes the image sensor to detect at least a portion of the light reflected from the head due to the light irradiation and output an image signal. The signal processing circuit generates brain activity data indicating the state of the user's brain based on the image signal, and stops outputting the brain activity data based on at least one signal selected from the group consisting of the image signal and a sensor signal output from a sensor that detects changes in the user's surrounding environment that affect the brain activity data.

[0016] [Item 2] In the biomeasuring device according to the first item, the light may be pulsed light, and the control circuit may cause the image sensor to output, as the image signal, a first signal obtained by detecting components contained in the reflected pulsed light during a period from when the intensity of the reflected pulsed light returning from the head due to irradiation of the pulsed light starts to decrease until the decrease ends.

[0017] [Item 3] In the biomeasurement device according to the first or second item, the control circuit may cause the light source to repeatedly emit the light and the image sensor to repeatedly output the image signal for a predetermined period of time, and the signal processing circuit may further calculate a first value based on at least one selected from the group consisting of the image signal and the sensor signal, and may stop outputting the brain activity data during a first period of the predetermined period in which the first value satisfies a predetermined condition.

[0018] [Item 4] In the biomeasurement device according to the third aspect, the signal processing circuit may output a signal indicating that the brain activity data is invalid during the first period.

[0019] [Item 5] In the biomeasurement device according to the third aspect, the signal processing circuit may output, during the first period, the same brain activity data as the brain activity data generated before the first period.

[0020] [Item 6] In the biomeasurement device according to the third item, the signal processing circuit may output data obtained by interpolating the brain activity data generated before the first period and the brain activity data generated after the first period as the brain activity data for the first period.

[0021] [Item 7] In the biomeasurement device according to the third item, the signal processing circuit may stop outputting the brain activity data during at least one period selected from the group consisting of a second period before the start of the first period and a third period after the end of the first period, in addition to the first period.

[0022] [Item 8] In the bioinstrumentation device according to the seventh aspect, the third period may be longer than the second period.

[0023] [Item 9] In the biomeasurement device according to any one of the third to eighth items, the frequency at which the first value is calculated may be equal to or greater than the frequency at which the brain activity data is generated.

[0024] [Item 10] In the biomeasurement device according to any one of items 1 to 9, the light is pulsed light, and the control circuit causes the image sensor to output, as the image signal, a second signal obtained by detecting a component contained in the reflected pulsed light returned from the head due to irradiation of the pulsed light before the intensity of the reflected pulsed light begins to decrease, and the signal processing circuit may stop outputting the brain activity data based on the second signal.

[0025] [Item 11] In the biomeasurement device according to the tenth item, the signal processing circuit may further calculate the amount of displacement of the head from a reference position or the speed of movement of the head based on the second signal, and stop outputting the brain activity data when the absolute value of the amount of displacement or the absolute value of the speed of movement exceeds a threshold value.

[0026] [Item 12] In the biomeasurement device according to the tenth item, the signal processing circuit may further calculate the brightness value of the head or the rate of change of the brightness value of the head based on the second signal, and stop outputting the brain activity data when the absolute value of the brightness value or the absolute value of the rate of change of the brightness value exceeds a threshold value.

[0027] [Item 13] In the biomeasurement device according to the tenth item, the signal processing circuit may further calculate the area of ​​a predetermined region in the head based on the second signal, and stop outputting the brain activity data if the area is smaller than a threshold value.

[0028] [Item 14] In the biomeasurement device according to any one of items 1 to 9, the signal processing circuit may further calculate a second value using the brain activity data, and stop outputting the brain activity data when the absolute value of the rate of change of the second value exceeds a threshold value.

[0029] [Item 15] In the bioinstrumentation device according to the first aspect, the sensor may be an acceleration sensor installed in the surrounding environment.

[0030] [Item 16] In the bioinstrumentation device according to the first aspect, the sensor may be an illuminance sensor installed in the surrounding environment.

[0031] [Item 17] In the biometric measurement device according to the first item, the sensor may be at least one selected from the group consisting of a steering angle sensor, a gear position sensor, and a speed sensor arranged in a vehicle driven by the user.

[0032] [Item 18] A biometric measurement device according to an eighteenth item includes a light source for irradiating light onto a user's head, an image sensor, a control circuit for controlling the light source and the image sensor, and a signal processing circuit. The control circuit causes the light source to emit the light and causes the image sensor to detect at least a portion of the light reflected from the head due to the light irradiation and output an image signal. The signal processing circuit generates brain activity data indicating the state of the user's brain based on the image signal, calculates reliability of the brain activity data based on at least one selected from the group consisting of the image signal and a sensor signal output from a sensor that detects changes in the user's surrounding environment that affect the brain activity data, and outputs reliability data indicating the reliability.

[0033] [Item 19] In the biomeasurement device according to the eighteenth aspect, the signal processing circuit may output the reliability data together with the brain activity data.

[0034] [Item 20] In the biomeasuring device according to the 18th or 19th item, the signal processing circuit may further calculate a first value based on at least one selected from the group consisting of the image signal and the sensor signal, and may calculate a lower reliability as the first value deviates from a preset value.

[0035] [Item 21] In the biomeasuring device according to the 18th or 19th item, the signal processing circuit may further calculate a first value based on at least one selected from the group consisting of the image signal and the sensor signal, and the longer the period during which the first value exceeds a predetermined value, the lower the reliability calculated.

[0036] [Item 22] In the biomeasurement device according to the 18th or 19th item, the signal processing circuit may further calculate a first value based on at least one selected from the group consisting of the image signal and the sensor signal, and may calculate a lower reliability the longer the period during which the first value exceeds a predetermined value during a certain period.

[0037] [Item 23] In the biometric measurement device according to any one of items 18 to 22, the sensor may be at least one selected from the group consisting of a steering angle sensor, a gear position sensor, and a speed sensor arranged in a vehicle driven by the user.

[0038] [Item 24] In the biomeasuring device according to any one of items 18 to 23, the light is pulsed light, and the control circuit causes the image sensor to output, as the image signal, a second signal obtained by detecting a component contained in the reflected pulsed light returning from the head due to irradiation of the pulsed light before the intensity of the reflected pulsed light begins to decrease, and the signal processing circuit may calculate the reliability based on the second signal.

[0039] [Item 25] In the biomeasurement device according to the first or second item, the control circuit may cause the light source to repeatedly emit the light and the image sensor to repeatedly output the image signal for a predetermined period of time, and the signal processing circuit may further calculate a first value based on at least one selected from the group consisting of the image signal and the sensor signal, and may stop outputting the brain activity data from after a delay time has elapsed since the start of a first period in which the first value satisfies a predetermined condition within the predetermined period, until the end of the first period.

[0040] [Item 26] The biometric measurement method according to the 26th item includes causing a light source to emit light that illuminates a user's head; causing an image sensor to detect at least a portion of the reflected light returning from the head due to the light irradiation and outputting an image signal; generating brain activity data indicating the state of the user's brain based on the image signal; and stopping the output of the brain activity data based on at least one selected from the group consisting of the image signal and a sensor signal output from a sensor that detects changes in the user's surrounding environment that affect the brain activity data.

[0041] [Item 27] The biometric measurement method according to the 27th item includes causing a light source to emit light that illuminates a user's head; causing an image sensor to detect at least a portion of the reflected light returning from the head due to the light irradiation and outputting an image signal; generating brain activity data indicating the state of the user's brain based on the image signal; calculating the reliability of the brain activity data based on at least one selected from the group consisting of the image signal and a sensor signal output from a sensor that detects changes in the user's surrounding environment that affect the brain activity data, and outputting reliability data indicating the reliability.

[0042] [Item 28] The program related to the 28th item is a program used in a biomeasurement device, the biomeasurement device comprising a light source that emits light toward a target part of a user, an image sensor, a control circuit that controls the light source and the image sensor, and a signal processing circuit that processes a signal output from the image sensor, wherein the control circuit causes the light source to emit light and the image sensor to detect at least a portion of the light emitted from the light source and returned from the target part of the user and output an image signal, and the program causes the signal processing circuit to generate biomeasurement data indicating the user's condition based on the image signal output from the image sensor, and decide whether to output the biomeasurement data based on a sensor signal output from the image sensor and / or another sensor that detects changes in the user's surrounding environment that affect the biomeasurement data, or calculate the reliability of the biomeasurement data based on the sensor signal and output reliability data indicating the reliability.

[0043] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, and component placement positions shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components.

[0044] In this disclosure, all or part of a circuit, unit, device, component, or part, or all or part of a functional block in a block diagram, may be implemented by one or more electronic circuits, including a semiconductor device, a semiconductor integrated circuit (IC), or an LSI (large scale integration). An LSI or IC may be integrated on a single chip or may be configured by combining multiple chips. For example, functional blocks other than memory elements may be integrated on a single chip. While the terms LSI and IC are used here, the term may be changed depending on the degree of integration, and may be referred to as a system LSI, a VLSI (very large scale integration), or an ULSI (ultra large scale integration). A field programmable gate array (FPGA), which is programmable after LSI fabrication, or a reconfigurable logic device, which can reconfigure connections within an LSI or set up circuit partitions within an LSI, may also be used for the same purpose.

[0045] Furthermore, all or part of the functions or operations of a circuit, unit, device, component, or section may be implemented by software processing. In this case, the software is recorded on one or more non-transitory recording media such as ROMs, optical disks, hard disk drives, etc., and when the software is executed by a processor, the functions specified in the software are performed by the processor and peripheral devices. A system or device may include one or more non-transitory recording media on which the software is recorded, a processor, and necessary hardware devices, such as interfaces.

[0046] Hereinafter, the bioinstrumentation device according to this embodiment will be specifically described with reference to the drawings.

[0047] (First embodiment) [1. Biometric measurement device] First, the configuration and operation of a bioinstrumentation device 10 according to the first embodiment will be described with reference to FIGS. 1A to 3. FIG.

[0048] 1A is a diagram schematically illustrating an example of a biomeasurement device 10 according to this embodiment. The biomeasurement device 10 includes an imaging unit 121, a measurement unit 110, and a signal processing circuit 122. The imaging unit 121 includes a light source 101, an image sensor 102 including a photoelectric conversion unit 103 and a charge accumulation unit 104, a control circuit 105 including a light source control unit 106 and a sensor control unit 107, and an image signal acquisition unit 108. The signal processing circuit 122 includes a biomeasurement data generation unit 109 and an output determination unit 111.

[0049] [1-1.Light source 101] The light source 101 emits light toward a target area of ​​the user 100. The target area of ​​the user 100 is, for example, the head, more specifically, the forehead. The light emitted from the light source 101 and reaching the user 100 is divided into a surface reflection component I1 that is reflected on the surface of the user 100 and an internal scattering component I2 that is scattered inside the user 100. The internal scattering component I2 is a component that is reflected once, scattered, or multiple times inside the living body. When light is irradiated onto the forehead of the user 100, the internal scattering component I2 refers to the component that reaches a region approximately 8 mm to 16 mm behind the surface of the forehead, such as the brain, and then returns to the biometric measurement device 10. The surface reflection component I1 includes three components: a direct reflection component, a diffuse reflection component, and a scattered reflection component. The direct reflection component is a reflection component whose angle of incidence and angle of reflection are equal. The diffuse reflection component is a component that is diffusely reflected due to the unevenness of the surface. The scattered reflection component is a component that is scattered and reflected by internal tissue near the surface. When light is emitted toward the forehead of user 100, the scattered reflection component is a component that is scattered and reflected within the epidermis. Hereinafter, in this disclosure, the surface reflection component I1 that is reflected from the surface of user 100 is assumed to include these three components. The surface reflection component I1 and the internal scattering component I2 change direction due to reflection or scattering, and some of them reach the image sensor 102.

[0050] First, a method for acquiring the internally scattered component I2 will be described. In accordance with instructions from the light source control unit 106, the light source 101 repeatedly emits pulsed light multiple times at predetermined time intervals or at predetermined timing. The pulsed light emitted from the light source 101 may be, for example, a rectangular wave with a fall period close to zero. In this specification, the "fall period" refers to the period from when the intensity of the pulsed light starts to decrease until the decrease ends. Generally, light incident on the user 100 propagates through the user 100 via various paths and exits the surface of the user 100 with a time lag. Therefore, the rear end of the internally scattered component I2 of the pulsed light has a spread. When the target area is the forehead, the spread of the rear end of the internally scattered component I2 is approximately 4 ns. Taking this into consideration, the fall period of the pulsed light can be set to, for example, half of that, or less, 2 ns or less. The fall period may even be half that, or less, 1 ns or less. The rise period of the pulsed light emitted from the light source 101 is arbitrary. In this specification, the "rise period" refers to the period from when the intensity of the pulsed light starts to increase until the increase ends. In this embodiment, the falling portion of the pulsed light is used to detect the internal scattering component I2, and the rising portion is not used. The rising portion of the pulsed light can be used to detect the surface reflection component I1. The light source 101 can be, for example, a laser such as an LD. The light emitted from the laser has a steep time response characteristic in which the falling portion of the pulsed light is approximately perpendicular to the time axis.

[0051] The wavelength of the light emitted from the light source 101 may be any wavelength within the wavelength range of, for example, 650 nm to 950 nm. This wavelength range falls within the red to near-infrared wavelength range. In this specification, the term "light" is used to refer to not only visible light but also infrared light. This wavelength range is called the "biological window" and has the property of being relatively difficult to absorb by water and skin in a living body. When detecting a living body, using light within this wavelength range can improve detection sensitivity. When detecting changes in blood flow in the skin and brain of the user 100, as in this embodiment, the light used is considered to be absorbed mainly by oxygenated hemoglobin (HbO2) and deoxygenated hemoglobin (Hb). Oxygenated hemoglobin and deoxygenated hemoglobin have different wavelength dependences of light absorption. Generally, changes in blood flow cause changes in the concentrations of oxygenated hemoglobin and deoxygenated hemoglobin. This changes the degree of light absorption. Therefore, changes in blood flow also cause changes in the amount of detected light over time.

[0052] The light source 101 may emit light of two or more wavelengths included in the above wavelength range. Such light of multiple wavelengths may be emitted from multiple light sources, respectively.

[0053] In the bioinstrumentation device 10 of this embodiment, a light source 101 designed with consideration of the impact on the retina can be used to measure the user 100 in a non-contact manner. For example, a light source 101 that satisfies Class 1 of the laser safety standards established in various countries can be used. When Class 1 is satisfied, light with such low illuminance that the accessible emission limit (AEL) is below 1 mW is emitted toward the user 100. Note that the light source 101 itself does not have to satisfy Class 1. For example, the laser safety standard Class 1 may be satisfied by placing a diffuser or ND filter in front of the light source 101 to diffuse or attenuate the light.

[0054] The pulsed light emitted from the light source 101 does not need to be ultrashort pulsed light. The pulse width can be any value. When light is applied to the forehead to measure cerebral blood flow, the amount of light of the internally scattered component I2 is extremely small, ranging from several thousandths to several tens of thousands of times smaller than that of the surface-reflected component I1. Furthermore, considering laser safety standards, the small amount of light emitted makes it difficult to detect the internally scattered component I2. Therefore, if the light source 101 emits pulsed light with a relatively large pulse width, the accumulated amount of the internally scattered component I2, which involves a time delay, can be increased, the amount of detected light can be increased, and the signal-to-noise ratio can be improved.

[0055] The light source 101 emits pulsed light with a pulse width of, for example, 3 ns or more. Generally, the temporal spread of light scattered within biological tissue such as the brain is about 4 ns.

[0056] The light source 101 may emit pulsed light with a pulse width of 5 ns or more, or even 10 ns or more. On the other hand, if the pulse width is too large, the amount of unused light increases and is wasted. For this reason, the light source 101 emits pulsed light with a pulse width of, for example, 50 ns or less. Alternatively, the light source 101 may emit pulsed light with a pulse width of 30 ns or less, or even 20 ns or less.

[0057] The irradiation pattern of the light source 101 may have, for example, a uniform intensity distribution within the irradiation area. In the biomeasurement device 10 of this embodiment, the surface reflection component I1 can be separated and reduced from the internal scattering component I2 over time. For this reason, a light source 101 with an emission pattern having a uniform intensity distribution can be used. An irradiation pattern having a uniform intensity distribution may be formed by diffusing the light emitted from the light source 101 with a diffuser.

[0058] Unlike conventional techniques, this embodiment can also detect the internal scattering component I2 emitted from the position on the user 100 where the light from the light source 101 is irradiated. By irradiating the user 100 with light over a spatially wide range, the measurement resolution can also be increased.

[0059] [1-2. Image Sensor 102] The image sensor 102 detects at least a portion of the reflected pulsed light emitted from the light source 101 and returned from the target area of ​​the user 100. The image sensor 102 outputs one or more image signals corresponding to the intensity of the detected light. The image signal acquisition unit 108 acquires the image signals output from the image sensor 102. The image signal acquisition unit 108 sends the acquired image signals to a biomeasurement data generation unit 109 and a measurement unit 110, which will be described later. When brain activity data indicating the state of brain activity is generated from the image signal as biomeasurement data, the image signal is a signal corresponding to the intensity and included in at least a portion of the falling period of the reflected pulsed light.

[0060] The image sensor 102 includes a photoelectric conversion unit 103 including a plurality of photoelectric conversion elements, and a charge accumulation unit 104. Specifically, the image sensor 102 has a plurality of photodetection cells arranged two-dimensionally, and simultaneously acquires two-dimensional information of the user 100. In this specification, the photodetection cells are also referred to as "pixels." The image sensor 102 may be any image sensor, such as a CCD image sensor or a CMOS image sensor.

[0061] The image sensor 102 is equipped with an electronic shutter. The electronic shutter is a circuit that controls the timing of imaging. In this embodiment, the sensor control unit 107 in the control circuit 105 has the function of the electronic shutter. The electronic shutter controls the period of one signal accumulation, in which received light is converted into an effective electrical signal and accumulated, and the period during which signal accumulation is stopped. The signal accumulation period can also be referred to as the "exposure period" or "shooting period." In the following description, the width of the exposure period may be referred to as the "shutter width." The time from the end of one exposure period to the start of the next exposure period may be referred to as the "non-exposure period." Hereinafter, the exposed state may be referred to as "OPEN," and the stopped exposure state may be referred to as "CLOSE."

[0062] The image sensor 102 can adjust the exposure and non-exposure periods by using an electronic shutter in the subnanosecond range, for example, from 30 ps to 1 ns. With the bioinstrumentation device 10 of this embodiment, it is not necessary to correct the amount of light from the subject, at least when generating brain activity data. Therefore, the shutter width does not necessarily need to be larger than the pulse width. Therefore, the shutter width can be set to a value between 1 ns and 30 ns, for example. The bioinstrumentation device 10 of this embodiment can reduce the shutter width. This reduces the influence of dark current contained in the detection signal.

[0063] When detecting information such as cerebral blood flow by emitting light toward the forehead of the user 100, the internal attenuation rate of light is extremely large. For example, compared to the incident light, the emitted light may attenuate to approximately one millionth. As a result, a single pulse of light may not be enough to detect the internal scattering component I2. Irradiation under Class 1 laser safety standards requires a very weak amount of light. In this case, the light source 101 emits multiple pulses of light, and the image sensor 102 is exposed multiple times by the electronic shutter accordingly. This allows the detection signals to be integrated to improve sensitivity.

[0064] An example of the configuration of the image sensor 102 will be described below.

[0065] The image sensor 102 may include a plurality of pixels arranged two-dimensionally on an imaging surface. Each pixel may include a photoelectric conversion element such as a photodiode and one or more charge accumulation units. Below, an example will be described in which each pixel includes a photoelectric conversion element and two charge accumulation units. The photoelectric conversion element generates signal charges according to the amount of received light through photoelectric conversion. One of the two charge accumulation units accumulates signal charges generated by the surface reflection component I1 of the pulsed light, and the other accumulates signal charges generated by the internal scattering component I2 of the pulsed light.

[0066] The control circuit 105 causes the light source 101 to emit one or more pulsed lights to acquire the internal scattering component I2. The control circuit 105 causes the image sensor 102 to detect, for each pixel of the image sensor 102, a component included in the falling edge period of each pulsed light returned from the target area of ​​the user 100. The component includes the internal scattering component I2. The control circuit 105 causes the image sensor 102 to output a signal obtained by the detection. In this embodiment, an image signal based on the internal scattering component I2 is used to generate biomeasurement data. Here, the light source 101 may emit light of two different wavelengths.

[0067] The control circuit 105 controls the light source 101 to emit one or more pulsed lights to acquire the surface reflection component I1. The control circuit 105 controls the image sensor 102 to detect, for each pixel of the image sensor 102, the components of each pulsed light returned from the target area of ​​the user 100, which are included before the falling edge of the pulsed light. The components include the surface reflection component I1. Note that "before the falling edge" refers to the time before the intensity of each pulsed light starts to decrease. The control circuit 105 controls the image sensor 102 to output a signal obtained by this detection. Detecting the components of each pulsed light included before the falling edge can improve the signal-to-noise ratio of the image signal. In this specification, "detecting the components of each pulsed light included before the falling edge" includes, for example, detecting the components of each pulsed light included in the rising edge of the pulsed light or detecting the entirety of each pulsed light. In this embodiment, the image signal based on the surface reflection component I1 is used to measure a condition related to the validity of biometric data.

[0068] 1B is a diagram showing a schematic configuration example of one pixel 201 of the image sensor 102. Note that FIG. 1B shows the configuration of one pixel 201 in a schematic manner and does not necessarily reflect the actual structure. The pixel 201 shown in FIG. 1B includes a photodiode 203 that performs photoelectric conversion, a first floating diffusion layer 204, a second floating diffusion layer 205, a third floating diffusion layer 206, and a fourth floating diffusion layer 207 that are charge accumulation sections, and a drain 202 that discharges signal charges.

[0069] Photons incident on each pixel due to the emission of one pulse of light are converted into signal electrons, which are signal charges, by the photodiode 203. The converted signal electrons are either discharged to the drain 202 or distributed to any of the first floating diffusion layer 204 to the fourth floating diffusion layer 207 in accordance with a control signal input from the control circuit 105.

[0070] The light source 101 emits pulsed light, the signal charges are accumulated in the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207, and the signal charges are discharged to the drain 202. This repetitive operation is fast and can be repeated tens of thousands to hundreds of millions of times within the time of one frame of a moving image. The time of one frame is, for example, approximately 1 / 30 seconds. The pixel 201 ultimately generates and outputs four image signals based on the signal charges accumulated in the first floating diffusion layer 204 to the fourth floating diffusion layer 207.

[0071] In this example, the control circuit 105 causes the light source 101 to repeatedly emit pulsed light having a wavelength λ1 and pulsed light having a wavelength λ2 in sequence. By selecting two wavelengths, λ1 and λ2, that have different absorption rates in the internal tissues of the user 100, the condition of the user 100 can be analyzed. For example, a wavelength longer than 805 nm may be selected as the wavelength λ1, and a wavelength shorter than 805 nm may be selected as the wavelength λ2. This makes it possible to detect changes in the oxygenated hemoglobin concentration and the deoxygenated hemoglobin concentration in the blood of the user 100.

[0072] The control circuit 105 first causes the light source 101 to emit a pulsed light beam with a wavelength λ1. The control circuit 105 causes the first floating diffusion layer 204 to accumulate signal charges during a first period during which an internally scattered component I2 of the pulsed light beam with wavelength λ1 is incident on the photodiode 203. In order to acquire the internally scattered component I2, the pulsed light beam is emitted at a predetermined timing. This pulsed light beam is referred to as a first pulsed light beam.

[0073] Next, the control circuit 105 accumulates signal charges in the second floating diffusion layer 205 during a second period in which the surface reflection component I1 of the pulsed light with wavelength λ1 is incident on the photodiode 203. To acquire the surface reflection component I1, a pulsed light is emitted at a predetermined timing different from that for acquiring the internal scattering component. This pulsed light is referred to as the second pulsed light.

[0074] Next, the control circuit 105 causes the light source 101 to emit a pulsed light having a wavelength λ2. The control circuit 105 causes the third floating diffusion layer 206 to accumulate signal charges during a third period in which the internally scattered component I2 of the pulsed light having a wavelength λ2 is incident on the photodiode 203.

[0075] Subsequently, the control circuit 105 causes the fourth floating diffusion layer 207 to accumulate signal charges during a fourth period in which the surface reflection component I1 of the pulsed light having the wavelength λ2 is incident on the photodiode 203.

[0076] In this manner, the control circuit 105 sequentially accumulates signal charges from the photodiode 203 in the first floating diffusion layer 204 and the second floating diffusion layer 205 at predetermined time intervals after starting the emission of pulsed light with wavelength λ1. Then, the control circuit 105 sequentially accumulates signal charges from the photodiode 203 in the third floating diffusion layer 206 and the fourth floating diffusion layer 207 at predetermined time intervals after starting the emission of pulsed light with wavelength λ2. This operation is repeated multiple times. To estimate the amount of ambient light and turbulence light, a period during which signal charges are accumulated in other floating diffusion layers (not shown) while the light source 101 is turned off may be provided. By subtracting the signal charge amounts in the other floating diffusion layers 204 to 207 from the signal charge amounts in the first to fourth floating diffusion layers 204 to 207, a signal from which the ambient light and turbulence light components have been removed can be obtained.

[0077] In this embodiment, the number of charge accumulation units is four, but may be two or more depending on the purpose. For example, when only one type of wavelength is used, the number of charge accumulation units may be two. Furthermore, when only one type of wavelength is used and the surface reflection component I1 is not detected, the number of charge accumulation units per pixel may be one. Furthermore, even when two or more types of wavelengths are used, the number of charge accumulation units may be one if imaging using each wavelength is performed in a different frame. Furthermore, as will be described later, the number of charge accumulation units may be one if detection of the surface reflection component I1 and detection of the internal scattering component I2 are performed in different frames.

[0078] FIG. 1C is a diagram showing an example of the configuration of the image sensor 102. In the example shown in FIG. 1C, the area surrounded by a two-dot chain line frame corresponds to one pixel 201. The pixel 201 includes one photodiode. Although FIG. 1C shows only four pixels arranged in two rows and two columns, in reality, many more pixels may be arranged. The pixel 201 includes four floating diffusion layers, namely, the first floating diffusion layer 204 to the fourth floating diffusion layer 207. The signals accumulated in the four floating diffusion layers, namely, the first floating diffusion layer 204 to the fourth floating diffusion layer 207, are treated as if they were four pixel signals of a typical CMOS image sensor and are output from the image sensor 102.

[0079] Each pixel 201 has four signal detection circuits. Each signal detection circuit includes a source follower transistor 309, a row selection transistor 308, and a reset transistor 310. In this example, the reset transistor 310 corresponds to the drain 202 shown in FIG. 1B, and a pulse input to the gate of the reset transistor 310 corresponds to a drain discharge pulse. Each transistor is, for example, but not limited to, a field-effect transistor formed on a semiconductor substrate. As shown in the figure, one of the input terminal and output terminal of the source follower transistor 309 is connected to one of the input terminal and output terminal of the row selection transistor 308. The one of the input terminal and output terminal of the source follower transistor 309 is typically the source. The one of the input terminal and output terminal of the row selection transistor 308 is typically the drain. The gate, which is the control terminal of the source follower transistor 309, is connected to the photodiode 203. The signal charge of holes or electrons generated by the photodiode 203 is accumulated in a floating diffusion layer, which is a charge accumulation portion between the photodiode 203 and the source follower transistor 309 .

[0080] Although not shown in FIG. 1C , the first floating diffusion layer 204 to the fourth floating diffusion layer 207 are connected to the photodiode 203. A switch may be provided between the photodiode 203 and each of the first floating diffusion layer 204 to the fourth floating diffusion layer 207. This switch switches the conduction state between the photodiode 203 and each of the first floating diffusion layer 204 to the fourth floating diffusion layer 207 in response to a signal accumulation pulse from the control circuit 105. This controls the start and stop of accumulation of signal charge in each of the first floating diffusion layer 204 to the fourth floating diffusion layer 207. The electronic shutter in this embodiment has a mechanism for such exposure control.

[0081] The signal charges accumulated in the first to fourth floating diffusion layers 204 to 207 are read out by the row selection circuit 302 turning on the gate of the row selection transistor 308. At this time, the current flowing from the source follower power supply 305 to the source follower transistor 309 and the source follower load 306 is amplified according to the signal potential of the first to fourth floating diffusion layers 204 to 207. An analog signal resulting from this current read out from the vertical signal line 304 is converted into digital signal data by the analog-to-digital conversion circuit 307 connected to each column. This digital signal data is read out column by column by the column selection circuit 303 and output from the image sensor 102. After reading out one row, the row selection circuit 302 and the column selection circuit 303 read out the signal charges in the floating diffusion layers of all rows in the same manner. After reading out all the signal charges, the control circuit 105 resets all the floating diffusion layers by turning on the gate of the reset transistor 310. This completes the imaging of one frame. By repeating the same high-speed imaging of frames, the image sensor 102 completes the imaging of a series of frames.

[0082] In this embodiment, an example of a CMOS type image sensor 102 has been described, but the image sensor 102 may be a CCD type, a single photon counting element, or an amplification type image sensor such as an EMCCD or ICCD.

[0083] 1D is a diagram schematically illustrating an example of operation within one frame in this embodiment. As shown in FIG. 1D, within one frame, the emission of pulsed light having a wavelength λ1 and the emission of pulsed light having a wavelength λ2 may be alternately switched multiple times. This reduces the time difference between the acquisition timing of detection images using the two wavelengths, making it possible to capture images using pulsed light of two wavelengths almost simultaneously.

[0084] In this embodiment, the image sensor 102 detects both the surface reflection component I1 and the internal scattering component I2 of the pulsed light. Biometric data of the user 100 can be generated from temporal or spatial changes in the internal scattering component I2. Meanwhile, data related to the validity of the biometric data can be measured from temporal or spatial changes in the surface reflection component I1.

[0085] In this specification, a signal related to the validity of biometric data may be referred to as a "validity signal."

[0086] [1-3. Control circuit 105] The control circuit 105 controls the above-described operations of the light source 101 and the image sensor 102. Specifically, the control circuit 105 adjusts the time difference between the emission timing of the pulsed light from the light source 101 and the shutter timing of the image sensor 102. Hereinafter, this time difference may be referred to as the "phase" or "phase delay." The "emission timing" of the pulsed light from the light source 101 is the time at which the pulsed light emitted from the light source 101 starts to rise. The control circuit 105 may adjust the phase by changing the emission timing, or may adjust the phase by changing the shutter timing.

[0087] The control circuit 105 may be configured to remove an offset component from the signal detected by the light receiving element of the image sensor 102. The offset component is a signal component due to ambient light such as sunlight or fluorescent light, or disturbance light. For example, the offset component due to ambient light or disturbance light can be estimated by detecting a signal using the image sensor 102 with the light source 101 turned off and not emitting light.

[0088] The control circuit 105 may be an integrated circuit including a processor such as a central processing unit (CPU) or a microcomputer, and a memory. The control circuit 105 executes a program stored in the memory to perform, for example, adjustment of emission timing and shutter timing, estimation of offset components, and removal of offset components.

[0089] Fig. 1E is a flowchart showing an outline of the operation of the control circuit 105 regarding the light source 101 and the image sensor 102. The control circuit 105 includes a light source control unit 106 and a sensor control unit 107, and generally performs the operation shown in Fig. 1G, which will be described later. Note that the operation for detecting only the internal scattering component I2 will be described here.

[0090] In step S101, the light source control unit 106 first causes the light source 101 to emit pulsed light for a predetermined period of time. At this time, the electronic shutter of the image sensor 102 is in a state where exposure is stopped. The sensor control unit 107 causes the electronic shutter to stop exposure until the period in which a portion of the pulsed light is reflected by the surface of the user 100 and reaches the image sensor 102 is completed. Next, in step S102, the sensor control unit 107 causes the electronic shutter to start exposure at the timing when another portion of the pulsed light is scattered inside the user 100 and reaches the image sensor 102. After the predetermined time has elapsed, in step S103, the sensor control unit 107 causes the electronic shutter to stop exposure. Next, in step S104, the control circuit 105 determines whether the number of times the above signal accumulation has been performed has reached a predetermined number of times. If this determination is No, steps S101 to S103 are repeated until the determination is Yes. If the determination in step S104 is Yes, in step S105, the sensor control unit 107 causes the image sensor 102 to generate and output a signal representing an image based on the signal charges accumulated in each floating diffusion layer.

[0091] By the above operation, the components of light scattered inside the measurement object can be detected with high sensitivity. Note that multiple emission and exposure are not essential and are performed as needed.

[0092] [1-4. Measurement unit 110] The measurement unit 110 measures a validity signal indicating the validity of the biometric data, and sends the measurement result as a signal to the output determination unit 111, which will be described later.

[0093] In this embodiment, the validity signal is measured from an image containing temporal or spatial variations of the surface reflectance component I1, which is acquired from the image sensor 102.

[0094] The measurement unit 110 may include an arithmetic circuit that performs arithmetic processing such as image processing. Such an arithmetic circuit can be realized by, for example, a digital signal processor (DSP), a programmable logic device (PLD) such as a field programmable gate array (FPGA), or a central processing unit (CPU) or a graphics processing unit (GPU) in combination with a computer program.

[0095] In another embodiment described below, the validity signal may be measured from an image containing temporal or spatial variations in the internal scattering component I2.

[0096] In another embodiment described later, the validity signal may be measured from a source other than the image output from the image sensor 102. In this case, the measurement unit 110 may include a sensor disposed inside or outside the biometric device 10. The sensor measures changes in the user 100's surrounding environment and outputs a sensor signal indicating the changes. The sensor signal may be, for example, a signal indicating the amount of change in a physical change in the surrounding environment that affects the biometric data. The sensor signal may be, for example, a signal indicating that the surrounding environment is in a specific state that affects the biometric data. "Affecting the biometric data" includes the introduction of noise into the biometric data or the biometric data becoming unmeasurable. The sensor may be at least one selected from the group consisting of an illuminance sensor, an acceleration sensor, a speed sensor, a steering angle sensor, and a gear position sensor.

[0097] [1-4. Signal processing circuit 122] The signal processing circuit 122 processes the signal output from the image sensor 102. The signal processing circuit 122 includes a biometric data generation unit 109 and an output determination unit 111. In the example shown in Fig. 1A, the signal processing circuit 122 and the measurement unit 110 are separate, but they may also be integrated.

[0098] The biometric data generation unit 109 generates biometric data of the user 100 based on the image signal output from the image sensor 102. When the biometric data is brain activity data of the user 100, the biometric data generation unit 109 processes the image signal including the temporal or spatial change of the internal scattering component I2 to generate moving image data indicating the change in cerebral blood flow over time, and sends a signal based on the generation result to the output determination unit 111. The change in cerebral blood flow over time is, for example, the change in the concentration of oxygenated hemoglobin and / or deoxygenated hemoglobin over time.

[0099] When the biometric data is brain activity data of the user 100, the biometric data generating unit 109 may generate other data related to cerebral blood flow, not limited to video data of cerebral blood flow. The data related to cerebral blood flow is, for example, the psychological state of the user 100 estimated from the video data of cerebral blood flow.

[0100] It is known that changes in cerebral blood flow or blood components such as hemoglobin are closely related to human neural activity. For example, changes in neural cell activity in response to changes in human emotions result in changes in cerebral blood flow or blood components. Therefore, if biological information such as changes in cerebral blood flow or blood components can be measured, the psychological state of user 100 can be estimated. The psychological state of user 100 relates to, for example, mood, emotion, health status, or thermal sensation. Mood, for example, is pleasant or unpleasant. Emotion, for example, is relief, anxiety, sadness, or anger. Health status, for example, is vitality or fatigue. Thermal sensation, for example, is hot, cold, or muggy. Furthermore, derived indicators representing the degree of brain activity are also included in the psychological state. Such indicators include, for example, proficiency, mastery, or concentration. In this specification, such data related to cerebral blood flow are collectively referred to as brain activity data.

[0101] The output determination unit 111 determines whether or not to output the biometric data sent from the biometric data generation unit 109, based on the measurement results sent from the measurement unit 110. The measurement results include values ​​calculated by the measurement unit 110 based on the image signals output from the image sensor 102 and / or the sensor signals output from the sensors described above.

[0102] At the timing to stop output of biometric data, the output determination unit 111 outputs a signal indicating that the biometric data at that timing is invalid, or continues to output the same data as the last valid biometric data, or at the timing when the biometric data becomes valid again, the output determination unit 111 outputs data interpolated from the last valid biometric data.

[0103] Like the measurement unit 110, the signal processing circuit 122 may include an arithmetic circuit for performing arithmetic processing such as image processing. Such an arithmetic circuit may be realized, for example, by a programmable logic device (PLD) such as a digital signal processor (DSP) or a field programmable gate array (FPGA), or by a combination of a central processing unit (CPU) or a graphics processing unit (GPU) for image processing and a computer program. The signal processing circuit 122 and the control circuit 105 may be integrated into a single circuit or may be separate, individual circuits. The signal processing circuit 122 may be a component of an external device, such as a server located in a remote location. In this case, the external device, such as the server, transmits and receives data to and from the light source 101, the image sensor 102, and the control circuit 105 via wired or wireless communication.

[0104] [1-5.Other] The biometric measurement device 10 may include an imaging optical system that forms a two-dimensional image of the user 100 on the light-receiving surface of the image sensor 102. The optical axis of the imaging optical system is approximately perpendicular to the light-receiving surface of the image sensor 102. The imaging optical system may include a zoom lens. Changing the position of the zoom lens changes the magnification of the two-dimensional image of the user 100. This changes the resolution of the two-dimensional image on the image sensor 102. Therefore, even if the user 100 is far away, it is possible to enlarge the desired measurement area and observe it in detail.

[0105] The biometric device 10 may also include a bandpass filter between the user 100 and the image sensor 102 that passes only light in the wavelength band emitted by the light source 101 or light in the vicinity of that wavelength band. This reduces the effects of disturbance components such as ambient light. The bandpass filter may be configured with a multilayer filter or an absorption filter. Taking into account the temperature of the light source 101 and band shifts caused by oblique incidence on the filter, the bandwidth of the bandpass filter may be approximately 20 to 100 nm.

[0106] Furthermore, the biometric device 10 may include a polarizing plate between the light source 101 and the user 100, and between the image sensor 102 and the user 100. In this case, the polarization directions of the polarizing plate arranged on the light source 101 side and the polarizing plate arranged on the image sensor 102 side are in a crossed Nicol relationship. This makes it possible to prevent specular reflection components, which have the same angle of incidence and reflection angle, from reaching the image sensor 102 among the surface reflection components I1 of the user 100. In other words, it is possible to reduce the amount of light that reaches the image sensor 102 as the surface reflection components I1.

[0107] [2. Operation of the light source and image sensor] The bioinstrumentation device 10 in this embodiment can distinguish and detect the surface reflection component I1 and the internal scattering component I2. If the user 100 is a human and the target area is the forehead, the signal intensity of the internal scattering component I2 to be detected is very small. This is due to the extremely small amount of emitted light required to meet laser safety standards, as described above, as well as the significant scattering and absorption of light by the scalp, cerebrospinal fluid, skull, gray matter, white matter, and blood flow. Furthermore, changes in signal intensity due to changes in blood flow or blood flow components during brain activity are very small, corresponding to a fraction of the signal intensity before the change. Therefore, in this embodiment, the surface reflection component I1, which is several thousand to several tens of thousands times stronger than the signal component to be detected, is removed as much as possible during imaging.

[0108] An example of the operation of the light source 101 and the image sensor 102 in the bioinstrumentation device 10 of this embodiment will be described below.

[0109] As shown in FIG. 1A, when the light source 101 irradiates the user 100 with pulsed light, a surface reflection component I1 and an internal scattering component I2 are generated. A portion of the surface reflection component I1 and the internal scattering component I2 reaches the image sensor 102. The internal scattering component I2 passes through the inside of the user 100 before it is emitted from the light source 101 and reaches the image sensor 102. For this reason, the optical path length of the internal scattering component I2 is longer than the optical path length of the surface reflection component I1. Therefore, the time it takes for the internal scattering component I2 to reach the image sensor 102 is delayed on average compared to the time it takes for the surface reflection component I1 to reach the image sensor 102.

[0110] FIG. 1F is a diagram showing a schematic diagram of an optical signal arriving at the image sensor 102 when a rectangular pulsed light beam emitted from the light source 101 returns from the user 100. The horizontal axis represents time (t) for signals (a) to (d). The vertical axis represents intensity for signals (a) to (c), and the OPEN or CLOSE state of the electronic shutter for signal (d). Signal (a) represents the surface reflection component I1. Signal (b) represents the internal scattering component I2. Signal (c) represents the sum of the surface reflection component I1 shown in signal (a) and the internal scattering component I2 shown in signal (b). As shown in signal (a), the surface reflection component I1 maintains its rectangular shape. On the other hand, as shown in signal (b), the internal scattering component I2 is the sum of light beams that have traveled various optical path lengths. For this reason, signal (b) exhibits a characteristic of a trailing tail at the rear end of the pulsed light. In other words, the fall period of the internally scattered component I2 is longer than that of the surface-reflected component I1, so a higher proportion of the internally scattered component I2 is extracted from the optical signal shown in signal (c). For this reason, as shown in signal (d), electronic shutter exposure begins after the trailing edge of the surface-reflected component I1. "After the trailing edge" refers to when or after the surface-reflected component I1 falls. The shutter timing of the electronic shutter is adjusted by the control circuit 105. As described above, the bioinstrumentation device 10 of this embodiment distinguishes between the surface-reflected component I1 and the internally scattered component I2 that has reached a deep portion of the object. Therefore, the emission pulse width and shutter width are optional. Therefore, unlike conventional methods using streak cameras, this simple configuration significantly reduces costs.

[0111] In the example shown in signal (a) of FIG. 1F, the trailing edge of the surface reflection component I1 falls vertically. In other words, the time from when the trailing edge of the surface reflection component I1 begins to fall until it ends is zero. However, in reality, if the waveform of the pulsed light emitted from the light source 101 is not perfectly vertical, if there are minute irregularities on the surface of the user 100, and / or if scattering occurs within the epidermis, the trailing edge of the surface reflection component I1 does not fall vertically. Furthermore, because the user 100 is an opaque object, the amount of light from the surface reflection component I1 is much greater than the amount of light from the internal scattering component I2. Therefore, even if the trailing edge of the surface reflection component I1 slightly deviates from the vertical falling position, the internal scattering component I2 may be obscured. Furthermore, a time delay due to electron movement may occur during the readout period of the electronic shutter. This may prevent ideal binary readout as shown in signal (d) of FIG. 1F from being achieved. Therefore, the control circuit 105 may delay the shutter timing of the electronic shutter slightly from immediately after the falling edge of the surface reflection component I1. For example, the shutter timing of the electronic shutter may be delayed by approximately 0.5 ns to 5 ns. Instead of adjusting the shutter timing of the electronic shutter, the control circuit 105 may adjust the emission timing of the light source 101. The control circuit 105 adjusts the time difference between the shutter timing of the electronic shutter and the emission timing of the light source 101. When measuring changes in blood flow or blood flow components during brain activity in a non-contact manner, delaying the shutter timing too much will further reduce the internal scattering component I2, which is already small. For this reason, the shutter timing may be kept near the trailing end of the surface reflection component I1. The time delay due to scattering by the user 100 is 4 ns. Therefore, the maximum amount of delay in the shutter timing is approximately 4 ns.

[0112] The exposure may be performed with each of the plurality of pulsed lights emitted from the light source 101 at shutter timings of the same phase, thereby amplifying the detected light amount of the internal scattering component I2.

[0113] Instead of or in addition to placing a bandpass filter between the user 100 and the image sensor 102, the control circuit 105 may estimate the offset component by capturing an image with the same exposure time without causing the light source 101 to emit light. The estimated offset component is subtracted from the signal detected by each pixel of the image sensor 102. This makes it possible to remove the dark current component generated in the image sensor 102.

[0114] The internal scattering component I2 includes internal information about the user 100, such as cerebral blood flow information. The amount of light absorbed by the blood changes depending on the temporal fluctuations in the cerebral blood flow of the user 100. As a result, the amount of light detected by the image sensor 102 also increases or decreases accordingly. Therefore, by monitoring the internal scattering component I2, it is possible to estimate the brain activity state of the user 100 from changes in the cerebral blood flow. In this specification, a signal that indicates the internal scattering component I2, among the signals output from the image sensor 102, may be referred to as a "brain activity signal." The brain activity signal may include information about increases or decreases in the cerebral blood flow of the user 100.

[0115] Next, an example of a method for detecting the surface reflection component I1 will be described. The surface reflection component I1 includes surface information of the user 100. The surface information is, for example, blood flow information on the face and scalp. The image sensor 102 detects the surface reflection component I1 from an optical signal that is generated when pulsed light emitted from the light source 101 reaches the user 100 and returns to the image sensor 102.

[0116] FIG. 1G is a diagram schematically illustrating an example of a timing chart for detecting the surface reflection component I1. To detect the surface reflection component I1, for example, as shown in FIG. 1G, the shutter may be opened before the pulsed light reaches the image sensor 102 and closed before the trailing edge of the pulsed light arrives. Controlling the shutter in this manner reduces the amount of internally scattered light I2. As a result, the proportion of light passing through the vicinity of the surface of the user 100 can be increased. The shutter may be closed immediately after the light reaches the image sensor 102. This enables signal detection that increases the proportion of the surface reflection component I1, which has a relatively short optical path length. Acquiring the signal of the surface reflection component I1 also makes it possible to detect the pulse rate or the degree of oxygenation of facial blood flow of the user 100. As another method for acquiring the surface reflection component I1, the image sensor 102 may detect the entire pulsed light emitted from the light source 101 or continuous light emitted from the light source 101.

[0117] 1H is a diagram showing an example of a timing chart for detecting the internal scattering component I2. By opening the shutter during the period when the trailing edge of the pulse reaches the image sensor 102, the signal of the internal scattering component I2 can be acquired.

[0118] The surface reflection component I1 may be detected by a device other than the bioinstrumentation device 10 that acquires the internal scattering component I2. A separate device, such as a pulse wave meter or Doppler blood flow meter, may be used. In this case, the separate device is used while taking into consideration timing synchronization between devices, optical interference, and the alignment of the detection points. Performing time-division imaging using the same camera or sensor, as in this embodiment, reduces temporal and spatial discrepancies. When acquiring signals for both the surface reflection component I1 and the internal scattering component I2 using the same sensor, the components acquired may be switched for each frame, as shown in FIGS. 1G and 1H. Alternatively, as described with reference to FIGS. 1B to 1D, the components acquired at high speed within a frame may be alternately switched. In this case, the detection time difference between the surface reflection component I1 and the internal scattering component I2 can be reduced. Furthermore, signals for both the surface reflection component I1 and the internal scattering component I2 may be acquired from the same pulsed light.

[0119] Furthermore, the signals for the surface reflection component I1 and the internal scattering component I2 may be acquired using light of two wavelengths. For example, pulsed light of two wavelengths, 750 nm and 850 nm, may be used. This allows the changes in the oxygenated hemoglobin and deoxygenated hemoglobin concentrations to be calculated from the changes in the amount of light detected at each wavelength. When the surface reflection component I1 and the internal scattering component I2 are acquired using two wavelengths, a method of rapidly switching between four types of charge accumulation within one frame may be used, as described with reference to Figures 1B to 1D. This method reduces the time lag in the detection signals.

[0120] The biometric device 10 emits pulsed near-infrared or visible light toward the forehead of the user 100 and can detect changes in the amount of oxygenated hemoglobin on the scalp or face or pulse rate from the temporal change in the surface reflection component I1. The light source 101 emits near-infrared or visible light to acquire the surface reflection component I1. Near-infrared light allows measurement day or night. Visible light, which has higher sensitivity, may be used to measure pulse rate. During the day, ambient light such as sunlight or an indoor light source may be used instead of lighting. If the amount of light is insufficient, a dedicated light source may be used to supplement the light. The internal scattering component I2 includes light components that reach the brain. By measuring the temporal change in the internal scattering component I2, the temporal increase or decrease in cerebral blood flow can be measured.

[0121] Light reaching the brain also passes through the scalp and facial surface. Therefore, fluctuations in scalp and facial blood flow are also detected as superimposed signals. To eliminate or reduce this effect, when brain activity data is used as biometric data, the biometric data generation unit 109 may subtract the surface reflection component I1 from the internal scattering component I2 detected by the image sensor 102. This allows pure cerebral blood flow information, excluding scalp and facial blood flow information, to be acquired. For example, a subtraction method may be used in which the signal of the surface reflection component I1 is multiplied by a coefficient of 1 or more determined in consideration of the optical path length difference, from the signal of the internal scattering component I2. This coefficient may be calculated by simulation or experiment, for example, based on the average optical constants of a typical human head. This subtraction process can be easily performed when measurements are taken using the same camera or sensor and light of the same wavelength. This is because it is easy to reduce temporal and spatial discrepancies and easily match the characteristics of the scalp blood flow component contained in the internal scattering component I2 with the characteristics of the surface reflection component I1.

[0122] The skull exists between the brain and the scalp. Therefore, the two-dimensional distribution of cerebral blood flow is independent of the two-dimensional distribution of scalp and facial blood flow. Therefore, based on the signal detected by the image sensor 102, the two-dimensional distribution of the internal scattering component I2 and the two-dimensional distribution of the surface reflection component I1 may be separated using a statistical method such as independent component analysis or principal component analysis.

[0123] [3. Biometric Measurement Sequence] Next, an example of a method for measuring biometric data of user 100 using the biometric device 10 described above will be described. In this embodiment, the measurement unit 110 measures the amount of body movement of user 100 from an image signal including surface reflection component I1. In this embodiment, the amount of body movement is the movement distance of the target part. In this case, the target part is the head of user 100. The movement distance can also be referred to as the amount of displacement from a reference position. In this embodiment, based on the measurement results, the output determination unit 111 determines whether or not to output the biometric data.

[0124] FIG. 2 is a flowchart showing an example of a process for measuring biometric data of the user 100 in this embodiment.

[0125] In step S201, the biomeasurement device 10 performs initial settings before measuring biomeasurement data. Step S201 includes a process in which the control circuit 105 optimally adjusts the timing of emitting pulsed light from the light source 101 and the shutter timing of the image sensor 102 according to the distance between the biomeasurement device 10 and the user 100. Step S201 also includes a process in which the control circuit 105 causes the image signal acquisition unit 108 to send an image signal including the surface reflection component I1 to the measurement unit 110, and a process in which the measurement unit 110 calculates the initial head position and stores the calculated position in a memory (not shown) within the measurement unit 110.

[0126] In step S202, the control circuit 105 causes the image signal acquisition unit 108 to send, to the biomeasurement data generation unit 109, an image signal representing an internal image including the internal scattering component I2.

[0127] In step S203, the control circuit 105 causes the image signal acquisition unit 108 to send to the measurement unit 110 an image signal representing a surface image including the surface reflection component I1.

[0128] The order of steps S202 and S203 may be reversed.

[0129] In step S204, the measurement unit 110 calculates the position of the target area from the image signal representing the surface image, and calculates the difference between the position of the target area in the initial state stored in memory to calculate the amount of body movement. This difference value is the amount of displacement of the target area. A known image processing method, such as feature point extraction using edge detection, can be used to calculate the position of the target area from the image signal. A two-dimensional image of a face contains more components with high spatial frequencies than a two-dimensional image of cerebral blood flow. Therefore, a two-dimensional image of a face is advantageous in that it is easier to extract feature points. The measurement unit 110 sends the amount of body movement to the output determination unit 111 as a difference value based on the initial state.

[0130] In step S205, the output determination unit 111 determines whether the amount of body movement is equal to or less than a threshold value. This threshold value is, for example, a value within the range of 1 mm to 30 mm. When obtaining a two-dimensional distribution of cerebral blood flow as biomeasurement data, if the desired resolution of the two-dimensional distribution is relatively low, the threshold value can be increased.

[0131] If the amount of body movement is equal to or less than the threshold, in step S206, the biometric data generating unit 109 generates biometric data from the image signal representing the internal image and sends the biometric data to the output determining unit 111. In step S207, the output determining unit 111 outputs the biometric data. This output may be displayed on a display unit (not shown) of the biometric device 10, for example, or may be used to control a higher-level system (not shown).

[0132] If the amount of body movement exceeds the threshold, steps S206 and S207 are skipped, and the output determination unit 111 stops outputting the biometric data. Here, instead of stopping the output of the biometric data, the output determination unit 111 may output a signal indicating that the biometric data at that timing is invalid. Alternatively, the output determination unit 111 may continue to output the same data as the last valid biometric data. Alternatively, the output determination unit 111 may output data interpolated from the last valid biometric data when the biometric data becomes valid again.

[0133] In step S208, it is determined whether measurement has been performed for a predetermined period. This determination may be made by the bioinstrumentation device 10 or by a higher-level system to which the output of the bioinstrumentation device 10 is connected.

[0134] The aforementioned "predetermined period" may be, for example, a period until the psychological state of user 100 can be estimated. Alternatively, the "predetermined period" may be a period until a series of tasks assigned to user 100 is completed. Alternatively, the "predetermined period" may be a period until user 100 completes a series of tasks. The series of tasks may be, for example, driving a car or operating a game console.

[0135] If the measurement for the predetermined period has not been completed, the bioinstrumentation device 10 repeats the sequence from step S202 to step S208. If the measurement for the predetermined period has been completed, the bioinstrumentation device 10 ends the measurement.

[0136] The above-described operations of the biomeasurement device 10 in this embodiment can be summarized as follows: The control circuit 105 causes the light source 101 to emit light and the image sensor 102 to output an image signal. The biomeasurement data generation unit 109 generates biomeasurement data based on the image signal. The output determination unit 111 determines whether to output the biomeasurement data. The control circuit 105, the biomeasurement data generation unit 109, and the output determination unit 111 repeat the above-described operations. The output determination unit 111 stops outputting the biomeasurement data for a period during which a value calculated based on the image signal output from the image sensor 102 satisfies a preset condition. In the above example, the value calculated based on the image signal output from the image sensor 102 is a difference value indicating the amount of displacement of the target area calculated by the measurement unit 110. The preset condition is that the difference value exceeds a threshold value.

[0137] [4. Operation during biometric measurement] 3A is a diagram illustrating an example of the operation of a process for measuring biometric data of user 100 in this embodiment. In the example shown in FIG.

[0138] In the example shown in parts (a) to (d) of Figure 3A, the horizontal axis represents the progression of frames or time. Part (a) schematically shows changes in a surface image, which is an image containing surface reflection component I1. Part (b) shows changes in body movement. Part (c) shows the determination result. Part (d) shows changes in oxygenated hemoglobin concentration in the forehead region, which is brain activity data, as an example of biometric data.

[0139] Although each frame is numbered, one or more frames may be inserted between each frame. The frame rate or measurement frequency may be, for example, between 1 fps (frame per second) and 30 fps. The measurement frequency of body movement and the generation frequency of biometric data may be different. Changes in cerebral blood flow occur gradually over a period of one to several seconds. On the other hand, body movement acquired as an image without contact changes at a faster rate than cerebral blood flow. Therefore, to detect body movement more precisely, the measurement frequency of body movement may be equal to or greater than the generation frequency of cerebral blood flow data.

[0140] In the example shown in part (a) of FIG. 3A, the amount of body movement measured by the measurement unit 110 exceeds a threshold due to movement of the subject between frames 3 and 4. There is a correlation between the amount of body movement and the irregularity of the generated biometric data. Therefore, the output determination unit 111 determines that the biometric data is invalid during the period in which the amount of body movement exceeds the threshold, and stops outputting the biometric data. As a result, as shown in part (d) of FIG. 3A, brain activity data, which is biometric data, is output in frames 1, 2, and 5. In the example shown in part (b) of FIG. 3A, the period in which the amount of body movement exceeds the threshold is described as an "invalid period."

[0141] The above-described embodiment makes it possible to measure biometric data non-contact, without prompting the user 100 to measure again. This allows for the realization of a biometric device that can be used for daily measurements. Furthermore, calculations associated with the reconstruction of cerebral blood flow distribution are not required. This reduces calculation costs, enabling biometric measurements to be performed at low cost.

[0142] FIG. 3B is a diagram showing a schematic diagram of the relationship between the change in the amount of body movement and the invalid period. In the movement shown in part (b) of FIG. 3A, the period in which the amount of body movement exceeds the threshold is set as the invalid period. On the other hand, as shown in part (a) of FIG. 3B, in addition to the period in which the amount of body movement exceeds the threshold, the period d before the amount of body movement exceeds the threshold is also set as the invalid period. f1 , and the period after the amount of body movement falls below the threshold d r1In addition to the period in which the amount of body movement exceeds the threshold, the period d f1 and period d r1 For example, a period including either one of the following may be set as an invalid period: f1 and period d r1 In this way, the output determination unit 111 determines whether the value calculated based on the image signal output from the image sensor 102 satisfies the preset condition, and also whether the value calculated based on the image signal output from the image sensor 102 satisfies the preset condition, and whether the value calculated based on the image signal satisfies the preset condition, and also whether the value calculated based on the image signal satisfies the preset condition. f1 and / or the period after termination d r1 During this period, the output of the biometric data may be stopped even if the value does not meet the condition.

[0143] The timing to start the invalid period may be after the amount of body movement exceeds the threshold. f1 may be a negative value because there may be a delay due to system processing after the amount of body movement exceeds the threshold value until the invalid period is set.

[0144] Furthermore, as shown in part (b) of FIG. 3B, the period after the end d r2 , the period before the start d f2 For example, the period d f2 is a period equivalent to 0.5 frames, and the period d r2 The reason for this will be described later in the description of the embodiment.

[0145] In this embodiment, the amount of body movement is calculated from an image signal including the surface reflection component I1. Alternatively, the amount of body movement may be calculated from an image signal including the internal scattering component I2. The image signal including the internal scattering component I2 also includes information about the outer shape of the target area. However, the image signal including the surface reflection component I1 can detect a larger amount of light. Therefore, the image signal including the surface reflection component I1 is more advantageous in terms of signal-to-noise ratio than the image signal including the internal scattering component I2.

[0146] Furthermore, in this embodiment, the displacement of the target part is the measurement target, but the measurement target is not limited to this. The measurement target may also be the movement speed of the target part. In a frame in which the absolute value of the movement speed of the target part exceeds a threshold value, the output of brain activity data may be stopped. This is because it is considered that body movement occurs in that frame. In other words, the movement speed of the target part is the movement distance of the target part between frames.

[0147] 3C is a diagram schematically illustrating the relationship between the moving speed of the target part and the invalid period. As an example of a method for setting the invalid period, as shown in FIG. 3C, the invalid period may be set to the period from when the moving speed exceeds a positive threshold value and then falls below the positive threshold value, and then falls below a negative threshold value and then exceeds the negative threshold value again, or the period from when the moving speed falls below a negative threshold value and then exceeds the negative threshold value, and then exceeds a positive threshold value and then falls below the positive threshold value again.

[0148] Furthermore, the measurement object does not have to be related to the movement of the target part, but may be the brightness value of the target part.

[0149] FIG. 3D is a diagram illustrating an example of the operation of a process for measuring biometric data of a user 100 in this embodiment. In the example shown in FIG. 3D, the target area is the head of the user 100. In the example shown in part (a) of FIG. 3D, the measurement unit 110 measures the luminance value of the target area from an internal image, which is an image including an internal scattering component I2. A period in which the luminance value of the target area exceeds a threshold is determined as an invalid period. The output determination unit 111 stops outputting brain activity data in frames 2 and 3 included in the invalid period. When the luminance value of the target area in an image including the internal scattering component I2 approaches a saturation value, accurate biometric data cannot be generated. Therefore, by determining validity based on the luminance value of the target area, the biometric device 10 can output only accurate biometric data.

[0150] In the example shown in Figure 3D, the period in which the luminance value of the target area exceeds the threshold is set as the invalid period. Conversely, the period in which the luminance value of the target area is below the threshold may also be set as the invalid period. This is because if the luminance value of the target area in an image containing the internal scattering component I2 is extremely low, the signal-to-noise ratio of the image is insufficient. In this case, accurate biometric data cannot be generated.

[0151] The measurement target may also be the rate of change of the luminance value of the target area. In a frame where the absolute value of the rate of change of the luminance value exceeds a threshold, the output of brain activity may be stopped. This is because it is considered that the luminance in that frame is either extremely high or extremely low.

[0152] Furthermore, the measurement unit 110 may measure the area of ​​a predetermined region of the target part from which biometric data is measured, from the image signal. The area of ​​the predetermined region of the target part is, for example, the area of ​​the forehead. To calculate the area of ​​the forehead from the image signal, a known image processing method such as the feature point extraction using the edge detection described above may be used. If the forehead part contains a lot of hair and the area of ​​the forehead part is below a threshold, the output of biometric data may be stopped. This is because if the area of ​​the target part is small, the accuracy of the generated biometric data decreases.

[0153] Furthermore, the measurement target may be biometric data itself.

[0154] FIG. 3E is a schematic diagram illustrating an example of the biomeasurement device 10 according to this embodiment. Unlike the example illustrated in FIG. 1A, the example illustrated in FIG. 3E also transmits the generated biomeasurement data to the measurement unit 110. The measurement unit 110 measures the rate of change of the biomeasurement data. The biomeasurement data may be, for example, brain activity data, such as oxygenated hemoglobin concentration or deoxygenated hemoglobin concentration. Cerebral blood flow changes gradually over a period of one to several seconds. On the other hand, body movement and changes in brightness of the target area affect the measured value of cerebral blood flow at a rate faster than cerebral blood flow. Therefore, by determining the validity based on the rate of change of the biomeasurement data, the biomeasurement device 10 can output only accurate biomeasurement data. The output determination unit 111 may stop outputting the biomeasurement data if the absolute value of the rate of change of the value calculated from the biomeasurement data exceeds a threshold value.

[0155] (Second embodiment) The configuration and operation of the bioinstrumentation device 20 in the second embodiment will be described with reference to FIGS.

[0156] FIG. 4 is a diagram schematically illustrating an example of a biometric measurement device 20 according to this embodiment. In this embodiment, the measurement unit 110 measures the amount of body movement of the user 100 from an image signal including a surface reflection component I1. In this embodiment, the amount of body movement of the user 100 is the distance traveled by the target part. The output determination unit 401 calculates the reliability of the biometric data based on the measurement results, and outputs reliability data indicating the reliability together with the biometric data. Thus, in the second embodiment, unlike the first embodiment, the output determination unit 401 outputs not only the biometric data but also the reliability data.

[0157] FIG. 5A is a flowchart showing an example of a process for measuring biometric data of user 100 in this embodiment.

[0158] The operations of performing initial settings in step S501, acquiring a surface image in step S502, acquiring an internal image in step S503, and calculating body movement in step S504 are the same as those in steps S201 to S204 in the first embodiment.

[0159] In step S505, the output determination unit 401 calculates the reliability based on the amount of body movement calculated from the difference value from the initial state.

[0160] FIG. 5B is a diagram showing an example of the relationship between the difference value and the reliability. When the difference value is 0 mm, the measurement result can be most reliable. Therefore, as shown in FIG. 5B, when the difference value is 0 mm, the reliability may be set to 100%. As shown in FIG. 5B, the larger the difference value, the lower the reliability may be. In this way, the output determination unit 401 may calculate a lower reliability as the value calculated from the image signal output from the image sensor 102 deviates from a preset value.

[0161] The biomeasurement device 20 in this embodiment may output biomeasurement data regardless of the magnitude of the amount of body movement. In step S506, the biomeasurement data generation unit 109 generates biomeasurement data from the image signal and sends the biomeasurement data to the output determination unit 401.

[0162] In step S507, the output determination unit 401 outputs the reliability data together with the biometric data. The output may be displayed on a display unit (not shown) of the biometric device 20, or may be used to control a higher-level system (not shown).

[0163] The operation in step S508 is similar to the operation in step S208 in the first embodiment.

[0164] 6 is a diagram illustrating an example of the operation of a process for measuring biometric data of the user 100 in this embodiment. In the example shown in FIG.

[0165] In the example shown in parts (a) to (d) of Figure 6, similar to the example shown in parts (a) to (d) of Figure 3A, the horizontal axis represents the progression of frames or time. Part (a) schematically shows a surface image that includes the surface reflection component I1. Part (b) shows changes in the amount of body movement. Part (c) shows changes in the reliability value. Part (d) shows changes in the oxygenated hemoglobin concentration in the forehead region, which is brain activity data, as an example of biomeasurement data.

[0166] In the second embodiment, unlike the first embodiment, the reliability is calculated according to the amount of body movement of each frame. In the example shown in part (c) of Fig. 6, the output determination unit 401 calculates a lower reliability value as the amount of body movement shown in part (b) of Fig. 6 increases. The amount of body movement is a difference value of the position of the target part based on the initial state.

[0167] In frames 3 and 4, the reliability is low due to movement of the target area. On the other hand, the output determination unit 401 may output biometric data in all frames. The output determination unit 401 outputs reliability data along with the biometric data.

[0168] The above-described embodiment makes it possible to measure biometric data non-contact, without prompting the user 100 to measure again. This allows for the realization of a biometric device that can be used for daily measurements. Furthermore, calculations associated with restoring cerebral blood flow distribution are not required. This reduces calculation costs, enabling biometric measurements to be performed at low cost. Furthermore, a special effect is achieved in that a higher-level system can comprehensively determine the validity of biometric data using reliability data.

[0169] In this embodiment, the reliability is calculated as a lower value as the measurement result from the measurement unit 110 deviates from the threshold value. However, the method for calculating the reliability is not limited to this.

[0170] The reliability may be calculated based on the temporal trend of the measurement result, rather than the value of the measurement result itself. For example, the longer the invalid period during which the measurement result exceeds the threshold, the lower the reliability may be calculated. In other words, the longer the period during which the value calculated from the image signal output from the image sensor 102 exceeds a preset value, the lower the reliability may be calculated by the output determination unit 111.

[0171] Furthermore, the higher the proportion of invalid periods in a certain period, the lower the reliability may be calculated. In other words, the longer the period during which the value calculated from the image signal output from the image sensor 102 exceeds a preset value during a certain period, the lower the reliability may be calculated by the output determination unit 111. When the validity and invalidity of a measurement value change frequently in a short period of time, it is effective to use such a reliability calculation method.

[0172] (Third embodiment) The configuration and operation of a bioinstrumentation device 30 according to the third embodiment will be described with reference to FIGS. 7A to 9. FIG.

[0173] 7A is a diagram schematically illustrating an example of a biomeasurement device 30 according to this embodiment. The biomeasurement device 30 according to this embodiment includes an imaging unit 701, a signal processing circuit 702, and a measurement unit 703. In this embodiment, the measurement unit 703 includes an acceleration sensor installed near the user 100, which is the surrounding environment, and measures acceleration. The output determination unit 111 determines whether or not to output biomeasurement data based on the measurement result.

[0174] 7B is a diagram showing an example of the arrangement of each part when the biometric measurement device 30 is installed inside a car. The imaging unit 701, signal processing circuit 702, and measurement unit 703 may be arranged separately. The measurement unit 703 is arranged near the user 100. Here, "arranged near" means arranged at a distance where there is at least a correlation between the acceleration measured by the measurement unit 703 and the body movement of the user 100 himself.

[0175] FIG. 7C is a diagram schematically illustrating an example of the arrangement of each part when the biometric measurement device is attached to a game machine or an attraction device. Similar to the example of the arrangement shown in FIG. 7B, the imaging unit 701, signal processing circuit 702, and measurement unit 703 may be arranged separately. The user 100 operates the controller 705 based on visual information from, for example, a display 704. The measurement unit 703 is attached to, for example, headphones or a head-mounted display worn on the head of the user 100. The measurement unit 703 detects acceleration accompanying the movement of the target part of the user. The measurement unit 703 may also detect, for example, a change in illuminance of the target part accompanying a change in the display information on the display 704.

[0176] FIG. 8 is a flowchart showing an example of a process for measuring biometric data of the user 100 in this embodiment.

[0177] In step S801, the biomeasurement device 30 performs initial settings before measuring biomeasurement data. As in the first embodiment, step S801 includes a process in which the control circuit 105 optimally adjusts the timing of emitting pulsed light from the light source 101 and the shutter timing of the image sensor 102 according to the distance between the biomeasurement device 30 and the user 100.

[0178] In step S802, the measurement unit 703 measures acceleration. If the bioinstrumentation device 30 is installed inside a car, measuring the acceleration corresponds to measuring the vibrations of the car.

[0179] The acceleration measurement frequency may be equal to or greater than the biometric data generation frequency. This improves the ability of the measurement value to track sudden vibrations. As a result, the accuracy of the determination by the output determination unit 111 can be improved. In this embodiment, as an example, the acceleration measurement frequency is set to 10 times the brain activity data calculation frequency.

[0180] In step S803, acceleration values ​​are sequentially stored in a memory (not shown) in the measurement unit 703. In step S804, the number of times acceleration has been measured is determined, and a loop of steps S802 and S803 is repeated, for example, 10 times. There is no limit to the number of measurements.

[0181] In step S805, the measurement unit 703 refers to the contents of the memory and sends the maximum value from among the accumulated 10 acceleration values ​​to the output determination unit 111. The average value of the 10 acceleration values ​​may be used instead of the maximum value. In this embodiment, a value that is at least correlated with the body movement of the user 100 himself / herself is calculated from the 10 acceleration values.

[0182] In step S806, the output determination unit 111 determines whether the maximum value of the acceleration value received from the measurement unit 703 is equal to or less than a threshold value. This threshold value is, for example, the acceleration in any of the directions of forward / backward, left / right, and vertical, and is in the range of 0.1 G to 1 G (G=9.8 m / s 2 ) is a value in the range

[0183] In step S807, the control circuit 105 causes the image signal acquisition unit 108 to send an image signal including the internal scattering component I2 to the biomeasurement data generation unit 109. The operations in steps S808, S809, and S810 are the same as the operations in steps S206, S207, and S208 in the first embodiment, respectively.

[0184] 9 is a diagram illustrating an example of the operation of a process for measuring biometric data of the user 100 in this embodiment. In the example shown in FIG.

[0185] In the examples shown in parts (a) to (d) of Figure 9, the horizontal axis represents the progression of frames or time. Part (a) schematically shows the state of the housing of a car or attraction device. Part (b) shows the change in acceleration. Part (c) shows the determination result. Part (d) shows the change in oxygenated hemoglobin concentration in the forehead region, which is brain activity data, as an example of biometric data.

[0186] In the third embodiment, unlike the first embodiment, the measurement unit 703 measures acceleration because there is a correlation between the acceleration in the vicinity of the user 100 and the amount of body movement of the user 100 himself.

[0187] In the example shown in FIG. 9, the housing of the car or attraction device vibrates between frames 3 and 4. As a result, the acceleration measured by the measurement unit 703 exceeds a threshold value. There is a correlation between the amount of body movement and the irregularity of the generated biometric data. Therefore, the output determination unit 111 determines that the biometric data is invalid during the invalid period in which the acceleration exceeds the threshold value, and stops outputting the biometric data. As a result, as shown in part (d) of FIG. 9, brain activity data, which is biometric data, is output in frames 1, 2, and 5.

[0188] The above-described embodiment makes it possible to measure biometric data non-contact, without prompting the user 100 to measure again. This allows for the realization of a biometric device that can be used for daily measurements. Furthermore, calculations associated with restoring cerebral blood flow distribution are not required. Therefore, calculation costs can be reduced, and biometric measurements can be made using an inexpensive method. Furthermore, since there is no need to measure body movement from image signals, a special effect is obtained in that calculation costs can be further reduced.

[0189] The measurement target of the measurement unit 703 is not limited to acceleration, as long as it is correlated with at least body movement. For example, if the biomeasurement device 30 is installed in a car, the measurement unit 703 may include a speed sensor to measure the speed. Furthermore, the measurement unit 703 may include a steering angle sensor to measure the steering angle of the car.

[0190] The measurement unit 703 may also include a gear position sensor to measure the gear position of the vehicle. In this case, the output determination unit 111 stops outputting biometric data during a period in which the gear position measurement result indicates a preset state. As shown in FIG. 3B , the output determination unit 111 may stop outputting biometric data for a period before the start of the preset period and / or a period after the end of the preset period, in addition to the preset period. The preset state is, for example, a state in which the gear is in reverse. In this state, the user 100 drives the vehicle while looking back to reverse. This causes the target part of the user 100 to be displaced from the reference position, resulting in body movement of the user 100.

[0191] Furthermore, the measurement target of the measurement unit 703 does not have to be related to the body movement of the user 100. The measurement unit 703 may include an illuminance sensor installed near the user 100 in the surrounding environment to measure illuminance. In this case, the output determination unit 111 determines a period in which the illuminance value exceeds a threshold as an invalid period and stops output of biomeasurement data. When the luminance value of the target part in an image containing the internal scattering component I2 approaches a saturation value, accurate biomeasurement data cannot be generated. Therefore, when there is at least a correlation between the illuminance value and the luminance value of the target part, the biomeasurement device 30 can output only accurate biomeasurement data by determining validity based on the luminance value of the target part.

[0192] (Example) Below, examples will be described that were carried out to confirm the principles of the present disclosure.

[0193] The biometric measurement device 10 shown in FIG. 1A according to the first embodiment was placed facing the head of a user 100, which was the subject. The face of the user 100 was placed on a chin rest to prevent unintended body movements. After the user 100 was allowed to relax, the user's head was photographed for 30 seconds, and brain activity data was measured as biometric data.

[0194] The wavelengths of the light sources were 750 nm and 850 nm. Image signals containing the surface reflection component I1 were acquired at 30 fps. Image signals containing the internal scattering component I2 were acquired at 5 fps. The resolution of the image signals was 320 pixels x 240 pixels. A 50 pixel x 50 pixel area including the center of the forehead was used as the target region, and spatial averaging and time averaging were performed in real time. The oxygenated hemoglobin concentration was calculated based on the image signals containing the internal scattering component I2 of the two wavelengths, and changes in the calculated oxygenated hemoglobin concentration were monitored.

[0195] At the same time, based on the image signal containing the surface reflection component I1, the change in the moving distance of the face from the measurement unit 110 was monitored. The Kanade-Lucas-Tomasi (KLT) algorithm was used to extract and track the facial feature points.

[0196] Only once during the measurement, the user 100 intentionally moved his head to the right and then immediately returned it to its original position.

[0197] FIG. 10A is a diagram showing the relationship between time and the head movement distance. FIG. 10B is a diagram showing the relationship between time and the oxygenated hemoglobin concentration. In the example shown in FIG. 10A, X represents the change in the left-right direction, Y represents the change in the up-down direction, and Z represents the change in the depth direction. The movement distance is the amount of displacement from the initial value in each of the X, Y, and Z directions. In the example shown in FIG. 10B, the oxygenated hemoglobin concentration is the amount of change from the initial value. The amount of change is expressed in arbitrary units.

[0198] 10A and 10B, large fluctuations in the oxygenated hemoglobin concentration occurred in synchronization with the head movement from approximately 11 to 12 seconds, indicating a correlation between body movements, such as head movement, and irregular fluctuations in brain activity data.

[0199] Furthermore, from the results shown in Figures 10A and 10B, it can be seen that fluctuations in oxyhemoglobin concentration occur with a slight delay after the start of head movement. Furthermore, it can be seen that fluctuations in oxyhemoglobin concentration subside with a delay after the head movement ends. There are two possible reasons for this: (1) The brain is floating in cerebrospinal fluid within the skull. Therefore, brain movement lags behind skull movement. (2) When measuring oxyhemoglobin concentration, time averaging is performed in real time. For this reason, the measured value is easily affected by immediately preceding fluctuations.

[0200] 10A and 10B, for example, when the following two conditions are met, validity may be determined and output of brain activity data may be stopped. (1) The threshold value is set to 5 mm for the movement distance in any of the X, Y, and Z directions. (2) The invalid period is set to 0.5 seconds before the threshold is exceeded and 4.5 seconds after the threshold is exceeded. These two conditions can eliminate irregular fluctuations in the measured values ​​of oxygenated hemoglobin concentration.

[0201] In the above example, cerebral blood flow data is used as an example of biometric data, but this is not limiting. The biometric data may be, for example, data indicating at least one selected from the group consisting of scalp blood flow, pulse rate, sweating, respiration, and body temperature. The data can be obtained from the surface reflection component I1.

[0202] This disclosure also includes programs and methods of operation performed by signal processing circuitry 122 and signal processing circuitry 702. [Industrial Applicability]

[0203] The biometric measurement device of the present disclosure can be used in a camera or measuring instrument that acquires internal information of a user without contact. The biometric measurement device can also be applied to, for example, biometric or medical sensing, sensing of a car driver, sensing of a user in a game machine or attraction device, sensing of a learner in an educational institution, or sensing of a worker in a workplace. [Explanation of symbols]

[0204] 10, 20, 30 Biometric measurement device 100 users 101 Light source 102 Image Sensor 103 Photoelectric conversion unit 104 Charge storage section 105 Control circuit 106 Light source control unit 107 Sensor control unit 108 Image signal acquisition unit 109 Biometric data generation unit 110, 703 Measurement section 111, 401 Output determination unit 121, 701 Imaging unit 122, 702 Signal processing circuit 201 pixels 202 Drain 203 Photodiode 204 First floating diffusion layer 205 Second floating diffusion layer 206 Third floating diffusion layer 207 Fourth floating diffusion layer 302 Row selection circuit 303 Column Selection Circuit 304 vertical signal line 305 Source Follower Power Supply 306 Source Follower Load 307 Analog-to-Digital Conversion Circuit 308 Row select transistor 309 Source Follower Transistor 310 Reset transistor 704 Display 705 Controller

Claims

1. a light source that emits light to a target part of a living body; a light detection cell that receives reflected light returning from the target portion due to the emission of the light and outputs a signal; a signal processing circuit; Equipped with The signal processing circuit determining whether a first value corresponding to an absolute value of a rate of change in the luminance value of the target portion based on the signal or an absolute value of a rate of change in the value of biometric data of the living body based on the signal exceeds a threshold value; outputting the biometric data is stopped during a first period in which it is determined that the first value exceeds the threshold value, and during a second period which is a predetermined period after the end of the first period; Biometric devices.

2. The biometric data includes at least one piece of information selected from the group consisting of scalp blood flow and pulse rate. The bioinstrumentation device according to claim 1 .

3. A light source that emits light to a target part of a living body; a light detection cell that receives reflected light returning from the target portion due to the emission of the light and outputs a signal; a signal processing circuit; Equipped with The signal processing circuit determining whether a body movement of the living body is occurring based on a brightness value of the target portion based on the signal or a first value corresponding to a value of biometric data of the living body based on the signal; the output of the biological measurement data is stopped during a first period in which it is determined that a body movement of the living body is occurring, and during a second period which is a predetermined period after the end of the first period. Biometric devices.

4. A biometric measurement device that measures biometric data without contacting a living body, a light source that emits light including near-infrared light to a target part of a living body; a light detection cell that receives reflected light returning from the target portion due to the emission of the light and outputs a signal; a signal processing circuit; Equipped with The signal processing circuit determining whether a first value corresponding to a brightness value of the target portion based on the signal or a value of biometric data of the living body based on the signal exceeds a threshold; outputting the biometric data is stopped during a first period in which it is determined that the first value exceeds the threshold value, and during a second period which is a predetermined period after the end of the first period; Biometric devices.

5. causing a light source to emit light to a target portion of a living body; causing a light detection cell to receive reflected light returning from the target portion due to the emission of the light and outputting a signal; determining whether a first value corresponding to an absolute value of a rate of change in the luminance value of the target portion based on the signal or an absolute value of a rate of change in the value of biometric data of the living body based on the signal exceeds a threshold; Stopping output of the biometric data during a first period in which it is determined that the first value exceeds the threshold value and during a second period that is a predetermined period after the end of the first period; Including, Biometric methods.

6. A method of detecting a target part of a living body by using a light source to emit light to the target part of the living body; causing a light detection cell to receive reflected light returning from the target portion due to the emission of the light and outputting a signal; determining whether a body movement of the living body is occurring based on a brightness value of the target portion based on the signal or a first value corresponding to a value of biometric data of the living body based on the signal; Stopping output of the biological measurement data during a first period in which it is determined that a body movement of the living body is occurring and during a second period which is a predetermined period after the end of the first period; Including, Biometric methods.

7. A biometric measurement method for measuring biometric data without contacting a living body, comprising: causing a light source to emit light including near-infrared light to a target part of a living body; causing a light detection cell to receive reflected light returning from the target portion due to the emission of the light and outputting a signal; determining whether a first value corresponding to a brightness value of the target portion based on the signal or a value of biometric data of the living body based on the signal exceeds a threshold; Stopping output of the biometric data during a first period in which it is determined that the first value exceeds the threshold value and during a second period that is a predetermined period after the end of the first period; Including, Biometric methods.

Images