Biomedical measuring device and biomedical measuring method

Abstract

To provide a biomedical measuring device for measuring a state of an object by an inexpensive method compared to a conventional method.SOLUTION: A biomedical measuring device includes a light source for emitting light onto an object part of a living body, a light detection cell for receiving reflection light returning from the object part attributed to the emission of the light and outputting a signal, and a signal processing circuit. The signal processing circuit determines whether or not a first value exceeds a threshold on the basis of a brightness value of the object part based on the signal, or the first value corresponding to a value of biomedical measurement data on the living body based on the signal, and stops outputting the biomedical measurement data during a first period in which the first value is determined to exceed the threshold, and in a second period, which is a predetermined period after the end of the first period.SELECTED DRAWING: Figure 1A

Description

[Technical field]

[0001] The present disclosure relates to a biomeasurement device and a biomeasurement method. [Background technology]

[0002] In the field of biomeasurement, a method is used in which light is emitted toward an object and internal information of the object is obtained from the light that passes through the object. In this method, surface reflection components reflected from the surface of the object can become noise. If the noise caused by the surface reflection components is removed, the desired internal information can be obtained with high accuracy.

[0003] Patent Document 1 discloses an imaging device that measures internal information of an object in a non-contact manner while suppressing noise due to surface reflection components. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-202328 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the above-mentioned conventional imaging device, if the object moves during measurement, the correction requires a high calculation cost.

[0006] The present disclosure provides a bioinstrumentation device that measures the condition of a target object using a method that is less expensive than conventional methods. [Means for solving the problem]

[0007] A bioinstrumentation device according to an embodiment of the present disclosure includes a light source for irradiating light onto a user's head, an image sensor, a control circuit for controlling the light source and the image sensor, and a signal processing circuit. The control circuit causes the light source to emit the light, and causes the image sensor to detect at least a portion of the reflected light returning from the head due to the irradiation of the light, and output an image signal. The signal processing circuit generates brain activity data indicating a state of the user's brain based on the image signal, and stops outputting the brain activity data based on at least one selected from the group consisting of the image signal and a sensor signal output from a sensor that detects a change in the user's surrounding environment that affects the brain activity data. Effect of the Invention

[0008] According to an embodiment of the bioinstrumentation device of the present disclosure, internal information of an object can be measured in an inexpensive manner even if the object moves during measurement or if the surrounding environment of the object changes. [Brief description of the drawings]

[0009] [Figure 1A] FIG. 1A is a diagram illustrating a schematic example of a bioinstrumentation device according to the present embodiment. [Figure 1B] FIG. 1B is a diagram showing a schematic configuration example of one pixel of an image sensor. [Figure 1C] FIG. 1C is a diagram illustrating an example of the configuration of an image sensor. [Figure 1D] FIG. 1D is a diagram illustrating an example of an operation within one frame in this embodiment. [Figure 1E] FIG. 1E is a flowchart showing an outline of the operation of the control circuit with respect to the light source and the image sensor. [Figure 1F] FIG. 1F is a diagram illustrating a schematic optical signal that reaches an image sensor when rectangular pulse light emitted from a light source returns from a user. [Figure 1G] FIG. 1G is a diagram illustrating an example of a timing chart for detecting a surface reflection component. [Figure 1H] FIG. 1H is a diagram illustrating an example of a timing chart for detecting an internal scattering component. [Diagram 2] FIG. 2 is a flowchart showing an example of a process for measuring biometric data of a user in this embodiment. [Figure 3A] FIG. 3A is a diagram illustrating an example of the operation of a process for measuring biometric data of a user in this embodiment. [Figure 3B] FIG. 3B is a diagram illustrating the relationship between the change in the amount of body movement and the invalid period. [Figure 3C] FIG. 3C is a diagram showing a schematic diagram of the relationship between the moving speed of the head and the invalid period. [Figure 3D] FIG. 3D is a diagram illustrating an example of the operation of a process for measuring biometric data of a user in this embodiment. [Figure 3E] FIG. 3E is a diagram illustrating a schematic example of a bioinstrumentation device according to this embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a bioinstrumentation device according to this embodiment. [Figure 5A] FIG. 5A is a flowchart showing an example of a process for measuring biometric data of a user in this embodiment. [Figure 5B] FIG. 5B is a diagram illustrating an example of the relationship between the difference value and the reliability. [Figure 6] FIG. 6 is a diagram for explaining an example of the operation of a process for measuring biometric data of a user in this embodiment. [Figure 7A] FIG. 7A is a diagram illustrating an example of a bioinstrumentation device according to this embodiment. [Figure 7B] FIG. 7B is a diagram showing a schematic example of an arrangement of the various parts when the bioinstrumentation device is installed inside an automobile. [Figure 7C] FIG. 7C is a diagram showing a schematic example of an arrangement of each part when the biomeasurement device is attached to a game machine or an attraction device and installed. [Figure 8]FIG. 8 is a flowchart showing an example of a process for measuring biometric data of a user in this embodiment. [Figure 9] FIG. 9 is a diagram for explaining an example of the operation of a process for measuring biometric data of a user in this embodiment. [Figure 10A] FIG. 10A is a diagram showing the relationship between time and the moving distance of the head. [Figure 10B] FIG. 10B is a graph showing the relationship between time and oxygenated hemoglobin concentration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (How one aspect of the present disclosure was arrived at) In the field of biomeasurement, it has long been known that changes in the user's condition affect the measurement results. For example, in measuring brain activity information using near-infrared spectroscopy (NIRS), the movement of the user's head is called body movement. Body movement has been thought to cause irregular fluctuations in signals indicating changes in cerebral blood flow.

[0011] When measuring brain activity information for research purposes, a sequence is possible in which the operator of the NIRS device determines the irregularity of the signal, notifies the user that "this measurement was an error," and prompts the user to measure again. However, it is not easy to use such a sequence for everyday purposes such as measuring a user's brain activity information.

[0012] Furthermore, when a measurement device measures brain activity information from the user's head in a non-contact manner, head movement changes the relative positional relationship between the measurement device and the user, which further increases the impact of changes in cerebral blood flow on the detection signal.

[0013] Patent Document 1 discloses a method for correcting positional deviations due to movement by pattern matching of cerebral blood flow distribution between frames. The pattern of cerebral blood flow distribution has a low spatial frequency. In other words, the cerebral blood flow distribution shows a spatially gentle distribution. For this reason, it is not necessarily easy to detect the feature points required for matching. In addition, it is computationally expensive to restore the cerebral blood flow distribution in all frames as if there was no head movement.

[0014] Based on the above investigations, the inventors have come up with the bioinstrumentation device described in the following items.

[0015] [Item 1] The bioinstrumentation device according to the first aspect includes a light source for irradiating light onto a user's head, an image sensor, a control circuit for controlling the light source and the image sensor, and a signal processing circuit. The control circuit causes the light source to emit the light, and causes the image sensor to detect at least a portion of the reflected light returning from the head due to the irradiation of the light, and output an image signal. The signal processing circuit generates brain activity data indicating the state of the user's brain based on the image signal, and stops outputting the brain activity data based on at least one selected from the group consisting of the image signal and a sensor signal output from a sensor that detects a change in the user's surrounding environment that affects the brain activity data.

[0016] [Item 2] In the biomeasurement device relating to the first item, the light may be pulsed light, and the control circuit may cause the image sensor to output, as the image signal, a first signal obtained by detecting a component contained in the reflected pulsed light during a period from when the intensity of the reflected pulsed light returning from the head due to irradiation of the pulsed light starts to decrease until when the decrease ends.

[0017] [Item 3] In the biomeasurement device relating to the first or second item, the control circuit may cause the light source to repeatedly emit the light and the image sensor to repeatedly output the image signal for a predetermined period of time, and the signal processing circuit may further calculate a first value based on at least one selected from the group consisting of the image signal and the sensor signal, and may stop outputting the brain activity data during a first period of the predetermined period during which the first value satisfies a preset condition.

[0018] [Item 4] In the biomeasurement device according to the third aspect, the signal processing circuit may output a signal indicating that the brain activity data is invalid during the first period.

[0019] [Item 5] In the biomeasurement device according to the third aspect, the signal processing circuit may output, during the first period, the same brain activity data as the brain activity data generated before the first period.

[0020] [Item 6] In the biomeasurement device relating to the third item, the signal processing circuit may output data obtained by interpolating the brain activity data generated before the first period and the brain activity data generated after the first period as the brain activity data for the first period.

[0021] [Item 7] In the biomeasurement device relating to the third item, the signal processing circuit may stop outputting the brain activity data during at least one period selected from the group consisting of a second period before the start of the first period and a third period after the end of the first period, in addition to the first period.

[0022] [Item 8] In the bioinstrumentation device according to the seventh aspect, the third period may be longer than the second period.

[0023] [Item 9] In the biomeasurement device according to any one of the third to eighth aspects, the frequency at which the first value is calculated may be equal to or greater than the frequency at which the brain activity data is generated.

[0024] [Item 10] In the biomeasurement device according to any one of items 1 to 9, the light is pulsed light, and the control circuit causes the image sensor to output, as the image signal, a second signal obtained by detecting a component contained in the reflected pulsed light before the intensity of the reflected pulsed light returning from the head due to irradiation of the pulsed light begins to decrease, and the signal processing circuit may stop outputting the brain activity data based on the second signal.

[0025] [Item 11] In the biomeasurement device related to the 10th item, the signal processing circuit may further calculate the amount of displacement of the head from a reference position or the movement speed of the head based on the second signal, and stop outputting the brain activity data when the absolute value of the amount of displacement or the absolute value of the movement speed exceeds a threshold value.

[0026] [Item 12] In the biomeasurement device related to the 10th item, the signal processing circuit may further calculate a luminance value of the head or a rate of change of the luminance value of the head based on the second signal, and stop outputting the brain activity data when the absolute value of the luminance value or the absolute value of the rate of change of the luminance value exceeds a threshold value.

[0027] [Item 13] In the biomeasurement device related to the 10th item, the signal processing circuit may further calculate an area of ​​a specified region in the head based on the second signal, and stop outputting the brain activity data when the area is smaller than a threshold value.

[0028] [Item 14] In the biomeasurement device relating to any of items 1 to 9, the signal processing circuit may further calculate a second value using the brain activity data, and stop outputting the brain activity data when an absolute value of a rate of change of the second value exceeds a threshold value.

[0029] [Item 15] In the biomeasurement device according to the first aspect, the sensor may be an acceleration sensor installed in the surrounding environment.

[0030] [Item 16] In the bioinstrumentation device according to the first aspect, the sensor may be an illuminance sensor installed in the surrounding environment.

[0031] [Item 17] In the biomeasurement device according to the first item, the sensor may be at least one selected from the group consisting of a steering angle sensor, a gear position sensor, and a speed sensor arranged in a vehicle driven by the user.

[0032] [Item 18] The bioinstrumentation device according to the eighteenth item includes a light source for irradiating light onto a user's head, an image sensor, a control circuit for controlling the light source and the image sensor, and a signal processing circuit. The control circuit causes the light source to emit the light, and causes the image sensor to detect at least a portion of the reflected light returning from the head due to the irradiation of the light, and outputs an image signal. The signal processing circuit generates brain activity data indicating the state of the user's brain based on the image signal, calculates the reliability of the brain activity data based on at least one selected from the group consisting of the image signal and a sensor signal output from a sensor that detects a change in the user's surrounding environment that affects the brain activity data, and outputs reliability data indicating the reliability.

[0033] [Item 19] In the biomeasurement device according to an eighteenth aspect, the signal processing circuit may output the reliability data together with the brain activity data.

[0034] [Item 20] In the biomeasurement device relating to the 18th or 19th item, the signal processing circuit may further calculate a first value based on at least one selected from the group consisting of the image signal and the sensor signal, and may calculate a lower reliability as the first value deviates from a preset value.

[0035] [Item 21] In the biomeasurement device relating to the 18th or 19th item, the signal processing circuit may further calculate a first value based on at least one selected from the group consisting of the image signal and the sensor signal, and the longer the period during which the first value exceeds a predetermined value, the lower the reliability calculated.

[0036] [Item 22] In the biomeasurement device relating to the 18th or 19th item, the signal processing circuit may further calculate a first value based on at least one selected from the group consisting of the image signal and the sensor signal, and may calculate a lower reliability the longer the period during which the first value exceeds a predetermined value during a certain period of time.

[0037] [Item 23] In the biomeasurement device relating to any of items 18 to 22, the sensor may be at least one selected from the group consisting of a steering angle sensor, a gear position sensor, and a speed sensor arranged in a vehicle driven by the user.

[0038] [Item 24] In a biomeasurement device relating to any of items 18 to 23, the light is pulsed light, and the control circuit causes the image sensor to output, as the image signal, a second signal obtained by detecting a component contained in the reflected pulsed light before the intensity of the reflected pulsed light returning from the head due to the irradiation of the pulsed light begins to decrease, and the signal processing circuit may calculate the reliability based on the second signal.

[0039] [Item 25] In the biomeasurement device relating to the first or second item, the control circuit may cause the light source to repeatedly emit the light and the image sensor to repeatedly output the image signal for a predetermined period of time, and the signal processing circuit may further calculate a first value based on at least one selected from the group consisting of the image signal and the sensor signal, and may stop outputting the brain activity data from after a delay time has elapsed since the start of a first period in which the first value meets a preset condition within the predetermined period, until the end of the first period.

[0040] [Item 26] The biomeasurement method relating to the 26th item includes causing a light source to emit light that illuminates a user's head; causing an image sensor to detect at least a portion of the reflected light returning from the head due to the irradiation of the light and outputting an image signal; generating brain activity data indicative of a state of the user's brain based on the image signal; and stopping the output of the brain activity data based on at least one selected from the group consisting of the image signal and a sensor signal output from a sensor that detects changes in the user's surrounding environment that affect the brain activity data.

[0041] [Item 27] The biomeasurement method relating to the 27th item includes causing a light source to emit light that illuminates a user's head; causing an image sensor to detect at least a portion of the reflected light returning from the head due to the irradiation of the light and outputting an image signal; generating brain activity data indicative of a state of the user's brain based on the image signal; calculating a reliability of the brain activity data based on at least one selected from the group consisting of the image signal and a sensor signal output from a sensor that detects changes in the user's surrounding environment that affect the brain activity data, and outputting reliability data indicative of the reliability.

[0042] [Item 28] The program related to the 28th item is a program used in a biomeasurement device, the biomeasurement device comprising a light source that emits light toward a target part of a user, an image sensor, a control circuit that controls the light source and the image sensor, and a signal processing circuit that processes a signal output from the image sensor, the control circuit causing the light source to emit light and the image sensor to detect at least a portion of the light emitted from the light source and returned from the target part of the user and output an image signal, the program causing the signal processing circuit to generate biomeasurement data indicating the status of the user based on the image signal output from the image sensor, and decide whether to output the biomeasurement data based on a sensor signal output from the image sensor and / or another sensor that detects changes in the user's surrounding environment that affect the biomeasurement data, or calculates the reliability of the biomeasurement data based on the sensor signal and outputs reliability data indicating the reliability.

[0043] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, and arrangement positions of the components shown in the following embodiments are merely examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in an independent claim showing a top concept are described as optional components.

[0044] In the present disclosure, all or part of a circuit, unit, device, member or part, or all or part of a functional block in a block diagram may be implemented by one or more electronic circuits including a semiconductor device, a semiconductor integrated circuit (IC), or an LSI (large scale integration). The LSI or IC may be integrated into one chip, or may be configured by combining multiple chips. For example, functional blocks other than memory elements may be integrated into one chip. Here, although it is called an LSI or an IC, the name may change depending on the degree of integration, and it may be called a system LSI, a VLSI (very large scale integration), or an ULSI (ultra large scale integration). A Field Programmable Gate Array (FPGA), which is programmed after the manufacture of an LSI, or a reconfigurable logic device, which can reconfigure the connection relationship inside the LSI or set up the circuit partition inside the LSI, may also be used for the same purpose.

[0045] Furthermore, all or part of the functions or operations of the circuit, unit, device, member, or part can be executed by software processing. In this case, the software is recorded in one or more non-transitory recording media such as ROMs, optical disks, hard disk drives, etc., and when the software is executed by a processor, the functions specified in the software are executed by the processor and peripheral devices. The system or device may include one or more non-transitory recording media on which the software is recorded, a processor, and necessary hardware devices, such as interfaces.

[0046] Hereinafter, the bioinstrumentation device according to this embodiment will be specifically described with reference to the drawings.

[0047] (First embodiment) [1. Biometrics equipment] First, the configuration and operation of a bioinstrumentation device 10 in the first embodiment will be described with reference to FIGS. 1A to 3. FIG.

[0048] 1A is a diagram illustrating an example of a biomeasurement device 10 according to the present embodiment. The biomeasurement device 10 includes an imaging unit 121, a measurement unit 110, and a signal processing circuit 122. The imaging unit 121 includes a light source 101, an image sensor 102 including a photoelectric conversion unit 103 and a charge storage unit 104, a control circuit 105 including a light source control unit 106 and a sensor control unit 107, and an image signal acquisition unit 108. The signal processing circuit 122 includes a biomeasurement data generation unit 109 and an output determination unit 111.

[0049] [1-1.Light source 101] The light source 101 emits light toward a target part of the user 100. The target part of the user 100 is, for example, the head, and more specifically, the forehead. The light emitted from the light source 101 and reaching the user 100 is divided into a surface reflection component I1 that is reflected on the surface of the user 100 and an internal scattering component I2 that is scattered inside the user 100. The internal scattering component I2 is a component that is reflected or scattered once or multiple times inside the living body. When light is irradiated to the forehead of the user 100, the internal scattering component I2 refers to a component that reaches a part about 8 mm to 16 mm deep from the surface of the forehead, such as the brain, and returns to the biomeasurement device 10 again. The surface reflection component I1 includes three components: a direct reflection component, a diffuse reflection component, and a diffuse reflection component. The direct reflection component is a reflection component whose incidence angle and reflection angle are equal. The diffuse reflection component is a component that is diffusely reflected due to the uneven shape of the surface. The diffuse reflection component is a component that is scattered and reflected by internal tissue near the surface. When light is emitted toward the forehead of user 100, the diffuse reflection component is a component that is scattered and reflected inside the epidermis. In the following, in this disclosure, the surface reflection component I1 that is reflected on the surface of user 100 includes these three components. The surface reflection component I1 and the internal scattering component I2 change their traveling direction due to reflection or scattering, and some of them reach the image sensor 102.

[0050] First, a method of acquiring the internal scattering component I2 will be described. The light source 101 repeatedly emits pulsed light multiple times at a predetermined time interval or at a predetermined timing according to an instruction from the light source control unit 106. The pulsed light emitted from the light source 101 may be, for example, a rectangular wave with a fall period close to zero. In this specification, the "fall period" is the period from when the intensity of the pulsed light starts to decrease to when the decrease ends. In general, light incident on the user 100 propagates through the user 100 by following various paths, and is emitted from the surface of the user 100 with a time difference. For this reason, the rear end of the internal scattering component I2 of the pulsed light has a spread. When the target part is the forehead, the spread of the rear end of the internal scattering component I2 is about 4 ns. In consideration of this, the fall period of the pulsed light can be set to, for example, 2 ns or less, which is half of the fall period. The fall period may be further half of the fall period, which is 1 ns or less. The rise period of the pulsed light emitted from the light source 101 is arbitrary. In this specification, the "rise period" refers to the period from when the intensity of the pulsed light starts to increase until the increase ends. In the detection of the internal scattering component I2 in this embodiment, the falling portion of the pulsed light is used, and the rising portion is not used. The rising portion of the pulsed light can be used to detect the surface reflection component I1. The light source 101 can be, for example, a laser such as an LD. The light emitted from the laser has a steep time response characteristic in which the falling portion of the pulsed light is approximately perpendicular to the time axis.

[0051] The wavelength of the light emitted from the light source 101 may be any wavelength within a wavelength range of, for example, 650 nm to 950 nm. This wavelength range is included in the wavelength range from red to near-infrared. In this specification, the term "light" is used for not only visible light but also infrared light. The above wavelength range is called the "biological window" and has the property of being relatively difficult to be absorbed by moisture and skin in a living body. When a living body is to be detected, the detection sensitivity can be increased by using light within the above wavelength range. When detecting changes in blood flow in the skin and brain of the user 100 as in this embodiment, it is considered that the light used is mainly absorbed by oxygenated hemoglobin (HbO2) and deoxygenated hemoglobin (Hb). Oxygenated hemoglobin and deoxygenated hemoglobin have different wavelength dependences of light absorption. In general, when a change occurs in blood flow, the concentrations of oxygenated hemoglobin and deoxygenated hemoglobin change. Therefore, the degree of light absorption also changes. Therefore, when the blood flow changes, the amount of light detected also changes over time.

[0052] The light source 101 may emit light having two or more wavelengths included in the above wavelength range. Such light having multiple wavelengths may be emitted from multiple light sources, respectively.

[0053] In the bioinstrumentation device 10 of this embodiment, in order to measure the user 100 in a non-contact manner, a light source 101 designed in consideration of the effect on the retina may be used. For example, a light source 101 that satisfies Class 1 of the laser safety standards established in various countries may be used. When Class 1 is satisfied, light with such low illuminance that the exposed emission limit (AEL) is below 1 mW is emitted toward the user 100. Note that the light source 101 itself does not have to satisfy Class 1. For example, the laser safety standard Class 1 may be satisfied by disposing a diffusion plate or an ND filter in front of the light source 101 to diffuse or attenuate the light.

[0054] The pulsed light emitted from the light source 101 does not need to be ultrashort pulsed light. The pulse width can be any width. When light is applied to the forehead to measure cerebral blood flow, the amount of light of the internally scattered component I2 is extremely small, ranging from several thousandths to several tens of thousands, compared to that of the surface reflected component I1. Furthermore, in consideration of laser safety standards, the amount of light emitted is small, making it difficult to detect the internally scattered component I2. Therefore, if the light source 101 emits pulsed light with a relatively large pulse width, the accumulated amount of the internally scattered component I2 with a time delay can be increased, the amount of detected light can be increased, and the signal-to-noise ratio can be improved.

[0055] The light source 101 emits pulsed light with a pulse width of, for example, 3 ns or more. In general, the temporal spread of light scattered in biological tissue such as the brain is about 4 ns.

[0056] The light source 101 may emit a pulsed light having a pulse width of 5 ns or more, or even 10 ns or more. On the other hand, if the pulse width is too large, the amount of unused light increases and is wasted. For this reason, the light source 101 emits a pulsed light having a pulse width of, for example, 50 ns or less. Alternatively, the light source 101 may emit a pulsed light having a pulse width of 30 ns or less, or even 20 ns or less.

[0057] The irradiation pattern of the light source 101 may have a uniform intensity distribution within the irradiation region, for example. In the bioinstrumentation device 10 of this embodiment, the surface reflection component I1 can be separated from the internal scattering component I2 over time and reduced. For this reason, the light source 101 having an emission pattern with a uniform intensity distribution can be used. The irradiation pattern with a uniform intensity distribution may be formed by diffusing the light emitted from the light source 101 with a diffusion plate.

[0058] In this embodiment, unlike the conventional technology, it is possible to detect the internal scattering component I2 emitted from the position on the user 100 where the light from the light source 101 is irradiated. By irradiating the user 100 with light over a spatially wide range, it is also possible to improve the measurement resolution.

[0059] [1-2.Image sensor 102] The image sensor 102 detects at least a portion of the reflected pulse light emitted from the light source 101 and returned from the target area of ​​the user 100. The image sensor 102 outputs one or more image signals according to the intensity of the detected light. The image signal acquisition unit 108 acquires the image signal output from the image sensor 102. The image signal acquisition unit 108 sends the acquired image signal to a biomeasurement data generation unit 109 and a measurement unit 110, which will be described later. When brain activity data indicating the state of brain activity is generated from the image signal as the biomeasurement data, the image signal is a signal corresponding to the intensity and included in at least a portion of the falling period of the reflected pulse light.

[0060] The image sensor 102 includes a photoelectric conversion unit 103 including a plurality of photoelectric conversion elements, and a charge accumulation unit 104. Specifically, the image sensor 102 has a plurality of photodetection cells arranged two-dimensionally, and acquires two-dimensional information of the user 100 at once. In this specification, the photodetection cells are also referred to as "pixels." The image sensor 102 is, for example, any image sensor such as a CCD image sensor or a CMOS image sensor.

[0061] The image sensor 102 includes an electronic shutter. The electronic shutter is a circuit that controls the timing of imaging. In this embodiment, the sensor control unit 107 in the control circuit 105 has the function of the electronic shutter. The electronic shutter controls the period of one signal accumulation in which the received light is converted into an effective electric signal and accumulated, and the period in which the signal accumulation is stopped. The signal accumulation period can also be called the "exposure period" or the "shooting period". In the following description, the width of the exposure period may be called the "shutter width". The time from the end of one exposure period to the start of the next exposure period may be called the "non-exposure period". Hereinafter, the exposed state may be called "OPEN", and the stopped exposure state may be called "CLOSE".

[0062] The image sensor 102 can adjust the exposure period and non-exposure period by an electronic shutter in the range of sub-nanoseconds, for example, 30 ps to 1 ns. In the biomeasurement device 10 of this embodiment, at least when generating brain activity data, it is not necessary to correct the amount of light from the subject. Therefore, the shutter width does not necessarily need to be larger than the pulse width. Therefore, the shutter width can be set to a value of, for example, 1 ns or more and 30 ns or less. According to the biomeasurement device 10 of this embodiment, the shutter width can be reduced. Therefore, the influence of dark current contained in the detection signal can be reduced.

[0063] When light is emitted toward the forehead of the user 100 to detect information such as cerebral blood flow, the attenuation rate of light inside is very large. For example, compared with the incident light, the emitted light may attenuate to about one millionth. For this reason, the amount of light may be insufficient to detect the internal scattering component I2 with only one pulse of irradiation. The amount of light is weak in the irradiation of Class 1 of the laser safety standard. In this case, the light source 101 emits pulsed light multiple times, and the image sensor 102 is exposed multiple times by the electronic shutter accordingly. This allows the detection signal to be accumulated to improve sensitivity.

[0064] An example of the configuration of the image sensor 102 will now be described.

[0065] The image sensor 102 may include a plurality of pixels arranged two-dimensionally on an imaging surface. Each pixel may include a photoelectric conversion element such as a photodiode, and one or more charge accumulation units. Below, an example will be described in which each pixel includes a photoelectric conversion element and two charge accumulation units. The photoelectric conversion element generates a signal charge according to the amount of received light by photoelectric conversion. One of the two charge accumulation units accumulates a signal charge generated by a surface reflection component I1 of the pulsed light, and the other accumulates a signal charge generated by an internal scattering component I2 of the pulsed light.

[0066] The control circuit 105 causes the light source 101 to emit one or more pulsed lights in order to acquire the internal scattering component I2. The control circuit 105 causes the image sensor 102 to detect, for each pixel of the image sensor 102, a component included in a falling period of each pulsed light returned from the target part of the user 100. The component includes the internal scattering component I2. The control circuit 105 causes the image sensor 102 to output a signal obtained by the detection. In this embodiment, an image signal based on the internal scattering component I2 is used to generate biomeasurement data. Here, the light source 101 may emit light of two types of wavelengths.

[0067] The control circuit 105 causes the light source 101 to emit one or more pulsed lights in order to acquire the surface reflection component I1. The control circuit 105 causes the image sensor 102 to detect, for each pixel of the image sensor 102, a component included before the falling period of each pulsed light returned from the target part of the user 100. The component includes the surface reflection component I1. Note that, before the falling period means before the intensity of each pulsed light starts to decrease. The control circuit 105 causes the image sensor 102 to output a signal obtained by the detection. If the component included before the falling period of each pulsed light is detected, the S / N ratio of the image signal can be improved. In this specification, "detecting the component included before the falling period of each pulsed light" includes, for example, a case where a component included in the rising period of each pulsed light is detected, or a case where the entirety of each pulsed light is detected. In this embodiment, the image signal based on the surface reflection component I1 is used to measure a state related to the validity of biomeasurement data.

[0068] Fig. 1B is a diagram showing a schematic configuration example of one pixel 201 of the image sensor 102. Note that Fig. 1B shows a schematic configuration of one pixel 201 and does not necessarily reflect an actual structure. The pixel 201 shown in Fig. 1B includes a photodiode 203 that performs photoelectric conversion, a first floating diffusion layer 204, a second floating diffusion layer 205, a third floating diffusion layer 206, and a fourth floating diffusion layer 207 that are charge storage units, and a drain 202 that discharges signal charges.

[0069] Photons incident on each pixel due to emission of one pulse light are converted into signal electrons, which are signal charges, by the photodiode 203. The converted signal electrons are either discharged to the drain 202 or distributed to any one of the first floating diffusion layer 204 to the fourth floating diffusion layer 207 in accordance with a control signal input from the control circuit 105.

[0070] The emission of pulsed light from the light source 101, the accumulation of signal charges in the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207, and the discharge of the signal charges to the drain 202 are repeated in this order. This repetitive operation is fast and can be repeated tens of thousands to hundreds of millions of times within the time of one frame of a moving image, for example. The time of one frame is, for example, about 1 / 30 seconds. The pixel 201 finally generates and outputs four image signals based on the signal charges accumulated in the first floating diffusion layer 204 to the fourth floating diffusion layer 207.

[0071] In this example, the control circuit 105 causes the light source 101 to repeatedly emit pulsed light having a wavelength λ1 and pulsed light having a wavelength λ2 in sequence. By selecting two wavelengths having different absorption rates in the internal tissue of the user 100 as the wavelength λ1 and the wavelength λ2, the condition of the user 100 can be analyzed. For example, a wavelength longer than 805 nm may be selected as the wavelength λ1, and a wavelength shorter than 805 nm may be selected as the wavelength λ2. This makes it possible to detect changes in the oxygenated hemoglobin concentration and the deoxygenated hemoglobin concentration in the blood of the user 100.

[0072] First, the control circuit 105 causes the light source 101 to emit a pulsed light having a wavelength λ1. The control circuit 105 causes the first floating diffusion layer 204 to accumulate signal charges during a first period during which the internally scattered component I2 of the pulsed light having the wavelength λ1 is incident on the photodiode 203. In order to acquire the internally scattered component I2, the pulsed light is emitted at a predetermined timing. This pulsed light is referred to as a first pulsed light.

[0073] Next, the control circuit 105 accumulates signal charges in the second floating diffusion layer 205 during a second period in which the surface reflection component I1 of the pulsed light with wavelength λ1 is incident on the photodiode 203. In order to acquire the surface reflection component I1, a pulsed light is emitted at a predetermined timing different from the timing when the internal scattering component is acquired. This pulsed light is referred to as a second pulsed light.

[0074] Next, the control circuit 105 causes the light source 101 to emit a pulsed light having a wavelength λ2. The control circuit 105 causes the third floating diffusion layer 206 to accumulate signal charges during a third period in which the internal scattering component I2 of the pulsed light having the wavelength λ2 is incident on the photodiode 203.

[0075] Next, the control circuit 105 causes the fourth floating diffusion layer 207 to accumulate signal charges during a fourth period in which the surface reflection component I1 of the pulsed light having the wavelength λ2 is incident on the photodiode 203.

[0076] In this way, the control circuit 105 sequentially accumulates the signal charge from the photodiode 203 in the first floating diffusion layer 204 and the second floating diffusion layer 205 with a predetermined time difference after starting the emission of the pulsed light of wavelength λ1. Thereafter, the control circuit 105 sequentially accumulates the signal charge from the photodiode 203 in the third floating diffusion layer 206 and the fourth floating diffusion layer 207 with the predetermined time difference after starting the emission of the pulsed light of wavelength λ2. The above operation is repeated multiple times. In order to estimate the amount of disturbance light and environmental light, a period may be provided in which signal charge is accumulated in other floating diffusion layers (not shown) with the light source 101 turned off. A signal from which disturbance light and environmental light components have been removed can be obtained by subtracting the amount of signal charge in the other floating diffusion layers from the amount of signal charge in the first floating diffusion layer 204 to the fourth floating diffusion layer 207.

[0077] In this embodiment, the number of charge accumulation units is set to four, but may be set to two or more depending on the purpose. For example, when only one type of wavelength is used, the number of charge accumulation units may be two. In addition, when only one type of wavelength is used and the surface reflection component I1 is not detected, the number of charge accumulation units per pixel may be one. Even when two or more types of wavelengths are used, the number of charge accumulation units may be one if imaging using each wavelength is performed in a different frame. In addition, as described later, the number of charge accumulation units may be one if the detection of the surface reflection component I1 and the detection of the internal scattering component I2 are performed in different frames.

[0078] FIG. 1C is a diagram showing an example of the configuration of the image sensor 102. In the example shown in FIG. 1C, an area surrounded by a two-dot chain line frame corresponds to one pixel 201. The pixel 201 includes one photodiode. Although only four pixels arranged in two rows and two columns are shown in FIG. 1C, a larger number of pixels may be arranged in practice. The pixel 201 includes four floating diffusion layers, namely, a first floating diffusion layer 204 to a fourth floating diffusion layer 207. The signals accumulated in the four floating diffusion layers, namely, the first floating diffusion layer 204 to the fourth floating diffusion layer 207, are treated as if they were signals of four pixels of a general CMOS image sensor, and are output from the image sensor 102.

[0079] Each pixel 201 has four signal detection circuits. Each signal detection circuit includes a source follower transistor 309, a row selection transistor 308, and a reset transistor 310. In this example, the reset transistor 310 corresponds to the drain 202 shown in FIG. 1B, and a pulse input to the gate of the reset transistor 310 corresponds to a drain discharge pulse. Each transistor is, for example, but not limited to, a field effect transistor formed on a semiconductor substrate. As shown in the figure, one of the input terminal and the output terminal of the source follower transistor 309 is connected to one of the input terminal and the output terminal of the row selection transistor 308. The one of the input terminal and the output terminal of the source follower transistor 309 is typically a source. The one of the input terminal and the output terminal of the row selection transistor 308 is typically a drain. The gate, which is the control terminal of the source follower transistor 309, is connected to the photodiode 203. The signal charge of holes or electrons generated by the photodiode 203 is accumulated in a floating diffusion layer, which is a charge accumulation portion between the photodiode 203 and the source follower transistor 309 .

[0080] Although not shown in FIG. 1C, the first floating diffusion layer 204 to the fourth floating diffusion layer 207 are connected to the photodiode 203. A switch may be provided between the photodiode 203 and each of the first floating diffusion layer 204 to the fourth floating diffusion layer 207. This switch switches the conduction state between the photodiode 203 and each of the first floating diffusion layer 204 to the fourth floating diffusion layer 207 in response to a signal accumulation pulse from the control circuit 105. This controls the start and stop of accumulation of signal charges in each of the first floating diffusion layer 204 to the fourth floating diffusion layer 207. The electronic shutter in this embodiment has a mechanism for such exposure control.

[0081] The signal charges accumulated in the first floating diffusion layer 204 to the fourth floating diffusion layer 207 are read out by turning on the gate of the row selection transistor 308 by the row selection circuit 302. At this time, the current flowing from the source follower power supply 305 to the source follower transistor 309 and the source follower load 306 is amplified according to the signal potential of the first floating diffusion layer 204 to the fourth floating diffusion layer 207. The analog signal due to this current read out from the vertical signal line 304 is converted into digital signal data by the analog-digital conversion circuit 307 connected to each column. This digital signal data is read out for each column by the column selection circuit 303 and output from the image sensor 102. After reading out one row, the row selection circuit 302 and the column selection circuit 303 read out the next row, and so on in the same manner to read out the information of the signal charges of the floating diffusion layers of all the rows. After reading out all the signal charges, the control circuit 105 resets all the floating diffusion layers by turning on the gate of the reset transistor 310. This completes the imaging of one frame. Similarly, high-speed imaging of frames is repeated thereafter, until the imaging of a series of frames by the image sensor 102 is completed.

[0082] In this embodiment, an example of the CMOS type image sensor 102 has been described, but the image sensor 102 may be a CCD type, a single photon counting element, or an amplification type image sensor such as an EMCCD or ICCD.

[0083] Fig. 1D is a diagram showing a schematic example of an operation in one frame in this embodiment. As shown in Fig. 1D, in one frame, the emission of pulsed light of wavelength λ1 and the emission of pulsed light of wavelength λ2 may be alternately switched multiple times. This reduces the time difference between the acquisition timing of detection images using two types of wavelengths, making it possible to capture images using pulsed light of two wavelengths almost simultaneously.

[0084] In this embodiment, the image sensor 102 detects both the surface reflection component I1 and the internal scattering component I2 of the pulsed light. Biomeasurement data of the user 100 can be generated from a temporal or spatial change in the internal scattering component I2. Meanwhile, data related to the validity of the biomeasurement data can be measured from a temporal or spatial change in the surface reflection component I1.

[0085] In this specification, a signal related to the validity of biometric data may be referred to as a "validity signal."

[0086] [1-3. Control circuit 105] The control circuit 105 controls the above operations of the light source 101 and the image sensor 102. Specifically, the control circuit 105 adjusts the time difference between the emission timing of the pulsed light from the light source 101 and the shutter timing of the image sensor 102. Hereinafter, this time difference may be referred to as "phase" or "phase delay." The "emission timing" of the pulsed light from the light source 101 is the time at which the pulsed light emitted from the light source 101 starts to rise. The control circuit 105 may adjust the phase by changing the emission timing, or may adjust the phase by changing the shutter timing.

[0087] The control circuit 105 may be configured to remove an offset component from a signal detected by the light receiving element of the image sensor 102. The offset component is a signal component due to ambient light such as sunlight or fluorescent light, or disturbance light. For example, the offset component due to ambient light or disturbance light can be estimated by detecting a signal by the image sensor 102 in a state in which the driving of the light source 101 is turned off and no light is emitted.

[0088] The control circuit 105 may be an integrated circuit including a processor, such as a central processing unit (CPU) or a microcomputer, and a memory. The control circuit 105 executes a program recorded in the memory, for example, to adjust the emission timing and the shutter timing, estimate the offset component, and remove the offset component.

[0089] Fig. 1E is a flowchart showing an outline of the operation of the control circuit 105 with respect to the light source 101 and the image sensor 102. The control circuit 105 includes a light source control unit 106 and a sensor control unit 107, and generally executes the operation shown in Fig. 1G, which will be described later. Note that the operation for detecting only the internal scattering component I2 will be described here.

[0090] In step S101, the light source control unit 106 first causes the light source 101 to emit pulsed light for a predetermined time. At this time, the electronic shutter of the image sensor 102 is in a state where exposure is stopped. The sensor control unit 107 causes the electronic shutter to stop exposure until a period in which a part of the pulsed light is reflected by the surface of the user 100 and reaches the image sensor 102 is completed. Next, in step S102, the sensor control unit 107 causes the electronic shutter to start exposure at the timing when another part of the pulsed light is scattered inside the user 100 and reaches the image sensor 102. After a predetermined time has elapsed, in step S103, the sensor control unit 107 causes the electronic shutter to stop exposure. Next, in step S104, the control circuit 105 determines whether the number of times the above signal accumulation has been performed has reached a predetermined number. If this determination is No, steps S101 to S103 are repeated until it is determined to be Yes. If the result of the determination in step S104 is Yes, in step S105, the sensor control unit 107 causes the image sensor 102 to generate and output a signal indicating an image based on the signal charges accumulated in each floating diffusion layer.

[0091] By the above operation, the components of the light scattered inside the measurement object can be detected with high sensitivity. Note that multiple emission and exposure are not essential and are performed as necessary.

[0092] [1-4. Measurement unit 110] The measurement unit 110 measures a validity signal indicating the validity of the biomeasurement data, and sends the measurement result as a signal to an output determination unit 111, which will be described later.

[0093] In this embodiment, the validity signal is measured from an image containing the temporal or spatial variation of the surface reflectance component I1. The image is acquired from the image sensor 102.

[0094] The measurement unit 110 may include an arithmetic circuit that performs arithmetic processing such as image processing. Such an arithmetic circuit may be realized by, for example, a digital signal processor (DSP), a programmable logic device (PLD) such as a field programmable gate array (FPGA), or a central processing unit (CPU) or a graphics processing unit (GPU) in combination with a computer program.

[0095] In another embodiment described below, the efficacy signal may be measured from an image containing temporal or spatial variations in the internal scattering component I2.

[0096] In another embodiment described later, the validity signal may be measured from something other than the image output from the image sensor 102. In this case, the measurement unit 110 may include a sensor disposed inside or outside the biomeasurement device 10. The sensor measures a change in the surrounding environment of the user 100 and outputs a sensor signal indicating the change. The sensor signal may be, for example, a signal indicating the amount of change in a physical change that affects the biomeasurement data among physical changes in the surrounding environment. The sensor signal may be, for example, a signal indicating that the surrounding environment is in a specific state that affects the biomeasurement data. "Affecting the biomeasurement data" includes the mixing of noise into the biomeasurement data or the biomeasurement data becoming in a state where it cannot be measured. The sensor may be at least one selected from the group consisting of an illuminance sensor, an acceleration sensor, a speed sensor, a steering angle sensor, and a gear position sensor.

[0097] [1-4. Signal processing circuit 122] The signal processing circuit 122 processes the signal output from the image sensor 102. The signal processing circuit 122 includes a biomeasurement data generating unit 109 and an output determining unit 111. In the example shown in Fig. 1A, the signal processing circuit 122 and the measuring unit 110 are separate, but may be integrated.

[0098] The biomeasurement data generating unit 109 generates biomeasurement data of the user 100 based on the image signal output from the image sensor 102. When the biomeasurement data is brain activity data of the user 100, the biomeasurement data generating unit 109 processes the image signal including the temporal or spatial change of the internal scattering component I2 to generate moving image data indicating the time change of cerebral blood flow, and sends a signal based on the generation result to the output determining unit 111. The time change of cerebral blood flow is, for example, the time change of the concentration of oxygenated hemoglobin and / or deoxygenated hemoglobin.

[0099] When the biometric data is brain activity data of the user 100, the biometric data generating unit 109 may generate other data related to the cerebral blood flow, not limited to the video data of the cerebral blood flow. The data related to the cerebral blood flow is, for example, the psychological state of the user 100 estimated from the video data of the cerebral blood flow.

[0100] It is known that there is a close relationship between changes in blood components such as cerebral blood flow or hemoglobin and human neural activity. For example, changes in neural cell activity in response to changes in human emotions cause changes in cerebral blood flow or blood components. Therefore, if biological information such as changes in cerebral blood flow or blood components can be measured, the psychological state of the user 100 can be estimated. The psychological state of the user 100 relates to, for example, mood, emotion, health state, or temperature sensation. The mood is, for example, pleasant or unpleasant. The emotion is, for example, relief, anxiety, sadness, or anger. The health state is, for example, vitality or fatigue. The temperature sensation is, for example, hot, cold, or muggy. In addition, derived from this, an index indicating the degree of brain activity is also included in the psychological state. The index is, for example, proficiency, mastery, or concentration. In this specification, such data related to cerebral blood flow is collectively referred to as brain activity data.

[0101] The output decision unit 111 decides whether or not to output the biomeasurement data sent from the biomeasurement data generation unit 109 based on the measurement result sent from the measurement unit 110. The measurement result includes a value calculated by the measurement unit 110 based on the image signal output from the image sensor 102 and / or the sensor signal output from the above-mentioned sensor.

[0102] At the timing to stop output of the biomeasurement data, the output decision unit 111 outputs a signal indicating that the biomeasurement data at that timing is invalid. Alternatively, the output decision unit 111 continues to output the same data as the last valid biomeasurement data. Alternatively, at the timing when the biomeasurement data becomes valid again, the output decision unit 111 outputs data interpolated from the last valid biomeasurement data.

[0103] The signal processing circuit 122 may include an arithmetic circuit for performing arithmetic processing such as image processing, as in the case of the measurement unit 110. Such an arithmetic circuit may be realized by, for example, a digital signal processor (DSP), a programmable logic device (PLD) such as a field programmable gate array (FPGA), or a combination of a central processing unit (CPU) or an image processing processor (GPU) and a computer program. The signal processing circuit 122 and the control circuit 105 may be integrated into one circuit, or may be separate individual circuits. The signal processing circuit 122 may be a component of an external device such as a server provided in a remote location. In this case, the external device such as the server transmits and receives data to and from the light source 101, the image sensor 102, and the control circuit 105 by wired or wireless communication.

[0104] [1-5.Other] The biomeasurement device 10 may include an imaging optical system that forms a two-dimensional image of the user 100 on the light receiving surface of the image sensor 102. The optical axis of the imaging optical system is approximately perpendicular to the light receiving surface of the image sensor 102. The imaging optical system may include a zoom lens. By changing the position of the zoom lens, the magnification ratio of the two-dimensional image of the user 100 changes. This changes the resolution of the two-dimensional image on the image sensor 102. Therefore, even if the distance to the user 100 is far, it is possible to enlarge a desired measurement area and observe it in detail.

[0105] The bioinstrumentation device 10 may also include a bandpass filter between the user 100 and the image sensor 102 that passes only light in the wavelength band emitted from the light source 101 or light in the vicinity of that wavelength band. This can reduce the influence of disturbance components such as ambient light. The bandpass filter may be configured with a multilayer filter or an absorption filter. Taking into account the temperature of the light source 101 and band shifts caused by oblique incidence on the filter, the bandwidth of the bandpass filter may be about 20 to 100 nm.

[0106] Furthermore, the biomeasurement device 10 may include a polarizing plate between the light source 101 and the user 100, and between the image sensor 102 and the user 100. In this case, the polarization directions of the polarizing plate arranged on the light source 101 side and the polarizing plate arranged on the image sensor 102 side are in a crossed Nicol relationship. This makes it possible to prevent the specular reflection component, which has the same angle of incidence and reflection angle, of the surface reflection component I1 of the user 100 from reaching the image sensor 102. In other words, the amount of light that the surface reflection component I1 reaches the image sensor 102 can be reduced.

[0107] [2. Operation of light source and image sensor] The bioinstrumentation device 10 in this embodiment can detect the surface reflection component I1 and the internal scattering component I2 separately. If the user 100 is a person and the target part is the forehead, the signal intensity due to the internal scattering component I2 to be detected is very small. This is because, as described above, the amount of emitted light that meets the safety standards for lasers is very small, and light is scattered and absorbed by the scalp, cerebrospinal fluid, skull, gray matter, white matter, and blood flow. Furthermore, the change in signal intensity due to changes in blood flow or blood flow components during brain activity corresponds to a few tenths of the signal intensity before the change, and is very small. Therefore, in this embodiment, the surface reflection component I1, which is several thousand to several tens of thousands times the signal component to be detected, is removed as much as possible during imaging.

[0108] An example of the operation of the light source 101 and the image sensor 102 in the bioinstrumentation device 10 in this embodiment will be described below.

[0109] As shown in FIG. 1A, when the light source 101 irradiates the user 100 with pulsed light, a surface reflection component I1 and an internal scattering component I2 are generated. A part of the surface reflection component I1 and the internal scattering component I2 reaches the image sensor 102. The internal scattering component I2 passes through the inside of the user 100 before being emitted from the light source 101 and reaching the image sensor 102. For this reason, the optical path length of the internal scattering component I2 is longer than the optical path length of the surface reflection component I1. Therefore, the time at which the internal scattering component I2 reaches the image sensor 102 is delayed on average from the time at which the surface reflection component I1 reaches the image sensor 102.

[0110] FIG. 1F is a diagram showing a schematic diagram of an optical signal that reaches the image sensor 102 when a rectangular pulsed light emitted from the light source 101 returns from the user 100. The horizontal axis represents time (t) in all of the signals (a) to (d). The vertical axis represents intensity in the signals (a) to (c), and the OPEN or CLOSE state of the electronic shutter in the signal (d). The signal (a) shows the surface reflection component I1. The signal (b) shows the internal scattering component I2. The signal (c) shows the sum of the surface reflection component I1 shown in the signal (a) and the internal scattering component I2 shown in the signal (b). As shown in the signal (a), the surface reflection component I1 maintains a rectangular shape. On the other hand, as shown in the signal (b), the internal scattering component I2 is the sum of light that has passed through various optical path lengths. For this reason, the signal (b) shows a characteristic in which the trailing end of the pulsed light is trailing. In other words, the fall period of the internal scattering component I2 is longer than the fall period of the surface reflection component I1, so that the ratio of the internal scattering component I2 is increased and extracted from the optical signal shown in signal (c). For this reason, as shown in signal (d), the exposure of the electronic shutter is started after the rear end of the surface reflection component I1. After the rear end means when or after the surface reflection component I1 falls. The shutter timing of the electronic shutter is adjusted by the control circuit 105. As described above, the bioinstrumentation device 10 in this embodiment distinguishes between the surface reflection component I1 and the internal scattering component I2 that has reached the depth of the object. For this reason, the emission pulse width and the shutter width are arbitrary. Therefore, unlike the conventional method using a streak camera, the cost can be significantly reduced by the simple configuration.

[0111] In the example shown in the signal (a) of FIG. 1F, the rear end of the surface reflection component I1 falls vertically. In other words, the time from when the surface reflection component I1 starts to fall to when it ends is zero. However, in reality, if the waveform of the pulsed light itself emitted from the light source 101 is not completely vertical at the fall, if there are fine irregularities on the surface of the user 100, and / or if scattering occurs within the epidermis. In these cases, the rear end of the surface reflection component I1 does not fall vertically. In addition, since the user 100 is an opaque object, the light amount of the surface reflection component I1 is much greater than the light amount of the internal scattering component I2. Therefore, even if the rear end of the surface reflection component I1 slightly protrudes from the vertical fall position, the internal scattering component I2 may be buried. In addition, a time delay due to electron movement may occur during the readout period of the electronic shutter. For this reason, it may not be possible to realize an ideal binary readout as shown in the signal (d) of FIG. 1F. Therefore, the control circuit 105 may delay the shutter timing of the electronic shutter slightly from immediately after the fall of the surface reflection component I1. For example, the shutter timing of the electronic shutter may be delayed by about 0.5 ns to 5 ns. Instead of adjusting the shutter timing of the electronic shutter, the control circuit 105 may adjust the emission timing of the light source 101. The control circuit 105 adjusts the time difference between the shutter timing of the electronic shutter and the emission timing of the light source 101. When measuring the change in blood flow rate or blood flow components during brain activity in a non-contact manner, if the shutter timing is delayed too much, the internal scattering component I2, which is originally small, will further decrease. For this reason, the shutter timing may be kept near the rear end of the surface reflection component I1. The time delay due to scattering by the user 100 is 4 ns. Therefore, the maximum amount of delay in the shutter timing is about 4 ns.

[0112] Each of the multiple pulsed lights emitted from the light source 101 may be exposed at the same phase shutter timing, thereby amplifying the detected light amount of the internal scattering component I2.

[0113] Instead of or in addition to placing a bandpass filter between the user 100 and the image sensor 102, the control circuit 105 may estimate the offset component by capturing an image with the same exposure time in a state where the light source 101 is not emitting light. The estimated offset component is subtracted from the signal detected by each pixel of the image sensor 102. This makes it possible to remove the dark current component generated on the image sensor 102.

[0114] The internal scattering component I2 includes internal information of the user 100, such as cerebral blood flow information. The amount of light absorbed by the blood changes according to the temporal fluctuation of the cerebral blood flow of the user 100. As a result, the amount of light detected by the image sensor 102 also increases or decreases accordingly. Therefore, by monitoring the internal scattering component I2, it is possible to estimate the brain activity state from the change in the cerebral blood flow of the user 100. In this specification, a signal indicating the internal scattering component I2 among the signals output from the image sensor 102 may be referred to as a "brain activity signal." The brain activity signal may include information on the increase or decrease in the cerebral blood flow of the user 100.

[0115] Next, an example of a method for detecting the surface reflection component I1 will be described. The surface reflection component I1 includes surface information of the user 100. The surface information is, for example, blood flow information on the face and scalp. The image sensor 102 detects the surface reflection component I1 from an optical signal that is generated when pulsed light emitted from the light source 101 reaches the user 100 and returns to the image sensor 102.

[0116] FIG. 1G is a diagram showing an example of a timing chart for detecting the surface reflection component I1. For example, as shown in FIG. 1G, in order to detect the surface reflection component I1, the shutter may be opened before the pulsed light reaches the image sensor 102, and closed before the trailing end of the pulsed light reaches the image sensor 102. By controlling the shutter in this manner, it is possible to reduce the amount of internally scattered components I2 mixed in. As a result, it is possible to increase the proportion of light that passes through the vicinity of the surface of the user 100. The timing of closing the shutter may be immediately after the light reaches the image sensor 102. This enables signal detection with an increased proportion of the surface reflection component I1, which has a relatively short optical path length. By acquiring the signal of the surface reflection component I1, it is also possible to detect the pulse of the user 100 or the degree of oxygenation of the facial blood flow. As another method of acquiring the surface reflection component I1, the image sensor 102 may detect the entire pulsed light emitted from the light source 101, or may detect continuous light emitted from the light source 101.

[0117] 1H is a diagram showing an example of a timing chart for detecting the internal scattering component I2. The signal of the internal scattering component I2 can be acquired by opening the shutter during the period when the trailing end portion of the pulse reaches the image sensor 102.

[0118] The surface reflection component I1 may be detected by a device other than the biomeasurement device 10 that acquires the internal scattering component I2. A device other than the device that acquires the internal scattering component I2, or a separate device such as a pulse wave meter or a Doppler blood flow meter may be used. In this case, the separate device is used in consideration of timing synchronization between devices, optical interference, and alignment of detection points. If time-division imaging is performed using the same camera or the same sensor as in this embodiment, temporal and spatial deviations are unlikely to occur. When signals of both the surface reflection component I1 and the internal scattering component I2 are acquired using the same sensor, the components to be acquired may be switched for each frame as shown in Figures 1G and 1H. Alternatively, as described with reference to Figures 1B to 1D, the components to be acquired at high speed within one frame may be alternately switched. In that case, the detection time difference between the surface reflection component I1 and the internal scattering component I2 can be reduced. In addition, signals of both the surface reflection component I1 and the internal scattering component I2 may be acquired from the same pulsed light.

[0119] Furthermore, the signals of the surface reflection component I1 and the internal scattering component I2 may be obtained using light of two wavelengths. For example, pulsed light of two wavelengths, 750 nm and 850 nm, may be used. This allows the concentration changes of oxygenated hemoglobin and deoxygenated hemoglobin to be calculated from the change in the amount of detected light at each wavelength. When the surface reflection component I1 and the internal scattering component I2 are obtained using two wavelengths, a method of switching between four types of charge accumulation at high speed within one frame may be used, as described with reference to Figures 1B to 1D. Such a method can reduce the time lag of the detection signals.

[0120] The biomeasurement device 10 emits pulsed near-infrared light or visible light toward the forehead of the user 100, and can detect changes in the amount of oxygenated hemoglobin on the scalp or face or the pulse from the temporal change in the surface reflection component I1. The light source 101 emits near-infrared light or visible light to obtain the surface reflection component I1. Measurement is possible with near-infrared light regardless of day or night. When measuring the pulse, visible light, which has higher sensitivity, may be used. During the day, sunlight or an indoor light source, which is external light, may be used instead of lighting. If the amount of light is insufficient, it may be supplemented by a dedicated light source. The internal scattering component I2 includes light components that have reached the brain. By measuring the temporal change in the internal scattering component I2, the temporal increase or decrease in cerebral blood flow can be measured.

[0121] The light that reaches the brain also passes through the scalp and the face surface. Therefore, the fluctuations in the blood flow of the scalp and face are also detected superimposed. In order to remove or reduce the influence, the biomeasurement data generating unit 109 may perform a process of subtracting the surface reflection component I1 from the internal scattering component I2 detected by the image sensor 102 when brain activity data is used as the biomeasurement data. This makes it possible to obtain pure brain blood flow information excluding the blood flow information of the scalp and face. For example, a method of subtracting a value obtained by multiplying the signal of the surface reflection component I1 by a certain coefficient of 1 or more determined in consideration of the optical path length difference from the signal of the internal scattering component I2 may be used as the subtraction method. This coefficient may be calculated by simulation or experiment based on, for example, the average value of the optical constants of a general person's head. Such subtraction processing can be easily performed when the same camera or sensor is used for measurement using light of the same wavelength. This is because it is easy to reduce temporal and spatial deviations and it is easy to match the characteristics of the scalp blood flow component contained in the internal scattering component I2 with the characteristics of the surface reflection component I1.

[0122] The skull is present between the brain and the scalp. Therefore, the two-dimensional distribution of cerebral blood flow is independent of the two-dimensional distribution of scalp and facial blood flow. Therefore, based on the signal detected by the image sensor 102, the two-dimensional distribution of the internal scattering component I2 and the two-dimensional distribution of the surface reflection component I1 may be separated using a statistical method such as independent component analysis or principal component analysis.

[0123] [3. Biometric sequence] Next, an example of a method for measuring biomeasurement data of the user 100 using the biomeasurement device 10 described above will be described. In this embodiment, the measurement unit 110 measures the amount of body movement of the user 100 from an image signal including the surface reflection component I1. In this embodiment, the amount of body movement is the moving distance of the target part. Here, the target part is the head of the user 100. The moving distance can also be referred to as the amount of displacement from a reference position. In this embodiment, based on the measurement result, the output decision unit 111 decides whether or not to output the biomeasurement data.

[0124] FIG. 2 is a flowchart showing an example of a process for measuring biometric data of user 100 in this embodiment.

[0125] In step S201, the biomeasurement device 10 performs initial setting before measuring biomeasurement data. Step S201 includes a process in which the control circuit 105 optimally adjusts the timing of emitting pulsed light from the light source 101 and the shutter timing of the image sensor 102 according to the distance between the biomeasurement device 10 and the user 100. Step S201 also includes a process in which the control circuit 105 causes the image signal acquisition unit 108 to transmit an image signal including the surface reflection component I1 to the measurement unit 110, and a process in which the measurement unit 110 calculates the head position in the initial state and stores it in a memory (not shown) in the measurement unit 110.

[0126] In step S202, the control circuit 105 causes the image signal acquisition unit 108 to send, to the biomeasurement data generation unit 109, an image signal representing an internal image including the internal scattering component I2.

[0127] In step S203, the control circuit 105 causes the image signal acquisition unit 108 to send to the measurement unit 110 an image signal representing a surface image including the surface reflection component I1.

[0128] The order of steps S202 and S203 may be reversed.

[0129] In step S204, the measuring unit 110 calculates the position of the target part from the image signal representing the surface image, and calculates the amount of body movement by calculating a difference value between the position of the target part in the initial state stored in the memory. The difference value is the amount of displacement of the target part. A known image processing method such as feature point extraction using edge detection can be used to calculate the position of the target part from the image signal. A two-dimensional image of a face contains more components with high spatial frequencies than a two-dimensional image of cerebral blood flow. Therefore, the two-dimensional image of a face is advantageous in that it is easier to extract feature points. The measuring unit 110 sends the amount of body movement to the output determining unit 111 as a difference value based on the initial state.

[0130] In step S205, the output determination unit 111 determines whether the amount of body movement is equal to or less than a threshold value. The threshold value is, for example, a value in the range of 1 mm to 30 mm. When obtaining a two-dimensional distribution of cerebral blood flow as biomeasurement data, if the desired resolution of the two-dimensional distribution is relatively low, the threshold value can be increased.

[0131] If the amount of body movement is equal to or less than the threshold, in step S206, the biomeasurement data generating unit 109 generates biomeasurement data from the image signal representing the internal image, and sends the biomeasurement data to the output determining unit 111. In step S207, the output determining unit 111 outputs the biomeasurement data. The output may be displayed on a display unit (not shown) of the biomeasurement device 10, for example, or may be used to control a higher-level system (not shown).

[0132] If the amount of body movement exceeds the threshold, steps S206 and S207 are skipped, and the output decision unit 111 stops outputting the biomeasurement data. Here, instead of stopping the output of the biomeasurement data, the output decision unit 111 may output a signal indicating that the biomeasurement data at that timing is invalid. Alternatively, the output decision unit 111 may continue to output the same data as the last valid biomeasurement data. Alternatively, the output decision unit 111 may output data interpolated from the last valid biomeasurement data at the timing when the biomeasurement data becomes valid again.

[0133] In step S208, it is determined whether or not measurement has been performed for a predetermined period of time. This determination may be made by the bioinstrumentation device 10, or may be made by a higher-level system to which the output of the bioinstrumentation device 10 is connected.

[0134] The aforementioned "predetermined period" may be, for example, a period until the psychological state of the user 100 can be estimated. Alternatively, the "predetermined period" may be a period until a series of tasks given to the user 100 is completed. Alternatively, the "predetermined period" may be a period until the user 100 completes a series of operations. The series of operations is, for example, driving a car or operating a game machine.

[0135] If the measurement for the predetermined period has not been completed, the bioinstrumentation device 10 repeats the sequence from step S202 to step S208. If the measurement for the predetermined period has been completed, the bioinstrumentation device 10 ends the measurement.

[0136] The above-mentioned operation of the biomeasurement device 10 in this embodiment can be summarized as follows. The control circuit 105 causes the light source 101 to emit light and causes the image sensor 102 to output an image signal. The biomeasurement data generation unit 109 generates biomeasurement data based on the image signal. The output decision unit 111 judges whether or not to output the biomeasurement data. The control circuit 105, the biomeasurement data generation unit 109, and the output decision unit 111 repeat the above-mentioned operation. The output decision unit 111 stops outputting the biomeasurement data during a period in which a value calculated based on the image signal output from the image sensor 102 corresponds to a preset condition. In the above-mentioned example, the value calculated based on the image signal output from the image sensor 102 is a difference value indicating the amount of displacement of the target part calculated by the measurement unit 110. The preset condition is that the difference value exceeds a threshold value.

[0137] [4. Operation during bio-measurement] 3A is a diagram illustrating an example of the operation of a process for measuring biometric data of user 100 in this embodiment. In the example shown in FIG.

[0138] In the example shown in parts (a) to (d) of FIG. 3A, the horizontal axis direction represents the progression of frames or time. Part (a) shows a schematic diagram of a change in a surface image, which is an image including a surface reflection component I1. Part (b) shows a change in the amount of body movement. Part (c) shows the determination result. Part (d) shows a change in the oxygenated hemoglobin concentration in the forehead region, which is brain activity data, as an example of biomeasurement data.

[0139] Although each frame is numbered, one or more frames may be inserted between each frame. The frame rate or measurement frequency may be in the range of, for example, 1 fps (frame per second) to 30 fps. The measurement frequency of the amount of body movement and the generation frequency of the biomeasurement data may be different. The change in cerebral blood flow changes gradually over one to several seconds. On the other hand, the body movement acquired as an image without contact changes at a faster rate than the cerebral blood flow. Therefore, in order to detect the body movement more precisely, the measurement frequency of the amount of body movement may be equal to or higher than the generation frequency of the cerebral blood flow data.

[0140] In the example shown in part (a) of FIG. 3A, the amount of body movement measured by the measuring unit 110 exceeds a threshold value due to movement of the subject between frames 3 and 4. There is a correlation between the amount of body movement and the irregularity of the generated biomeasurement data. Therefore, the output determination unit 111 determines that the biomeasurement data is invalid during the period in which the amount of body movement exceeds the threshold value, and stops outputting the biomeasurement data. As a result, as shown in part (d) of FIG. 3A, brain activity data, which is the biomeasurement data, is output in frames 1, 2, and 5. In the example shown in part (b) of FIG. 3A, the period in which the amount of body movement exceeds the threshold value is described as an "invalid period."

[0141] The above-described embodiment makes it possible to measure biometric data in a non-contact manner without prompting the user 100 to measure again. This makes it possible to realize a biometric device that can perform measurements on a daily basis. In addition, calculations associated with the restoration of cerebral blood flow distribution are not required. Therefore, calculation costs can be reduced, making it possible to perform biometric measurements using an inexpensive method.

[0142] FIG. 3B is a diagram showing a schematic diagram of the relationship between the change in the amount of body movement and the invalid period. In the movement shown in FIG. 3A part (b), the period in which the amount of body movement exceeds the threshold is set as the invalid period. On the other hand, as shown in FIG. 3B part (a), in addition to the period in which the amount of body movement exceeds the threshold, the period d before the amount of body movement exceeds the threshold is set as the invalid period. f1 , and the period after the amount of body movement falls below the threshold d r1The period including the period d may be set as an invalid period. f1 and period d r1 For example, a period including either one of the following may be set as an invalid period: f1 and period d r1 In this way, the output determination unit 111 determines whether the value calculated based on the image signal output from the image sensor 102 satisfies a preset condition, and also determines whether the value calculated based on the image signal output from the image sensor 102 satisfies a preset condition, and whether the value calculated based on the image signal satisfies a preset condition, and also determines whether the value calculated based on the image signal satisfies a preset condition. f1 and / or the period after termination d r1 During this period, the output of the biometric data may be stopped even if the value does not meet the condition.

[0143] The timing to start the invalid period may be after the amount of body movement exceeds the threshold. In other words, when the amount of body movement exceeds the threshold, d f1 may be a negative value because there may be a delay due to system processing from when the amount of body movement exceeds the threshold value until the invalid period is set.

[0144] Furthermore, as shown in part (b) of FIG. 3B, in the period d r2 d before the start of the period f2 For example, the period d f2 is a period equivalent to 0.5 frames, and period d r2 The reason for this will be described later in the description of the embodiment.

[0145] In this embodiment, the amount of body movement is calculated from an image signal including the surface reflection component I1. Alternatively, the amount of body movement may be calculated from an image signal including the internal scattering component I2. The image signal including the internal scattering component I2 also includes information on the outer shape of the target area. However, the image signal including the surface reflection component I1 can detect a larger amount of light. Therefore, the image signal including the surface reflection component I1 is more advantageous in terms of signal-to-noise ratio than the image signal including the internal scattering component I2.

[0146] In addition, in this embodiment, the displacement amount of the target part is the measurement target, but the measurement target is not limited to this. The measurement target may be the movement speed of the target part. In a frame in which the absolute value of the movement speed of the target part exceeds a threshold value, the output of brain activity data may be stopped. This is because it is considered that a body movement occurs in that frame. In other words, the movement speed of the target part is the movement distance of the target part between frames.

[0147] Fig. 3C is a diagram showing a relationship between the moving speed of the target part and the invalid period. As an example of a method for setting the invalid period, as shown in Fig. 3C, the invalid period may be set to a period from when the moving speed exceeds a positive threshold value, falls below the positive threshold value, falls below a negative threshold value, and then exceeds the negative threshold value, or from when the moving speed falls below a negative threshold value, exceeds the negative threshold value, exceeds the positive threshold value, and then falls below the positive threshold value.

[0148] Furthermore, the measurement object does not have to be related to the movement of the target part, but may be the brightness value of the target part.

[0149] FIG. 3D is a diagram for explaining an example of the operation of the process of measuring biomeasurement data of the user 100 in this embodiment. In the example shown in FIG. 3D, the target part is the head of the user 100. In the example shown in part (a) of FIG. 3D, the measurement unit 110 measures the luminance value of the target part from an internal image, which is an image including an internal scattering component I2. A period in which the luminance value of the target part exceeds a threshold value is set as an invalid period. The output determination unit 111 stops outputting brain activity data in frames 2 and 3 included in the invalid period. When the luminance value of the target part in the image including the internal scattering component I2 approaches a saturation value, it becomes impossible to generate biomeasurement data accurately. Therefore, by determining the validity based on the luminance value of the target part, the biomeasurement device 10 can output only accurate biomeasurement data.

[0150] In the example shown in FIG. 3D, the period in which the luminance value of the target part exceeds the threshold value is set as the invalid period. On the other hand, conversely, the period in which the luminance value of the target part is below the threshold value may be set as the invalid period. This is because when the luminance value of the target part of the image including the internal scattering component I2 is extremely low, the signal-to-noise ratio of the image is insufficient. In this case, accurate biomeasurement data cannot be generated.

[0151] The measurement target may be the rate of change of the luminance value of the target area. In a frame where the absolute value of the rate of change of the luminance value exceeds a threshold, the output of brain activity may be stopped. This is because it is considered that the luminance is either extremely high or extremely low in that frame.

[0152] Furthermore, the measuring unit 110 may measure the area of ​​a predetermined region in the target part from which biometric data is measured, from the image signal. The area of ​​the predetermined region in the target part is, for example, the area of ​​the forehead. A known image processing method such as the feature point extraction using edge detection described above may be used to calculate the area of ​​the forehead from the image signal. If the forehead part contains a lot of hair and the area of ​​the forehead part is below a threshold value, the output of the biometric data may be stopped. This is because if the area of ​​the target part is small, the accuracy of the generated biometric data decreases.

[0153] Furthermore, the measurement target may be biomeasurement data itself.

[0154] FIG. 3E is a diagram showing a schematic example of a biomeasurement device 10 in this embodiment. In the example shown in FIG. 3E, unlike the example shown in FIG. 1A, the generation result of the biomeasurement data is also sent to the measurement unit 110. The measurement unit 110 measures the change rate of the biomeasurement data. The biomeasurement data is, for example, oxygenated hemoglobin concentration or deoxygenated hemoglobin concentration, which is brain activity data. The change in cerebral blood flow changes slowly over one second to several seconds. On the other hand, the body movement and the change in brightness of the target part affect the measurement value of the cerebral blood flow at a speed faster than the cerebral blood flow. Therefore, by determining the validity based on the change rate of the biomeasurement data, the biomeasurement device 10 can output only accurate biomeasurement data. The output determination unit 111 may stop outputting the biomeasurement data when the absolute value of the change rate of the value calculated from the biomeasurement data exceeds a threshold value.

[0155] Second embodiment The configuration and operation of a bioinstrumentation device 20 in the second embodiment will be described with reference to FIGS.

[0156] FIG. 4 is a diagram illustrating an example of the biomeasurement device 20 in this embodiment. In this embodiment, the measurement unit 110 measures the amount of body movement of the user 100 from an image signal including a surface reflection component I1. In this embodiment, the amount of body movement of the user 100 is the moving distance of the target part. The output decision unit 401 calculates the reliability of the biomeasurement data based on the measurement result, and outputs reliability data indicating the reliability together with the biomeasurement data. Thus, in the second embodiment, unlike the first embodiment, the output decision unit 401 outputs not only the biomeasurement data but also the reliability data.

[0157] FIG. 5A is a flowchart showing an example of a process for measuring biometric data of user 100 in this embodiment.

[0158] The operations of performing initial settings in step S501, acquiring a surface image in step S502, acquiring an internal image in step S503, and calculating body movement in step S504 are the same as those in steps S201 to S204 in the first embodiment.

[0159] In step S505, the output determination unit 401 calculates the reliability based on the amount of body movement calculated from the difference value from the initial state.

[0160] FIG. 5B is a diagram showing an example of the relationship between the difference value and the reliability. When the difference value is 0 mm, the measurement result can be most reliable. Therefore, as shown in FIG. 5B, when the difference value is 0 mm, the reliability may be set to 100%. As shown in FIG. 5B, the reliability may be set to a lower value as the difference value increases. In this way, the output determination unit 401 may calculate a lower reliability as the value calculated from the image signal output from the image sensor 102 deviates from a preset value.

[0161] The biomeasurement device 20 in this embodiment may output biomeasurement data regardless of the magnitude of the amount of body movement. In step S506, the biomeasurement data generating unit 109 generates biomeasurement data from the image signal and sends the biomeasurement data to the output determining unit 401.

[0162] In step S507, the output determination unit 401 outputs the reliability data together with the biomeasurement data. The output may be displayed on a display unit (not shown) of the biomeasurement device 20, for example, or may be used to control a higher-level system (not shown).

[0163] The operation in step S508 is similar to the operation in step S208 in the first embodiment.

[0164] 6 is a diagram illustrating an example of the operation of a process for measuring biometric data of the user 100 in this embodiment. In the example shown in FIG.

[0165] In the example shown in parts (a) to (d) of Fig. 6, the horizontal axis represents the progression of frames or time, similarly to the example shown in parts (a) to (d) of Fig. 3A. Part (a) shows a schematic representation of a surface image that is an image including a surface reflection component I1. Part (b) shows a change in the amount of body movement. Part (c) shows a change in the reliability value. Part (d) shows a change in the oxygenated hemoglobin concentration in the forehead region, which is brain activity data, as an example of biomeasurement data.

[0166] In the second embodiment, unlike the first embodiment, the reliability is calculated according to the amount of body movement of each frame. In the example shown in part (c) of Fig. 6, the output determination unit 401 calculates a lower reliability value as the amount of body movement shown in part (b) of Fig. 6 increases. The amount of body movement is a difference value of the position of the target part based on the initial state.

[0167] In frames 3 and 4, the reliability is low due to movement of the target area. On the other hand, the output determination unit 401 may output biomeasurement data in all frames. The output determination unit 401 outputs reliability data together with the biomeasurement data.

[0168] The above-described embodiment makes it possible to measure biomeasurement data in a non-contact manner without prompting the user 100 to measure again. This makes it possible to realize a biomeasurement device that can be used for daily measurements. In addition, calculations associated with the restoration of cerebral blood flow distribution are not required. Therefore, calculation costs can be reduced, and biomeasurement can be performed using an inexpensive method. Furthermore, a special effect is obtained in that a higher-level system can comprehensively determine the validity of biomeasurement data using reliability data.

[0169] In this embodiment, the reliability is calculated to be a lower value as the measurement result by the measuring unit 110 deviates from the threshold value. However, the method of calculating the reliability is not limited to this.

[0170] The reliability may be calculated based on the time trend of the measurement result, instead of the value of the measurement result itself. For example, the longer the invalid period during which the measurement result exceeds the threshold, the lower the reliability may be calculated. In other words, the longer the period during which the value calculated from the image signal output from the image sensor 102 exceeds a preset value, the lower the reliability may be calculated by the output determination unit 111.

[0171] Also, the higher the ratio of invalid periods in a certain period, the lower the reliability may be calculated. In other words, the longer the period during which the value calculated from the image signal output from the image sensor 102 exceeds a preset value during a certain period, the lower the reliability may be calculated by the output determination unit 111. When the measurement value switches between valid and invalid in a short period of time, it is effective to use such a reliability calculation method.

[0172] (Third embodiment) The configuration and operation of a bioinstrumentation device 30 according to the third embodiment will be described with reference to FIGS. 7A to 9. FIG.

[0173] 7A is a diagram showing a schematic example of a biomeasurement device 30 in this embodiment. The biomeasurement device 30 in this embodiment includes an imaging unit 701, a signal processing circuit 702, and a measurement unit 703. In this embodiment, the measurement unit 703 includes an acceleration sensor installed in the vicinity of the user 100, which is the surrounding environment, and measures acceleration. The output decision unit 111 decides whether or not to output biomeasurement data based on the measurement result.

[0174] 7B is a diagram showing an example of the arrangement of each part when the biomeasurement device 30 is installed inside an automobile. The imaging unit 701, the signal processing circuit 702, and the measurement unit 703 may be arranged separately from each other. The measurement unit 703 is arranged in the vicinity of the user 100. Here, "arranged in the vicinity" means arranged at a distance where there is at least a correlation between the acceleration measured by the measurement unit 703 and the body movement of the user 100 himself.

[0175] FIG. 7C is a diagram showing an example of the arrangement of each part when the biomeasurement device is added to a game machine or an attraction device and installed. As in the example of the arrangement shown in FIG. 7B, the imaging unit 701, the signal processing circuit 702, and the measurement unit 703 may be arranged separately. The user 100 operates the controller 705 based on visual information from the display 704, for example. The measurement unit 703 is attached to headphones or a head-mounted display worn on the head of the user 100, for example. The measurement unit 703 detects acceleration accompanying the movement of the target part of the user. The measurement unit 703 may also detect a change in illuminance of the target part accompanying a change in the display information of the display 704, for example.

[0176] FIG. 8 is a flowchart showing an example of a process for measuring biometric data of user 100 in this embodiment.

[0177] In step S801, the biomeasurement device 30 performs initial settings before measuring biomeasurement data. This step S801 includes a process in which the control circuit 105 optimally adjusts the timing of emitting pulsed light from the light source 101 and the shutter timing of the image sensor 102 according to the distance between the biomeasurement device 30 and the user 100, as in the first embodiment.

[0178] In step S802, the measurement unit 703 measures acceleration. If the bioinstrumentation device 30 is installed inside a car, measuring the acceleration corresponds to measuring the vibration of the car.

[0179] The measurement frequency of the acceleration may be equal to or higher than the generation frequency of the biomeasurement data. This improves the followability of the measurement value when a sudden vibration occurs. As a result, the accuracy of the determination in the output determination unit 111 can be improved. In this embodiment, as an example, the measurement frequency of the acceleration is set to 10 times the calculation frequency of the brain activity data.

[0180] In step S803, acceleration values ​​are sequentially stored in a memory (not shown) in the measurement unit 703. In step S804, a loop of steps S802 and S803 is repeated, for example, ten times, by a step of determining the number of times acceleration is measured. There is no limit to the number of times measurement is performed.

[0181] In step S805, the measurement unit 703 refers to the contents of the memory, and sends the maximum value from among the accumulated acceleration values ​​for the 10 times to the output determination unit 111. The average value of the 10 times may be used instead of the maximum value. In this embodiment, a value that is at least correlated with the body movement of the user 100 himself is calculated from the acceleration values ​​for the 10 times.

[0182] In step S806, the output determination unit 111 judges whether the maximum value of the acceleration value received from the measurement unit 703 is equal to or less than a threshold value. This threshold value is, for example, an acceleration in any one of the directions of forward / backward, left / right, or vertical, and is in the range of 0.1 G to 1 G (G=9.8 m / s 2 ) range.

[0183] In step S807, the control circuit 105 causes the image signal acquisition unit 108 to send an image signal including the internal scattering component I2 to the biomeasurement data generation unit 109. The operations in steps S808, S809, and S810 are the same as the operations in steps S206, S207, and S208 in the first embodiment, respectively.

[0184] 9 is a diagram illustrating an example of the operation of a process for measuring biometric data of the user 100 in this embodiment. In the example shown in FIG.

[0185] In the examples shown in parts (a) to (d) of FIG. 9, the horizontal axis represents the progression of frames or time. Part (a) shows a schematic representation of the state of the housing of a car or an attraction device. Part (b) shows the change in acceleration. Part (c) shows the result of the determination. Part (d) shows the change in oxygenated hemoglobin concentration in the forehead region, which is brain activity data, as an example of biomeasurement data.

[0186] In the third embodiment, unlike the first embodiment, the measurement unit 703 measures acceleration. The reason for measuring acceleration is that there is a correlation between the acceleration in the vicinity of the user 100 and the amount of body movement of the user 100 himself / herself.

[0187] In the example shown in FIG. 9, the housing of the car or attraction device vibrates in frames 3 and 4. As a result, the acceleration measured by the measuring unit 703 exceeds the threshold value. There is a correlation between the amount of body movement and the irregularity of the generated biomeasurement data. Therefore, during the invalid period in which the acceleration exceeds the threshold value, the output determining unit 111 determines that the biomeasurement data is invalid, and stops outputting the biomeasurement data. As a result, as shown in part (d) of FIG. 9, brain activity data, which is the biomeasurement data, is output in frames 1, 2, and 5.

[0188] According to the embodiment described above, it is possible to measure biometric data in a non-contact manner without prompting the user 100 to measure again. This makes it possible to realize a biometric device that can perform measurements on a daily basis. In addition, calculations associated with the restoration of cerebral blood flow distribution are not required. Therefore, calculation costs can be reduced, and biometric measurements can be performed in an inexpensive manner. In addition, since there is no need to measure the amount of body movement from image signals, a special effect is obtained in that calculation costs can be further reduced.

[0189] The measurement target of the measuring unit 703 is not limited to acceleration as long as it is correlated with at least body movement. For example, when the bioinstrumentation device 30 is installed in a car, the measuring unit 703 may include a speed sensor to measure the speed. Also, the measuring unit 703 may include a steering angle sensor to measure the steering angle of the car.

[0190] Moreover, the measurement unit 703 may include a gear position sensor and measure the gear position of the automobile. In this case, the output determination unit 111 stops outputting biomeasurement data during a period in which the measurement result of the gear position indicates a preset state. As shown in FIG. 3B, the output determination unit 111 may stop outputting biomeasurement data during the period in addition to the period, a period before the start of the period, and / or a period after the end of the period. The preset state is, for example, a state in which the gear is in reverse. In this state, the user 100 drives while looking back in order to reverse the automobile. This causes the target part of the user 100 to be displaced from the reference position, causing the body movement of the user 100.

[0191] Furthermore, the measurement target of the measuring unit 703 may not be related to the body movement of the user 100. The measuring unit 703 may include an illuminance sensor installed near the user 100, which is the surrounding environment, and measure illuminance. In this case, the output determining unit 111 determines the period in which the illuminance value exceeds the threshold value as an invalid period and stops outputting the biomeasurement data. When the luminance value of the target part in the image including the internal scattering component I2 approaches the saturation value, the biomeasurement data cannot be generated accurately. Therefore, when there is at least a correlation between the illuminance value and the luminance value of the target part, the biomeasurement device 30 can output only accurate biomeasurement data by determining the validity based on the luminance value of the target part.

[0192] (Example) Below, examples will be described that were carried out to confirm the principles of the present disclosure.

[0193] The biomeasurement device 10 shown in FIG. 1A in the first embodiment was placed so as to face the head of the user 100, which is the subject. The face of the user 100 was placed on a chin rest to prevent unintended body movements. After the user 100 was made to relax, the user's head was photographed for 30 seconds, and brain activity data was measured as biomeasurement data.

[0194] The wavelengths of the light sources were 750 nm and 850 nm. The image signals including the surface reflection component I1 were acquired at 30 fps. The image signals including the internal scattering component I2 were acquired at 5 fps. The resolution of the image signals was 320 pixels × 240 pixels. The target region was a 50 pixel × 50 pixel area including the center of the forehead, and spatial averaging was performed, as well as time averaging in real time. The oxyhemoglobin concentration was calculated based on the image signals including the internal scattering component I2 of the two wavelengths, and the change in the calculated oxyhemoglobin concentration was monitored.

[0195] At the same time, based on the image signal including the surface reflection component I1, the change in the moving distance of the face from the measurement unit 110 was monitored. The Kanade-Lucas-Tomasi (KLT) algorithm was used to extract and track the facial feature points.

[0196] Only once during the measurement, the user 100 intentionally moved his head to the right and then immediately returned it to the original position.

[0197] FIG. 10A is a diagram showing the relationship between time and the head movement distance. FIG. 10B is a diagram showing the relationship between time and the oxygenated hemoglobin concentration. In the example shown in FIG. 10A, X represents the change in the left-right direction, Y represents the change in the up-down direction, and Z represents the change in the depth direction. The movement distance is the amount of displacement from the initial value in each of the X direction, Y direction, and Z direction. In the example shown in FIG. 10B, the oxygenated hemoglobin concentration is the amount of change from the initial value. The amount of change is expressed in arbitrary units.

[0198] 10A and 10B, large fluctuations in the oxygenated hemoglobin concentration occur in synchronization with the time period from about 11 to 12 seconds when head movement occurs, indicating that there is a correlation between body movements such as head movement and irregular fluctuations in brain activity data.

[0199] Moreover, from the results shown in Figures 10A and 10B, it can be seen that the fluctuations in the oxyhemoglobin concentration occur with a slight delay from the start of head movement. Furthermore, it can be seen that the fluctuations in the oxyhemoglobin concentration subside with a delay from the end of head movement. There are two possible reasons for this. (1) The brain is in a state where it is floating in cerebrospinal fluid inside the skull. Therefore, the movement of the brain is delayed in time compared to the movement of the skull. (2) When measuring the oxyhemoglobin concentration, time averaging is performed in real time. For this reason, the measured value is easily affected by the previous fluctuations.

[0200] 10A and 10B, for example, when the following two conditions are met, the output of brain activity data may be stopped upon determining validity. (1) The threshold value is set to 5 mm for the moving distance in any of the X, Y, and Z directions. (2) The invalid period is set to 0.5 seconds before the threshold value is exceeded to 4.5 seconds after the threshold value is exceeded. These two conditions can eliminate irregular fluctuations in the measured value of the oxygenated hemoglobin concentration.

[0201] In the above example, cerebral blood flow data is used as an example of biometric data, but is not limited thereto. The biometric data may be, for example, data indicating at least one selected from the group consisting of scalp blood flow, pulse, sweat, respiration, and body temperature. The data can be obtained from the surface reflection component I1.

[0202] This disclosure also includes programs and methods of operation performed by signal processing circuitry 122 and signal processing circuitry 702. [Industrial Applicability]

[0203] The biometric device in the present disclosure can be used in a camera or measuring device that obtains internal information of a user in a non-contact manner. The biometric device can also be applied to, for example, biometric or medical sensing, sensing of a driver of a car, sensing of a user in a game machine or an attraction device, sensing of a learner in an educational institution, or sensing of a worker in a workplace. [Explanation of symbols]

[0204] 10, 20, 30 Biometrics 100 users 101 Light source 102 Image Sensor 103 Photoelectric conversion unit 104 Charge storage section 105 Control circuit 106 Light source control unit 107 Sensor control unit 108 Image signal acquisition unit 109 Biometric data generation unit 110, 703 Measurement section 111, 401 Output determination unit 121, 701 Imaging unit 122, 702 Signal processing circuit 201 pixels 202 Drain 203 Photodiode 204 First floating diffusion layer 205 Second floating diffusion layer 206 Third floating diffusion layer 207 Fourth Floating Diffusion Layer 302 Row selection circuit 303 Column Selection Circuit 304 Vertical signal line 305 Source Follower Power Supply 306 Source Follower Load 307 Analog-to-Digital Conversion Circuit 308 Row selection transistor 309 Source Follower Transistor 310 Reset transistor 704 Display 705 Controller

Claims

1. A light source that emits light to a target part of a living body; a light detection cell that receives reflected light returning from the target portion due to the emission of the light and outputs a signal; A signal processing circuit; Equipped with The signal processing circuit includes: determining whether a first value corresponding to a luminance value of the target portion based on the signal or a value of biometric data of the living body based on the signal exceeds a threshold value; stop outputting the biomeasurement data during a first period during which it is determined that the first value exceeds the threshold value, and during a second period which is a predetermined period after the end of the first period; Biometrics device.

2. the first value indicates an absolute value of a rate of change of the luminance value or an absolute value of a rate of change of the value of the biometric data; The bioinstrumentation device according to claim 1 .

3. the first value indicates the brightness value or the value of the biometric data; The bioinstrumentation device according to claim 1 .

4. The biometric data includes at least one piece of information selected from the group consisting of scalp blood flow, pulse rate, sweating, respiration, and body temperature; The bioinstrumentation device according to claim 1 .

5. causing a light source to emit light to a target portion of a living body; causing a light detection cell to receive reflected light returning from the target portion due to the emission of the light, and outputting a signal; determining whether a first value corresponding to a brightness value of the target portion based on the signal or a value of biometric data of the living body based on the signal exceeds a threshold; Stopping output of the biomeasurement data during a first period during which it is determined that the first value exceeds the threshold value and during a second period that is a predetermined period after the end of the first period; Including, Biometric methods.

Images