Biological information detection device, biological information detection method, and storage medium

Abstract

The present application provides a biological information detection device, a biological information detection method and a storage medium. The biological information detection device comprises: a light source, which projects a plurality of points formed by a first light to a living body; a camera system, which detects a second light generated by the projection of the plurality of points and generates an image signal containing a plurality of pixel signals; and an operation circuit; the second light contains surface reflection light as light reflected on the surface of the living body and internal scattering light scattered in the interior of the living body; the operation circuit selects a plurality of first pixel signals and a plurality of second pixel signals from the plurality of pixel signals, the plurality of first pixel signals correspond to a first region in the living body as a region emitting the surface reflection light, and the plurality of second pixel signals correspond to a second region in the living body as a region emitting the internal scattering light; and the operation circuit generates biological information related to the living body based on the plurality of first pixel signals and the plurality of second pixel signals.

Description

[0001] This application is a divisional application of the invention patent application filed on September 21, 2017, with application number 201710857842.2 and invention title "Biological Information Detection Device". Technical Field

[0002] This invention relates to a biological information detection device. For example, it relates to a biological information detection device that detects biological information such as heartbeat in a non-contact manner. Background Technology

[0003] As fundamental parameters for assessing a person's health status, heart rate, blood flow, blood pressure, and blood oxygen saturation are widely used. These biological parameters of blood are typically measured using contact-type measuring devices. Because these devices are attached to the subject's body, they can cause discomfort, especially during prolonged, continuous measurements.

[0004] Various attempts have been made to simply measure basic biological information used to determine a person's health status. For example, Japanese Patent Application Publication No. 2005-218507 discloses a method for non-contact detection of heart rate based on image information such as a face captured by a camera. Japanese Patent Application Publication No. 2003-517342 discloses a method for measuring blood oxygen saturation using a white light source and a laser source, utilizing the laser Doppler effect of laser light backscattered on the surface of a living organism. Japanese Patent Application Publication No. 2014-527863 discloses a method for measuring blood oxygen saturation using a conventional color camera to eliminate the influence of peripheral light.

[0005] On the other hand, many methods for speculating on changes in human psychology have been proposed. For example, Japanese Patent Application Publication No. 6-54836 and No. 2008-237244 disclose a method for detecting a decrease in temperature of the periphery of the nose caused by stress (tension) or concentration using a body surface temperature distribution measuring device. Summary of the Invention

[0006] A biological information detection device according to a technical solution of this disclosure includes: a light source that projects a plurality of points formed by a first light onto a biological body; a camera system that detects a second light generated by the projection of the plurality of points and generates an image signal including a plurality of pixel signals; and a computing circuit; the second light includes surface-reflected light as light reflected from the surface of the biological body and internally scattered light scattered inside the biological body; the computing circuit selects a plurality of first pixel signals and a plurality of second pixel signals from the plurality of pixel signals, the plurality of first pixel signals corresponding to a first region in the biological body as a region emitting surface-reflected light, and the plurality of second pixel signals corresponding to a second region in the biological body as a region emitting internally scattered light; the computing circuit generates biological information related to the biological body based on the plurality of first pixel signals and the plurality of second pixel signals. Attached Figure Description

[0007] Figure 1A This is a schematic diagram showing the structure of a biological information detection device according to one embodiment of the present disclosure.

[0008] Figure 1B It is a diagram used to illustrate the characteristics of images of the surface of an organism obtained by a camera.

[0009] Figure 2 It is a diagram used to illustrate the characteristics of different wavelengths of biological information obtained by a camera.

[0010] Figure 3 This is a graph showing the absorption and scattering coefficients of hemoglobin, melanin, and water—major components of living organisms—from visible light to near-infrared light.

[0011] Figure 4A This is a schematic diagram showing the structure of the biological information detection device of Embodiment 1 and the output image data.

[0012] Figure 4B This is a diagram illustrating the characteristics of the near-infrared image sensor according to Embodiment 1.

[0013] Figure 4C This is a diagram illustrating the characteristics of the color image sensor according to Embodiment 1.

[0014] Figure 4D This is a block diagram showing the structure of the computer in Implementation Method 1.

[0015] Figure 5A This is an explanatory diagram showing a general flow of signal processing in Implementation Method 1.

[0016] Figure 5B This is an explanatory diagram showing the effect of body movement correction in Implementation Method 1.

[0017] Figure 5C This is an explanatory diagram showing the effect of signal processing for acquiring biological information according to Embodiment 1.

[0018] Figure 6 This is an explanatory diagram illustrating the method for calculating contrast in Implementation Method 1.

[0019] Figure 7A This is a diagram illustrating an example of the structure of the biological information detection device according to Embodiment 2.

[0020] Figure 7B This is a diagram illustrating an example of the structure of the biological information detection device according to Embodiment 2.

[0021] Figure 8 This is a graph showing an example of the time-varying intensity of the internal scattered light in Embodiment 2.

[0022] Figure 9 This is a graph showing the results of measuring blood oxygen saturation using the method of Embodiment 2 and conventional methods.

[0023] Figure 10 This is a diagram showing the structure of the biological information detection device according to Embodiment 3.

[0024] Figure 11A This is a diagram showing the nose and cheek in the image obtained in Embodiment 3.

[0025] Figure 11B This is a graph showing the results of pressure sensing using a body surface temperature distribution measuring device.

[0026] Figure 11C This is a graph showing the changes in blood flow and blood oxygen saturation obtained using the biological information detection device of Embodiment 3.

[0027] Figure 12 This is a diagram showing the structure of the biological information detection device according to Embodiment 4.

[0028] Figure 13A This is an explanatory diagram of a monitoring system using the biological information detection device of Embodiment 4.

[0029] Figure 13B This is a diagram illustrating the algorithm of the monitoring system using the biological information detection device of Embodiment 4.

[0030] Figure 14 This is a diagram illustrating the driver monitoring process in Implementation 5.

[0031] Figure 15A This is a diagram schematically illustrating the structure of the biological information detection device according to Embodiment 6.

[0032] Figure 15B This is a diagram showing multiple color filters in embodiment 6.

[0033] Figure 15C This is a graph showing the wavelength dependence of the transmittance of the color filter in Embodiment 6.

[0034] Figure 15D This is a cross-sectional structural diagram of the image sensor in Embodiment 6.

[0035] Figure 16A This is a diagram schematically illustrating the structure of another biological information detection device according to Embodiment 6.

[0036] Figure 16B This is a diagram showing a plurality of other color filters in embodiment 6.

[0037] Figure 16C This is a graph showing the wavelength dependence of the relative sensitivity of the image sensor in Embodiment 6. Detailed Implementation

[0038] (Based on the understanding that forms the basis of this disclosure)

[0039] Before describing the embodiments of this disclosure, the understanding that forms the basis of this disclosure will be explained.

[0040] As mentioned above, various attempts have been made to measure fundamental biological information used to determine a person's health status. For example, Japanese Patent Application Publication No. 2005-218507 discloses a method for non-contact detection of heart rate based on image information such as a face captured by a camera. The method in Japanese Patent Application Publication No. 2005-218507 calculates the heart rate by analyzing the spatial frequency components of the acquired color image. However, in this method, accuracy decreases due to stray light such as indoor lighting, making stable detection difficult.

[0041] Pulse wave oxygen saturation meters are commonly used to measure blood oxygen saturation. These meters involve inserting a finger and shining light of two wavelengths within the red to near-infrared range onto the finger, measuring the transmittance. From this, the ratio of oxidized hemoglobin concentration to reduced hemoglobin concentration in the blood can be determined. Pulse wave oxygen saturation meters offer a simple way to measure blood oxygen saturation. However, because they are contact devices, they can cause a feeling of constraint.

[0042] Japanese Patent Application Publication No. 2003-517342 discloses an example of a non-contact blood oxygen saturation measuring device. This device uses a white light source and a laser source, utilizing the laser Doppler effect caused by the backscattering of laser light on the surface of a living organism to measure blood oxygen saturation. However, this method suffers from problems such as a complex device structure and a relatively weak signal.

[0043] Japanese Patent Publication No. 2014-527863 discloses a method for measuring heart rate and blood oxygen saturation using a conventional color camera to eliminate the influence of ambient light. However, this method suffers from significant variations in ambient light, making it difficult to measure heart rate and blood oxygen saturation with high precision and stability.

[0044] Thus, previous non-contact methods for measuring blood oxygen saturation have problems with accuracy and stability. Currently, there is no practical non-contact blood oxygen saturation measurement device.

[0045] On the other hand, many methods have been proposed for inferring psychological changes in individuals using thermography (e.g., Japanese Patent Application Publication No. 6-54836 and No. 2008-237244). These methods detect a decrease in nasal temperature using a thermography device. The nasal region contains numerous arteriovenous anastomoses, making it susceptible to obstruction of blood circulation due to the influence of the autonomic nervous system. Psychological changes such as stress or tension can lead to a decrease in blood flow to the nasal region due to the influence of the autonomic nervous system, resulting in a drop in nasal temperature. The devices disclosed in Japanese Patent Application Publication No. 6-54836 and No. 2008-237244 infer psychological changes in the subject by detecting temperature changes using a thermography device. However, methods using thermography devices have drawbacks such as low responsiveness due to the time required for the temperature to drop, and susceptibility to ambient temperature fluctuations.

[0046] If blood flow to the face can be accurately measured, it could be possible to establish a more responsive method for predicting psychological changes that is unaffected by ambient temperature.

[0047] Of the methods described above, the method using a conventional camera is considered the most promising due to its advantages of inexpensive equipment, fast response speed, and high resolution. Since the stability of the measurement is a concern, a method for stable and high-precision detection of biological information using a conventional camera was investigated. First, a color camera sensitive to visible light and a near-infrared camera sensitive to near-infrared light were used to photograph a human face. Pixel regions corresponding to the forehead were extracted, and the average value of the pixel signals for each color was measured. Signals for blue, green, and red were obtained using the color camera, and signals at 750nm and 850nm were obtained using the near-infrared camera. Figure 2 The average signal value of each pixel in the forehead of each color is shown corresponding to time. Figure 2In the diagram, signals A, B, C, D, and E represent blue, green, red, and wavelengths of 750 nm and 850 nm, respectively. The subject's heart rate at the time of the scan, measured by a contact pulse oximeter, was 80 beats per minute (0.75 seconds per beat). A clear 0.75-second period corresponding to the heart rate was obtained in the green signal, but no heartbeat-like signal was obtained in the red and blue signals. Compared to the red and blue signals, a slight trace of the heartbeat signal can be seen in the near-infrared signals at wavelengths of 750 and 850 nm. By storing the near-infrared signals at wavelengths of 750 and 850 nm within a certain time range and performing frequency analysis, the frequency corresponding to the heartbeat can be obtained. Methods for practically using near-infrared light to measure heart rate have been proposed. However, according to... Figure 2 As shown in the signal waveform, the signal-to-noise ratio is poor when using near-infrared light, making it difficult to improve measurement accuracy. Therefore, the mainstream biometrics technique using cameras employs green signals obtained from visible light cameras. By selectively using green signals, the signal-to-noise ratio can be improved, resulting in high-precision biometrics acquisition. However, this method using green signals has problems. There are requirements for acquiring biometrics in the dark, such as for monitoring during sleep (infants, elderly individuals, hospitalized patients, patients with sleep apnea syndrome, etc.) and monitoring drivers. However, biometrics using green signals are not suitable for such purposes. This is because signals cannot be acquired in the dark. Furthermore, it is difficult to reliably measure biometrics using green signals under conditions such as changes in ambient light.

[0048] The inventors of this application, focusing on the aforementioned problems, have researched a structure to solve them. Considering measurements at night or in darkness, it is desirable to utilize near-infrared light, which is invisible to the human eye. However, as... Figure 2 As shown, even using near-infrared light images alone yields signals with poor signal-to-noise ratios, making it difficult to achieve sufficient measurement accuracy. One technical solution disclosed herein relates to a method for improving signal quality by spatially separating biological information using patterned near-infrared illumination. Furthermore, another technical solution disclosed herein relates to a method for obtaining near-infrared light images by projecting a near-infrared light pattern onto the surface of a biological organism, simultaneously obtaining a visible light image that does not contain near-infrared light, and then processing the near-infrared light image and the visible light image to obtain biological information.

[0049] (principle)

[0050] The following explains the principle of a biological information detection device capable of achieving high-precision acquisition of biological information. A biological information detection device according to this disclosure comprises a near-infrared patterned illumination system, a near-infrared imaging system, and a visible light imaging system. By appropriately calculating the signals from both imaging systems, stable biological information sensing can be achieved regardless of the environment. By increasing the signal ratio from the visible light imaging system in stable, bright lighting environments, and conversely increasing the signal ratio from the near-infrared imaging system under unstable or dim lighting conditions, stable biological information sensing independent of the environment is achieved.

[0051] Figure 1A This diagram illustrates the schematic structure of a biological information detection device according to an exemplary embodiment of the present disclosure. The device includes a light source 1 that projects near-infrared light L0 onto an object including a biological organism, and cameras 201 and 202 as imaging systems. Here, camera 201 captures images using near-infrared light, and camera 202 captures images using visible light. Figure 1A In this process, near-infrared patterned light projects a dot pattern, consisting of multiple discretely arranged dots, onto the organism. The light source 1, illuminating the near-infrared light, is configured to project the multiple dots onto the organism 3. Cameras 201 and 202, each equipped with an image sensor, capture images of the organism's surface 4, generating and outputting image signals. Camera 202 captures reflected light L3 from the organism 3 within its visible wavelength range.

[0052] exist Figure 1A The diagram shows the structure of the skin surface of the organism 3. The surface-reflected light, which is reflected by the organism's surface 4, retains the image of the dot pattern brought by the light source 1. In contrast, the internally scattered light, which penetrates into the interior of the organism 3 and is scattered within the organism before exiting the surface 4, loses the image of the dot pattern from the light source 1 due to stronger scattering within the organism. By using the light source 1 that illuminates the dot pattern, the surface-reflected light L1 and the internally scattered light L2 can be easily separated spatially.

[0053] Figure 1AThe organism 3 represented here is human skin, comprising the epidermis 33, dermis 34, and subcutaneous tissue 35. There are no blood vessels in the epidermis 33, but capillaries 31 and small arteries and veins 32 are present in the dermis 34. Because there are no blood vessels in the epidermis 33, the surface-reflected light L1 does not contain information about blood. Since the epidermis 33 contains melanin, which strongly absorbs light, the surface-reflected light L1 from the epidermis 33 becomes noise in acquiring blood information. Therefore, the surface-reflected light L1 not only has no effect on acquiring blood information but also hinders the acquisition of accurate blood information. To detect organism information with high accuracy, it is important to suppress the influence of surface-reflected light and efficiently acquire information from internally scattered light.

[0054] One embodiment of this disclosure features a novel structure that spatially separates surface-reflected light and internally scattered light using a light source 1 that illuminates patterned light in the near-infrared range and a camera 201 that captures near-infrared light. This enables the measurement of information within a living organism in a non-contact manner with high precision. Figure 1B This is a schematic diagram illustrating the two-dimensional distribution of the image output from camera 201. Multiple discretely arranged point images projected by light source 1 are represented by black circles. Surface-reflected light is reflected from the portion of the skin surface within the black circles. Conversely, internally scattered light diffuses within the skin, spreading outwards from the portion within the black circles. The distribution of internally scattered light is shown using... Figure 1B The white circles represent the areas. Surface reflected light and internal scattered light can be easily separated from the image captured by camera 201. In the 2D image, the areas with stronger light intensity, represented by black circles, are primarily areas containing surface reflected light, while the areas outside the black circles are primarily areas containing internal scattered light. By using a light source that illuminates the pattern in this way, surface reflected light and internal scattered light can be easily separated.

[0055] Conventionally, to separate surface-reflected light from the surface of such organisms, methods using polarized light illumination, such as those disclosed in Japanese Patent Application Publication No. 2002-200050, have been employed. In these methods, a polarizer with a polarized light transmission axis orthogonal to the polarization direction of the illumination light reflected from the subject is used. By capturing an image with a camera via such a polarizer, the influence of surface-reflected light can be suppressed. However, regarding reflections from uneven surfaces like skin, the degree of polarization of the surface-reflected light varies depending on its position, leading to insufficient separation of the surface-reflected light. Furthermore, since skin is a strong scatterer, light scattered and reflected from the shallower parts of the skin surface does not contain blood information and is therefore a signal that should be separated. However, because polarized light is lost in the shallower parts of the skin, this component cannot be separated, thus failing to improve the signal-to-noise ratio. In contrast, in the method of this disclosure, since the scattered light in the shallower parts of the skin is located near the pattern of the light source, it can be easily spatially separated from the signal containing biological information reflected from the deeper parts of the skin. According to the method disclosed herein, since surface reflected light and internal scattered light can be spatially separated, the influence of surface reflected light can be suppressed more effectively.

[0056] In the biological information detection device of the embodiments of this disclosure, the wavelength of the light from the light source that projects the dot pattern is also important. The wavelength of the light source can be set, for example, to be between approximately 650 nm and approximately 950 nm. This wavelength range is encompassed within the wavelength range from red to near-infrared. In this specification, the term "light" is used not only for visible light but also for infrared light. The aforementioned wavelength range is referred to as the "biological window," which is known through the low absorption rate within the body.

[0057] Figure 3 This graph shows the wavelength dependence of the light absorption coefficients of oxidized hemoglobin, deoxidized hemoglobin, melanin, and water, as well as the light scattering coefficients within the body. In the visible light region below 650 nm, blood (i.e., hemoglobin) has high absorption, while water has high absorption in wavelengths longer than 950 nm. Therefore, light in these wavelength ranges is unsuitable for acquiring information within the body. On the other hand, in the wavelength range from approximately 650 nm to approximately 950 nm, the absorption coefficients of hemoglobin and water are relatively low, while their scattering coefficients are relatively high. Therefore, light in this wavelength range, after penetrating the body, is strongly scattered and returns to the body surface. Thus, information within the body can be acquired more efficiently.

[0058] The biological information detection device of this disclosure utilizes light in the wavelength range that conforms to the "window of life". Therefore, since the light reflected from the surface of the organism and the light scattered back within the body can be separated and detected with high precision, information within the body can be obtained efficiently.

[0059] By using near-infrared light, biological information can be stably acquired in more situations, including at night, compared to conventional methods using visible light. However, problems arise when using near-infrared light in environments with sunlight or light bulb illumination. Sunlight and light bulb light are not only visible light but also include a significant amount of near-infrared light. In such environments, near-infrared light from the ambient sunlight or light bulbs can also illuminate areas outside the illuminated near-infrared pattern. While it is possible to separate surface reflected light and internal scattered light from the two-dimensional distribution of the image, there are concerns about signal-to-noise ratio degradation and decreased measurement accuracy due to the influence of sunlight or light bulb light. In such cases, accuracy can be improved by using signals in the visible wavelength range simultaneously acquired from the camera 202. As already described, higher measurement accuracy can be obtained in bright and stable lighting environments by using green signals. On the other hand, higher measurement accuracy can be achieved in dark environments by using near-infrared light. By switching between near-infrared and visible light information, or using both in combination, depending on the measurement environment, biological information can be stably acquired in a wide variety of environments. Since measurements can be stably performed using the green signal under bright and stable ambient light, the power consumption required for illumination can be eliminated by turning off the near-infrared illumination in such environments, enabling energy-efficient measurements.

[0060] This disclosure includes, for example, the technical solutions described in the following projects.

[0061] [Project 1]

[0062] The biological information detection device of Item 1 of this disclosure comprises: a light source that projects a pattern formed by near-infrared light onto an object including a biological organism; one or more camera systems including a plurality of first light detection units for detecting light in the near-infrared wavelength range and a plurality of second light detection units for detecting light in the visible wavelength range, generating a first image signal representing a first image and generating a second image signal representing a second image, wherein the first image is a near-infrared wavelength image of the object on which the pattern is projected and the second image is a visible wavelength image of the object; and a computing circuit that uses at least one selected from a group consisting of the first image signal and the second image signal to compute biological information related to the biological organism.

[0063] [Project 2]

[0064] In the biological information detection device described in Project 1, the above pattern may also contain multiple points.

[0065] [Project 3]

[0066] In the biological information detection device described in Project 1 or 2, the near-infrared light may include light with a wavelength of 650 nm to 950 nm; and the light in the visible wavelength range may include light with a wavelength of 500 nm to 620 nm.

[0067] [Project 4]

[0068] In any of the biological information detection devices described in items 1 to 3, the biological information may include at least one selected from the group consisting of the organism's heart rate, blood pressure, blood flow, blood oxygen saturation, melanin concentration in the organism's skin, presence or absence of spots on the organism's skin, and presence or absence of moles on the organism's skin.

[0069] [Project 5]

[0070] In any of the biological information detection devices described in items 1 to 4, the above-mentioned computing circuit may use the first image signal to detect the first part in the first image that corresponds to the biological organism.

[0071] [Project 6]

[0072] In the organism information detection device described in Project 5, the above-mentioned computing circuit may determine whether an organism exists at the position corresponding to the pixel based on the ratio of the intensity of the first image signal corresponding to the pixel contained in the first image and the ratio of the deviation to the average value of the intensity of the first image signal corresponding to the multiple pixels arranged around the pixel.

[0073] [Project 7]

[0074] In the biological information detection device described in Item 5 or 6, the above-mentioned computing circuit may calculate the biological information based on the ratio of the average intensity of the first image signal corresponding to a subset of pixels selected in descending order of the intensity of the first image signal corresponding to each pixel from the first portion of the first image and a plurality of pixels arranged around the first pixel, and (i) the average intensity of the first image signal corresponding to another subset of pixels selected in ascending order of the intensity of the first image signal corresponding to each pixel, and (ii) the average intensity of the first image signal corresponding to another subset of pixels selected in ascending order of the intensity of the first image signal corresponding to each pixel.

[0075] [Project 8]

[0076] In the biological information detection device described in item 5 or 6, the above-mentioned computing circuit may calculate the biological information by using the average value of the intensity of the second image signal corresponding to each of the multiple pixels contained in the second part of the second image corresponding to the first part of the first image.

[0077] [Project 9]

[0078] In any of the biological information detection devices described in items 1 to 8, the above-mentioned one or more camera systems may further include: a camera element having a camera surface divided into a first region where a plurality of first light detection units are arranged and a second region where a plurality of second light detection units are arranged; a first optical system forming the first image in the first region; and a second optical system forming the second image in the second region.

[0079] [Project 10]

[0080] In the biological information detection device described in Project 9, the above-mentioned one or more camera systems may also include: a first bandpass filter that allows light in the near-infrared wavelength range to pass through; and a second bandpass filter that allows light in the visible wavelength range to pass through.

[0081] [Project 11]

[0082] In the biological information detection device described in Project 9, the above-mentioned one or more camera systems may also include: a first bandpass filter that allows light in the near-infrared wavelength range to pass through; a linear polarization filter that is configured to be perpendicular to the polarization direction of the light source; and a second bandpass filter that allows light in the visible wavelength range to pass through; wherein the near-infrared light is linearly polarized light; and the linear polarization filter is configured such that the polarization direction of the linearly polarized light transmitted by the linear polarization filter is perpendicular to the polarization direction of the near-infrared light.

[0083] [Project 12]

[0084] In the biological information detection device described in Project 9, the imaging element may include a first color filter that is opposite to the plurality of first light detection units and transmits light in the near-infrared wavelength range, a second color filter that is opposite to the plurality of second light detection units and transmits light in the visible wavelength range, and a near-infrared absorption filter that is opposite to the plurality of second light detection units and the second color filter and absorbs light in the near-infrared wavelength range.

[0085] [Project 13]

[0086] In any of the biological information detection devices described in items 1 to 8, the aforementioned one or more camera systems may include a first camera system and a second camera system; the first camera system includes: a first camera element having a first imaging surface with the aforementioned plurality of first light detection units disposed thereon; and a first optical system forming the aforementioned first image on the aforementioned first imaging surface; the second camera system includes: a second camera element having a second imaging surface with the aforementioned plurality of second light detection units disposed thereon; and a second optical system forming the aforementioned second image on the aforementioned second imaging surface.

[0087] [Project 14]

[0088] In the biological information detection device described in Project 13, the first camera system may also include a first bandpass filter that allows light in the near-infrared wavelength range to pass through; and the second camera system may also include a second bandpass filter that allows light in the visible wavelength range to pass through.

[0089] [Project 15]

[0090] In any of the biological information detection devices described in items 1 to 14, the aforementioned computing circuit may calculate the blood flow and blood oxygen saturation of the organism based on the first image signal and the second image signal; and based on the blood flow and blood oxygen saturation of the organism, generate information representing at least one selected from a group consisting of the organism's physical condition, emotions, and level of concentration.

[0091] [Project 16]

[0092] In any of the biological information detection devices described in items 1 to 14, when the first image and the second image contain at least one part selected from the group consisting of the cheek and nose of the organism, the computing circuit uses the first image signal and the second image signal to calculate the time change of blood flow and the time change of blood oxygen saturation of the at least one part selected from the group consisting of the cheek and the nose; and uses the time change of blood flow and the time change of blood oxygen saturation to generate information representing at least one of the group consisting of the organism's physical condition, emotions, and level of concentration.

[0093] [Project 17]

[0094] In any of the biological information detection devices described in items 1 to 14, when the first image and the second image contain the cheek and nose of the organism, the computing circuit uses the first image signal and the second image signal to calculate the time changes in blood flow and blood oxygen saturation of the cheek, and the time changes in blood flow and blood oxygen saturation of the nose; based on the comparison between the time changes in blood flow and blood oxygen saturation of the cheek and the time changes in blood flow and blood oxygen saturation of the nose, information representing at least one selected from the group consisting of the organism's physical condition, emotions, and level of concentration is generated.

[0095] [Project 18]

[0096] In any of the biological information detection devices described in items 1 to 17, the above-mentioned one or more camera systems may further include: a camera element having a camera surface with the above-mentioned plurality of first light detection units disposed thereon; an optical system forming the above-mentioned first image on the camera surface; and an adjustment mechanism for adjusting the focus of the above-mentioned optical system; the adjustment mechanism adjusting the focus to maximize the contrast of the above-mentioned first image.

[0097] [Project 19]

[0098] In the biological information detection device described in Project 1, the above-mentioned computing circuit may calculate the reliability of the biological information calculated using the first image signal and the reliability of the biological information calculated using the second image signal, and then compare them.

[0099] [Project 20]

[0100] In any of the biological information detection devices described in items 1 to 8, the above-mentioned one or more camera systems may further include: a camera element having a camera surface divided into a first region where a plurality of first light detection units are arranged and a second region where a plurality of second light detection units are arranged; a projection optical system for forming an image in the first region and the second region; and a reflection optical system for incident light in the near-infrared wavelength range and light in the visible wavelength range onto the projection optical system.

[0101] In this disclosure, all or part of a circuit, unit, device, component, or section, or all or part of a functional block of a block diagram, may be implemented by one or more electronic circuits, including semiconductor devices, semiconductor integrated circuits (ICs), and LSIs (large scale integration). An LSI or IC can be integrated onto a single chip or constructed by combining multiple chips. For example, functional blocks other than memory elements can also be integrated onto a single chip. It is referred to herein as an LSI or IC, but the terminology varies depending on the degree of integration; it may also be called a system LSI, VLSI (very large scale integration), or ULSI (ultra large scale integration). Field-programmable gate arrays (FPGAs) programmed after the LSI is manufactured, or reconfigurable logic devices capable of reconfiguring the internal bonding relationships of an LSI or setting up the internal circuit partitioning of an LSI, can also be used for the same purpose.

[0102] Furthermore, all or part of the functions or operations of a circuit, unit, device, component, or part can be executed through software processing. In this case, the software is recorded on one or more non-transitory recording media such as ROM, optical disk, hard disk, etc. When the software is executed by a processor, the functions determined by the software are executed by the processor and peripheral devices. The system or device may also include one or more non-transitory recording media containing the software, a processor, and necessary hardware devices, such as interfaces.

[0103] The embodiments of this disclosure are described in more detail below. These embodiments pertain to a bio-information detection device that primarily uses the human face as the surface of a living organism to measure bio-information non-contactly. However, the technology of this disclosure is not limited to the human face; it can also be applied to other body parts or parts of the skin of animals other than humans.

[0104] (Implementation Method 1)

[0105] As a first embodiment, a system for non-contact heart rate monitoring is described. With increasing health awareness, the importance of stable biometric information sensing is growing. Systems capable of non-contact and frequent measurement of biometric information are important not only for hospitals but also for health management in daily life. The system of this embodiment can monitor heart rate and heart rate variability non-contactly.

[0106] Figure 4A This is a diagram illustrating the schematic structure of the biological detection system according to this embodiment. The biological detection system of this embodiment is as follows: Figure 4A As shown, the system includes: a light source 1 that emits light in the near-infrared wavelength range and is positioned at a location away from the organism 3; a camera 201 capable of recording images of the surface of the irradiated organism and taking near-infrared photographs; a camera 202 adjusted to have approximately the same imaging range and taking visible light photographs; and a computer 20 that calculates and outputs information about the organism based on the photographed images. Cameras 201 and 202 correspond to a camera system. The computer 20 includes arithmetic circuitry.

[0107] A light source 1, illuminating a near-infrared pattern, projects the dot pattern onto the surface of a living organism. The dot pattern is typically a collection of tiny bright spots arranged in a two-dimensional pattern. Depending on the application, a one-dimensional dot pattern may also be used. In this embodiment, for example, an Osela random dot pattern laser projector RPP017ES can be used as the light source 1. This laser source emits 830nm near-infrared laser light and projects a laser dot pattern of 57,446 dots into a 45° × 45° field of view.

[0108] Camera 201 has a lens 501 and a housing 601 as a first optical system. Lens 501 may be an assembly of multiple lenses. Inside housing 601, an image sensor 701, serving as a solid-state imaging element, and a bandpass filter 801 that allows light with a wavelength of only 830 nm ± 10 nm to pass through are mounted. Image sensor 701 receives near-infrared light. On the other hand, camera 202, like camera 201, has a lens 502, a housing 602, an image sensor 702, and a bandpass filter 802 as a second optical system. The bandpass filter 802 allows green wavelengths from 520 to 600 nm to pass through. Image sensor 702 receives visible light.

[0109] exist Figure 4B Solid lines are used to represent the characteristics of image sensors 701 and 702. In this embodiment, neither image sensor 701 nor 702 has a color filter; instead, they use monochrome silicon image sensors sensitive to light ranging from ultraviolet to near-infrared (wavelengths 300-1200 nm). Typical monochrome image sensors, such as those in… Figure 4BAs indicated by the dotted line, it has a sensitivity peak at wavelengths around 550-600 nm. In contrast, image sensors 701 and 702 have a sensitivity wavelength range roughly the same as typical monochrome image sensors, but they are designed to increase sensitivity on the longer wavelength side by increasing the depth of the photodiode. In this specification, image sensors with such characteristics are referred to as near-infrared image sensors. To detect near-infrared signals with higher sensitivity, not only image sensor 701 but also image sensor 702 uses a near-infrared image sensor. This is to facilitate subsequent signal processing by making the characteristics of the two image sensors consistent. Regarding image sensor 702, a less expensive, typical monochrome image sensor can also be used.

[0110] Furthermore, regarding the image sensor 702, it is also possible to use... Figure 4C The image shown is a typical color image sensor. A color image sensor can separate and detect the three color signals by configuring red, green, and blue color filters on pixels. However, in this case, the transmission wavelength of the bandpass filter 802, configured in front of the image sensor 702, is expanded to approximately 400 to 650 nm, allowing the entire visible light spectrum to pass through. As already described, using green light is effective for sensing biometric information, and pixels with blue and red light can be considered unnecessary. However, as will be discussed later, for correction of subject movement (body movement), it is effective to perform calculations using a green signal that contains more biometric information and a red or blue signal that does not contain biometric information. Therefore, in situations involving significant body movement, Figure 4C The use of the color image sensor shown is effective.

[0111] When people are taken as objects, such as Figure 4AAs shown in part (a), camera 201 acquires an image containing multiple point images with brightness corresponding to the infrared reflectivity of each location. As mentioned above, living organisms possess special optical characteristics for wavelengths from red to near-infrared, known as "bio-windows." Because human skin has a low absorption coefficient and a high scattering coefficient in this wavelength range, light passing through the skin surface is repeatedly scattered and diffused within the body, escaping over a wide area from the skin surface. In the aforementioned wavelength range, a high proportion of internally scattered light relative to surface-reflected light is characteristic of living organisms. In contrast, in objects other than living organisms, the proportion of surface-reflected light is higher than that of internally scattered light. Therefore, living organisms can be detected based on the ratio of surface-reflected light to internally scattered light. Thus, biological regions within an image can be detected. Furthermore, biological information can be rapidly acquired using only pixel signals of biological regions within an image. This is because the detection of human body regions using the optical characteristics of skin is faster and more accurate than conventional image recognition methods. Here, as... Figure 4A As shown in section (b), the forehead (shown in a rectangular box) that enables stable acquisition of biological information is extracted from the detected skin region for biological information detection. Since camera 202 is adjusted to have approximately the same imaging range, the forehead position of camera 202 can be determined based on the coordinates of the forehead region obtained from camera 201. The signal within the forehead region is used in biological information detection from the visible light image. Here, since the near-infrared light image and the visible light image capture the same object, the forehead position can also be accurately determined on the visible image with high precision through image recognition of the green image. However, since the image recognition processing has a large computational load, the processing can be simplified. In the near-infrared image, the biological information detection region can be detected quickly and accurately based on the diffuse light component and the surface reflected light component. Using this image signal, the biological information detection region of the green signal can be determined. If the configuration of the two cameras is adjusted with sufficient high precision, the biological information detection region of the green image can be determined by shifting the coordinates from the near-infrared image by a certain value. Without fully adjusting the configuration of the two cameras, the biological information detection area of ​​the green image can be determined by performing AND processing on the biological information detection area obtained from the near-infrared image and the area of ​​reflected light intensity above the threshold of the green image.

[0112] like Figure 4AAs shown in section (c), surface reflected light L1 and internal scattered light L2 are spatially separated from the near-infrared light image acquired by camera 201, and biological information is calculated using both. On the other hand, biological information is calculated based on the green image acquired by camera 202. As described above, by switching between near-infrared information and visible light information, or by using both in combination, depending on the measurement environment, biological information can be stably acquired for a wide variety of environments.

[0113] Figure 4D This is a block diagram illustrating the structure of a computer 20, which serves as a biological information processing device. The computer 20 includes: an input interface 21 electrically connected to cameras 201 and 202; a processing circuit 22 for signal processing of detected human bodies within images; a processing circuit 23 for calculating biological information using image data from the detected human body areas; a memory 25 for recording various data; a control circuit 26 for controlling the overall operation of the device; an output interface 24 for outputting data; and a display 27 for displaying the processing results. The processing circuits 22 and 23 may be, for example, image processing circuits such as digital signal processors (DSPs). The control circuit 26 may be, for example, an integrated circuit such as a central processing unit (CPU) or a microcomputer. The control circuit 26 executes, for example, a control program recorded in the memory 25, to control functions such as indicating the illumination of the light source 1, indicating the imaging of cameras 201 and 202, and indicating the processing of the processing circuits 22 and 23. The control circuit 26 and the processing circuits 22 and 23 may also be implemented by a single, combined circuit. In this example, the computer 20 includes a display 27, but the display may also be an external device connected via wired or wireless connection. Computer 20 can also acquire image information from a remote camera via a communication circuit (not shown).

[0114] Here, the method and its effects of using a light source 1 that illuminates patterned light in the near-infrared region will be explained. For example... Figure 2 As shown, near-infrared images captured using a uniform near-infrared light source have a low signal-to-noise ratio relative to biological information, making it difficult to obtain sufficient accuracy. This disclosure uses patterned near-infrared light. As described above, based on an image captured by a camera of reflected light from the surface of a biological organism using patterned near-infrared light, it is easy to spatially separate the surface reflected light from the biological organism's surface and the internally scattered light that is incident on the organism's interior, diffuses within the organism, and is emitted from the surface. The surface reflected light does not contain information about the organism's interior, but since the internally scattered light is light scattered and re-emitted within the skin, it contains information about the organism's interior. By selectively using internally scattered light, biological information can be detected with high accuracy. Regarding the specific detection method, [the following is an explanation of the method described]. Figure 5A , Figure 5B , Figure 5C A detailed explanation. Figure 5APart (a) is an image captured by projecting a near-infrared pattern illumination consisting of random dot patterns onto the human body and using a near-infrared camera. Here, a region capable of stably detecting biological information is selected for acquiring biological information. The skin region is extracted from the near-infrared image, and calculations are performed. If observed... Figure 5A The image of the forehead region shown in part (b) can then be used to determine the density corresponding to the illumination pattern. For example... Figure 5A As shown in sections (c) and (d), the area is divided into multiple parts of 5×5 pixels. Figure 5A As shown in section (e), for each section, the light intensity of pixels is compared among 25 pixels, and the average light intensity of the top 10 pixels with the strongest light intensity (denoted by A) is calculated, defined as the intensity of the surface reflected light component in that region. On the other hand, the average light intensity of the bottom 10 pixels with the weakest light intensity (denoted by B) among the 25 pixels is calculated, defined as the intensity of the internal scattered light component in that region. By calculating the surface reflected light and internal scattered light components of the entire region for each 25 pixels and averaging them separately, the intensity of the average surface reflected light and average internal scattered light components of the region is calculated. Here, surface reflected light and internal scattered light are similar signals because they are information from pixels that are spatially close. In non-contact biometric sensing, the influence of interference factors (variations in ambient light, measurement system, and subject movement), which are always a major problem, is roughly the same for both surface reflected light and internal scattered light. The only difference is that internal scattered light contains more information from within the biometric system. This characteristic can be used to correct for interference factors when detecting signals such as body movement. Specific correction methods include using the intensity ratio of internally scattered light to surface-reflected light, or subtracting the intensity of surface-reflected light multiplied by a certain value from the intensity of internally scattered light. To determine the constant multiplied by the intensity of surface-reflected light, independent component analysis can also be used. Here, a simple division of the intensity of internally scattered light to the intensity of surface-reflected light yields sufficient correction; therefore, correction for interference factors is performed through division. Figure 5B The text in the middle indicates the effect of the correction. Figure 5B The signal A represents the average value of all pixels in the region over time, and the signal is expressed as follows: Figure 5AThe signal B, representing the intensity ratio of internal scattered light to surface reflected light, is obtained using the method shown. It represents data from a state of rest after 35 seconds of head-shaking motion. For both types of data, the forehead region was tracked, making it the measurement area. The average value of all pixels varies significantly due to head-shaking, indicating that small heartbeat signals, seemingly like swaying, are difficult to detect. In contrast, while the effect of head-shaking is visible in the ratio of internal scattered light to surface reflected light, a pulsating signal is clearly visible in the output signal, demonstrating that biological information can be detected even with significant body movement. Figure 5C The image shows the obtained heartbeat signal. Signal A, obtained using the conventional method of utilizing all pixels, has a low signal-to-noise ratio (SNR), making it difficult to obtain stable biological information. Signal B, obtained by separating internal scattered light from surface reflected light using a near-infrared patterned light source and then performing signal processing, shows a significantly improved SNR. Figure 5C In the example shown, the signal-to-noise ratio is improved by 30 times compared to using the average of all pixels. This method of using a near-infrared patterned light source is very effective for detecting biological information using near-infrared light. Furthermore, in the example above, signals from 5×5 pixels, the first 10 pixels, and the last 10 pixels are used. However, the optimal values ​​for these values ​​vary depending on the light source pattern, camera resolution, and the distance between the subject and the camera. Therefore, these values ​​can be variably set according to the system and usage conditions.

[0115] Up to this point, the signal processing for biometric sensing using a near-infrared camera with a near-infrared patterned light source has been explained. In biometric sensing using green light, as described above, since green light has a high signal-to-noise ratio, no special light source is required, and ambient light can be used as the light source for biometric sensing. Therefore, the signal processing for green light can be simply calculated using the average light intensity within the determined measurement area (e.g., the foreground region). However, when using green light, the average light intensity is also strongly affected by body motion. For stable signal detection, body motion correction can also be applied to green light. Regarding the removal of the influence of body motion, as already described in the body motion correction method for near-infrared light images, the influence of body motion can be removed by calculating image data with a lower proportion of biometric information in the same scene of the photographed subject and image data containing more biometric information. In the case of near-infrared light images, surface reflected light is used as the image data with a lower proportion of biometric information, and internal scattered light is used as the image data containing more biometric information. In the case of green light, since images obtained from green light contain a relatively high amount of biological information, the surface reflection component obtained from near-infrared light can be used as image data with a lower proportion of biological information. Body motion correction for green light can be performed by calculating between the average intensity of green light and the surface reflection component of near-infrared light. Specifically, as described above, by dividing the two or subtracting after multiplying by a certain value, the influence of body motion can be removed, resulting in high-precision measurements. Other methods can also be used for body motion correction of green light. Figure 4A In the measurement system shown, when the image sensor 702 is a color image sensor, it outputs three-color image data (red, green, and blue) from a visible light camera. By using the green data, which contains more biological information, and the red or blue signals, which contain less biological information, for calculation, body motion correction can be performed. This method has the advantages of correcting not only body motion but also changes in ambient light, and it can achieve high alignment accuracy within the same field of view. However, it also has the problems of reduced green light signal and decreased resolution.

[0116] In the aforementioned biometric information sensing, near-infrared patterned illumination is used to detect the skin region of the human body in order to efficiently determine the measurement area. In conventional methods, image recognition is used to extract facial features, but image recognition has a high computational load, requiring high-performance computers for high-speed processing, leading to larger and more expensive devices. In a technical solution disclosed herein, by using near-infrared patterned illumination, human body detection can be performed with high accuracy and high speed with a lighter computational load. Regarding the specific method of human body detection, [the following is a description of the method used]. Figure 6 Please provide an explanation. Figure 6 This is an example diagram showing the pixel region used in the contrast calculation within the detection area. The image data is recorded as 2D intensity data in memory 25. Let P be the data of the i-th pixel in the horizontal (x) direction and the j-th pixel in the vertical (y) direction. ij The contrast C of pixel (i, j) is defined as follows: ij .

[0117] C ij =S ij / A ij

[0118] Here, S ij and A ij These are the standard deviation and average values ​​of the pixel data within a 7×7 pixel region centered at pixel (i, j), respectively. Since a higher ratio of internal scattered light to surface reflected light results in a higher standard deviation S... ij The smaller, the better. ij The value decreases. After repeating this process on all pixels, the arithmetic circuit 22 extracts only C. ij The value is a specified range of pixels. As an example, in... Figure 5A In the near-infrared image shown, by setting it to 0.2 <C ij <0.47, indicating that the human body region was correctly extracted.

[0119] Thus, according to this embodiment, by utilizing the unique optical properties of skin, the skin region can be efficiently detected from an object. Here, the average value and dispersion standard deviation value within a 5×5 pixel region are calculated to determine the image contrast (i.e., the contrast between surface reflected light and scattered light), but this is just one example. The size of the pixel region used in the contrast calculation (i.e., the number of pixels) is appropriately set according to the density of the multiple point images formed by the near-infrared patterned light and the resolution of the camera 202. To suppress deviations in the calculation results, multiple (e.g., more than 3 points) illumination point images may be included within the pixel region of the object to be processed. By increasing the number of pixels in the region of the object to be processed, the accuracy of the contrast calculation value is improved, but the resolution of the resulting image of the organism decreases. Therefore, the number of pixels in the region of the object to be processed is appropriately set according to the system structure and intended use. Furthermore, not only the number of pixels in the object to be processed, but also the interval between pixels that repeat the process also affects the processing speed. In the above processing, the calculation is performed at a 5-pixel interval for high speed. Reducing the pixel interval slows down the processing speed, but improves the resolution. The interval between repeating pixels in the calculation can be set appropriately based on the system architecture and intended use. Similarly, the specified range for contrast is not limited to 0.2. <C ij <0.47, can be set appropriately according to the system structure and purpose of use.

[0120] Many methods have been proposed for non-contact heartbeat monitoring using conventional visual or near-infrared cameras. In these conventional methods, the separation of the reflected and scattered light components is insufficient, making stable and high-precision measurements difficult due to stray light interference during non-contact operation. By spatially separating the reflected and scattered light components as described in this embodiment, stable and high-precision heartbeat measurements can be achieved. For example, in conventional remote heartbeat measurements using cameras, detection becomes unstable and inaccurate when there is body movement. Using this method, stable heartbeat measurements can be performed even with body movement.

[0121] By using the biometric detection device of this embodiment, heart rate or blood pressure can be monitored regularly without interfering with the subject's movement, including while sleeping. Therefore, for example, a system can be built in a hospital to regularly monitor a patient's condition and alert medical staff when abnormalities occur. In the home, it is also possible to monitor the heart rate at night for patients, for example, those suffering from asystole syndrome. Furthermore, stress sensing can be easily performed in daily life as described above, allowing for a more fulfilling daily life.

[0122] (Implementation Method 2)

[0123] As a second embodiment, a system for non-contact measurement of blood oxygen saturation is described. The primary function of blood is to extract oxygen from the lungs and transport it to the tissues, and to extract carbon dioxide from the tissues and circulate it back to the lungs. Approximately 15g of hemoglobin is present in 100ml of blood. Hemoglobin that binds to oxygen is called oxidized hemoglobin (HbO2), and hemoglobin that does not bind to oxygen is called reduced hemoglobin (Hb). Figure 3 As shown, oxidized hemoglobin and deoxidized hemoglobin have different light absorption characteristics. In a system according to a technical solution of this disclosure, the oxygen saturation, which is the ratio of the two hemoglobins, can be determined based on the signals of reflected light in the infrared wavelength domain and reflected light in the green wavelength domain. Oxygen saturation is a value that indicates how much of the hemoglobin in the blood has been bound to oxygen. Oxygen saturation is defined by the following formula.

[0124] Oxygen saturation = C(HbO2) / [C(HbO2)+C(Hb)]×100 (%)

[0125] Here, C(Hb) represents the concentration of reduced hemoglobin, and C(HbO2) represents the concentration of oxidized hemoglobin.

[0126] Within a living organism, in addition to blood, there are components that absorb light of wavelengths from the visible to the near-infrared. However, the variation in light absorption over time is primarily due to hemoglobin in arterial blood. Therefore, blood oxygen saturation can be measured with high precision based on variations in absorption. Arterial blood, pulsating from the heart, forms a pulse wave that moves within the blood vessels. Venous blood, on the other hand, does not have a pulse wave. Light illuminating a living organism is absorbed in various layers of the organism, including arteries, veins, and tissues other than blood, and passes through the organism. However, the thickness of tissues other than arteries does not change over time. Therefore, the internally scattered light from within the organism exhibits a change in intensity over time corresponding to the change in the thickness of the arterial blood layer caused by pulsation. This change reflects the change in the thickness of the arterial blood layer, without including the influence of venous blood and tissues. Thus, information about arterial blood can be obtained by focusing only on the varying components of internally scattered light. The pulse can also be determined by measuring the period of the time-varying components.

[0127] As an embodiment, an example of measuring biological information using a single camera will be described. In the first embodiment, two cameras are used, each acquiring signals of different light source wavelengths. While this method has the advantage of utilizing existing cameras, the system structure becomes complex due to the linkage of the two cameras. The acquired data is also independent motion image data from two cameras, making time-matching data processing complicated. In a technical solution of this disclosure, a biological information detection device capable of simultaneously acquiring image data of two wavelengths using a single camera is realized.

[0128] In this embodiment, an example of measuring blood oxygen saturation will be described. Figure 7AThis diagram illustrates the structure of the bio-information detection device according to this embodiment. The device is configured as a binocular stereo camera with two built-in cameras 201 and 202 serving as an imaging system. Therefore, in this specification, this configuration is referred to as a "stereo camera configuration." In the bio-information detection device, reflected light from a bio-organism illuminated by a light source 1 emitting patterned light with a wavelength of 830 nm near-infrared passes through bandpass filters 801 and 802, and its direction of travel is bent by 90 degrees by mirrors 901 and 902. The light is then imaged onto the imaging surfaces of image sensors 701 and 702 by lenses 501 and 502. Bandpass filters 801 and 802 are respectively designed to allow only near-infrared light with a wavelength of 830 ± 15 nm and green light with a wavelength of 520 to 600 nm to pass through. Here, a near-infrared image sensor can be used as the image sensor 701 that receives near-infrared light, and a conventional black-and-white image sensor, a near-infrared image sensor, or a conventional color image sensor can be used as the image sensor 702 that receives visible light. However, when using a color image sensor, the bandpass filter 802 is changed to allow only visible light wavelengths from 400 to 650 nm to pass through. Furthermore, as... Figure 7B As shown, a linear polarization filter 1001 can also be inserted into the optical path of a near-infrared camera. In this embodiment, a patterned light source 1 with a near-infrared wavelength of 830 nm is used as the laser source. Laser light has the characteristic of linear polarization, and the surface-reflected light reflected from the skin surface maintains the linear polarization property of the light source. On the other hand, internally scattered light that penetrates into the skin and is repeatedly scattered and emitted from the skin loses the linear polarization property. As described above, since information inside the organism (e.g., blood flow information) is contained in the scattered reflected light, by arranging the linear polarization filter 1001 perpendicular to the polarization direction of the laser source (i.e., orthogonal Nicol configuration), it is possible to prevent surface-reflected light from the skin surface from reaching the near-infrared image sensor 701 and efficiently obtain internally scattered light. However, by using a polarization filter, the amount of signal that can be obtained is reduced. Therefore, whether to use a linear polarization filter can be determined appropriately according to the measurement conditions and system specifications. In cases where light from a light source with a pattern of illumination points leaks significantly into areas outside the area that should be illuminated (i.e., areas that are not the areas that should be illuminated), the signal-to-noise ratio of biological information can be significantly improved by utilizing a polarization filter.

[0129] If the shutter button 11 is pressed, the image sensors 701 and 702 acquire images of the organism's motion.

[0130] Similar to Embodiment 1, the arithmetic circuit 22 within computer 20 first performs skin detection on the human body based on a near-infrared motion image, extracting specific parts of the face (e.g., the forehead). Based on the near-infrared image output from image sensor 701, it separates and measures the surface reflected light component from the internal scattered light component, and obtains a signal representing heartbeat information based on these calculations. A signal representing heartbeat information is also obtained based on a green motion image output from image sensor 702.

[0131] Figure 8 This is a graph illustrating an example of the time-varying intensity of the obtained signals. The intensity of signal A obtained from the near-infrared motion image and the intensity of signal B obtained from the green motion image both vary with time. Here, let the intensities of near-infrared light and green light on the biological surface be Ii(IR) and Ii(G), respectively, and let the time average values ​​of the varying components of the internal scattered light in the near-infrared and green wavelength domains be ΔI(IR) and ΔI(G), respectively. Blood oxygen saturation SpO2 is calculated using the following formula.

[0132] SpO2=a+b×(log(ΔI(G) / Ii(G)) / (log(ΔI(IR) / Ii(IR)))

[0133] The values ​​of a and b in the above formula can be determined based on their relationship with the measured values ​​of an existing pulse oxygen saturation meter.

[0134] To confirm the accuracy of the measuring device, the system was used to measure oxygen saturation at the fingertip. The upper arm was pressurized with pressure (200 mmHg) using a band similar to that used in blood pressure measurement to stop blood flow, and oxygen saturation was measured at the fingertip.

[0135] A commercially available pulse oximeter, which is inserted into the index finger, is used to non-contactly measure the oxygen saturation of the middle finger using this system. After determining a and b above through initial measurement, the blood oxygen saturation SpO2 is measured.

[0136] Figure 9 The vertical axis shows a comparison between the measured values ​​using a pulse oximeter and the horizontal axis shows the measurement values ​​of this embodiment. Since the results are largely consistent, it can be seen that accurate measurement is possible. In this embodiment, not only blood oxygen saturation is measured, but also... Figure 8 The pulse wave shown also measures the pulse rate.

[0137] According to this embodiment, by making the imaging system a stereo camera structure, the overall system becomes compact, and the structure of the signal processing system from the subsequent image signal processing to oxygen saturation calculation is simplified. Thus, both ease of operation and high speed can be achieved.

[0138] (Implementation Method 3)

[0139] As a third embodiment, another method for measuring blood oxygen saturation using a single camera will be described. In the second embodiment, a stereo camera structure was used, comprising two optical systems and two image sensors. In this embodiment, a system is employed that segments the image using multiple lenses and acquires two different images corresponding to two different wavelengths using a single image sensor. This method is referred to as the "stereo lens method." (See reference...) Figure 10 The system using a stereo lens is described.

[0140] Figure 10 This is a schematic cross-sectional view showing a portion of the bio-information detection device according to this embodiment. The bio-information detection device has a light source 1 that illuminates patterned light in the near-infrared range of 850 nm, and two sets of lenses 501 and 502 inside a lens 5, designed to image different areas of the imaging surface of an image sensor 7. In front of lenses 501 and 502, two bandpass filters 801 and 802 are respectively arranged to allow light with a wavelength of 850 nm to pass through and light with wavelengths from 520 to 600 nm to pass through.

[0141] With this structure, it is possible to acquire two images formed by two wavelengths of light simultaneously using a single image sensor 7. This is used as the image sensor 7. Figure 4B The near-infrared image sensor shown. Control circuit 26 calculates organism information based on these two images using the same method as in embodiments 1 to 3. According to this embodiment, since information from two images corresponding to two different wavelengths at the same time is contained in a single image signal, computational processing becomes easier.

[0142] The following describes the results of pressure sensing using this stereoscopic lens system. A method for detecting a decrease in nasal temperature due to pressure or concentration is proposed using a body surface temperature distribution measuring device. Due to psychological changes, blood flow to the nose decreases, resulting in a drop in nasal temperature. This is typically detected using a body surface temperature distribution measuring device. Changes in facial temperature occur through changes in blood flow. If changes in blood flow can be measured with higher accuracy, pressure sensing can be performed with greater accuracy and better responsiveness compared to measuring changes in surface temperature that result from changes in blood flow.

[0143] Pressure was sensed using a conventional body surface temperature distribution measuring device and a biological information detection device according to a present disclosure, and the results were compared. The pressure was measured by immersing the right hand in cold water. Regarding... Figure 11AThe nose and cheeks, surrounded by dots and lines, are used to measure blood flow information using image signals obtained from a biological information detection device according to a technical solution of this disclosure, and to measure temperature changes using a body surface temperature distribution measuring device. Figure 11B This graph shows the results of pressure sensing using a conventional body surface temperature distribution measuring device. The temperature of the nose gradually decreased over approximately 3 minutes after the cold water load began, dropping by about 1.2°C before stabilizing. After the load ended, the temperature returned to normal over approximately 3 minutes. On the other hand, the temperature of the cheeks remained stable with almost no effect from the cold water load.

[0144] Figure 11C This is a graph showing the changes in blood flow and blood oxygen saturation obtained using the bio-information detection device of this embodiment, which employs a stereoscopic lens method. From the blood flow and oxygen saturation (SpO2) data of the face, the changes in blood flow and oxygen saturation (SpO2) are extracted. Figure 11A Data for the nose and cheek areas are represented by dotted lines. Solid lines represent changes in blood flow over time, and dotted lines represent changes in oxygen saturation (ΔSpO2) over time. Figure 11C As shown, blood flow in the nose decreased immediately after cold stimulation, indicating a high time-response characteristic. On the other hand, blood flow in the cheeks remained almost unchanged. Regarding oxygen saturation, a decrease in oxygen saturation was observed in the nose along with the decrease in blood flow, while it remained almost unchanged in the cheeks.

[0145] These results demonstrate that measuring blood flow and oxygen saturation at different points on the face yields a wealth of data. Based on this data, it is possible to detect mood changes, physical condition, or level of concentration with high accuracy. Since blood flow variations due to the autonomic nervous system differ across facial areas, measuring these variations using a camera is particularly important. Furthermore, simultaneously measuring areas with less variation in blood flow as a reference can improve measurement accuracy.

[0146] In this structure, there is only one image sensor, and it is not possible to distinguish between near-infrared light and visible light. In the above example, a near-infrared image sensor is used as image sensor 7, but a color image sensor could also be used. Figure 4CAs shown, in a color image sensor, red, green, and blue pixels also have sensitivity to near-infrared light above 800 nm. Therefore, bandpass filter 801 is configured to allow light with a wavelength of 850 nm, corresponding to the wavelength of the light source, to pass through, and bandpass filter 802 is configured to allow light with wavelengths from 400 to 650 nm, which are visible light wavelengths, to pass through. Thus, the image sensor area imaged using bandpass filter 801 can acquire a near-infrared image, and the image sensor area imaged using bandpass filter 802 can acquire a three-color image (red, green, and blue). In reality, in the near-infrared region, subtle differences in near-infrared light sensitivity sometimes occur due to the subtle differences in the near-infrared transmission characteristics of the red, green, and blue color filters.

[0147] In such cases, sensitivity correction can be performed simply by pre-observing the sensitivity differences between the individuals. As described above, high-precision body motion correction can be achieved by using a color image sensor.

[0148] (Implementation Method 4)

[0149] As a fourth embodiment, an example of combining human body detection and human body information sensing is shown. A system according to a technical solution of this disclosure can rapidly detect human bodies and measure biometric information such as heart rate with high speed and high precision based on data from the detected human body area. Utilizing this feature, a monitoring system can be realized in personal spaces such as bathrooms, toilets, and bedrooms. In personal spaces, privacy considerations are particularly important. In systems that use high-resolution cameras to continuously photograph and use images of subjects, there are concerns about privacy violations due to image leakage, as well as psychological burdens caused by the presence of the camera during photography.

[0150] With the aging population, it is estimated that 10,000 to 20,000 people die in bathhouses in Japan each year. This number far exceeds the 4,000 to 5,000 deaths from traffic accidents. The causes of death in bathhouses are both accidents and illnesses. The deceased are predominantly elderly, and the number of deaths is higher in winter, increasing annually with the aging population. Regarding deaths in bathhouses, whether due to accidents or illnesses, there are many cases where lives could have been saved if the abnormalities had been detected earlier. Because bathhouses are enclosed and private spaces, many deaths are due to delayed detection. There is a strong desire for a system that can monitor those in bathhouses while respecting their privacy.

[0151] The camera system in this embodiment adopts a different approach than that in embodiment 3, which uses a single camera to measure information about organisms. Figure 12This is a cross-sectional view schematically showing the structure of the bio-information detection device according to this embodiment. The device includes a stereo adapter 100 that can be mounted onto a conventional camera lens. The stereo adapter 100 is an accessory including four mirrors 101, 102, 103, and 104 and two bandpass filters 801 and 802. By using the stereo adapter 100, two images corresponding to two wavelengths can be formed in two different regions of the imaging surface of the image sensor 7. This method is referred to as the "stereo adapter method".

[0152] In the stereo adapter configuration, two sets of opposing mirrors are used, enabling a single image sensor 7 to acquire two different images corresponding to two wavelengths. An image of the subject illuminated with patterned near-infrared light at a wavelength of 830 nm is acquired by a camera attached to the front end of the lens 5 using a stereo adapter 100. Pairs of mirrors 101 and 102, and pairs of mirrors 103 and 104, bend the light path twice before guiding it into the lens 5. Bandpass filters 801 and 802, which allow light at a wavelength of 830 nm and light at wavelengths of 520–600 nm to pass through, are mounted between the lens 5 and the mirrors 101, 102, 103, and 104, respectively.

[0153] This bio-information detection device can acquire images of two wavelengths simultaneously using a single image sensor 7. Here, the image sensor 7 also uses... Figure 4B The near-infrared image sensor shown is based on the same basic considerations as in implementation method 3. The stereo lens approach allows for lens miniaturization, thus enabling overall system miniaturization. Conversely, while the stereo adapter approach results in a larger overall system, it offers advantages such as the ability to use high-performance camera lenses for improved resolution, and the ability to use lenses with different magnifications and zoom lenses. This increased system flexibility is a key advantage of the stereo adapter approach. Furthermore, the stereo lens approach suffers from low sensitivity due to its smaller numerical aperture, resulting in insufficient light capture. In contrast, this approach allows for the use of lenses with larger numerical apertures, enabling the construction of a highly sensitive system capable of sensing even in darker conditions.

[0154] In this embodiment, as described in the third embodiment, a color image sensor can also be used as the image sensor 7. In this case, as the bandpass filter 802, any bandpass filter that allows light of wavelengths from 400 to 650 nm, which are visible light wavelengths, to pass through can be used.

[0155] use Figure 13A , Figure 13B The actual monitoring algorithm of this embodiment will be explained. Figure 12 The biometric information detection device shown is installed in a corner of the bathroom, such as... Figure 13A As shown in section (a), it is capable of monitoring the entire bathroom. Human detection is performed based on the near-infrared light images captured by the camera. Figure 13A (b) of the), body movement detection ( Figure 13A (c) and detection of abnormal heartbeats Figure 13A (d)). Upon detecting a human body, if there is no physical movement, for example, an alarm 1 (alarm 1) is issued to the person in the bath to draw their attention. Furthermore, if an abnormal heartbeat is detected, an alarm 2 (alarm 2) is issued to someone outside the bathroom, for example. See below for further details. Figure 13B The flowchart below provides a more detailed explanation of the operation of the monitoring system in this embodiment.

[0156] Figure 13B This is a flowchart illustrating the operation of the monitoring system according to this embodiment. First, the arithmetic circuit 22 detects a human body based on the acquired near-infrared image data using the same method as in Embodiment 1 (step S201). Here, if a human body is detected, the process proceeds to the next step, body motion detection, S202. At this time, the image data used in human body detection is not recorded in the storage device; only the data of the human body area is retained and rewritten using the image data of the next frame of the motion image. In this way, image data that can identify an individual is not retained, thus protecting privacy.

[0157] Next, the processing circuit 23 detects body movement in the detected human body area by comparing data between multiple consecutive frames (step S202). For example, if there is no body movement for a certain period of time (e.g., 30 seconds), an alarm 1 is issued to the person (step S203). This could be an alarm such as, "Are you awake? It is dangerous in the bathroom. If you are awake, please press the OK button." Alarm 1 is for the purpose of arousing the person's attention and confirming their status. In the absence of body movement, the processing circuit 23 then measures pulses (step S204). If there are few pulses or no pulses can be detected, an alarm 2 is issued (step S205). This is an alarm for people outside the bathroom (family members, caregivers, ambulances, etc.). It could be an alarm intended to confirm and request assistance from a person pre-set by the system via sound alarm, telephone, or Internet.

[0158] According to this embodiment, since it is possible to perform three-stage detection (1) human body detection, (2) body movement detection and (3) heart rate measurement with a simple system structure, it is possible to achieve highly reliable monitoring.

[0159] In the above example, the detection is carried out in three stages: (1) human body detection, (2) body movement detection, and (3) heart rate measurement. However, body movement detection and heart rate measurement can also be carried out in parallel after human body detection. This allows for stable monitoring of the bather's heart rate and provides appropriate advice to the bather. Drowning is more common due to changes in heart rate caused by vasoconstriction due to the temperature difference between the undressing area and the bathroom, and to orthostatic hypotension caused by a decrease in blood flow to the brain and heart due to an increase in blood flow to the body surface, resulting in orthostatic hypotension. By measuring changes in the bather's physical condition in real time using heart rate monitoring and providing feedback to the bather, such accidents can be prevented. For example, in cases of a significant increase in heart rate, a message such as "Beware of orthostatic hypotension. Please hold onto the handrail and stand up slowly" can be issued.

[0160] In monitoring systems located in private spaces such as bathrooms, toilets, and bedrooms, privacy protection is particularly important. In this embodiment, the images captured during image signal processing are only used for human detection and heart rate measurement; the image data itself is not recorded on a storage medium and is always overwritten by the next frame's data after human detection processing. Furthermore, the system in this embodiment is designed without an image data output mechanism. Therefore, a structure that prevents image data from being obtained externally is designed to ensure privacy is not compromised even in the event of malicious attacks from hackers. Such hardware and psychological privacy assurance is especially important in monitoring systems located in private spaces. This system enables privacy-conscious monitoring within the home.

[0161] Furthermore, regarding long-term monitoring in bedrooms, hospital wards, and similar settings, there are situations where the measurement environment is bright during the day and exposed to direct sunlight, while at night, the lights are turned off to create a dark environment. Stable detection of biological information and long-term monitoring under such significantly different environmental conditions is difficult with conventional biological information detection systems. However, the biological information detection device using near-infrared and green light, as disclosed in this disclosure, enables a monitoring system unaffected by environmental changes.

[0162] (Implementation Method 5)

[0163] As a fifth embodiment, an application example of driver monitoring will be described. Currently, the development of autonomous driving technology for automobiles is progressing. Beyond fully autonomous driving where no human is driving, there is a desire for smooth switching between human driving and autonomous driving. In particular, a mechanism is required to continuously monitor the driver's state and quickly switch to autonomous driving when the driver is in a state unsuitable for driving. To achieve such a driving switch, a driver monitoring system capable of continuously monitoring the driver's state is desired. Under driver monitoring, the measurement environment also varies considerably. Brightness varies significantly; it is bright during the day, there is direct sunlight in the evening, and the interior of the car is dark at night, making detection through visible light difficult. Furthermore, the subject vibrates relative to the camera due to the car's movement. In addition, it is affected by body movements caused by driving operations. Thus, the environment for driver monitoring as a biometric detection is very demanding. As described above, the biometric detection device of the present disclosure has strong tolerance to changes in the lighting environment and the advantage of being able to eliminate the influence of body movements, making it suitable for driver monitoring.

[0164] As the camera system in this embodiment, the one described in Embodiment 4 is adopted. Figure 12 The stereo adapter method is shown. However, in the near-infrared patterned light, a wavelength of 940nm, which is present in a relatively small proportion in sunlight due to absorption by water vapor, is used. Correspondingly, in bandpass filters 801 and 802, bandpass filters are used that allow light of wavelengths from 940nm and 520 to 600nm to pass through, respectively.

[0165] use Figure 12 The biometric detection device shown monitors the driver. It measures pulse rate, blood flow, and oxygen saturation based on near-infrared and green light images. Here, blood flow is calculated based on the magnitude of the pulse, i.e., the amplitude of the pulse wave signal. By analyzing the fluctuations in pulse over time, it is possible to determine stress, level of concentration, and drowsiness. Figure 14 The flowchart in the middle represents the driver monitoring process.

[0166] The system measures pulse rate, blood flow, and oxygen saturation using a biometric detection device (step S301). Based on this information, if a sharp drop in pulse rate, blood flow, or oxygen saturation above a first threshold is detected, it is determined to be a change in the driver's physical condition, and the driver is forcibly and automatically switched to driving mode after a warning (step S302). Since a gradual decrease in pulse rate and an increase in pulse rate fluctuations can be considered as a sign of decreased concentration or drowsiness, the pulse rate is measured again (step S303). If a change in pulse rate fluctuation above a second threshold is detected, an alert to attract the driver's attention is issued (step S304). The pulse rate is measured again (step S305). If the change in pulse rate fluctuation does not decrease below the second threshold after the alert, the driver is forcibly and automatically switched to driving mode after a warning (step S306). This driver monitoring system is applicable not only to autonomous driving but also to driving a regular car. By issuing alerts to the driver only when physical condition changes or concentration decreases, driving safety can be increased.

[0167] (Implementation Method 6)

[0168] As a sixth embodiment, a method for measuring blood oxygen saturation using a single camera without image segmentation using an optical system will be described. In embodiments 3 to 5, methods were described for sensing biological information such as oxygen saturation by segmenting light into two wavelengths and calculating the result. The biological information detection device of this embodiment does not perform image segmentation, but instead uses an image sensor to acquire two image signals of different wavelengths.

[0169] Figure 15A This diagram schematically illustrates the structure of the bio-information detection device according to this embodiment. The device separates two images corresponding to two wavelengths using an image sensor 703, rather than an optical system. Near-infrared light and green light returning from the subject illuminated with patterned light of near-infrared wavelength (860 nm) are imaged onto the imaging surface of the image sensor 703 using a lens 5. The image sensor 703 used here differs from conventional image sensors, having a color filter G that allows green light to pass through and a color filter IR that allows near-infrared light to pass through.

[0170] Figure 15B This diagram shows multiple color filters opposite to multiple light detection units arranged on the imaging surface of the image sensor 703. The image sensor 703 has a color filter G that selectively transmits light in the 520-600 nm range and a color filter IR that selectively transmits light with wavelengths above 800 nm. The color filters G and IR are arranged in a checkerboard pattern. Figure 15CThis diagram illustrates an example of the wavelength dependence of the transmittance of color filters G and IR. Image sensor 703 uses multiple light detection units (also called pixels) to detect two images formed by green light and 860nm near-infrared light. Here, if a color filter that selectively allows light with wavelengths of 520-600nm to pass through can be selected, an image sensor can be formed with a simple structure. However, since the green color filter used in typical image sensors has the characteristic of allowing infrared light to pass through, even using only the green color filter, a green image cannot be obtained in the presence of near-infrared light. Instead, an image is obtained by combining the green light and near-infrared light. One solution is to calculate the green image by subtracting the near-infrared light image signal from the image signal combining the green light and near-infrared light for each pixel. However, in this method, the signal-to-noise ratio deteriorates, making high-precision detection difficult. Therefore, in this embodiment, as... Figure 15D As shown, a near-infrared absorption filter 805 is formed on a green filter 804 adjacent to a near-infrared light filter 803, such that green light is incident on a green pixel. By employing this structure, the green pixel becomes... Figure 15B As shown, it is only sensitive to green.

[0171] exist Figure 16A The middle indicates the use of having with Figure 15A Examples of image sensors 704 with different color filter structures. Here, images of visible light and near-infrared light are imaged onto the imaging surface of the image sensor 704 by the lens 5. The image sensor 704 used here includes a light detection unit for acquiring color images and a light detection unit for acquiring near-infrared light images.

[0172] Figure 16B This is a diagram showing multiple color filters arranged on the imaging surface of the image sensor 704. Figure 16C This represents the wavelength dependence of the relative sensitivity of the pixels opposite each filter. For example... Figure 16B As shown, three color filters (R, G, B) that allow red, green, and blue light to pass through, respectively, and a color filter (IR) that allows light above 650nm to pass through are arranged on the imaging surface. Here, a near-infrared absorption filter is actually formed above the color filters R, G, and B, realizing pixels that do not contain near-infrared light (R, G, B). In a typical Bayer color filter configuration, two green filters are arranged diagonally adjacent to each other, and red and blue filters are arranged diagonally opposite them. However, in this embodiment, one of the two green pixels is a near-infrared light pixel. Figure 2 As shown, red and blue signals are not effective for detecting biological information. Red and blue pixels are used here to remove the effects of body movement and changes in ambient light from the green signal using blue and red pixel data. Figures 5A to 5CAs shown, the effects of body motion are removed in near-infrared light using surface reflection and internal scattering. Similarly, by using red or blue signals, the effects of body motion and ambient light variations can be eliminated from the green signal.

[0173] (Other implementation methods)

[0174] The embodiments of this disclosure have been illustrated above, but this disclosure is not limited to the above embodiments and various modifications are possible. The processing described in the above embodiments may also be applicable to other embodiments. Examples of other embodiments will be described below.

[0175] In the above embodiments, a laser source is used as the light source for projecting the dot pattern, but other types of light sources can also be used. For example, a cheaper LED light source can also be used. However, compared with a laser source, an LED light source has lower linear propagation and is more prone to diffusion. Therefore, it is sufficient to use a dedicated focusing optical system or limit the distance between the object being photographed and the camera.

[0176] The bio-information detection device can also have an adjustment mechanism for adjusting the focus of the optical system. Such an adjustment mechanism could, for example, consist of a motor (not shown) and... Figure 4D The control circuit 26 shown implements this. This adjustment mechanism adjusts the focus of the optical system to maximize the contrast of the image of the dot pattern projected from the light source onto the object. Therefore, the accuracy of the contrast calculation described in Embodiment 1 is improved.

[0177] The processing circuit 22 can also extract the surface reflectance light component from the image signal of the organism's surface, and generate epidermal information based on this surface reflectance light component, including at least one of the following: melanin concentration, presence or absence of spots, and presence or absence of moles. The surface reflectance light component can be obtained, for example, based on whether the contrast exceeds a predetermined threshold as described in Embodiment 1, or by removing low-frequency components from the image signal.

[0178] This disclosure describes a dual-camera setup using two cameras. Figure 1A A stereo camera system that incorporates two optical systems and two image sensors within a single camera. Figure 7A and Figure 7B Stereo lens method using two sets of lenses and one image sensor ( Figure 10 ), using a lens adapter and a stereo adapter method that uses one lens and one image sensor ( Figure 12 ), using image sensors to segment images ( Figure 15A , Figure 16A As already described, each method has its advantages and disadvantages, so the optimal method can be chosen based on the intended use.

[0179] This disclosure illustrates an example of using a dot pattern as a near-infrared patterned light source, but other patterns may also be used. For example, line-spaced patterns, checkerboard patterns, grid patterns, etc., may also be used.

[0180] Furthermore, in this disclosure, a bandpass filter with a wavelength of 520-600 nm is used in the green pixel, but it is not limited to this wavelength range. The wavelength that best contains biological information is 570-590 nm, and the signal-to-noise ratio deteriorates at greater distances. Therefore, if only the signal-to-noise ratio is considered, a bandpass filter with a wavelength of 570-590 nm or a narrow-band light source of 580 nm can also be used. However, since green light is efficient in utilizing ambient light, the use of a narrow-band bandpass filter reduces sensitivity and measurement accuracy in darker environments. Based on the balance between sensitivity and signal-to-noise ratio, a bandpass filter with a wavelength of 520-600 nm is used in the embodiments of this disclosure, but the wavelength range can be varied depending on the environment. If the application is intended for use in a bright environment, a narrow-band filter with a wavelength of 570-590 nm can also be used. On the other hand, a wider wavelength range is preferable in the case of a darker environment, but if the signal-to-noise ratio is considered, the wavelength range can also be controlled to be from 500 to 620 nm.

[0181] As explained above, according to the embodiments of this disclosure, it is possible to measure not only heart rate and blood flow, but also blood oxygen saturation, in various environments without restraining the subject or allowing the detection device such as sensors to come into contact with the subject. Furthermore, it is possible to infer the subject's emotional changes or physical condition based on the measured blood flow and oxygen saturation values ​​at different sites on the subject.

[0182] Label Explanation

[0183] 1. Light source

[0184] 2, 201, 202 Cameras

[0185] 3. Organisms

[0186] 4. Surface of organisms

[0187] 5. Lenses 501 and 502

[0188] 6, 601, 602 Housing

[0189] 7, 701, 702, 703, 704 Image Sensors

[0190] 8, 801, 802 bandpass filters

[0191] Mirrors 901 and 902

[0192] 100 Stereo Adapter

[0193] Mirrors 101, 102, 103, and 104

[0194] 11. Shutter button

[0195] 20 Computers

[0196] 21 Input Interface

[0197] Operational circuits 22 and 23

[0198] 24 Output Interfaces

[0199] 25. Memory

[0200] 26 Control Circuit

[0201] 27 Monitors

[0202] 31 Capillaries

[0203] 32. Small arteries and veins

[0204] 33 Epidermis

[0205] 34 Genuine Leather

[0206] 35 Subcutaneous tissue

[0207] L0 Light from the array of point sources

[0208] L1 surface reflected light

[0209] L2 internal scattered light

[0210] L3 Reflected light in the visible wavelength range

Claims

1. A biological information detection device, characterized in that, have: The light source projects a pattern of light containing multiple points, formed by the first light corresponding to the near-infrared wavelength domain, onto the organism. The camera system detects the second light generated by the projection through the aforementioned multiple points and generates an image signal containing multiple pixel signals; as well as Operational circuits; The second light mentioned above includes surface-reflected light, which is light reflected from the surface of the organism, and internally scattered light, which is scattered from the interior of the organism. The aforementioned processing circuit selects a plurality of first pixel signals and a plurality of second pixel signals from the plurality of pixel signals. The plurality of first pixel signals correspond to a first region in the organism that is a region emitting the surface reflected light, and the plurality of second pixel signals correspond to a second region in the organism that is a region emitting the internal scattered light. The aforementioned processing circuit generates biological information of the organism based on the aforementioned plurality of first pixel signals and the aforementioned plurality of second pixel signals.

2. The biological information detection device as described in claim 1, characterized in that, The aforementioned organism information includes at least one selected from the group consisting of the organism's heart rate, blood pressure, blood flow, blood oxygen saturation, melanin concentration in the organism's skin, presence or absence of spots on the organism's skin, and presence or absence of moles on the organism's skin.

3. The biological information detection device as described in claim 1, characterized in that, The first light mentioned above includes light with wavelengths above 650 nm and below 950 nm.

4. The biological information detection device as described in claim 1, characterized in that, The aforementioned processing circuit generates the aforementioned biological information based on the average intensity of each of the plurality of first pixel signals and the average intensity of each of the plurality of second pixel signals.

5. The biological information detection device as described in claim 1, characterized in that, The aforementioned processing circuit generates the aforementioned biological information based on the ratio of the intensity of the plurality of first pixel signals to the intensity of the plurality of second pixel signals.

6. The biological information detection device as described in claim 1, characterized in that, The aforementioned processing circuit generates the aforementioned biological information by subtracting the value obtained by multiplying the intensity of the aforementioned plurality of first pixel signals by a certain value from the intensity of the aforementioned plurality of second pixel signals.

7. The biological information detection device as described in claim 1, characterized in that, The aforementioned plurality of first pixel signals are pixel signals with relatively strong intensity among the plurality of pixel signals; The aforementioned multiple second pixel signals are pixel signals with relatively weak intensity among the aforementioned multiple pixel signals.

8. A method for detecting biological information, which is a computer-executed method, characterized in that, Includes the following processing: Using a light source, a pattern of light containing multiple points, formed by a first light corresponding to the near-infrared wavelength domain, is projected onto a living organism; and The camera system detects the second light generated by the projection through the aforementioned multiple points, and the camera system generates an image signal containing multiple pixel signals; The second light mentioned above includes surface-reflected light, which is light reflected from the surface of the organism, and internally scattered light, which is scattered from the interior of the organism. The above-mentioned biological information detection methods also include the following processing: From the plurality of pixel signals, a plurality of first pixel signals and a plurality of second pixel signals are selected, wherein the plurality of first pixel signals correspond to a first region in the organism that is a region emitting surface-reflected light, and the plurality of second pixel signals correspond to a second region in the organism that is a region emitting internally scattered light; and Based on the aforementioned multiple first pixel signals and multiple second pixel signals, the biological information of the aforementioned organism is generated.

9. A storage medium, characterized in that, The record contains a program that causes a computer to execute the biological information detection method of claim 8.

Images