Biological information analysis equipment

Abstract

To continuously monitor a change in biological information of a subject.SOLUTION: A biological information analysis device 1 comprises: a camera 10 which images an eye of a target person A; a light emitting element 13a which is provided in an opening 12 of the camera 10 and emits light in the first center wavelength toward the pupil of the target person A; a light emitting element 13b which is provided at the same position as the light emitting element 13a with respect to the opening 12 and emits light in the second center wavelength toward the pupil of the target person A; a light source controller 40 which controls the light emitting elements 13a, 13b; and a computing device 20. The computing device 20 acquires a first eye image in which the eye irradiated with light by the light emitting element 13a is reflected and a second eye image in which the eye irradiated with light by the light emitting element 13b is reflected from the camera 10, and calculates and outputs an analysis amount about biological information of the target person A on the basis of the luminance in a region corresponding to the pupil in the first eye image and the luminance in a region corresponding to the pupil in the second eye image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a biological information analyzer that analyzes values ​​relating to biological information. [Background technology]

[0002] Conventionally, devices have been used that acquire images of specific locations such as the fundus of a subject and generate biometric information such as the subject's blood oxygen saturation based on the images (Patent Documents 1 to 3 below). It is generally known that generating biometric information based on an image of a subject is useful for diagnosing the subject's diseases, such as age-related macular degeneration, diabetic retinopathy, retinitis pigmentosa, and retinal vascular occlusion. In addition, in the recent outbreak of the novel coronavirus, the importance of measuring blood oxygen saturation, which is an indicator of the severity of the disease, has been pointed out, and pulse oximeters, which are devices that measure blood oxygen saturation using near-infrared light by attaching a probe to a fingertip, are being used. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2002-238850 A [Patent Document 2] JP 2001-145603 A [Patent Document 3] JP 2018-68484 A Summary of the Invention [Problem to be solved by the invention]

[0004] The conventional devices described above require the measurement of biological information while the device is close to the subject or while the subject is wearing the device, making it difficult to continuously monitor changes in the subject's biological information in daily life.

[0005] The present invention has been made in consideration of the above problems, and has an object to provide a biological information analysis device capable of continuously monitoring changes in a subject's biological information. [Means for solving the problem]

[0006] In order to solve the above problems, a bioinformation analysis device according to one embodiment of the present invention includes a camera that acquires an eye image by capturing an image of a subject's eye, a first light source that is provided in an opening of the camera and irradiates light of a first central wavelength toward the subject's pupil, a second light source that is provided in the same position as the first light source with respect to the opening of the camera and irradiates light of a second central wavelength different from the first central wavelength toward the subject's pupil, a light source controller that controls the irradiation of light by the first light source and the second light source, and a calculation device that processes the eye image, wherein the calculation device acquires from the camera a first eye image that is an eye image reflecting the eye illuminated by the first light source and a second eye image that is an eye image reflecting the eye illuminated by the second light source, calculates an analysis amount related to the bioinformation of the subject based on a first luminance in a region of the first eye image corresponding to the pupil and a second luminance in a region of the second eye image corresponding to the pupil, and outputs the analysis amount.

[0007] The "camera opening" here means a portion for taking in the light of an image from outside the camera to the image sensor inside the camera, and is not necessarily limited to the circular lens part of the camera barrel, but means the opening when the lens part of the camera is covered with a cover member having an opening narrower than the lens part. The shape of the opening is not limited to a circle, and may be various shapes such as a polygon such as a rectangle, an ellipse, etc.

[0008] According to the biometric information analysis device of the above embodiment, a first eye image reflecting the pupil of the subject illuminated by light of a first central wavelength and a second eye image reflecting the pupil of the subject illuminated by light of a second central wavelength from the same position as the light of the first central wavelength toward the entire pupil of the subject are acquired, and an analysis amount related to the biometric information of the subject is calculated and output based on a first luminance in a region of the first eye image corresponding to the pupil and a second luminance in a region of the second eye image corresponding to the pupil. This allows the subject to acquire biometric information simply by facing the camera, making it easy to continuously monitor changes in the biometric information.

[0009] Here, the first light source and the second light source may be evenly arranged around the opening of the camera. In this case, the light of the first central wavelength and the light of the second central wavelength can be evenly irradiated onto the pupil of the subject, and highly accurate biological information reflecting the characteristics of the entire pupil can be obtained.

[0010] The light source controller may control the first light source and the second light source to alternately irradiate light at a first timing and a second timing, which are different timings, and the computing device may acquire, from the camera, a first eye image that is an eye image captured at the first timing and a second eye image that is an eye image captured at the second timing. In this case, the first eye image reflecting the pupil of the subject illuminated by light of the first central wavelength and the second eye image reflecting the pupil of the subject illuminated by light of the second central wavelength can be acquired with little noise, so that highly accurate biological information can be acquired.

[0011] The camera may also have an optical element in which a first dividing element combining a bandpass filter that passes light of a first central wavelength and a first polarizer that transmits linearly polarized light at a predetermined angle and a second dividing element combining a bandpass filter that passes light of a second central wavelength and a second polarizer that transmits linearly polarized light at an angle different from the predetermined angle are divided and arranged inside the lens of the opening, and an image sensor in which polarizers that transmit linearly polarized light of at least two different angles are attached to adjacent pixels, wherein the light source controller controls the first light source and the second light source to be irradiated simultaneously, and the arithmetic device calculates a luminance corresponding to the light of the first central wavelength based on the luminance of adjacent pixels in the eye image and combines the calculated luminances to obtain a first eye image, and calculates a luminance corresponding to the light of the second central wavelength based on the luminance of adjacent pixels in the eye image and combines the calculated luminances to obtain a second eye image. In this case, a first eye image reflecting the subject's pupil illuminated by light of the first central wavelength and a second eye image reflecting the subject's pupil illuminated by light of the second central wavelength can be obtained by calculation as images reflecting the pupils at the same time, and highly reliable biometric information can be obtained.

[0012] The computing device may also calculate the analysis amount of the blood oxygen of the subject by calculating the ratio between the value based on the first luminance and the value based on the second luminance. In this case, the change in the biological information of the subject regarding the blood oxygen can be continuously monitored.

[0013] The first central wavelength and the second central wavelength may be in the near infrared region of 760 nm or more and 1000 nm or less. In this way, the subject does not feel dazzled during measurement, and analysis of biological information with less burden on the subject can be realized.

[0014] The arithmetic device may also acquire an unilluminated image, which is an eye image captured at a third timing when the first light source and the second light source are not irradiating light, and calculate the analysis amount based on an image calculated using the first eye image and the unilluminated image and an image calculated using the second eye image and the unilluminated image. In this case, even if disturbance light such as natural light or illumination light is incident on the measurement environment, the influence of the disturbance light can be removed from the first eye image and the second eye image, and highly accurate biological information can be acquired.

[0015] The computing device may also calculate the analysis amount based on the first eye image excluding the range of the subject's corneal reflection and the second eye image excluding the range of the subject's corneal reflection. In this way, the analysis amount related to the biological information is calculated using only the image from the pupil, so that the analysis amount that reflects the biological information with high accuracy can be obtained.

[0016] The camera may further include a third light source that is located farther from the camera opening than the first and second light sources and irradiates light in the near-infrared region toward the subject's pupil, wherein the light source controller controls the third light source to irradiate at a fourth timing shifted from the first and second timings, and the computing device acquires a third eye image which is an eye image captured at the fourth timing, compares the third eye image with the first eye image or the second eye image to identify the pupil area on the eye image, and calculates an analysis amount based on a first luminance in the pupil area in the first eye image and a second luminance in the pupil area in the second eye image. In this case, in addition to the first eye image and the second eye image, a third eye image reflecting the subject's pupil illuminated by light in the near-infrared region from an angle direction larger than the light of the first central wavelength based on the optical axis of the camera is acquired, and the analysis amount is calculated for the area of ​​the pupil identified from the comparison result between the third eye image and the first eye image or the second eye image. In this way, the area to be calculated for the analysis amount can be stably limited to the area of ​​the pupil, and more accurate biological information can be acquired.

[0017] Furthermore, the arithmetic device may acquire an unilluminated image, which is an eye image captured at a timing when the first light source and the second light source are not irradiating light, calculate a luminance corresponding to light of a first central wavelength based on the luminance of adjacent pixels in the unilluminated image, combine the calculated luminances to acquire a first unilluminated image, calculate a luminance corresponding to light of a second central wavelength based on the luminance of adjacent pixels in the unilluminated image, combine the calculated luminances to acquire a second unilluminated image, and calculate an analysis amount based on an image calculated using the first eye image and the first unilluminated image and an image calculated using the second eye image and the second unilluminated image. In this case, even if disturbance light such as natural light or illumination light is incident on the measurement environment, the influence of the disturbance light can be removed from the first eye image and the second eye image, and highly accurate biological information can be acquired.

[0018] The camera may further include a third light source that is located farther away from the camera opening than the first and second light sources and irradiates light in the near-infrared region toward the subject's pupil, and the optical element may further include a third dividing element that combines a bandpass filter that passes the central wavelength of the light irradiated by the third light source and a third polarizer that transmits linearly polarized light at an angle different from that of the first polarizer and the second polarizer, and is divided and provided inside the lens of the opening. The light source controller controls the third light source to irradiate simultaneously with the first and second light sources. The arithmetic device may calculate a luminance corresponding to the light irradiated by the third light source based on the luminance of adjacent pixels in the eye image, combine the calculated luminances to obtain a third eye image, and identify the pupil area on the eye image by comparing the third eye image with the first eye image or the second eye image, and calculate an analysis amount based on a first luminance in the pupil area in the first eye image and a second luminance in the pupil area in the second eye image. In this case, in addition to the first eye image and the second eye image, a third eye image reflecting the subject's pupil illuminated by light in the near-infrared region from an angle direction larger than the light of the first central wavelength based on the optical axis of the camera is acquired by calculation, and the analysis amount is calculated for the area of ​​the pupil identified from the comparison result between the third eye image and the first eye image or the second eye image. In this way, the area to be calculated for the analysis amount can be stably limited to the area of ​​the pupil, and more accurate biological information can be acquired.

[0019] Furthermore, the computing device may calculate an average value of the luminance of pixels in the pupil region in the first eye image as the first luminance, and calculate an average value of the luminance of pixels in the pupil region in the second eye image as the second luminance. With this configuration, the influence of luminance noise in the pupil can be removed, and the tendency of changes in the biological information can be stably monitored.

[0020] Furthermore, the computing device may calculate an analysis amount for each pixel based on the luminance of the pixel in the pupil area in the first eye image and the luminance of the pixel in the pupil area in the second eye image, and output the analysis amount by averaging the analysis amount between the pixels in the pupil area. This also makes it possible to remove the influence of luminance noise in the pupil, and therefore to stably monitor the tendency of changes in biological information.

[0021] The computing device may also calculate the pupil position or gaze direction of the subject based on the first or second eye image and the third eye image, and perform a stabilization process of the pupil luminance detected when the analysis amount is acquired. In this way, the pupil luminance of the subject is stabilized, and the analysis amount with high sensitivity and high accuracy can be acquired.

[0022] The pupil luminance stabilization process may be a process of guiding the pupil position or the gaze direction to a predetermined position or a predetermined direction. In this way, the pupil position or the gaze direction of the subject is stabilized, so that an analysis amount with high sensitivity and high accuracy can be obtained. Effect of the Invention

[0023] According to the present invention, changes in biological information of a subject can be continuously monitored. [Brief description of the drawings]

[0024] [Figure 1] 1 is a perspective view showing a biological information analysis device according to an embodiment. [Diagram 2] FIG. 2 is a plan view showing a lens portion of the camera. [Diagram 3] FIG. 2 is a diagram illustrating a hardware configuration of a calculation device according to an embodiment. [Figure 4] FIG. 2 is a block diagram showing a functional configuration of a calculation device according to an embodiment. [Diagram 5] 1 is a timing chart showing an example of the irradiation timing of the light source 13 controlled by the timing control unit 21 and the photographing timing of the eye image acquired by the image acquisition unit 22. [Figure 6] 1 is a graph showing the absorption characteristics of oxygenated hemoglobin (HbO2) and reduced hemoglobin (Hb) present in blood versus the wavelength of incident light. [Figure 7] FIG. 13 is a diagram showing a schematic configuration of a camera 110 according to a modified example. [Figure 8] 8 is a plan view of the optical element in FIG. 7 as seen from outside the housing. [Figure 9] 8 is a plan view of the optical element in FIG. 7 as seen from outside the housing. [Figure 10] 1 is a diagram showing the structure of an image sensor 116 disposed inside an opening 12, as viewed from the opening 12 side. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Hereinafter, preferred embodiments of the bioinformation analyzer according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same or corresponding parts are designated by the same reference numerals, and duplicated description will be omitted.

[0026] First, a configuration of a bioinformation analysis device 1 according to an embodiment of the present disclosure will be described with reference to Figs. 1 to 4. The bioinformation analysis device 1 is a computer system that acquires an image of the eye of a subject A and generates an analysis amount related to the bioinformation of the subject A based on the image. A subject is a person whose bioinformation is to be analyzed, and can also be called a test subject. The bioinformation analysis device 1 of this embodiment analyzes blood oxygen saturation (SO2) in blood vessels near the fundus as an example of the bioinformation of the subject A.

[0027] As shown in FIG. 1, the biometric information analysis device 1 includes a pair of cameras 10 functioning as a stereo camera, a computing device 20, and a light source controller 40. In the following description, the pair of cameras 10 will be referred to as a left camera 10 on the left side of a subject A, as necessary. L and the right camera 10 on the right side of the subject A. RIn this embodiment, the biometric analysis device 1 further includes a display device 30 that is the object that the subject A looks at, but the object in the line of sight of the subject A is not limited to the display device 30 and may be, for example, a windshield of a car, a mirror, a smartphone, a tablet terminal, etc. Therefore, the display device 30 is not an essential element of the biometric analysis device 1. Each camera 10 is connected to the calculation device 20 wirelessly or by wire, and various data or commands are transmitted and received between the camera 10 and the calculation device 20. Camera calibration is performed in advance for each camera 10.

[0028] The camera 10 is used to capture an eye image by photographing the eye, including the pupil and the surrounding area, of the subject A. The pair of cameras 10 are arranged at a predetermined interval along the horizontal direction, and are installed at a position lower than the face of the subject A in order to prevent reflected light from appearing in the facial image when the subject A is wearing glasses. The elevation angle of the camera 10 with respect to the horizontal direction is set to, for example, a range of 20 to 35 degrees, taking into consideration both reliable detection of the pupil and avoidance of obstruction of the visual field range of the subject A.

[0029] In this embodiment, the cameras 10 are high-speed cameras capable of capturing images at a predetermined time interval (for example, at a time interval of 2,000 fps). The cameras 10 capture images of the subject A in response to commands from the computing device 20, and output image data to the computing device 20.

[0030] FIG. 2 shows a schematic view of the lens portion of the camera 10. As shown in this figure, in the camera 10, the objective lens 11 is accommodated in a circular opening 12, and a light source 13 is attached to the outside of the opening 12. The light source 13 is a device for irradiating illumination light toward the eye (pupil) of the subject A, and is composed of a plurality of light-emitting elements 13a (first light source), a plurality of light-emitting elements 13b (second light source), and a plurality of light-emitting elements 13c (third light source). The light-emitting element 13a is a semiconductor light-emitting element (LED) whose output light has a central wavelength of 810 nm (first central wavelength). The light-emitting element 13b is a semiconductor light-emitting element whose output light has a central wavelength of 940 nm (second central wavelength). The light-emitting element 13c is a semiconductor light-emitting element whose output light has a central wavelength of 940 nm (near-infrared region). The plurality of light-emitting elements 13a and the plurality of light-emitting elements 13b are arranged to form a double ring shape. In detail, the light-emitting elements 13a and the light-emitting elements 13b are alternately arranged at equal intervals at a position about a predetermined distance D1 away from the center of the opening 12 to form an inner ring, and are alternately arranged at equal intervals at a position adjacent to the inner ring about a predetermined distance D2 (>D1) away from the center of the opening 12 to form an outer ring. In this way, the light-emitting elements 13a and the light-emitting elements 13b are arranged so that their positions relative to the opening 12 are the same as a whole, specifically, so that their distances from the opening 12 are equal. Here, "arrangement at the same position as a whole" means that the entirety of the light-emitting elements 13a and the entirety of the light-emitting elements 13b are arranged so that the illumination light is irradiated from the same irradiation position (the same ring-shaped position in this embodiment) to the eye (retina) of the subject A. Here, "equal distance as a whole" means that the light-emitting elements 13a of the multiple groups obtained by dividing the multiple light-emitting elements 13a and the light-emitting elements 13b of the multiple groups obtained by dividing the multiple light-emitting elements 13b are the same distance from the center of the opening 12 between the corresponding groups. The plurality of light-emitting elements 13c are arranged in a ring shape at equal intervals at a position about a predetermined distance D3, which is greater than the predetermined distances D1 and D2, from the center of the opening 12. Each of the plurality of light-emitting elements 13a, 13b, and 13c is provided so as to emit illumination light along the optical axis of the camera 10.In this way, each of the multiple light-emitting elements 13a, 13b, and 13c is evenly arranged around the opening 12, so that the intensity (amount of light) of the illumination light irradiated by each of the multiple light-emitting elements 13a, 13b, and 13c is uniform across the entire pupil of the subject A.

[0031] The above-mentioned multiple light-emitting elements 13a and 13b are light sources for illuminating the eye (pupil) of subject A with illumination light to obtain a bright pupil image. A bright pupil image refers to an image in which the pupil of subject A appears relatively brighter than a dark pupil image described below. The multiple light-emitting elements 13a and 13b are disposed at positions relatively close to the center of the opening 12. As a result, in the image of subject A illuminated by the multiple light-emitting elements 13a and 13b, the pupil appears brighter, making it easier to detect even a small pupil.

[0032] The light-emitting element 13c is a light source for illuminating the eye (pupil) of the subject A with illumination light for obtaining a dark pupil image. A dark pupil image refers to an image in which the pupil of the observed subject appears relatively dark compared to the above-mentioned bright pupil image. The multiple light-emitting elements 13c are made of multiple LEDs in the near-infrared region whose output light has a longer central wavelength than that of the light-emitting element 13a, and are arranged at a relatively far distance from the center of the opening 12. As a result, the pupil appears relatively dark in the image of the subject A illuminated by the multiple light-emitting elements 13c.

[0033] When illumination light is emitted from any one of the plurality of light-emitting elements 13a and 13b to the eyeball of the subject A and the pupil illuminated by the illumination light is imaged by the camera 10, a bright pupil image is acquired. When illumination light is emitted from the plurality of light-emitting elements 13c to the eyeball of the subject A and the pupil illuminated by the illumination light is imaged by the camera 10, a dark pupil image is acquired. This is due to the following property. That is, when illumination light to the eyeball is incident from a position relatively distant from the optical axis of the camera 10, illumination light that enters the pupil of the eyeball, is reflected inside the eyeball, and passes through the pupil again is unlikely to reach the camera 10, so that the pupil appears relatively dark.

[0034] The arrangement of the light sources 13 is not limited to the configuration shown in Fig. 2. For example, when the light amount of the light-emitting elements 13a and 13b is insufficient, the light-emitting elements 13a and 13b may be arranged in a ring shape of three or more layers, and the light-emitting element 13c may be arranged in a ring shape of two or more layers. In addition, the computing device 20 may be controlled via the light source controller 40 to control only some of the light-emitting elements of the light-emitting elements 13a, 13b, and 13c to emit light according to the luminance of the pupil of the subject A in the eye image.

[0035] Here, the light-emitting elements 13a and 13b are made of light-emitting elements with the same directivity. As a result, even if the direction of the pupil of the subject A as seen from the camera 10 changes, the light amount ratio of the light entering the pupil between the light-emitting elements 13a and the light-emitting elements 13b is stabilized. Preferably, the light-emitting elements 13a and 13b are made of light-emitting elements with weak directivity. As a result, even if the direction of the pupil as seen from the camera 10 changes within the range captured by the camera 10, the light amount of the light irradiated to the pupil from each of the light-emitting elements 13a and the light-emitting elements 13b can be stabilized. In addition, it is also preferable that the light-emitting elements 13a and the light-emitting elements 13b have the same outer shape. In this case, it is easy to design the light-emitting elements 13a and 13b so that the distances from the opening 12 are the same. In addition, a bandpass filter for narrowing the band of the output light may be provided on the front surface of each of the light-emitting elements 13a, 13b, and 13c. It is preferable to use super luminescent diodes having narrow-band emission wavelengths as the light-emitting elements 13a, 13b, and 13c. In this case, a bandpass filter for narrowing the band of output light may be omitted.

[0036] The light source controller 40 controls the left camera 10 L and right camera 10 RThe light source controller 40 is a circuit that controls the irradiation of illumination light from the plurality of light-emitting elements 13a, 13b, and 13c in each of the light sources 13. The light source controller 40 is electrically connected to the calculation device 20, and periodically receives a trigger signal from the calculation device 20 to control the timing of irradiation of illumination light from the plurality of light-emitting elements 13a, 13b, and 13c in response to the trigger signal. The light source controller 40 is also configured to be able to adjust the intensity of illumination light from the plurality of light-emitting elements 13a, 13b, and 13c. In detail, the light source controller 40 pre-sets the intensity of illumination light emitted from the plurality of light-emitting elements 13a and the plurality of light-emitting elements 13c so that the background other than the pupil has the same brightness (same luminance on the image) when irradiated to the subject A, and pre-sets the intensity of illumination light emitted from the plurality of light-emitting elements 13b so that the brightness (luminance on the image) of illuminating the entire pupil of the subject A is the same as that of the plurality of light-emitting elements 13a. In other words, by arranging the multiple light-emitting elements 13a, 13b as described above and adjusting the light source controller 40, the intensity of the illumination light irradiated by the multiple light-emitting elements 13a to the entire pupil of the subject A is set to be equivalent to the intensity of the illumination light irradiated by the multiple light-emitting elements 13b to the entire pupil of the subject A.

[0037] The arithmetic device 20 is a computer that controls the camera 10 and the light source 13, and executes arithmetic processing on the eye image of the subject A. The arithmetic device 20 may be constructed of a stationary or portable personal computer (PC), a workstation, or other types of computers. Alternatively, the arithmetic device 20 may be constructed by combining a plurality of computers of any type. When a plurality of computers are used, these computers are connected via a communication network such as the Internet or an intranet.

[0038] 3 shows a typical hardware configuration of the arithmetic device 20. The arithmetic device 20 includes a CPU (processor) 101 that executes an operating system, application programs, etc., a main memory unit 102 consisting of ROM and RAM, an auxiliary memory unit 103 consisting of a hard disk or flash memory, etc., a communication control unit 104 consisting of a network card or a wireless communication module, input devices 105 such as a keyboard or a mouse, and output devices 106 such as a display or a printer.

[0039] Each functional element of the arithmetic unit 20, which will be described later, is realized by loading predetermined software onto the CPU 101 or the main memory unit 102, operating the communication control unit 104, the input device 105, the output device 106, etc. under the control of the CPU 101, and reading and writing data in the main memory unit 102 or the auxiliary memory unit 103. Data and databases necessary for processing are stored in the main memory unit 102 or the auxiliary memory unit 103.

[0040] 4, the arithmetic device 20 includes, as functional components, a timing control unit 21, an image acquisition unit 22, a pupil detection unit 23, an analysis amount calculation unit 24, and a gaze point detection unit 25. There is no limitation on the output destination of the calculation result of the arithmetic device 20. For example, the arithmetic device 20 may display the calculation result on a monitor as an image, a figure, or text, may store the calculation result in a storage device such as a memory or a database, or may transmit the calculation result to another computer system via a communication network. The functions of each component of the arithmetic device 20 will be described in detail below.

[0041] The timing control unit 21 controls the left camera 10 L A plurality of light emitting elements 13a, 13b, 13c, and a right camera 10 R The timing of emitting illumination light from the light-emitting elements 13a and 13c is controlled via the light source controller 40. That is, the timing control unit 21 outputs a trigger signal to the light source controller 40 at the timing when measurement of the subject A is started, and in response to this, the right camera 10 R A plurality of light emitting elements 13a, 13c and a left camera 10L The timing control unit 21 controls the left camera 10 to emit light in a pulsed manner at different timings. L A plurality of light emitting elements 13a, 13b, 13c, and a right camera 10 R The illumination light is controlled so as to be repeatedly emitted from the plurality of light emitting elements 13a and 13c.

[0042] The image acquisition unit 22 is synchronized with the control of the illumination light irradiation timing by the timing control unit 21. L and right camera 10 R The image acquisition unit 22 repeatedly acquires eye images reflecting the subject's eyes illuminated by each illumination light from the left camera 10. L The left camera 10 is synchronized with the illumination timing of the plurality of light emitting elements 13c. L The eye image captured by the left camera 10 L Dark pupil image (third eye image) G LD Get the left camera as 10 L The left camera 10 is synchronized with the illumination timing of the plurality of light-emitting elements 13a. L The eye image captured by the left camera 10 L Bright pupil image (first eye image) G LBA Get the left camera as 10 L The left camera 10 is synchronized with the illumination timing of the plurality of light-emitting elements 13b. L The eye image captured by the left camera 10 L Bright pupil image (second eye image) G LBB The image acquisition unit 22 acquires the image from the right camera 10 as follows. R The right camera 10 is synchronized with the illumination timing of the plurality of light emitting elements 13c. R The eye image captured by the right camera 10 R Dark pupil image G RD Get the right camera as 10 R The right camera 10 is synchronized with the illumination timing of the plurality of light-emitting elements 13a. R The eye image captured by the right camera 10 RBright pupil image G RB In addition, the image acquisition unit 22 acquires the image from the left camera 10 as follows. L The left camera 10 is synchronized with the timing immediately before the irradiation timing of the plurality of light emitting elements 13a and when none of the light sources 13 is irradiating light. L The eye image captured by the left camera 10 L No-lighting image G SA Get the left camera as 10 L The left camera 10 is synchronized with the timing immediately before the irradiation timing of the plurality of light emitting elements 13b and when none of the light sources 13 is irradiating light. L The eye image captured by the left camera 10 L No-lighting image G SB Obtain as.

[0043] FIG. 5 is a timing chart showing the irradiation timing of the light source 13 controlled by the timing control unit 21 and the photographing timing of the eye image acquired by the image acquisition unit 22. Part (a) shows the timing of the trigger signal, and part (b) shows the timing of the right camera 10. R The timing of irradiation of the multiple light emitting elements 13a is shown in FIG. R The timing of irradiation of the plurality of light emitting elements 13c is shown in part (d). The left camera 10 L The timing of irradiation of the plurality of light emitting elements 13c is shown in (e), and the left camera 10 is shown in (c). L The timing of irradiation of the multiple light-emitting elements 13b is shown in (f), and the left camera 10 is shown in (f). L In part (g), the timing of illuminating the right and left light emitting elements 13a is shown, and in part (g), the timing of acquiring an eye image from the camera 10 is shown. In this way, the timing control unit 21 sequentially controls the right and left camera 10 to acquire an eye image at each generation timing of the trigger signal. R The plurality of light-emitting elements 13a and the plurality of light-emitting elements 13c, the left camera 10 L The illumination light is emitted from the plurality of light-emitting elements 13c, the plurality of light-emitting elements 13b, and the plurality of light-emitting elements 13a. In synchronization with the emission timing of these elements, a bright pupil image G RB , Dark pupil image G RD , Dark pupil image G LD , Unilluminated image G SB , bright pupil image GLBB , No-lighting image G SA , and bright pupil image G LBA is acquired. Bright pupil image G LBB and bright pupil image G LBA The unilluminated image G is acquired immediately before each acquisition timing. SB and no-lighting image G SA is the bright pupil image G LBB and bright pupil image G LBA It is used to eliminate the influence of light from the external environment (sunlight, illumination light other than that from the light source 13, etc.) in the image sensor (details will be described later).

[0044] The pupil detection unit 23 calculates the range and position (range in the camera coordinate system) of the pupil area on the eye image by using the eye image periodically acquired by the image acquisition unit 22. That is, the pupil detection unit 23 calculates the range and position of the pupil area on the eye image (range in the camera coordinate system) on the dark pupil image G LD and bright pupil image G LBA More specifically, the pupil detection unit 23 uses the bright pupil image G LBA and dark pupil image G LD The pupil detection unit 23 then calculates the difference in brightness between corresponding pixels in the dark pupil image G to generate a difference image. The pupil detection unit 23 then binarizes the difference image to specify the range and position of the pupil region. LD and bright pupil image G LBB may be used to identify the extent and location of the pupil region.

[0045] The analysis amount calculation unit 24 calculates the biological information of the subject A using the eye images periodically acquired by the image acquisition unit 22. That is, the analysis amount calculation unit 24 calculates the biological information of the subject A using the bright pupil image G LBA The numerical value based on the brightness in the area corresponding to the pupil in the bright pupil image G LBB By calculating the ratio of the brightness in the area corresponding to the pupil in the image to the brightness in the area corresponding to the pupil in the image, an analysis quantity is generated that analyzes the blood oxygen saturation (SO2) near the fundus of subject A.

[0046] First, the analysis amount calculation unit 24 calculates the bright pupil image G LBB From the brightness value of each pixel in the unilluminated image G SB By subtracting the luminance values ​​of the corresponding pixels in the unilluminated image G SB The subtracted bright pupil image G is calculated so that the effect of light reflected by the pupil is removed. LBB The analysis amount calculation unit 24 also obtains the bright pupil image G LBA From the brightness value of each pixel in the unilluminated image G SA By subtracting the luminance values ​​of the corresponding pixels in the unilluminated image G SA The subtracted bright pupil image G is calculated so that the effect of light reflected by the pupil is removed. LBA This allows a bright pupil image to be obtained from which the influence of light from the external environment (sunlight, illumination light other than the light source 13, etc.) has been removed. The analysis amount calculation unit 24 then obtains the subtraction bright pupil image G LBB and subtraction bright pupil image G LBA For example, the analysis amount calculation unit 24 may calculate the subtraction bright pupil image G LBA The part where the brightness value of is higher than a predetermined threshold is identified as the position of the corneal reflection, and a predetermined range including that position (a predetermined circular range centered on the position of the corneal reflection, and a range as narrow as possible so that the corneal reflection is not affected by the pupil brightness) is defined as the subtracted bright pupil image G to be processed. LBA ,G LBB Exclude from.

[0047] Furthermore, the analysis amount calculation unit 24 calculates the subtraction bright pupil image G LBA The average brightness L of the pixels within the pupil region specified by the pupil detection unit 23 is calculated. 810 Calculate the bright pupil image G after subtraction LBB The average brightness of the pixels within the pupil area is calculated by L 940 However, the analysis amount calculation unit 24 calculates the bright pupil image G LBA The average brightness of the pixels within the pupil region specified by the pupil detection unit 23 is calculated, and the unilluminated image G is obtained from the average brightness. SA The average luminance value within the pupil area is subtracted to obtain the average value L 810Similarly, the bright pupil image G LBB The average brightness of the pixels within the pupil region specified by the pupil detection unit 23 is calculated, and the unilluminated image G is obtained from the average brightness. SB The average luminance value within the pupil area is subtracted to obtain the average value L 940 may be calculated.

[0048] Generally, the relationship between the pupil area and pupil luminance in an eye image can be approximated by a linear line. For example, an approximate line f that approximates the relationship between the pupil area x and pupil luminance in an eye image obtained by irradiating the eye with a plurality of light-emitting elements 13a can be expressed as l (x), an approximation line g that approximates the relationship between pupil area x and pupil luminance in an eye image obtained by illuminating a plurality of light-emitting elements 13b l (x) is the following formula (1) or (2); f l (x)=a l x+c l (1), g l (x)=b l x+d l (2) is given by (a l ,b l ,c l ,d 1 is a preset parameter). The analysis amount calculation unit 24 calculates the approximate straight line f by referring to the parameters stored in advance in the calculation device 20. l The value c corresponds to the intercept of (x) l and the approximate line g l The value d corresponds to the intercept of (x) l and the average brightness value L 810 From value c l Subtract the average luminance L 940 to value d l By subtracting and calculating the ratio of them, the analytical quantity B is calculated by the following formula (3). C Then, the analysis amount calculation unit 24 generates the calculated analysis amount B C is output to an output device such as a monitor each time. B C =(L 810 -cl ) / (L 940 -d l ) (3) The analysis amount calculation unit 24 calculates the approximate straight line f l (x),g l (x) parameter (value c l , d l In order to accurately set the pupil area and pupil luminance in advance, the luminance of the image displayed on the display device 30 is changed significantly for a predetermined period (for example, 10 seconds), and the relationship between the pupil area and pupil luminance in the eye images repeatedly acquired during the change is subjected to regression analysis to obtain an approximate straight line f l (x),g l The relationship in (x) may be set and stored.

[0049] Here, the analysis quantity B calculated by the analysis quantity calculation unit 24 C Figure 6 shows the characteristics of oxygenated hemoglobin (HbO 2 ) and reduced hemoglobin (Hb) with respect to the wavelength of incident light. In this way, generally, oxyhemoglobin has a higher absorption coefficient at longer wavelengths than reduced hemoglobin at wavelengths of about 810 nm, and at shorter wavelengths up to about 600 nm, oxyhemoglobin has a higher absorption coefficient. By utilizing this characteristic, when the amount of oxyhemoglobin in the blood vessels near the fundus increases, the light absorbance of oxyhemoglobin is greater than that of reduced hemoglobin in the 940 nm band, resulting in a decrease in pupil luminance in the eye image. In contrast, in the 810 nm wavelength band, the light absorbance of oxyhemoglobin and reduced hemoglobin are equal, so even if the ratio of the amount of oxyhemoglobin to reduced hemoglobin changes, the pupil luminance in the eye image should not change. Due to this property, the pupil luminance L in the 810 nm wavelength band is 810 The pupil luminance L in the 940 nm wavelength band 940 The analysis volume B corresponds to the value divided by C When the blood oxygen saturation (SO2) becomes high, the analysis amount B becomes large. CBy calculating the above, it is possible to derive an evaluation value of the blood oxygen saturation (SO2) in the blood vessels near the fundus of the subject A. Here, in this embodiment, the illumination light in the wavelength band of 940 nm is used instead of the illumination light in the wavelength band around 1000 nm where the difference in light absorbance between oxygenated hemoglobin and reduced hemoglobin is larger, and the bright pupil image G LBB Left camera in 10 L This has the advantages that it is possible to prevent a decrease in the sensitivity of the light emitting element 13a, prevent a shortage of the light emitting power of the light emitting element 13b, and prevent overlapping of the light emitting bands of the light emitting elements 13a and 13b.

[0050] Returning to FIG. 4, the gaze point detection unit 25 detects the gaze point of the right camera 10 acquired by the image acquisition unit 22. R Bright pupil image G RB and dark pupil image G RD and left camera 10 L Bright pupil image G LBA and dark pupil image G LD Based on this, the three-dimensional position, gaze direction, and gaze point of the pupil of the subject A are periodically detected. The method of calculating the three-dimensional position, gaze direction, and gaze point of the pupil may be a method developed by the present inventors (see International Publication WO2012 / 020760). The gaze point detection unit 25 uses the three-dimensional position, gaze direction, and gaze point of the pupil, which are continuously detected during the analysis processing of the biological information of the subject A, to perform a process of stabilizing the pupil luminance detected by the analysis amount calculation unit 24, and a process of guiding the subject A so that they are stabilized during the analysis processing. For example, such a process may include a process of displaying a symbol, character, optotype, or the like on the display device 30 for guiding the position of the pupil or the gaze point to a predetermined position, a process of instructing the movement of the pupil position or the gaze point by voice output, and the like. The analysis amount B calculated by the analysis amount calculation unit 24 C The eyeball of subject A is the left camera 10 LThis is because the illumination light from the light source 13 is irradiated onto a different area (position) of the fundus of the subject A. The gaze point detection unit 25 stabilizes the three-dimensional position of the pupil of the subject A, the gaze direction, and the gaze point, thereby obtaining a highly sensitive and highly accurate analysis quantity B C can be obtained. At this time, the analysis amount calculation unit 24 may perform a correction process of the pupil luminance as a process for stabilizing the detected pupil luminance by using the continuously detected three-dimensional position of the pupil, the gaze direction, or the gaze point. If the three-dimensional position of the pupil, the gaze direction, or the gaze point changes due to a change in the distance of the head of the subject A or a change in the direction of the face such as turning the face to the side or up, the irradiation state of the light from the light source changes, and the detected pupil luminance becomes unstable. The above correction process by the analysis amount calculation unit 24 prevents such problems and makes it possible to detect a stable pupil luminance.

[0051] The effects of the biological information analysis device 1 according to the embodiment of the present disclosure will be described.

[0052] According to the above-described biological information analysis device 1, a bright pupil image G reflecting the pupil of the subject A illuminated by the plurality of light emitting elements 13a is generated. LBA And left camera 10 L A bright pupil image G reflecting the pupil of the subject A illuminated by the plurality of light-emitting elements 13b from the same angle direction as the plurality of light-emitting elements 13a based on the optical axis of the light-emitting element 13b. LBB and a bright pupil image G LBA The average luminance L in the area corresponding to the pupil 810 and bright pupil image G LBB The average luminance L in the area corresponding to the pupil 940 Based on this, the amount of analysis B regarding the biological information of subject A Cis calculated and output. As a result, the subject A can obtain biometric information simply by facing the camera 10, and it becomes easy to continuously monitor changes in the biometric information. In the conventional device, it was necessary to measure the biometric information while the device was close to the subject A or while the subject A was wearing the device. According to this embodiment, it is possible to obtain biometric information from a distance from the subject A, and it is possible to easily analyze the biometric information without having the subject A approach the device. In the conventional device, it was necessary to measure the biometric information while the subject's eyes were close to the device or while the subject was wearing the device, but in this embodiment, it is possible to analyze the biometric information by macroscopically photographing the pupils of the subject from a distance (for example, from a position 20 cm or more away) without having the subject approach the device. In particular, in this embodiment, analysis is possible even when the camera 10 is not focused on the subject's pupils.

[0053] Here, the left camera 10 of the biometric information analysis device 1 L The multiple light emitting elements 13a, 13b provided in the are arranged so that they are equidistant from the opening 12 as a whole. Basically, the luminance of the pupil illuminated by the illumination light depends on the angle from the center of the opening 12 to the light source 13 when viewed from the pupil. If the multiple light emitting elements 13a, 13b are set at different distances from the opening 12, the positions on the fundus to which illumination light of different wavelengths is irradiated will differ, and therefore the luminance of the pupil illuminated by illumination light of different wavelengths will vary due to changes in pupil size or crystalline lens even if the light reflectance of the fundus is constant. In this embodiment, such fluctuations in pupil luminance can be prevented, making it possible to monitor the evaluation value related to blood oxygen saturation with high accuracy.

[0054] Also, the left camera 10 LThe light emitting elements 13a and 13b provided in the light emitting element 13 are arranged at equal intervals in a ring shape. The light emitting elements 13a and 13b, which are LEDs, may vary in the position of the light emitting part when the element body is viewed from the light emitting side due to manufacturing variations. In this embodiment, even if such manufacturing variations exist, the light emitting elements 13a and 13b are arranged alternately in a direction rotated by a predetermined angle at equal intervals, so that the distance from the opening 12 can be averaged, and the distance from the opening 12 can be set to be equal between the two types of light emitting elements.

[0055] The light source controller 40 controls the plurality of light emitting elements 13a and the plurality of light emitting elements 13b to emit light alternately at different timings. The calculation device 20 controls the left camera 10. L , and bright pupil image G, which is an eye image captured at a different time. LBA and bright pupil image G LBB In this case, a bright pupil image G reflecting the pupil of the subject illuminated by illumination light with a central wavelength of 810 nm is obtained. LBA and a bright pupil image G reflecting the subject's pupil illuminated by illumination light with a central wavelength of 940 nm. LBB Since the above and the above can be obtained with little noise, highly accurate biological information can be obtained.

[0056] In addition, the calculation device 20 calculates the average luminance L 810 Based on the average brightness L 940 By calculating the ratio of the value based on the above, the analysis amount B regarding the blood oxygen saturation of subject A is obtained. C In this case, the change in the biological information regarding the blood oxygen of the subject A can be continuously monitored.

[0057] In addition, in this embodiment, the central wavelength of the light emitted by the multiple light-emitting elements 13a, 13b, and 13c is set in the near-infrared region. In this way, the subject A does not feel glare during measurement, so that analysis of biological information with less burden on the subject A can be realized. Generally, in devices for observing or diagnosing the eyes of the subject A, visible light and near-infrared light are often used. When using such devices, even if the light source is used with flash emission, the subject A feels glare. In addition, when it is desired to measure the subject A while driving a vehicle, it is difficult to use a device using a visible light source because the light emission is an obstacle to the visual function of the subject A, and the measurement is limited to limited scenes such as during a break. In this embodiment, since measurement can be performed using only near-infrared light, it is possible to measure biological information with less burden on the subject A in various usage scenes such as while driving a vehicle. The emission wavelength of the light-emitting element 13a provided in the biological information analysis device 1 is 810 nm, and the illumination light output by the light-emitting element 13a appears slightly bright to the subject A, but is unlikely to be an obstacle. For example, the biometric analysis device 1 of this embodiment can be realized by a portable device such as a smartphone or tablet device, or can be incorporated into equipment in a vehicle or a bathroom mirror, making it possible to non-contactly monitor the changing trends in the biometric information of subject A in real time and continuously in the course of his or her daily life.

[0058] In addition, the calculation device 20 of this embodiment calculates a non-illumination image G, which is an eye image captured when the light source 13 is not irradiating the illumination light. SA ,G SB and obtain a bright pupil image G LBA No illumination image G SA The image with subtraction and the bright pupil image G LBB No illumination image G SB Based on the image with subtraction, the analysis amount B C When disturbance light such as natural light or indoor lighting is present in the environment in which the biometric analysis device 1 is used, the bright pupil image G LBA Or bright pupil image G LBBThe effect of ambient light is superimposed on the pupil luminance in the image. LBA and bright pupil image G LBB Therefore, the effects of such disturbance light can be eliminated, and highly accurate biological information can be obtained.

[0059] In this embodiment, a plurality of light-emitting elements 13c are further provided, and the light source controller 40 controls the plurality of light-emitting elements 13c to irradiate illumination light at a timing different from that of the plurality of light-emitting elements 13a and 13b. The calculation device 20 calculates a dark pupil image G, which is an eye image captured at that timing. LD and obtain a dark pupil image G LD and bright pupil image G LBA Or bright pupil image G LBB By comparing the pupil area on the eye image with the bright pupil image G, LBA The average luminance L in the pupil area 810 and bright pupil image G LBB The average luminance L in the pupil area 940 Based on this, the analysis amount B C In this case, the analysis quantity B C This makes it possible to stably limit the area for which calculation is performed to the pupil area, thereby making it possible to obtain biometric information with higher accuracy.

[0060] In addition, the calculation device 20 of this embodiment calculates the bright pupil image G LBA The average brightness of the pixels in the pupil area is calculated as the average brightness L 810 and the bright pupil image G LBB The average brightness of the pixels in the pupil area is calculated as the average brightness L 940 According to this configuration, the influence of luminance noise in the pupil can be removed, so that the tendency of changes in biological information can be stably monitored.

[0061] The present inventors performed an actual analysis of the amount B CIn detail, the percutaneous arterial blood oxygen saturation (SpO2) was actually measured for each of the subjects using a pulse oximeter, and the analysis amount B was obtained using the bioinformation analysis device 1 in the high SpO2 section and the low SpO2 section for each of the subjects. C As a result, in most of the measurement results, the analysis amount B was higher in the high SpO2 section than in the low SpO2 section. C was found to show significantly higher values.

[0062] The present invention is not limited to the above-described embodiment, and the configuration of the above-described embodiment can be modified in various ways.

[0063] In the above embodiment, the analysis amount calculation unit 24 of the calculation device 20 calculates the analysis amount B C Various formulas can be used to calculate. For example, the analysis amount calculation unit 24 uses the following formula (4); B C =(L 810 -c l ) / {(L 940 -d l )+(L 810 -c l )} (4) Analysis volume B C The blood oxygen saturation of subject A can also be evaluated using such a formula.

[0064] In addition, in the bioinformation analyzer 1 according to the embodiment described above, light emitting elements having output light with central wavelengths of 810 nm and 940 nm in the near infrared region are used as the light source 13, but light emitting elements with other emission wavelengths may be used as long as the central wavelength is 740 nm or more and 1,000 nm or less in the near infrared region. For example, the light emitting element 13b on the long wavelength side may be one having output light with a central wavelength of 870 nm, 890 nm, or 910 nm. Also, the light emitting element 13a on the short wavelength side may be one having a central wavelength of 780 nm, 760 nm, 740 nm, or the like, and equipped with a bandpass filter for narrowing the output light. In this case, the light emitting element 13b on the long wavelength side may be one having a central wavelength of 870 nm, 890 nm, 910 nm, or 940 nm, and equipped with a bandpass filter for narrowing the output light. Such light emitting elements 13a and 13b can also be used as a light source invisible to the subject A. In addition, as the short-wavelength light-emitting element 13a and the long-wavelength light-emitting element 13b, light-emitting elements with center wavelengths whose absorption coefficients change in opposite directions when the blood oxygen saturation (SO2) changes are used. C The fluctuation of analysis volume B becomes large. C The detection sensitivity of the eye image by the camera 10 is increased. In general, the power of the output light of the light-emitting element becomes smaller as the wavelength becomes longer, and the sensitivity of the camera 10 to light of long wavelengths also becomes lower. By providing a bandpass filter to the light-emitting element 13b on the long wavelength side and shortening the wavelength of the output light of the light-emitting element 13b to, for example, 850 nm, the power of the output light of the light-emitting element 13b can be increased, and the detection sensitivity of the eye image by the camera 10 can also be increased. As a result, monitoring of biological information can be achieved with a small number of light-emitting elements 13b.

[0065] Furthermore, the bioinformation analyzer 1 according to the embodiment described above may be provided with a light source 13 other than that shown in Fig. 2. For example, the light emitting elements 13a and the light emitting elements 13b may be provided at the same position overall, and the number of the light emitting elements 13a and the number of the light emitting elements 13b may be different. For example, a configuration in which one light emitting element 13a is sandwiched between two light emitting elements 13b is repeated may be used. Also, a configuration in which a plurality of light emitting elements are arranged, each of which is equipped with a plurality of LED chips that generate light of different wavelengths.

[0066] In addition, in the pair of cameras 10 of this embodiment, a filter for cutting light of unnecessary wavelength bands may be provided in front of the objective lens 11. By providing such a filter, it is possible to prevent saturation of brightness in the eye image acquired by the camera 10 due to disturbance light, and stable measurement of biological information is realized.

[0067] In addition, the analysis amount calculation unit 24 of the calculation device 20 of this embodiment calculates the average value L 810 ,L 940 Based on the analysis amount B C was calculated, but two bright pupil images G LBA ,G LBB For each pixel in the pupil region, the brightness L 810 ,L 940 and the brightness L for each pixel in the pupil region is obtained. 810 ,L 940 Based on this, the analytical quantity B is calculated using the above formula (3). C Calculate the analysis quantity B between the pixels in the pupil region. C Average the average analysis amount B C may be output as the final result. In such a case, by calculating (dividing) the luminance ratio for each pixel, the luminance ratio value of the pupil region can be emphasized compared to the values ​​of regions other than the pupil region. By calculating the luminance ratio as the analysis amount, the analysis amount can be made to easily follow changes in biological information (especially changes in blood oxygen saturation) compared to calculating the luminance difference. Note that the luminance L for each pixel in the pupil region is 810 ,L 940 Based on the analysis amount BC When calculating, the analysis amount calculation unit 24 calculates the luminance L 810 ,L 940 Detect outliers by creating a histogram for the target, and exclude pixels with outliers as pixels corresponding to the position of the corneal reflection to obtain the analysis quantity B C The average value of may be calculated.

[0068] In addition, the analysis amount calculation unit 24 calculates the average value L 810 ,L 940 Based on the analysis amount B C Other functions may be used as the approximation function taken into consideration when calculating. That is, in the above embodiment, the relationship between the pupil area x and pupil luminance in the eye image is approximated by a linear function, but it may also be approximated by a curve defined by a quadratic function, exponential function, or the like. More specifically, the relationship between the pupil area x and pupil luminance in the eye image obtained by the multiple light-emitting elements 13a and the relationship between the pupil area x and pupil luminance in the eye image obtained by the multiple light-emitting elements 13b are approximated by a quadratic approximation function f shown in the following expressions (5) and (6): q (x),g q It can be approximated by (x). f q (x)=a q x 2 +c q x+e q (5) g q (x)=b q x 2 +d q x+h q (6) In addition, the relationship between the pupil area x and the pupil luminance in the eye image obtained by the plurality of light-emitting elements 13a and the relationship between the pupil area x and the pupil luminance in the eye image obtained by the plurality of light-emitting elements 13b are expressed by the following approximate exponential functions f e (x),g e It can also be approximated by (x). f e (x)=a e exp(c e x) (7) g e(x)=b e exp(d e x) (8) The analysis amount calculation unit 24 then refers to parameters stored in advance to define these relationships, and calculates the average value L 810 ,L 940 Each of these is calculated from the area x of the pupil region detected by the pupil detection unit 23 by an approximation function f q (x),g q Normalization is performed according to (x) using the following equations (9) and (10). l 810 =L 810 / f q (x) (9) l 940 =L 940 / g q (x) (10) Finally, the analysis amount calculation unit 24 calculates the average value l 810 The average value l 940 Analyze quantity B by dividing by C Even with this modification, the analysis quantity B C This reduces the effect of changes in pupil area during observation, making it possible to monitor trends in changes in biological information with high accuracy.

[0069] As described above, the analysis amount calculation unit 24 may set and store in advance the approximation functions shown in the above formulas (5) to (8) by performing regression analysis on the relationship between the pupil area and pupil luminance in the eye image. Also, the analysis amount calculation unit 24 may set and store in advance the approximation functions shown in the above formulas (5) to (8) by performing regression analysis on the relationship between the pupil area and pupil luminance in the eye image. 810 / L 940 The relationship between the two is approximated by one approximation function, and the luminance ratio L obtained for each image frame is 810 / L 940 is normalized by the value calculated by the approximation function, and the normalized luminance ratio L 810 / L 940 Based on the analysis amount B C It is also possible to calculate

[0070] The pair of cameras 10 included in the biological information analysis device 1 may be modified to have the following modified configuration of a camera 110.

[0071] FIG. 7 is a diagram showing a schematic configuration of a camera 110 according to this modified example. The camera 110 has a housing 115, an imaging element (image sensor) 116 such as a CCD or CMOS housed in the housing 115, and an objective lens 11 and an optical element 119 housed in the housing 115. The housing 115 has a circular opening 12 formed on a surface facing the eyeball of the subject A. The objective lens 11 and the optical element 119 are disposed between the opening 12 and the imaging element 116 inside the housing 115. The optical axis L0 of the objective lens 11 coincides with the central axis of the opening 12. The imaging element 116 is fixed so that its light receiving surface intersects perpendicularly with the optical axis L0 of the objective lens 11. The optical element 119 is disposed on the optical axis L0 of the objective lens 11 inside the objective lens 11 (on the imaging element 116 side). The image sensor 116 captures an image of the eyeball of the subject A to generate eye image data and output the data to the calculation device 20.

[0072] The diameter of the opening 12 is smaller than the diameter of the objective lens 11 and is approximately the same as the effective diameter of the objective lens 11. Moreover, the optical element 119 has an approximately circular shape as a whole and has a diameter approximately the same as the effective diameter of the objective lens 11. With this configuration, an image of the vicinity of the eyeball of the subject A is introduced toward the objective lens 11 and the image sensor 116 in the camera 110 through the opening 12, and then is imaged so as to converge on the light receiving surface of the image sensor 116 by the optical system including the objective lens 11 and the optical element 119 in the camera 110.

[0073] FIG. 8 shows the structure of the optical element 119 arranged inside the opening 12. The optical element 119 has a disk-like structure and is arranged so that the optical axis L0 of the objective lens 11 passes through the vicinity of its center and is approximately perpendicular to the optical axis L0. This optical element 119 is composed of semicircular plate-like splitting elements 119A and 119B split into two at the center. The splitting element 119A is an optical element in which a bandpass filter that passes a first illumination light having a central wavelength (first central wavelength) of the output light of the light-emitting element 13a and a first polarizer that transmits linearly polarized light having a polarization direction of 0 degrees based on the vertical direction of the opening 12 are combined in two layers. The splitting element 119B is an optical element in which a bandpass filter that passes a second illumination light having a central wavelength (second central wavelength) of the output light of the light-emitting element 13b and a second polarizer that transmits linearly polarized light having a polarization direction of 90 degrees based on the vertical direction are combined in two layers. That is, the polarization direction of the linearly polarized light transmitted through the first polarizer is approximately perpendicular to the polarization direction of the linearly polarized light transmitted through the second polarizer. In the optical element 119 having such a structure, the splitting elements 119A and 119B are arranged on both sides of a central axis (line) that passes horizontally through the center of the opening 12 along the opening surface of the opening 12, more specifically, are configured to be linearly symmetrical with respect to the central axis. Note that the shape of the optical element 119 combining the splitting elements 119A and 119B is not limited to a disk shape, and may be another shape such as a rectangle as long as it corresponds to the shape of the edge of the opening 12.

[0074] On the other hand, the optical element 119 may have a structure as shown in FIG. 9. The optical element 119 shown in FIG. 9 is composed of splitting elements 119A and 119B having a continuous shape corresponding to the shape of the edge of the opening 12, and these splitting elements 119A and 119B have a shape divided into two by a boundary line along the edge of the opening 12, and the splitting element 119B is disposed inside the splitting element 119A. That is, the splitting element 119B has a disk-like shape (circular shape) with a center located near the optical axis L0, and the splitting element 119A has a ring-like shape located outside the splitting element 119B. The optical element 119 having such a structure is configured such that the splitting elements 119A and 119B are point-symmetric with respect to the center of the opening 12, similar to the light source 13. Note that the shape of the optical element 119 combining the splitting elements 119A and 119B is not limited to a disk-like shape, and may have other shapes such as a rectangular shape as long as it corresponds to the shape of the edge of the opening 12.

[0075] FIG. 10 shows the structure of the image sensor 116 arranged inside the opening 12, as viewed from the opening 12 side. The image sensor 116 has a light receiving surface 120 arranged to be approximately perpendicular to the optical axis L0, a plurality of pixels 121 with light receiving elements (not shown) formed therein and arranged in a two-dimensional array on the light receiving surface 120, and four types of polarizers 122A, 122B, 122C, and 122D arranged on the surfaces of the plurality of pixels 121 so as to cover the pixels 121. That is, different types of polarizers 122A, 122B, 122C, and 122D are attached to the surfaces of four adjacent pixels 121 (hereinafter also referred to as a four-pixel group). The polarizer 122A is an optical element that transmits linearly polarized light with a polarization direction of 0 degrees relative to the vertical direction of the opening 12. The polarizer 122B is an optical element that transmits linearly polarized light with a polarization direction rotated 45 degrees counterclockwise relative to the vertical direction of the opening 12. The polarizer 122C is an optical element that transmits linearly polarized light having a polarization direction rotated 90 degrees counterclockwise with respect to the up-down direction of the opening 12. The polarizer 122D is an optical element that transmits linearly polarized light having a polarization direction rotated 135 degrees counterclockwise with respect to the up-down direction of the opening 12. The imaging element 116 has a structure in which such structures are repeated two-dimensionally (in the left-right and up-down directions in FIG. 10).

[0076] The range of the four pixel groups on the light receiving surface 120 of the image sensor 116 is sufficiently small compared with the size of the objective lens 11. Therefore, the optical element 119 and the image sensor 116 are configured so that the first illumination light passing through the dividing element 119A of the optical element 119 is incident on the four pixels constituting the four pixel groups with the same intensity, and the second illumination light passing through the dividing element 119B of the optical element 119 is incident on the four pixels constituting the four pixel groups with the same intensity.

[0077] In this modification, the light source controller 40 controls the multiple light emitting elements 13a and the multiple light emitting elements 13b to emit light at the same timing within a frame period that is determined by a trigger signal and is repeated at 30 Hz or 60 Hz. The image acquisition unit 22 of the calculation device 20 then uses the following principle to obtain a bright pupil image G LBA ,G LBB, and unilluminated image G SA ,G SB is obtained by calculation.

[0078] When an eye image is acquired, the light transmitted through the first dividing element 119A and the second dividing element 119B is incident with the same light intensity on four pixel groups each consisting of four adjacent pixels 121. The light intensity of each incident light is defined as I(λ a1 ),I(λ a2 ), the light intensities O detected by the pixels 121 to which the polarizers 122A, 122B, 122C, and 122D are attached among the four pixel groups are expressed as follows: i (i=1 to 4) is expressed by the following formula.

number

[0079] The above formula can be expressed as a determinant to obtain the following formula:

number

number

number

[0080] Using the above-mentioned formula, the image acquisition unit 22 calculates the luminance value [O i ] and the brightness value [O i ] and the preset pseudo-inverse matrix [M ij ] + Using the above, the luminance value I(λ) corresponding to the light of the first central wavelength incident on the four pixel groups is calculated. a1 ) and the luminance value I(λ a2 Then, the image acquisition unit 22 calculates the luminance value I(λ) calculated for each of the four pixel groups on the light receiving surface 120. a1 ) to generate two-dimensional image data, and a bright pupil image G LBA Then, the brightness value I(λ a2 ) to generate two-dimensional image data, and a bright pupil image G LBB Similarly, the image acquisition unit 22 acquires the luminance value I(λ) calculated for each of the four pixel groups based on the eye image captured at a timing immediately before the timing when the light emitting elements 13a and the light emitting elements 13b are irradiated and at a timing when the light source 13 is not irradiating light. a1 ) to generate two-dimensional image data. SA Then, the brightness value I(λ a2 ) to generate two-dimensional image data. SB In addition, the image acquisition unit 22 calculates a dark pupil image G based on the eye image captured at the timing when only the light emitting elements 13c are irradiated. LD can be obtained.

[0081] In this modification, the analysis amount calculation unit 24 of the arithmetic device 20 calculates the bright pupil image G LBA No illumination image G SA The light reflected from the pupil is subtracted from the pupil image G to remove the effect of the light reflected from the pupil image G. LBB No illumination image GSB The light reflected by the pupil is subtracted from the pupil image G. LBA ,G LBB Using the analysis quantity B C As a result, even if disturbance light such as natural light or illumination light is incident on the measurement environment, the effects of the disturbance light can be removed from the second eye image, making it possible to obtain highly accurate biological information.

[0082] In the above modification, the image sensor 116 has a structure in which four types of polarizers are provided for every four pixel groups, but it is sufficient that the image sensor 116 has a structure in which at least two types of polarizers are provided. For example, two types of polarizers may be provided for every two adjacent pixels 121, or three types of polarizers may be provided for every three adjacent pixels 121. Even in such a modification, the calculation device 20 can calculate the luminance value of the light component of the first central wavelength and the luminance value of the light component of the second central wavelength based on the pixel values ​​of two adjacent pixels or three adjacent pixels.

[0083] Furthermore, in the above modified example, the calculation device 20 calculates the luminance value of the light component having the first central wavelength and the luminance value of the light component having the second central wavelength based on the pixel values ​​of four adjacent pixels, but it is also possible to calculate the luminance value of the light component having the first central wavelength and the luminance value of the light component having the second central wavelength based on the pixel values ​​of at least two pixels (two pixels, or three pixels) among the four adjacent pixels.

[0084] In addition, the optical element 119 according to the above modification may further include a splitting element 119C, which is a combination of a bandpass filter that passes the central wavelength of the light irradiated by the light emitting element 13c and a third polarizer that transmits linearly polarized light at an angle different from that of the first polarizer and the second polarizer, and is provided inside the lens of the opening 12, by dividing the splitting element 119C together with the splitting elements 119A and 119B. In this case, the light source controller 40 controls the multiple light emitting elements 13c having output light with a central wavelength different from that of the light emitting elements 13a and 13b to irradiate light simultaneously with the multiple light emitting elements 13a and 13b within a frame period defined by the trigger signal. In addition, the image acquisition unit 22 of the calculation device 20 calculates the luminance corresponding to the light irradiated by the multiple light emitting elements 13c based on the luminance of adjacent pixels in the acquired eye image, and combines the calculated luminances to obtain a dark pupil image G. LD The luminance corresponding to the light emitted by the plurality of light-emitting elements 13a is calculated, and the calculated luminance is combined to obtain a bright pupil image G LBA The luminance corresponding to the light emitted by the plurality of light-emitting elements 13b is calculated, and the calculated luminance is combined to obtain a bright pupil image G LBB can be obtained.

[0085] The analysis amount calculation unit 24 of the arithmetic device 20 according to this modified example also calculates the dark pupil image G LD and bright pupil image G LBA Or bright pupil image G LBB By comparing the pupil area on the eye image with the bright pupil image G, LBA The luminance in the pupil region in the bright pupil image G LBB Based on the brightness in the pupil area in C With this configuration, the analytical quantity B C The region to be calculated can be stably limited to the pupil region, and more accurate biological information can be obtained. In the above modification, too, no-illumination images of three types of wavelength bands are calculated based on the eye image obtained immediately before the illumination timing of the light-emitting elements 13a, 13b, and 13b, and a dark pupil image G is obtained. LD and bright pupil image G LBA ,GLBB Alternatively, the unilluminated images of the corresponding wavelength bands may be subtracted from the corresponding images. [Explanation of symbols]

[0086] 1...Biometric analysis device, 10, 10 L , 110...camera, 12...opening, 13...light source, 13a...light-emitting element (first light source), 13b...light-emitting element (second light source), 13c...light-emitting element (third light source), 20...arithmetic unit, 40...light source controller, 116...imaging element (image sensor), 119...optical element, 119A, 119B, 119C...splitting element, 121...pixel, 122A, 122B, 122C, 122D...polarizer, A...subject, D1, D2, D3...predetermined distance, G LBA …bright pupil image (first eye image), G LBB …bright pupil image (second eye image), G LD … dark pupil image (third eye image), G SA ,G SB ...Unlit image. .

Claims

1. a camera that captures an image of a subject's eye to obtain an eye image; a first light source provided outside an opening of the camera and configured to irradiate light having a first central wavelength toward a pupil of the subject; a second light source that is provided at the same position as the first light source with respect to the opening of the camera and that irradiates light having a second central wavelength different from the first central wavelength toward the pupil of the subject; a light source controller that controls the irradiation of light by the first light source and the second light source; a computing device for processing the eye image, The computing device includes: obtaining a first eye image from the camera, the first eye image reflecting the eye illuminated by the first light source, and a second eye image reflecting the eye illuminated by the second light source; calculating an analysis amount related to biological information of the subject based on a first luminance in a region corresponding to the pupil in the first eye image and a second luminance in a region corresponding to the pupil in the second eye image, and outputting the analysis amount; Biometric analysis device.

2. The first light source and the second light source are evenly arranged along the same ring shape centered on the opening of the camera.

2. The biological information analysis device according to claim 1.

3. the light source controller controls the first light source and the second light source to alternately irradiate light at a first timing and a second timing that are separate timings; The computing device includes: acquiring, from the camera, a first eye image which is the eye image captured at the first timing, and a second eye image which is the eye image captured at the second timing; The biological information analysis device according to claim 1 .

4. The camera includes: an optical element in which a first dividing element is combined with a bandpass filter that passes light of the first central wavelength and a first polarizer that transmits linearly polarized light at a predetermined angle, and a second dividing element is combined with a bandpass filter that passes light of the second central wavelength and a second polarizer that transmits linearly polarized light at an angle different from the predetermined angle, are provided inside the lens of the opening; An image sensor in which a polarizer that transmits linearly polarized light having at least two different angles is attached to each of adjacent pixels; having the light source controller controls the first light source and the second light source to emit light simultaneously; The computing device includes: calculating a luminance corresponding to light of the first central wavelength based on the luminance of the adjacent pixels in the eye image, and combining the calculated luminances to obtain the first eye image; calculating a luminance corresponding to light of the second central wavelength based on the luminance of the adjacent pixels in the eye image, and combining the calculated luminances to obtain the second eye image. The biological information analysis device according to claim 1 .

5. The arithmetic device calculates the analysis amount related to blood oxygen in the subject by calculating a ratio between a numerical value based on the first luminance and a numerical value based on the second luminance. The biological information analysis device according to any one of claims 1 to 4.

6. The first central wavelength and the second central wavelength are in the near infrared region of 740 nm or more and 1000 nm or less. The biological information analysis device according to any one of claims 1 to 5.

7. the arithmetic device acquires an unilluminated image, which is the eye image captured at a third timing when the first light source and the second light source are not irradiating light; calculating the analysis quantity based on an image calculated using the first eye image and the no-illumination image and an image calculated using the second eye image and the no-illumination image; 4. The biological information analysis device according to claim 3.

8. The arithmetic device calculates the analysis amount based on an image obtained by excluding a range of the corneal reflection image of the subject from the first eye image and an image obtained by excluding a range of the corneal reflection image of the subject from the second eye image. The biological information analysis device according to any one of claims 1 to 7.

9. Further comprising a third light source that is provided at a greater distance from the opening of the camera than the first light source and the second light source and irradiates light in the near infrared region toward the pupil of the subject, the light source controller controls the third light source to emit light at a fourth timing shifted from the first timing and the second timing; the arithmetic device acquires a third eye image, which is the eye image captured at the fourth timing; comparing the third eye image to the first eye image or the second eye image to identify an area of ​​the pupil on the eye image; calculating the analysis quantity based on a first luminance in a region of the pupil of the first eye image and a second luminance in a region of the pupil of the second eye image; The biological information analysis device according to claim 3 or 7.

10. the computing device acquires a non-illumination image, which is the eye image captured at a timing when the first light source and the second light source are not irradiating light; calculating a luminance corresponding to light of the first central wavelength based on the luminance of the adjacent pixels in the unilluminated image, and combining the calculated luminances to obtain a first unilluminated image; calculating a luminance corresponding to light of the second central wavelength based on the luminance of the adjacent pixels in the unilluminated image, and combining the calculated luminances to obtain a second unilluminated image; calculating the analysis quantity based on an image calculated using the first eye image and the first no-illumination image and an image calculated using the second eye image and the second no-illumination image; 5. The biological information analysis device according to claim 4.

11. Further comprising a third light source that is provided at a greater distance from the opening of the camera than the first light source and the second light source and irradiates light in the near infrared region toward the pupil of the subject, The optical element further includes a third dividing element, which is a combination of a bandpass filter that passes a central wavelength of the light irradiated by the third light source and a third polarizer that transmits linearly polarized light at an angle different from that of the first polarizer and the second polarizer, and is divided and provided inside the lens of the opening, the light source controller controls the third light source to emit light simultaneously with the first light source and the second light source; The computing device includes: calculating a luminance corresponding to the light emitted by the third light source based on the luminance of the adjacent pixels in the eye image, and combining the calculated luminances to obtain a third eye image; comparing the third eye image to the first eye image or the second eye image to identify an area of ​​the pupil on the eye image; calculating the analysis quantity based on a first luminance in a region of the pupil of the first eye image and a second luminance in a region of the pupil of the second eye image; The biological information analysis device according to claim 4 or 10.

12. The computing device includes: calculating an average value of luminance of pixels in a region of the pupil in the first eye image as the first luminance, and calculating an average value of luminance of pixels in a region of the pupil in the second eye image as the second luminance. The biological information analysis device according to claim 9 or 11.

13. The computing device includes: calculating the analysis amount for each pixel based on the luminance of a pixel in the pupil region in the first eye image and the luminance of a pixel in the pupil region in the second eye image, and averaging and outputting the analysis amount between pixels in the pupil region; The biological information analysis device according to claim 9 or 11.

14. The computing device includes: Calculating a pupil position or a gaze direction of the subject based on the first eye image or the second eye image and the third eye image, and performing a stabilization process of the pupil luminance detected when the analysis amount is acquired. The biological information analysis device according to claim 9 or 11.

15. The pupil luminance stabilization process is a process of guiding the pupil position or the gaze direction to a predetermined position or a predetermined direction. The biological information analysis device according to claim 14.

Images