Blood flow analysis device, biological information analysis system

Abstract

The present invention provides a blood flow analysis device and a biological information analysis system capable of rapidly estimating a person's condition and obtaining various information without placing a burden on the subject. The blood flow analysis device uses visible light images to detect blood flow distribution, uses this blood flow distribution to compare blood flow between different measurement areas, and analyzes correlations between blood flows occurring simultaneously in different measurement areas.

Description

Technical Field

[0001] The present invention relates to a device for analyzing blood flow distribution of a subject. Background Art

[0002] There is a technology that extracts subtle changes in blood color from a series of images taken of human tissue to detect changes in blood flow. This technology is achieved by detecting the optical properties of the organism that change due to blood flow from the captured images.

[0003] As an example of the above-mentioned technology, there is Patent Document 1. Patent Document 1 takes "blood pressure measurement by non-contact method" as its subject and describes the following technology (see abstract): "It comprises: an image acquisition unit that acquires a skin image of a user's skin; a pulse wave timing calculation unit that uses the skin image to calculate the time change of brightness in the skin image, and calculates time information indicating the time when the brightness reaches a peak in the time change of brightness as pulse wave timing; a millimeter wave acquisition unit that acquires a signal of an electric wave reflected by the user and received by a receiving antenna; a heartbeat timing calculation unit that uses the signal of the electric wave acquired by the millimeter wave acquisition unit to calculate the time change of the distance between the user and the receiving antenna, and calculates time information indicating the time when the distance reaches a peak in the time change of the distance as heartbeat timing; and a blood pressure determination unit that determines the user's blood pressure based on the time difference between the pulse wave timing and the heartbeat timing."

[0004] Current non-contact blood flow status detection technology can measure the amplitude and peak timing of the pulse wave in a specific location. This technology is used to non-contactly measure biological information such as heart rate and blood pressure, replacing the contact-based measuring devices currently used. Currently, there is a growing social demand for monitoring mental and physical conditions such as fatigue and stress in a wider range of scenarios, without burdening the individuals being measured, in order to self-assess their health and improve the quality of living and working environments. In the future, to meet these needs, it is expected that other biological information will be inferred from blood flow status.

[0005] In order to meet the above needs, the following problems (1) and (2) need to be solved. In particular, when detecting a state change of a person without movement, the problem becomes significant because the state change cannot be visually determined.

[0006] Topic (1): Obtain more information from the state of blood flow.

[0007] Topic (2): Shorten the measurement time.

[0008] For problem (1), the amount of information related to blood flow needs to be increased spatially. For problem (2), an indicator that can analyze a large amount of information in real time is needed.

[0009] Patent Document 1 discloses a method equipped with a receiving antenna. Because pulse wave timing information derived solely from acquired skin images is insufficient, this receiving antenna uses millimeter waves reflected from the user to detect heartbeat timing. This structure requires multiple sensors, including an image acquisition unit and a receiving antenna, but only provides one type of biological information: blood flow. Furthermore, since this document detects the peak timing of the pulse and heartbeat, it cannot obtain information unless the time is necessarily longer than the pulse or heartbeat cycle.

[0010] Patent Document 1: Japanese Patent Application Laid-Open No. 2016-077890 Summary of the Invention

[0011] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a blood flow analysis device that can estimate a person's condition at high speed and obtain various information without placing a measurement burden on a subject.

[0012] The blood flow analysis device of the present invention detects blood flow distribution using visible light images, compares blood flows between different measurement areas using the blood flow distribution, and analyzes correlation between blood flows occurring simultaneously in different measurement areas.

[0013] According to the blood flow analysis device according to the present invention, it is possible to estimate the human condition at high speed and obtain various information without imposing a measurement burden on the subject. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a configuration diagram of the blood flow analysis device 1 according to the first embodiment.

[0015] Figure 2 This is a structural diagram of the biological information analysis system 2 according to the second embodiment.

[0016] Figure 3 Graphs showing experimental results of blood flow states in two measurement areas at rest.

[0017] Figure 4 Is to express Figure 3 Graph showing the correlation between blood flow states in two corresponding measurement areas at rest.

[0018] Figure 5 The graph shows the blood flow state in two measurement areas when the subject was resting for 60 seconds and then moving.

[0019] Figure 6 This is a diagram showing the correlation between blood flow states in two measurement areas when the subject transitions from a resting state to an active state.

[0020] Figure 7 This is a diagram showing the correlation between blood flow states in two measurement areas when the subject is moving.

[0021] Figure 8 This is a structural diagram of the biological information analysis system 2 according to the third embodiment.

[0022] Figure 9 This is a diagram showing the measurement area in the third embodiment.

[0023] Figure 10 This is a structural diagram of the biological information analysis system 2 according to the fourth embodiment.

[0024] Description of Reference Signs

[0025] 1: blood flow analysis device, 11: visible light imaging unit, 12: image analysis unit, 13: blood flow distribution detection unit, 14: blood flow distribution analysis unit, 15: signal output unit, 16: state estimation unit, 17: signal output unit, 18: display unit, 2: biological information analysis system. DETAILED DESCRIPTION

[0026] <Implementation Method 1>

[0027] Figure 1 This is a structural diagram of a blood flow analyzer 1 according to Embodiment 1 of the present invention. The blood flow analyzer 1 analyzes the blood flow of a subject. The blood flow analyzer 1 includes a visible light imaging unit 11, an image analysis unit 12, a blood flow distribution detection unit 13, a blood flow distribution analysis unit 14, and a signal output unit 15. The blood flow distribution analysis unit 14 also includes a blood flow comparison unit 141 and a correlation analysis unit 142.

[0028] The visible light imaging unit 11 has the function of receiving visible light in the measurement area and converting the spatially distributed light intensity at at least two points into a signal for output. A CCD sensor or a CMOS sensor can be used as the sensor for converting light intensity into a signal. An optical system such as a lens and a reflector can also be included to image the subject on the sensor. Furthermore, color filters arranged corresponding to the sensor pixels can be used to obtain the intensity of red, green, and blue light for each unit space. It is particularly preferable to select a color filter that matches the absorption spectrum of the pigments contained in blood. For humans, a color filter with sensitivity in the red region, where the absorbance of hemoglobin in blood is low, is preferably used. For example, a red color filter can be used that transmits light with a wavelength of approximately 600 to 700 nm. Alternatively, a green color filter can be used that transmits green light, which penetrates the capillaries present under the skin but has difficulty penetrating deeper than these. For example, a green color filter can be used that transmits light with a wavelength of approximately 546 nm. In addition, a blue color filter that is difficult to penetrate into the interior of a living body can be used together. For example, a blue color filter that allows light with a wavelength of around 436nm to pass can be used. A light receiving portion that is sensitive to infrared light can also be provided, or a color filter that allows only infrared light to pass can be assembled in a visible light camera. In addition, it can also be provided as an imaging portion independently of the visible light camera. For example, an infrared light filter that allows light with a wavelength of around 830nm to pass can be used. If infrared light is used, blood flow inside deep blood vessels can be detected because it penetrates deeply into the living body. In addition, when light in the infrared region is used, the oxygen saturation concentration in the blood can be evaluated. Hemoglobin has different absorbance depending on whether it is bound to oxygen. For example, if light with a wavelength of around 665nm is used, the difference in its optical properties is significant. On the other hand, in the infrared region, the absorbance is low and the absorption is small regardless of the oxygen saturation concentration. Taking advantage of this, the ratio of oxygen-bound hemoglobin in the blood, that is, the oxygen saturation, can be estimated based on the ratio of the absorbance of light in the infrared region and that of light near 665 nm.

[0029] The sensor included in the visible light imaging unit 11, which converts light intensity into a signal, has been described assuming a sensor with pixels arranged in a two-dimensional manner. However, a configuration may also be employed in which a plurality of photodetectors, which obtain a signal from the light intensity at a single point, are combined with an optical system that focuses light from the measurement object, thereby enabling imaging of measurement objects that are spatially separated. A color filter may also be included.

[0030] The image analysis unit 12 receives the image signal from the visible light imaging unit 11, performs image analysis, and then passes it to the blood flow distribution detection unit 13. For example, it extracts only the necessary information from specific areas of the received image signal, eliminating unnecessary processing by extracting only the image information required for processing. Furthermore, if the pixels near the target pixel are sufficiently homogeneous, the resolution can be reduced to speed up processing. As described above, receiving a video signal and performing image processing can reduce processing waste.

[0031] The blood flow distribution detection unit 13 obtains information related to the blood flow distribution based on the image signal sent from the image analysis unit 12. Specifically, the image signal is separated by color. For example, the blood flow distribution can be obtained using a red signal (hereinafter referred to as an R signal). In the case where the measurement object of the blood flow distribution is a human organism, red light penetrates into the human organism, a part of it is absorbed, and the other light is diffused and reflected. Among them, the visible light camera unit 11 can be used to capture the light emitted from the organism. If the blood flow changes, the thickness of the blood vessels changes, and the amount of blood flowing inside changes, so the optical characteristics will change accordingly. This change in optical characteristics will occur even in the red wavelength region reaching the blood vessels, so the change in blood flow can be seen. In addition, the visible light camera unit 11 can obtain the distribution of light intensity in space, so it can detect blood flow distribution.

[0032] The blood flow distribution analysis unit 14 has the function of analyzing information related to the blood flow state based on the blood flow distribution detected by the blood flow distribution detection unit 13. For example, it can calculate the pulse, which is biological information. For example, the pulse can be detected by performing a Fourier transform on the temporal changes in blood flow and performing frequency analysis. Furthermore, by recording the time when the pulse reaches its peak and recording the changes in the pulse cycle for a sufficient length, it can be used for analysis to estimate the balance of the autonomic nervous system.

[0033] The blood flow distribution analysis unit 14 includes a blood flow comparison unit 141 that compares information related to a plurality of blood flows, and a correlation analysis unit 142 that analyzes the relationship between the intensities of the plurality of blood flows at a certain point in time.

[0034] The blood flow comparison unit 141 has a function of comparing changes in the intensity of blood flow measured at different locations. Even in a living body, the ease with which blood flow changes varies depending on the location. By comparing the blood flow measured at a location that is easy to change and a location that is difficult to change, the ratio of the change in blood flow can be measured. One phenomenon in which the state of blood flow changes locally is the opening and closing of the arteriovenous anastomosis based on the autonomic nerves. The blood flow comparison unit 141 measures the local change in the state of blood flow for a period of time until the distribution of blood flow changes sufficiently, thereby inferring changes in the arteriovenous anastomosis and thereby detecting changes in the balance of the autonomic nerves.

[0035] The correlation analysis unit 142 has the following function: for the blood flow status of multiple regions at a specific moment, it obtains values ​​with the blood flow status of each region as the axis, and analyzes blood flow-related information based on this distribution. For example, assume there are a first measurement region and a second measurement region. In a two-axis graph, if the signal corresponding to the blood flow in the first measurement region at a specific moment (time t) is set as B1, and the signal corresponding to the blood flow in the second measurement region is set as B2, then a point with coordinates (B1, B2) is plotted. If these values ​​are sampled over a certain period of time, a distribution is obtained. The correlation analysis unit 142 has the function of calculating and outputting an indicator by analyzing the state of this distribution.

[0036] As evaluation items for evaluating the distribution state, for example, the center of gravity, slope, intercept, and correlation coefficient can be listed. The center of gravity represents the state of representative blood flow balance within the measurement time. The slope depends on the ratio of the amplitudes in different measurement objects. The intercept represents the offset related to the blood flow. If attention is paid to the temporal change of the blood flow, it is also possible to perform a process to offset the offset. The correlation coefficient depends on the ratio of the phase of the blood flow change of the measurement object to its amplitude. The correlation coefficient becomes high when the phase is consistent and the amplitude ratio does not change. In this way, the amount determined by the ratio of the phase to the amplitude can be analyzed each time, so multiple sampling can be performed within the cycle of blood flow pulsation. Through the above-mentioned function of the correlation analysis unit 142, a unit that analyzes and evaluates the blood flow distribution at a higher speed than the pulse cycle can be provided.

[0037] The signal output unit 15 has the function of outputting a signal representing at least one result (blood flow evaluation index) calculated by the blood flow distribution analysis unit 14. For example, the result output by the blood flow distribution analysis unit 14 may include heart rate, the correlation coefficient between the blood flow rates of the first measurement subject and the second measurement subject, and the like. The signal output unit 15 can also output these results by, for example, transmitting data to a visual display device such as a monitor or printer, or to a speaker for audio notification.

[0038] <Implementation 1: Summary>

[0039] The blood flow analysis device 1 according to this embodiment measures blood flow distribution based on images acquired non-contactly. Using this blood flow distribution, it compares blood flow between different measurement areas and analyzes correlations between blood flows occurring simultaneously in different measurement areas. This allows for rapid blood flow measurement and the acquisition of various blood flow-related information.

[0040] The blood flow analysis device 1 of this embodiment detects changes in the state of blood flow distribution in a time shorter than the pulse cycle. This makes it possible to obtain various information related to blood flow other than the pulse in a time shorter than the pulse.

[0041] <Implementation Method 2>

[0042] Figure 2 This is a structural diagram of a biological information analysis system 2 according to Embodiment 2 of the present invention. The biological information analysis system 2 uses information related to blood flow distribution analyzed by the blood flow analysis device 1 to estimate the condition of the subject. In addition to the blood flow analysis device 1, the biological information analysis system 2 also includes a condition estimation unit 16, a signal output unit 17, and a display unit 18. The results output by the signal output unit 15 are transmitted to the condition estimation unit 16.

[0043] The state estimation unit 16 includes at least one database storing indicators used to evaluate the results received from the signal output unit 15. The database records at least information related to the type of evaluation and the boundaries of the evaluation. If the number of data is small and the update frequency is low, requiring no management, a simple data table can be used. Here, an example is shown that includes an indicator DB 161, a personal correction value DB 162, and a composite estimation DB 163.

[0044] The indicator database 161 stores combinations of evaluation indicators, judgment boundary information, and subject states. The state estimation unit 16 uses the evaluation indicators received from the signal output unit 15 to refer to the indicator database 161 and retrieve the subject state corresponding to the evaluation indicators from the indicator database 161. This allows estimation of the subject state.

[0045] Even when the blood flow distribution evaluation index is the same, the state of the subject may differ from person to person. The personal correction value DB 162 stores personal correction values ​​used to correct for these individual differences. By using the personal correction value DB 162, the state estimation unit 16 can estimate the subject's state using the index DB 161 after correcting for these individual differences.

[0046] The composite estimation database 163 stores relationships between multiple index values ​​and the subject's state. The state estimation unit 16 inputs, for example, correlation coefficients of blood flow distribution in multiple target areas and pulse rates into the composite estimation database 163 to obtain the subject's state corresponding to these correlation coefficients. For example, if these correlation coefficients are high and the pulse rate is low, the composite estimation database 163 stores an estimate indicating that the subject is stable and resting. To prevent the number of combinations of index values ​​and estimation results required for recording from increasing, combinations of correlation parameters, correction coefficients, and coefficients of functions used for approximation may be stored.

[0047] The signal output unit 17 has the function of outputting a signal based on the result of estimating the state of at least one person. For example, the result output by the state estimation unit 16 may include a resting state, etc. The display unit 18 receives the estimation result and displays the signal in a manner that the user can visually see.

[0048] Figure 3 This graph shows the experimental results of blood flow in two measurement areas during rest. It shows changes in blood flow in the first and second measurement areas, which served as blood flow measurement targets, during 120 seconds of rest. The horizontal axis represents time, and the vertical axis represents the signal value indicating the increase or decrease in blood flow. In this experiment, the measurement areas were set to include the nose as the first measurement area and the cheeks as the second measurement area.

[0049] Figure 4 Is to express Figure 3 This graph shows the correlation between blood flow states in two corresponding measurement areas at rest. By plotting the blood flow signal value in the first measurement area at a given moment on the horizontal axis and the blood flow signal value in the second measurement area on the vertical axis, the blood flow states in both measurement areas at a given moment can be represented using a single point. Figure 4 The data were plotted using 120 seconds of resting measurements. These data can be evaluated using an approximate straight line derived from multiple measurement points. The approximate straight line consists of a slope and an intercept ( Figure 4 As another evaluation method, evaluation can be performed based on the number of measurement points within a region sandwiched between an upper limit 401 and a lower limit 402 separated by a certain distance D and separated by an approximate straight line, or the ratio thereof to all measurement points. Alternatively, evaluation can be performed using the correlation coefficient or variance of the measurement data.

[0050] Figure 5 This is a diagram showing the blood flow state in two measurement areas when the subject is resting for 60 seconds and then moving. Figure 3 To ensure consistency, the measurement areas were set so that the first measurement area included the nose and the second measurement area included the cheeks. As an activity, mental arithmetic involving two-digit addition (i.e., an activity not accompanied by physical movement) was performed. It can be seen that the two blood flow change trends diverged significantly starting at 65 seconds.

[0051] Figure 6 This is a graph showing the correlation between the blood flow states in the two measurement areas when the subject changes from a resting state to an active state. Unlike the resting state, it can be seen that the measurement data has a large dispersion. For reference, Figure 4 The upper limit 401 and lower limit 402 are shown overlapping in the diagram. It can be seen that there is a large amount of measurement data with large deviations. The correlation coefficient is as low as 0.33. In particular, since the measurement data is mixed with those during rest and activity, it is difficult to obtain a correlation.

[0052] The evaluation of the blood flow state is also related to the time required for the state of the subject to transition. For example, it is assumed that the phenomenon that occurs when the state changes suddenly is different from that when it changes slowly. Therefore, the time change rate of the state can also be used as an evaluation indicator. For example, when the correlation of the measurement data is suddenly destroyed, it becomes a benchmark for the change in the state of the subject. When there is no relevant change, the subject is considered to be in a stable state. The state estimation unit 16 estimates that the state of the subject has changed based on anomalies, for example, when the time change rate of the indicator value is above a threshold value. If the time change rate is less than the threshold value, it can be estimated that the state of the subject is stable.

[0053] Figure 7 This graph shows the correlation between blood flow states in two measurement areas when the subject was active. Specifically, it extracts measurement data only during activity. It can be seen that the measurement data has a large dispersion. The correlation coefficient is 0.74.

[0054] If you focus on Figures 3 to 7 The correlation coefficient of the experiment shown is as high as 0.94 during rest, as low as 0.33 during the transition from rest to activity, and 0.74 during activity. Although not shown, if the correlation coefficient is measured during rest at other times, it is 0.81. Therefore, the threshold value of the correlation coefficient for distinguishing between rest and activity can be set between 0.74 and 0.81. If it is lower than this, it is likely during transition. During transition, it is preferable to refer to the correlation coefficient and state changes before and after. State estimation using the correlation coefficient or evaluation index can be implemented by the state estimation unit 16 or the correlation analysis unit 142.

[0055] Figures 3 to 7 The relationship between the correlation coefficient shown and the subject's motion state is particularly strong between measurement areas where blood flow changes due to arteriovenous anastomosis are large and those where blood flow changes are small. Therefore, when using these correlation coefficients to estimate the subject's motion state, it is preferable to use the correlation coefficient between these measurement areas. The nose and cheek are one example of this.

[0056] <Implementation 2: Summary>

[0057] The biological information analysis system 2 according to the second embodiment calculates an index value indicating the correlation between the increase or decrease in blood flow in the first measurement region and the increase or decrease in blood flow in the second measurement region, and estimates the subject's condition based on this index value. This enables information about the subject's condition to be obtained that would be difficult to determine based solely on the blood flow distribution at a single location.

[0058] The biological information analysis system 2 according to the second embodiment estimates the subject's resting state when the index value (a) representing the correlation between the increase and decrease of blood flow in the first measurement area and the increase and decrease of blood flow in the second measurement area is high (approximately 0.94 in the example); estimates the subject's active state when (b) is slightly low (0.74 to 0.81 in the example); and estimates the subject's transition between resting and active states when (c) is low (0.33 in the example). This allows for highly accurate estimation of the subject's state.

[0059] <Implementation Method 3>

[0060] Embodiment 3 of the present invention describes a configuration example in which only a range required for blood flow distribution analysis is selected in a subject image acquired in a non-contact manner to reduce the analysis load of blood flow distribution.

[0061] Figure 8 2 is a structural diagram of a living body information analysis system 2 according to Embodiment 3. In addition to the configuration described in Embodiment 2, the living body information analysis system 2 according to Embodiment 3 further includes a noise removal unit 121, a subject tracking unit 122, and a face tracking unit 123 as submodules of the image analysis unit 12, and further includes a peak analysis unit 143 and a frequency component analysis unit 144 as submodules of the blood flow distribution analysis unit 14.

[0062] The noise removal unit 121 processes the image acquired by the image analysis unit 12 to remove unnecessary information. A specific method for removing unnecessary information is to calculate the average of a plurality of consecutive pixels and smooth the signal value as a value representing the pixel to remove noise. This method can be used to calculate the signal intensity of the area near the pixel, which can be processed by smoothing the signal error randomly added to each pixel due to noise from the electrical characteristics of the sensor included in the visible light imaging unit 11. This method can be used to process the signal intensity as a value representing the area near the pixel. This noise removal method produces an image with suppressed spatial noise amplitude.

[0063] Another possible method for noise removal is to use a histogram of the signal intensities of multiple pixels and select the most frequently occurring signal intensities as representative values ​​for noise removal. This method selects the signal intensities that occur most frequently in a nearby area. Therefore, after noise removal, the noise of consecutive occurrences of the same value is reduced, the variance is minimized, and a homogenized signal distribution is achieved. Because this method seeks the most frequently occurring value in the vicinity of the measurement object, it is possible that pixels with the most frequently occurring signal intensities are clustered together, making it difficult to uniquely identify them, even in conditions with large fluctuations in error. Therefore, this method is preferably applied to images after averaging and removing noise between pixels.

[0064] Another noise removal method involves balancing the signal intensity of each color in the image captured by the visible light imaging unit 11. For example, this description will focus on a camera that obtains the commonly used light intensity distribution of the three colors red (R), green (G), and blue (B). When obtaining the light intensity distributions for R, G, and B in the measurement area, the intensity ranges of the signal distributions between the colors can be aligned by aligning the highest and lowest intensities for each color and then compensating for the intermediate intensities. Alternatively, the aligned highest and lowest intensities can be scaled to match the upper and lower limits of the signal level. For example, if processing is performed as a percentage, the distribution ranges are 0% to 100%, and if processing is performed as 8 bits, the distribution ranges are 0 to 255. This method can offset the differences in sensitivity between the colors determined by the visible light imaging unit 11, resulting in a signal distribution that allows for equal processing of each color.

[0065] As another method of removing noise, a method is used to adjust the balance of the signal intensity of each color using the signal distribution of each color in a certain area of ​​the image obtained from the visible light camera unit 11. For example, when the light intensity distribution of R, G, and B is obtained respectively, the characteristic quantity representing the light intensity distribution of each color in the area is obtained, and the signal of each color is corrected in such a way that the characteristic quantity is aligned. As the characteristic quantity, the highest intensity, the lowest intensity, the most frequently occurring intensity, etc. can be cited. As the coefficient required for correction, the offset representing the difference of the signal distribution, the gain representing the intensity ratio of the signal distribution, etc. can be cited. As a method of performing correction in more detail, an arbitrary function that corresponds the corrected signal value for each color and each brightness can also be used, or an adjustment using the so-called tone curve can be used. According to this method, when the signal intensity between the colors is aligned, correction can be performed taking into account the distribution of the signal intensity of the intermediate color.

[0066] In order to correct the signal intensity of each color, a method can be used to determine the correction coefficient by measuring non-living objects. If this method is used, the non-living image captured by the visible light camera unit 11 will not change due to the reaction of the living body, but will be affected by changes in the optical environment, such as changes in the color and intensity of the uniformly illuminated ambient light. On the other hand, the image of the living body being measured is not only affected by changes in signal intensity caused by changes in optical properties due to the reaction of the living body, but also by changes in the color and intensity of the ambient light. In other words, by using the non-living image at a certain moment that serves as a reference to determine the correction coefficient of the signal intensity between each color, and using the correction coefficient of the signal intensity between each color of the non-living image measured at other times, it is possible to offset the changes in the optical environment and reduce the impact in accordance with the conditions at a certain reference moment. In this way, changes caused by the optical properties of the living body can be detected with good sensitivity.

[0067] In order to reduce the deviation of the intensity signal between each color and process it equally during photography, the non-living object used as the measurement object for correcting the signal intensity of each color is preferably an achromatic color with suppressed hue. If an achromatic color with no hue is used as a reference, it is helpful to determine which color is biased. In addition, in order to suppress the reflection peak of the illumination light from appearing in space, a material with high diffusion is preferred. For example, when observing a lighting fixture with a limited light-emitting surface reflecting the measurement object, the diffusion can be determined by confirming that the shape of the light-emitting surface is difficult to determine. Since the information required to calculate the conversion function (tone curve) of the signal intensity between each color for each intermediate signal intensity can be obtained, a non-living object with a grayscale of intensity distribution is preferred.

[0068] The subject tracking unit 122 has an image tracking function for determining where the subject is reflected in the image captured by the visible light imaging unit 11 when the subject moves in the image. As a method for image tracking of the subject, an object tracking algorithm can be used. The object tracking algorithm is a function that pre-records data related to the appearance of the object to be measured and determines whether the corresponding object exists in the image captured by the visible light imaging unit 11. It is expected that not only the presence or absence of the object in the image can be determined, but also the position and size of the object reflected in the image can be determined.

[0069] As a method for achieving object tracking, a machine learning mechanism can be used, in which data related to the object is pre-assigned for learning and the evaluation function obtained through learning is used to return the object's presence, position, and size. This method can determine the position of the measurement object within the captured image even if the captured object moves or the direction of the visible light imaging unit 11 shifts, thereby enabling stable measurement.

[0070] Another method for object tracking is an image tracking method using optical flow. Optical flow is a method that assumes that a pixel or area at a reference time has moved in a certain direction and distance over a certain period of time. An image generated from the image at the reference time, assuming movement in that direction, is compared with an image obtained after the actual time has elapsed. The most consistent movement direction and distance for each pixel or area are determined. This method makes it possible to track even if the subject has not been previously learned. In particular, when any scene is used as the subject, it can improve stability against movement, such as vibration, of the visible light imaging unit 11.

[0071] The face tracking unit 123 tracks faces within the image captured by the visible light imaging unit 11 and identifies the area within the captured image. By tracking facial areas, the measurement target area can be effectively limited, eliminating unnecessary image processing and achieving efficient processing. In addition to tracking faces, it can also track individual facial features (e.g., eyes, nose, etc.).

[0072] Face tracking is a function that learns from multiple facial images and uses the learned evaluation function to determine where a person's face is located within a photographic image. This function can also be implemented using evaluation functions derived from machine learning and deep learning. By limiting processing to the image area containing the facial region identified by this function, it is possible to avoid processing unnecessary areas for measurement, achieving efficient processing.

[0073] The position of facial parts can also be used as a reference for the measurement area. Alternatively, the area within the image reflected by specific facial parts, such as the forehead, eyelids, eyes, nose, cheeks, upper lip, lower lip, and ears, can be determined, with the position and size of these areas serving as a reference. For example, to set the measurement area between the eyebrows, a method can be adopted that first determines the position of the eyes and then calculates and determines the area between them. This method allows the definition of the measurement area based on relative position, even if there are parts that cannot be directly determined, using other parts that can be determined as a reference, thereby increasing the degree of measurement flexibility.

[0074] A function could also be provided to store data related to an individual's face in advance, compare it with face tracking, and determine whether the subject is the measurement target, thereby narrowing the measurement area for efficiency. Furthermore, a function to register data related to an individual's face would be very convenient.

[0075] Peak analysis unit 143 captures peaks in blood flow conditions, records their timing and magnitude, and analyzes them. Frequency component analysis unit 144 analyzes the pulse frequency during changes in blood flow conditions. With these capabilities, analyzing a sufficient number of pulse beats allows for estimating a person's condition. For example, these analysis results can be used to assess autonomic nervous system balance.

[0076] Figure 9 This figure shows the measurement areas in Embodiment 3. Face area 300 is determined by face tracking, and measurement area 301 is determined as the cheek and measurement area 302 is determined as the nose by identifying facial parts. For example, measurement area 301 can be used as the first measurement area, and measurement area 302 can be used as the second measurement area.

[0077] <Implementation 3: Summary>

[0078] The biological information analysis system 2 according to this third embodiment tracks the subject using the subject tracking unit 122 or the facial tracking unit 123, thereby limiting the image information required for blood flow distribution analysis. This reduces the load of analyzing blood flow distribution, enabling the detection of changes in blood flow distribution status in a shorter time.

[0079] <Implementation Method 4>

[0080] Embodiment 4 of the present invention will describe a configuration example for estimating a person's condition in more detail based on a plurality of different data using learning data received from outside the blood flow analysis device 100 .

[0081] Figure 10 This is a structural diagram of the biological information analysis system 2 according to the fourth embodiment. In this fourth embodiment, the state estimation unit 16 includes, in addition to the components described in Embodiments 2 and 3, a DB update unit 164 and a learning unit 165. The DB update unit 164 acquires learning data provided externally and updates the contents of the various databases included in the state estimation unit 16. The learning unit 165 uses the learning data to perform machine learning and other processes to obtain the evaluation function required for state estimation.

[0082] The DB update unit 164 updates the contents of the index DB 161, the personal correction value DB 162, and the composite estimation DB 163. Based on the provided instructions, the DB update unit 164 determines and updates the databases to be updated, along with the items and contents to be updated. It is preferable to include a function that determines whether the instructions are inconsistent, so as to avoid errors by not executing the instructions based on consistency with other databases.

[0083] The learning unit 165 has the function of updating the evaluation function required for state estimation using data from the signal output unit 15 of the blood flow analysis device 100 or learning data obtained from other units. For example, if the input data consists of measurement results and measurement conditions from the blood flow analysis device 100, and human state identification data indicating a clearly defined human state, a configuration can be employed in which the measurement results, measurement conditions, and human state identification data are associated with each other and used as training data to learn the evaluation function. This configuration uses the learned evaluation function to increase the probability of accurately estimating a human state, even for conditions that have not been measured.

[0084] For example, by using data from the signal output unit 15 of the blood flow analysis device 100 and the state of the subject obtained by other corresponding units as training data for machine learning, it is possible to construct a learning model using supervised learning that estimates the state of non-photographed persons based on the output of the signal output unit 15 of the blood flow analysis device 100. Alternatively, a dataset obtained by adding data related to the subject's movements to the training data can be used to improve the accuracy of the estimated state.

[0085] In addition to data from the signal output unit 15 of the blood flow analysis device 100 and datasets of the subject's condition obtained by other corresponding units, a model for estimating the subject's condition can also be constructed using deep learning, which uses time series information such as the order and timing of the measurements as learning data. For example, an RNN (Recurrent Neural Network) can be used as a learning method. In particular, by considering information related to the increase or decrease in blood flow changes and the speed of change, a model for estimating the subject's condition can be constructed, thereby increasing the variety of estimated conditions and improving accuracy.

[0086] The DB updating unit 164 can also be configured to reinforce learning by inserting new conditional items into the existing conditions used by the learning unit 165. This configuration allows for increased accuracy by adding conditions, even when the conditions have limited freedom and the evaluation function for accurate judgment is no longer modified. Learning data can be applied not only to the indicator DB 161 but also to the individual correction value DB 162 and the composite estimation DB 163.

[0087] Learning using data from the personal correction value database 162 yields an evaluation function that takes individual differences into account. When adding a new subject, based on the results of a simple measurement, the personal correction values ​​are initialized using pre-registered personal correction values ​​that reflect characteristics similar to those of the subject. Learning is then performed based on the results of the measurement under modified conditions and the subject's condition, enabling faster convergence.

[0088] When learning is performed using data stored in the composite estimation database 163, more detailed data can be obtained based on externally acquired learning data and various data obtained from the blood flow analysis device 100. For example, when measuring a subject, it is determined that the blood flow correlation value output by the correlation analysis unit 142 is as high as 0.97. If this value remains stable for one minute, the subject's condition is stable and unchanged. If the pulse measurement result is 60 bpm, and the subject's normal pulse rate stored in the individual correction value database 162 is 64 bpm, it can be determined that the pulse rate is lower than usual. Since it is known that a lower pulse rate generally indicates relaxation, a more detailed estimation of a relaxed and stable state can be made. If, as learning data, images taken with other cameras indicate a signal indicating no movement, the subject is estimated to be at rest, enabling a more accurate determination of a stable, relaxed state. In this way, combining different information allows for a more detailed estimation of a person's condition.

[0089] It is also possible to use multiple types of learning data, and when the types cover multiple aspects, a deep learning model can be used. Alternatively, methods can be used to improve computational efficiency by reducing dimensionality midway through the calculation. For example, when the data from the signal output unit 15 of the blood flow analysis device 100 and the appearance obtained by measuring the blood flow of the corresponding subject are combined as a learning data set, a data structure can be obtained in which the data dimensionality increases according to the spatial resolution of the appearance. This is effective in improving computational efficiency when the dimensionality of the learning data increases significantly. As a measurement example assuming this condition, consider the case where deformation based on muscle contraction obtained from the appearance, or components of an image containing facial expression information, is used together with data related to blood flow as learning data. Under such conditions, the dimensionality of the learning data increases, thereby enabling the realization of a learning model that can cope with more conditions, expanding the types of inferred subject conditions, and improving stability and accuracy.

[0090] <Implementation 4: Summary>

[0091] The biological information analysis system 2 of this fourth embodiment uses learning data to learn the relationship between the subject's motion state and measurement results, thereby continuously improving the accuracy of motion state estimation. Furthermore, by using data acquired from sources other than the blood flow analysis device 1 as learning data, a more detailed estimation of the subject's motion state can be made based on a variety of different data.

[0092] <Regarding Modifications of the Invention>

[0093] The present invention is not limited to the embodiments described above and includes various variations. For example, the embodiments described above are described in detail to facilitate understanding of the present invention and are not limited to necessarily having all the structures described. In addition, a portion of the structure of a certain embodiment can be replaced with the structure of another embodiment, and a structure of another embodiment can be added to the structure of a certain embodiment. In addition, for a portion of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0094] In the above embodiment, the image analysis unit 12, the blood flow distribution analysis unit 14, the blood flow distribution detection unit 13, the signal output unit 15, the state estimation unit 16, the signal output unit 17, and the display unit 18 can also be composed of hardware such as circuit devices installed with these functions, or can be composed of software installed with these functions executed by a computing device.

[0095] In the above embodiment, the state estimation unit 16 can also be a component of the blood flow analysis device 1. In this case, the state estimation unit 16 can be configured as an independent functional unit or as part of another functional unit. For example, the state estimation unit 16 can be configured as part of the blood flow distribution analysis unit 14, and the blood flow distribution analysis unit 14 can also estimate the state of the subject and output the result.

Claims

1. A blood flow analysis device for analyzing the blood flow of a subject, characterized in that: The blood flow analysis device has: a visible light imaging unit that captures a light distribution of visible light in a measurement area for measuring the blood flow; a blood flow distribution detecting unit configured to detect the blood flow distribution using the visible light image captured by the visible light imaging unit; a blood flow comparison unit that compares blood flows between different measurement areas using the blood flow distribution detected by the blood flow distribution detection unit; and a correlation analysis unit that analyzes correlations between blood flows occurring simultaneously in different measurement areas using the blood flow distribution detected by the blood flow distribution detection unit; The correlation analysis unit calculates an index value indicating a correlation between an increase or decrease in a first blood flow in a first measurement region and an increase or decrease in a second blood flow occurring simultaneously with the first blood flow in a second measurement region different from the first measurement region. The correlation analysis unit estimates whether the subject's current action is in a resting state, an active state, or a transitional state between the resting state and the active state based on the index value, and outputs the estimate.

2. The blood flow analysis device according to claim 1, characterized in that The correlation analysis unit calculates quantities represented by phases and amplitudes of respective blood flows simultaneously generated in different measurement regions at a frequency higher than a pulse cycle of the subject, thereby analyzing the correlation at a frequency higher than the pulse cycle.

3. The blood flow analysis device according to claim 1, characterized in that When the index value is equal to or greater than a first threshold value, the correlation analysis unit outputs an estimation result indicating that the subject is in a resting state.

4. The blood flow analysis device according to claim 3, characterized in that When the index value is smaller than the first threshold value and equal to or greater than a second threshold value which is smaller than the first threshold value, the correlation analysis unit outputs an estimation result indicating that the subject is in an active state.

5. The blood flow analysis device according to claim 4, characterized in that: When the index value is smaller than the second threshold value, the correlation analysis unit outputs an estimation result indicating that the subject is in a transitional state between a resting state and an active state.

6. The blood flow analysis device according to claim 1, characterized in that When the temporal change rate of the index value is equal to or greater than a third threshold value, the correlation analysis unit outputs an estimation result indicating that the motion state of the subject has changed. When the temporal change rate of the index value is smaller than the third threshold value, the correlation analysis unit outputs an estimation result indicating that the motion state of the subject is stable.

7. The blood flow analysis device according to claim 1, characterized in that The blood flow comparison unit compares the blood flow in a first measurement region where the blood flow rate changes due to the arteriovenous anastomosis with the blood flow in a second measurement region where the change in blood flow rate due to the arteriovenous anastomosis is smaller than that in the first measurement region, thereby estimating the change in the arteriovenous anastomosis and outputting the result.

8. The blood flow analysis device according to claim 7, characterized in that: The blood flow comparison unit detects changes in the balance of the subject's autonomic nerves by estimating changes in the arteriovenous anastomosis and outputs the results.

9. The blood flow analysis device according to claim 1, characterized in that The blood flow analysis device further includes a subject tracking unit that, when the subject shown in the visible light image moves, determines the position of the subject before and after the movement. The blood flow distribution detecting unit detects the blood flow distribution at the position identified by the subject tracking unit.

10. The blood flow analysis device according to claim 1, characterized in that The blood flow analysis device further includes a face tracking unit that identifies a facial portion of the subject reflected in the visible light image. The blood flow distribution detection unit detects the blood flow distribution of the facial portion identified by the face tracking unit.

11. A biological information analysis system, characterized in that: The biological information analysis system has: The blood flow analysis device according to claim 1; and A state estimating unit estimates whether the subject's current action is a resting state, an active state, or a transitional state between the resting state and the active state, using the blood flow analyzed by the blood flow analyzer.

12. The biological information analysis system according to claim 11, characterized in that The biological information analysis system further includes an index database that describes the relationship between an index value indicating the state of the blood flow and an action being performed by the subject. The state estimating unit refers to the index database using the index value to estimate whether the action currently being performed by the subject is a resting state, an active state, or a transitional state between the resting state and the active state.

13. The biological information analysis system according to claim 12, characterized in that The biological information analysis system further includes a correction value database in which correction values ​​for correcting the index values ​​are recorded for each of the subjects. The state estimating unit obtains the correction value corresponding to the subject from the correction value database. The state estimating unit refers to the index database using the index value corrected by the correction value obtained from the correction value database, thereby estimating whether the subject's current action is a resting state, an active state, or a transitional state between the resting state and the active state.

14. The biological information analysis system according to claim 12, wherein: The biological information analysis system further includes a learning unit that learns the relationship between the index value and the action being performed by the subject through machine learning. The index value is a value indicating the state of the blood flow obtained by the blood flow analysis device or a device other than the blood flow analysis device. The learning unit updates the indicator database according to the learned relationship.

Images