Measuring device, measuring method, and program

Abstract

This invention provides a measuring device that can appropriately calculate biological information in accordance with the function of the autonomic nervous system using images captured from living organisms. [Solution] The measuring device comprises an imaging unit that images a living organism and acquires an image; a feature calculation unit that calculates feature quantities related to the living organism and autonomic nerve feature quantities related to the autonomic nerves of the living organism from the image; a storage unit that stores a model for calculating biological information related to at least one of the blood vessels of the living organism and the blood of the living organism; and a biological information calculation unit that calculates the biological information using at least one of the feature quantities and autonomic nerve feature quantities and the model.

Description

[Technical Field]

[0001] The present disclosure relates to a measuring apparatus, a measuring method, and a program. [Background Art]

[0002] There are mainly two types of non-invasive methods for measuring biological information (vital signs) by detecting blood fluctuations in a living body as pulse waves through optical techniques. One method uses the fact that the propagation velocity or propagation time of a pulse wave propagating in an artery of a living body is proportional or inversely proportional to biological information, determines a relational expression in advance, and calculates biological information from the measured pulse wave. However, this method requires multiple measurement sensors that need to be synchronized, and for accurate measurement, the sensors need to be attached to the body and fixed so that they do not move, which imposes a burden on the patient.

[0003] The other method involves analyzing the pulse wave itself. This method uses a device capable of measuring pulse waves in a non-contact manner, such as a camera, and can measure biological information without the need to attach multiple sensors. More specifically, when light is irradiated onto the skin or when natural light hits the skin, changes in the intensity of reflected light, transmitted light, or absorbance are detected, so that volume changes of blood vessels can be measured as pulse waves. Since the volume pulse wave obtained by such an optical method reflects vascular dynamics, it is possible to calculate vital signs by learning the relationship between biological information (vital signs) related to blood vessels and the volume pulse wave using a method such as machine learning.

[0004] Patent Document 1 discloses a blood pressure monitoring apparatus that monitors the blood pressure value of a living body based on the propagation velocity or propagation time of a pulse wave indicating the flow of blood propagating in an artery of the living body. In the blood pressure monitoring apparatus disclosed in Patent Document 1, the velocity change of pulse wave propagation velocity is corrected and calculated based on the ratio of respective signal intensities, the difference in signal intensities, the change in the ratio of signal intensities, or the change in the difference in signal intensities of the heart rate cycle fluctuation high-frequency component and the propagation velocity fluctuation low-frequency component, and the correspondence relationship between the blood pressure value and the pulse wave propagation velocity is corrected. [Prior Art Literature] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 10-066680 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] In blood pressure monitoring devices that monitor a living body's blood pressure based on the propagation speed or propagation time of pulse waves propagating within the arteries of the living body, as described in Patent Document 1, changes in speed can be corrected using simple correction formulas or correction curves. However, the mechanisms by which the autonomic nervous system influences the propagation speed of pulse waves and the effects of the volume pulse wave are different. Because the waveform of the volume pulse wave changes in a complex way due to the superposition of reflected waves generated by reflection from the blood vessel wall, it is difficult to correct for vasoconstriction, dilation, and hardening caused by the action of the autonomic nervous system using simple correction formulas like those in Patent Document 1.

[0007] Therefore, one aspect of this disclosure aims to provide a measuring device, a measuring method, and a program that can appropriately calculate biological information in accordance with the function of the autonomic nervous system using images captured from a living organism. [Means for solving the problem]

[0008] A measuring device according to one embodiment of the present disclosure includes: an imaging unit that images a living organism and acquires an image; a feature calculation unit that calculates feature quantities related to the living organism and autonomic nerve feature quantities related to the autonomic nerves of the living organism from the image; a storage unit that stores a model for calculating biological information related to at least one of the blood vessels of the living organism and the blood of the living organism; and a biological information calculation unit that calculates the biological information using at least one of the feature quantities and autonomic nerve feature quantities and the model.

[0009] A measurement method according to one embodiment of the present disclosure involves imaging a living organism to acquire an image, calculating feature quantities related to the living organism and autonomic nerve feature quantities related to the autonomic nerves of the living organism from the image, reading out a model for calculating biological information related to at least one of the blood vessels of the living organism and the blood of the living organism, and calculating the biological information using at least one of the feature quantities and autonomic nerve feature quantities and the model.

[0010] A program according to one embodiment of the present disclosure causes a computer to perform the following functions: a function to image a living organism and acquire an image; a function to calculate feature quantities related to the living organism and autonomic nerve feature quantities related to the autonomic nerves of the living organism from the image; a function to read a model for calculating biological information related to at least one of the blood vessels of the living organism and the blood of the living organism; and a function to calculate the biological information using at least one of the feature quantities and autonomic nerve feature quantities and the model. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example of how the measuring device is used. [Figure 2] This is a block diagram showing an example of the configuration of a measuring device according to the first embodiment. [Figure 3] This figure shows an example of a volume pulse wave waveform. [Figure 4] This figure shows an example of a volume pulse wave waveform. [Figure 5] This figure shows an example of a volume pulse wave power spectrum. [Figure 6] This is a flowchart illustrating an example of the operation of the measuring device according to the first embodiment. [Figure 7] This figure shows an example of multiple areas in a multidimensional space formed by autonomic nervous system features. [Figure 8] This figure shows an example of the conditions for autonomic nervous system features linked to a model. [Figure 9] This is a block diagram showing an example of the configuration of a measuring device according to the second embodiment. [Figure 10]This flowchart shows an example of the operation of the measuring device according to the second embodiment. [Modes for carrying out the invention]

[0012] (First Embodiment) The first embodiment will be described with reference to Figures 1 to 6. In the drawings, the same or similar elements are denoted by the same reference numerals, and redundant explanations are omitted.

[0013] Figure 1 shows an example of how the measuring device 100 is used. As illustrated in Figure 1, the measuring device 100 includes an imaging unit 101.

[0014] The measuring device 100 measures the time-series changes in the surface and internal state of the skin of the living organism 102 from images acquired by the imaging unit 101, and calculates biological information. The biological information is information relating to at least one of the blood vessels and blood of the living organism 102. For example, the measuring device 100 is a PC (Personal Computer), smartphone, tablet terminal, dedicated biological information measurement terminal, or a monitoring robot equipped with an imaging unit 101. When the living organism 102 is irradiated with lighting or natural light, it is possible to measure the internal state of the skin, such as vital signs relating to at least one of blood vessels and blood flow, by measuring the light transmitted or reflected by the skin. In this embodiment, blood pressure is used as an example of a vital sign, but it is not limited to blood pressure as long as it is a vital sign relating to at least one of blood vessels and blood flow. In Figure 1, the living organism 102 is not holding the measuring device 100 in its hand, but this is not the only example; for example, it also includes cases where the living organism 102 or a person other than the living organism 102 holds a smartphone or tablet in their hand to take a picture.

[0015] The imaging unit 101 images the living body 102 and acquires an image 211. In this disclosure, still images and videos extracted from continuous or discontinuous live recordings that reflect the state of the blood vessels of the living body 102, as captured by the imaging unit 101, are referred to as images 211.

[0016] The imaging unit 101 is installed at a position where an exposed portion of the body surface of a living body 102 can be imaged. The exposed portion of the body surface of the living body 102 refers to the forehead, cheeks, fingertips, wrist, palm, etc. of the living body 102. For example, the imaging unit 101 is installed in a personal computer, a smartphone, a tablet, a display, a mirror, a washstand, or the like.

[0017] The imaging unit 101 is a camera including a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and a lens. The imaging unit 101 may be configured by an image sensor for a camera including an RGB filter. For example, the imaging unit 101 is provided with a color filter of an RGB Bayer array in order to detect minute changes in the skin of the living body 102. Alternatively, the imaging unit 101 may be provided with a color filter such as RGBCy or RGBIR. Color filters such as RGBCy and RGBIR are suitable for observing increases and decreases in blood volume indicated by reflected light of light transmitted into the skin.

[0018] FIG. 2 is a block diagram showing an example of the configuration of the measurement apparatus 100.

[0019] The measurement apparatus 100 includes the imaging unit 101, an input unit 201, an output unit 202, a storage unit 203, a control unit 204, and the like. The imaging unit 101, the input unit 201, the output unit 202, and the storage unit 203 are electrically connected to the control unit 204.

[0020] The imaging unit 101 images the living body 102 to acquire an image 211, and transmits the acquired image 211 to the control unit 204. For example, the imaging unit 101 images the living body 102 at 30 to 60 fps (frames per second) to acquire the image 211. The image 211 includes an image of the body surface of the living body 102. For example, the image of the body surface of the living body 102 is a face image.

[0021] The input unit 201 accepts input of information necessary for managing the measurement results and information necessary for the measuring device 100 to perform the measurement. For example, the information necessary for managing the measurement results includes at least one of the following: the name, ID, and age of the living organism 102. For example, the input unit 201 may be a keyboard, mouse, touch panel, etc.

[0022] The output unit 202 outputs images 211, a report 212 compiled by the control unit 204 containing biometric information according to the user's needs, a message to the user, the date and time, etc. For example, the output unit 202 may include a display, a speaker, etc.

[0023] The control unit 204 executes various processes according to the programs and data stored in the storage unit 203. For example, the control unit 204 is composed of processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).

[0024] The control unit 204 comprises a feature calculation unit 205, a biometric information calculation unit 206, and a model selection unit 207.

[0025] The feature calculation unit 205 calculates feature quantities 216 related to the organism 102 and autonomic nerve feature quantities 216a related to the autonomic nerves of the organism 102 from the image 211. More specifically, the feature calculation unit 205 calculates a biological signal related to the organism 102 from the image 211, and calculates feature quantities 216 and autonomic nerve feature quantities 216a from the biological signal. For example, the feature calculation unit 205 calculates a biological signal from a signal indicated by the RGB (Red Blue Green) pixel values ​​of pixels contained in the image 211. In this disclosure, the biological signal represents a volume pulse wave. The volume pulse wave reflects the state of the blood vessels and blood flow of the organism 102.

[0026] For example, the feature calculation unit 205 extracts the location of the region of interest within the face image using pattern recognition, machine learning, etc. Then, the feature calculation unit 205 calculates the volume pulse wave using the pixel values ​​within the extracted region of interest and the time-series data of those pixel values. In the measurement device 100 according to this embodiment, the image 211 is not limited to a face image. The image 211 may include images of areas where the surface of the living body 102 is exposed, and may include images of the forehead, cheeks, fingertips, wrists, palms, etc. of the living body 102.

[0027] In this embodiment, the measurement device 100 may appropriately determine the means for calculating specific biological signals depending on the vital signs to be measured. For example, the feature calculation unit 205 calculates the biological signal as a time change of a value calculated by substituting the brightness value of the image 211 into a predetermined mathematical formula. Alternatively, the feature calculation unit 205 may calculate the biological signal by converting it to absorbance or the like. Alternatively, the feature calculation unit 205 may calculate the biological signal using independent component analysis or the like. For example, if the vital sign to be measured is blood pressure, the feature calculation unit 205 may calculate the volume pulse wave from a conversion formula using absorbance, taking advantage of the correlation between hemoglobin concentration and the amount of light absorbed by blood vessels. The feature calculation unit 205 calculates various features 216 and also calculates the autonomic nervous system features 216a described later. In this embodiment, calculation methods and formulas, including mathematical formulas and coefficients necessary for these calculations, are stored in the memory unit 203, and the feature quantity calculation unit 205 can read the calculation methods and formulas from the memory unit 203 as needed to calculate the feature quantity 216 and the autonomic nervous system feature quantity 216a.

[0028] The model selection unit 207 selects a model for calculating biological information from among several types of machine learning models based on the autonomic nervous system features 216a. The model selection unit 207 outputs the information of the selected model as a determination result 217. In this disclosure, "model" refers to a machine learning model for calculating biological information.

[0029] The biological information calculation unit 206 calculates biological information based on a model and feature quantities 216 for calculating biological information. In other words, the biological information calculation unit 206 receives a judgment result 217 and calculates biological information using a model selected based on the judgment result 217. For example, the biological information may represent blood pressure.

[0030] The output unit 202 receives the report 212 created by the control unit 204 from the biological information and outputs the report 212 as needed. The report 212 shows values ​​representing the biological information and information created by processing and modifying the biological information according to the user's needs. Specifically, the report 212 shows values ​​representing the biological information, a graph showing the trend of the biological information, the results of the analysis of the biological information, an evaluation of the biological information, a summary of the biological information, etc. The procedures for the above processing and modification are stored in the storage unit 203 in advance.

[0031] The storage unit 203 is a recording medium capable of recording various data, programs, etc., and is composed of a hard disk, SSD (Solid State Drive), semiconductor memory, server, etc. The storage unit 203 includes a measurement information storage unit 213, a biological information storage unit 214, and a model storage unit 215.

[0032] The measurement information storage unit 213 stores pre-programmed information necessary for measuring biological information, information registered by the user, etc. For example, the measurement information storage unit 213 stores calculation formulas for converting images 211 to volume pulse waves, signal processing algorithms for reducing noise in volume pulse waves, and measurement conditions such as the measurement time required for calculating biological information. For example, the user may be the administrator of the biological specimen 102 or the measuring device 100, or the manufacturer of the measuring device 100.

[0033] The biological information storage unit 214 stores information related to the biological organism 102. This information includes input data, measured data, and calculated results. For example, the information related to the biological organism 102 includes images 211, information on the model used and its corresponding autonomic nervous system features 216a, biological information calculated from images 211, and reports 212.

[0034] The model storage unit 215 stores models for calculating biological information, methods and formulas for calculating the features used by the models, pre-stored programs related to the models, and information registered by the user regarding the models. Multiple types of machine learning models are stored in the model storage unit 215. Furthermore, the models stored in the model storage unit 215 can be changed by the user. In addition, the models for calculating biological information, methods and formulas for calculating the features used by the models, pre-stored programs related to the models, and information registered by the user regarding the models do not necessarily have to be stored in the same storage unit. For example, at least one of these may be stored in the storage unit of an information processing device different from the measurement device 100. In that case, the measurement device 100 can obtain the information necessary for calculating biological information from the information processing device.

[0035] Here, we will use Figure 3 to explain the effects of the autonomic nervous system on blood vessels and blood pressure.

[0036] Figure 3 shows an example of a volume pulse wave waveform. In Figure 3, the height of the volume pulse wave is indicated by a double-headed arrow. The height of the volume pulse wave is also called amplitude or wave height, but its definition varies depending on the context. For example, whether or not the baseline is corrected, and what range is defined as the height. Figure 3 shows the amplitude as an example, but this is not the only way, and the height of the volume pulse wave may be defined as needed.

[0037] When blood vessels constrict due to the autonomic nervous system's function, which reflects psychological states such as stress and tension, the height of the volume pulse wave may decrease from the amplitude P1 of the volume pulse wave to the amplitude P2 of the waveform within the dotted rectangle, as shown in Figure 3. In this case, although blood pressure remains almost unchanged due to the function of the autonomic nervous system, only the volume pulse wave changes. As a result, the correlation between the shape of the volume pulse wave and blood pressure becomes smaller, which can reduce the accuracy of estimating blood pressure from the volume pulse wave.

[0038] Therefore, in this embodiment, the state of blood vessels is quantified and clearly indicated by calculating autonomic nervous system information as a feature. In machine learning, it is more effective to clearly indicate features that are directly related to the vital signs that are the target of estimation in order to improve learning. Therefore, in this embodiment, features that directly represent the characteristics of volume pulse wave due to the function of the autonomic nervous system are clearly indicated, and the model to be used is selected based on the autonomic nervous system feature 216a.

[0039] For example, the model memory unit 215 stores multiple models that correspond to different cases of autonomic nervous system activity. These multiple models correspond to different states of the organism, such as a model used for organisms with active autonomic nervous system activity and a model used for organisms with inactive autonomic nervous system activity.

[0040] For example, a learning device different from the measuring device 100 generates a model using machine learning. For example, the learning device is an information processing device different from the measuring device 100. The learning device includes a processor that controls the learning device.

[0041] The learning device generates training data for each of several training organisms in a state of active autonomic nervous system activity. This data consists of feature quantities calculated from volume pulse wave data, autonomic nervous system feature quantities calculated from the volume pulse wave data, and biological information indicating the correct values ​​measured when the volume pulse wave data was measured. The learning device then performs machine learning using the training data from the multiple training organisms in a state of active autonomic nervous system activity, taking the feature quantities calculated from volume pulse wave data as input and the biological information indicating the correct values ​​as output, thereby generating a model for use in organisms with active autonomic nervous system activity. Furthermore, the learning device associates at least one of the following with the model for use in organisms with active autonomic nervous system activity: the value of the autonomic nervous system feature quantity indicating active autonomic nervous system activity, its range, and a value obtained using a predetermined formula from the value. This association can be used as a criterion or formula for model selection.

[0042] Similarly, for example, the learning device generates training data for each of several training organisms in a state where the autonomic nervous system is not active. This data consists of feature quantities calculated from volume pulse waves, autonomic nervous system feature quantities calculated from those volume pulse waves, and biological information indicating the correct values ​​measured when the volume pulse waves were measured. The learning device then performs machine learning using the training data for the multiple training organisms in a state where the autonomic nervous system is not active, taking the feature quantities calculated from volume pulse waves as input and biological information indicating the correct values ​​as output, thereby generating a model for use in organisms with an inactive autonomic nervous system. Furthermore, the learning device associates at least one of the following with the model for use in organisms with an inactive autonomic nervous system: the value of the autonomic nervous system feature quantity indicating an inactive state, its range, and a value obtained using a predetermined formula from that value. This association can be used as a criterion or formula for selecting the model. Here, we have explained two methods for creating models: one for use in organisms with active autonomic nervous system activity and another for use in organisms with inactive autonomic nervous system activity. However, these are merely examples, and the models are not necessarily limited to these two types. Alternatively, a model for classifying the state of autonomic nervous system activity in living organisms may be created using autonomic nervous system features 216a. For example, the learning device generates training data for multiple learning organisms in a state of active autonomic nervous system activity and multiple learning organisms in a state of inactive autonomic nervous system activity. This data consists of autonomic nervous system features, features calculated from the volume pulse wave, and biological information indicating the state of autonomic nervous system activity in the organism as the correct answer when the volume pulse wave was measured. The learning device then performs machine learning on the training data, taking the autonomic nervous system features and features calculated from the volume pulse wave as input, and outputs biological information indicating the correct answer, thereby generating a classification model.

[0043] Based on the above, the learning device generates a model corresponding to the state of the living organism. The measuring device 100 then stores the model generated by the learning device in the model storage unit 215.

[0044] The model selection unit 207 selects a model from a plurality of models stored in the model storage unit 215 based on the autonomic nervous system feature quantities 216a. The model selection is based on the values, ranges, or values ​​obtained using a predetermined formula from the autonomic nervous system feature quantities 216a associated with each model, or a classification model created using the autonomic nervous system feature quantities 216a. As a result, the measuring device 100 according to this embodiment can use a model that reflects the relationship between the function of the autonomic nervous system and the state of the blood vessels, and can calculate blood pressure with high accuracy.

[0045] The model selection unit 207 is not necessarily limited to one model; it may select multiple models. For example, the biological information calculation unit 206 can determine the final calculated value by multiplying the blood pressure values ​​calculated from each model selected by the model selection unit 207 by a weighting coefficient previously stored in the model storage unit 215 and summing the results. This allows the measuring device 100 according to this embodiment to more accurately represent the complex and changing state of blood vessels, thereby improving the accuracy of biological information calculation.

[0046] Thus, the measuring device 100 according to this embodiment calculates vital signs by taking into account the effects of autonomic nerve function on blood vessels and blood pressure using autonomic nerve feature quantities 216a. For example, the feature quantity calculation unit 205 acquires feature quantities 216 that represent the characteristics of the volume pulse wave and autonomic nerve feature quantities 216a using the following method.

[0047] For example, the autonomic nervous system feature 216a is represented by at least one of the amplitude and width of the volume pulse wave. Alternatively, the autonomic nervous system feature 216a is represented by at least one of the peak position, peak height, dip position, and dip size of the volume pulse wave. Alternatively, the autonomic nervous system feature 216a may be represented by a value obtained by arithmetic operations using at least one of the amplitude, width, peak position, peak height, dip position, and dip size of the volume pulse wave. Alternatively, the temporal variation of these values ​​may be calculated and represented.

[0048] The feature calculation unit 205 may also perform frequency analysis of the volume pulse wave using Fourier transform or the like to obtain at least one of the amplitude spectrum, phase spectrum, and power spectrum. The feature calculation unit 205 then calculates at least one of the following from the amplitude spectrum, phase spectrum, and power spectrum as the autonomic nervous system feature 216a: peak size, peak frequency, ratio of peak heights, and peak interval.

[0049] Alternatively, the feature calculation unit 205 may calculate the autonomic nervous system feature 216a from a waveform calculated by differentiating the volume pulse wave waveform. Alternatively, the feature calculation unit 205 may calculate the autonomic nervous system feature 216a from a pattern shown by at least one of the volume pulse wave image, spectrogram calculated from the volume pulse wave, scalogram calculated from the volume pulse wave, and correlogram calculated from the volume pulse wave. Specifically, the feature calculation unit 205 may acquire at least one of the volume pulse wave image, spectrogram, scalogram, and correlogram as an image, and calculate the autonomic nervous system feature 216a from the pattern of the acquired image. For example, the relationship between the pattern of at least one of the volume pulse wave image, spectrogram, scalogram, and correlogram and the autonomic nervous system feature is learned in advance by machine learning. In that case, the feature calculation unit 205 identifies a learned pattern that fits the pattern of the acquired image and calculates the autonomic nervous system feature from the identified pattern.

[0050] Figure 4 shows an example of a volume pulse wave waveform. It is known that heart rate fluctuates due to the autonomic nervous system. The peaks in the volume pulse wave are mainly caused by the rhythm of the heart's blood pumping, and the behavior of these peaks is also called pulse variability. When the autonomic nervous system is functioning, pulse variability can be influenced by heart rate variability.

[0051] Therefore, we will illustrate a method that utilizes the fact that pulse rate variability is influenced by heart rate variability to capture the interval between peaks as an autonomic nervous system feature 216a. For example, let the peaks of the two peaks shown in Figure 4 be A and B, respectively. In this case, the interval between the time at A and the time at B is the length indicated by the arrow. For example, the interval between peaks is indicated by time. Alternatively, the interval between peaks may be indicated by the number of camera frames. For example, the feature calculation unit 205 can calculate the autonomic nervous system feature 216a using time-series data obtained by linking the value of the peak interval to the time at B and arranging the peak intervals for each time. The feature calculation unit 205 may also calculate the autonomic nervous system feature 216a using the interval between peaks of the differential waveform obtained by differentiating the volume pulse wave.

[0052] As a method of using the time-series data of peak intervals as autonomic nerve feature 216a using the method described above, for example, the feature calculation unit 205 may detect multiple peak intervals and calculate the average value of the detected multiple peak intervals and the variance of the detected multiple peak intervals as autonomic nerve feature 216a. Alternatively, the feature calculation unit 205 may take advantage of the fact that it is time-series data and calculate the value obtained by frequency analysis as autonomic nerve feature 216a.

[0053] As an example, we will explain the use of the power spectrum of a volume pulse wave with reference to Figure 5. Figure 5 is a diagram showing an example of the power spectrum of a volume pulse wave. For example, the feature calculation unit 205 sets frequency bands as shown in C and D of Figure 5, and calculates a value representing at least one of the frequency, intensity, and area of ​​the maximum peak in the set frequency band as the autonomic nervous system feature 216a. Alternatively, if multiple frequency domains are set, the feature calculation unit 205 may calculate the autonomic nervous system feature 216a by performing arithmetic operations on the value representing at least one of the frequency, intensity, and area of ​​the maximum peak for each frequency domain. For example, if multiple frequency domains are set, the feature calculation unit 205 may calculate the autonomic nervous system feature 216a as the ratio of the values ​​representing at least one of the frequency, intensity, and area of ​​the maximum peak in different frequency bands.

[0054] Alternatively, the feature calculation unit 205 may calculate the autonomic nervous system feature 216a using the power spectrum obtained from the time-series data of the peak intervals of the volume pulse wave in Figure 4, rather than using the power spectrum obtained from the volume pulse wave in Figure 4, as illustrated in Figure 5. In this case as well, the feature calculation unit 205 can quantify the function of the autonomic nervous system and calculate the autonomic nervous system feature 216a in the same way as the method described in Figure 5.

[0055] As described above, the measuring device 100 according to this embodiment can quantify the function of the autonomic nervous system from the waveform of the volume pulse wave or the results of frequency analysis and calculate the autonomic nervous system characteristic quantity 216a.

[0056] Figure 6 is a flowchart showing an example of the operation of the measuring device 100 according to this embodiment.

[0057] In step S601, the imaging unit 101 images the living body 102 and acquires an image 211.

[0058] In step S602, the feature calculation unit 205 obtains feature quantities 216 from the image 211 acquired in step S601. For example, the feature calculation unit 205 calculates feature quantities 216 from the RGB pixel values ​​of the region of interest in the image 211 using a calculation formula that has been previously stored in the storage unit 203.

[0059] In step S603, the feature calculation unit 205 calculates the autonomic nervous system feature 216a using a calculation formula previously stored in the memory unit 203. For example, the feature calculation unit 205 calculates the autonomic nervous system feature 216a from at least one of the following: the waveform of the volume pulse wave, the result of the frequency analysis of the volume pulse wave, and an image showing the volume pulse wave. The image showing the volume pulse wave shows the waveform image of the volume pulse wave and at least one of the spectrogram, scalogram, and correlogram calculated from the volume pulse wave. In this flowchart, the autonomic nervous system feature 216a is calculated after the feature 216, but the order is not limited to this; the order may be reversed, or they may be calculated in the same step.

[0060] In step S604, the model selection unit 207 selects the optimal model from among the models stored in the model memory unit 215 based on the autonomic nervous system feature quantities 216a calculated in step S603. The model memory unit 215 has pre-stored criteria for model selection, mathematical formulas, classification models, etc.

[0061] In step S605, the biological information calculation unit 206 calculates biological information from the feature quantities 216 using the model selected in step S604. In this disclosure, the biological signal represents a volume pulse wave, so the biological information calculation unit 206 calculates biological information related to at least one of blood vessels and blood flow using the feature quantities 216 calculated from the volume pulse wave.

[0062] In step S606, the control unit 204 creates a report 212 based on the biometric information as necessary, and outputs the created report 212 to the output unit 202. The output unit 202 performs a process to present the report 212 to the user as necessary. Alternatively, the output unit 202 performs a process to send the report 212 to a terminal device (not shown) used by the user as necessary.

[0063] In step S607, the control unit 204 stores the biological information, feature quantities 216, autonomic nervous system feature quantities 216a, selected models, etc., in the storage unit 203 as needed. Then, the control unit 204 terminates the process.

[0064] As described above, the measuring device 100 according to this embodiment can appropriately calculate biological information by reflecting the user's autonomic nervous system information by selecting a model for calculating biological information from among several machine learning models based on the autonomic nervous system features 216a. For example, if the body 102 is in a state other than rest, the relationship between blood vessel diameter and blood pressure will change from the state in which the body 102 is at rest. Here, "rest" is not limited to a state of medical rest, but also includes a state of relaxation in daily life or a state of mental calmness. A state in which the body 102 is in a state other than rest is a state in which the autonomic nervous system constricts or dilates blood vessels to regulate blood pressure, or a state in which the blood vessels are temporarily hardened. For example, if the body 102 is in a tense state or under stress, the sympathetic nervous system becomes dominant, causing blood vessels to constrict and the amplitude of the volume pulse wave to decrease. Alternatively, if the blood vessels are temporarily hardened, the blood vessel diameter becomes less likely to change, so the relationship between the amplitude of the volume pulse wave and blood pressure changes from the state in which the body 102 is at rest. Due to these phenomena, the waveform of the volume pulse wave and blood pressure become less correlated. In that case, if biological information is calculated using a machine learning model created using data measured in a resting state, the accuracy of the calculation of biological information may decrease.

[0065] However, the measuring device 100 according to this embodiment can appropriately calculate biological information in accordance with the function of the autonomic nervous system by reflecting the user's autonomic nervous system information and calculating biological information related to at least one of blood vessels and blood flow, using the image 211 captured of the living body 102.

[0066] (First torture) As a first modification of the measuring device 100 according to this embodiment, the measuring device 100 may use a plurality of autonomic nerve feature quantities 216a and select a model by dividing an area within the multidimensional space formed by the plurality of autonomic nerve feature quantities 216a. The plurality of autonomic nerve feature quantities 216a are at least two or more autonomic nerve feature quantities 216a.

[0067] In this modified example, the model storage unit 215 stores in advance information indicating an area in a multidimensional space formed by multiple autonomic nerve feature quantities 216a, with the model associated with that information. In this modified example, the multidimensional space refers to a multidimensional space in which multiple autonomic nerve feature quantities 216a each form an axis of space. The feature quantity calculation unit 205 in this modified example calculates multiple autonomic nerve feature quantities 216a from the image 211. The model selection unit 207 in this modified example then selects a model from among multiple models based on the area identified by the multiple autonomic nerve feature quantities 216a. More specifically, the model selection unit 207 selects a model from among multiple pre-prepared models that is associated with an area identified by the values ​​on each axis corresponding to the multiple autonomic nerve feature quantities 216a in the multidimensional space formed by the multiple autonomic nerve feature quantities 216a.

[0068] This modified example will be explained in detail using Figure 7. Figure 7 shows an example of multiple areas in a multidimensional space formed by autonomic nerve feature quantities 216a. The multidimensional space illustrated in Figure 7 is assumed to be a two-dimensional space formed by feature quantities F1 and F2 among the multiple autonomic nerve feature quantities 216a.

[0069] As shown in Figure 7, the two-dimensional space formed by feature quantities F1 and F2 is divided by lines L1 and L2 into areas Area1, Area2, Area3, and Area4. The model storage unit 215 has models pre-stored for each of these areas.

[0070] For example, if the autonomic nervous system feature 216a calculated by the feature calculation unit 205 is plotted in a two-dimensional space as illustrated in Figure 7, and the resulting point is designated as plot point data1, then plot point data1 belongs to area Area1. Therefore, for the autonomic nervous system feature 216a corresponding to plot point data1, the model selection unit 207 selects a model that corresponds to area Area1.

[0071] Similarly, for example, if the autonomic nervous system feature 216a calculated by the feature calculation unit 205 is plotted in a two-dimensional space as illustrated in Figure 7, and the resulting point is designated as plot point data2, then plot point data2 belongs to area Area2. In that case, the model selection unit 207 selects a model corresponding to area Area2 for the autonomic nervous system feature 216a corresponding to plot point data2.

[0072] The measurement device 100 according to this modified example can calculate biological information with greater accuracy than when calculated using a single autonomic nerve feature 216a, because it selects a model based on areas divided in a multidimensional space using multiple autonomic nerve feature quantities 216a.

[0073] Generally, the autonomic nervous system has two functions: the sympathetic nervous system and the parasympathetic nervous system. Furthermore, the human body is regulated by the combination of the functions of these two nerves. For example, even if the sympathetic nervous system is dominant and stress levels are high, if both the sympathetic and parasympathetic nervous systems are activated, the body is in a state of tension in a positive sense. Conversely, if they are not activated, the body and mind feel fatigued. In this way, blood vessels are controlled by the complex control of the sympathetic and parasympathetic nervous systems. Because such complex functions also affect changes in the shape of the volume pulse wave, the measurement device 100 in this modified example expresses the state of the autonomic nervous system using multiple feature quantities to more clearly indicate the state of the blood vessels, and determines the model to be used based on these multiple feature quantities. For example, if the values ​​of feature quantity F1 and feature quantity F2 reflect the states of the sympathetic and parasympathetic nervous systems, respectively, the model to be used can be determined by considering the states of the sympathetic and parasympathetic nervous systems and their balance.

[0074] In Figure 7, a two-dimensional space is shown as the multidimensional space formed by multiple autonomic nerve feature quantities 216a, but it is not limited to this; the multidimensional space formed by multiple autonomic nerve feature quantities 216a may be three or more dimensions. Also, in Figure 7, two lines are shown dividing the area, but it is not limited to this; three or more lines may be used. Furthermore, in Figure 7, the lines dividing the area are straight lines, but curves may also be used.

[0075] (Second variation) As a second modification of the measuring device 100 according to this embodiment, the measuring device 100 may use a plurality of autonomic nerve feature quantities 216a, and if the value indicated by each of the plurality of autonomic nerve feature quantities 216a satisfies a predetermined condition, a model corresponding to that condition may be selected. The plurality of autonomic nerve feature quantities 216a are at least two or more autonomic nerve feature quantities 216a.

[0076] In this modified example, the model storage unit 215 stores in advance the conditions of the values ​​indicated by multiple autonomic nerve features 216a and the models associated with them. The feature calculation unit 205 in this modified example calculates multiple autonomic nerve features 216a from the image 211. The model selection unit 207 in this modified example selects a model from the multiple models based on the conditions that are satisfied by the values ​​indicated by the autonomic nerve features 216a.

[0077] This modified example will be explained in detail with reference to Figure 8. Figure 8 shows an example of a model associated with the conditions of the value indicated by the autonomic nervous system feature 216a. For example, the model memory unit 215 has models M1, M2, and M3 pre-stored. Furthermore, the feature calculation unit 205 calculates autonomic nervous system features F3, F4, and F5. In the table illustrated in Figure 8, models M1, M2, and M3 are linked to the conditions of the value indicated by the autonomic nervous system feature F3, the conditions of the value indicated by the autonomic nervous system feature F4, and the conditions of the value indicated by the autonomic nervous system feature F5. In this case, the model selection unit 207 selects a model that satisfies the conditions of the value indicated by the autonomic nervous system feature F3, the conditions of the value indicated by the autonomic nervous system feature F4, and the conditions of the value indicated by the autonomic nervous system feature F5, based on the table illustrated in Figure 8.

[0078] For example, if the value indicated by autonomic nervous system feature F3 is 0.2, the value indicated by autonomic nervous system feature F4 is 3.5, and the value indicated by autonomic nervous system feature F5 is 15, the model selection unit 207 will use model M1 by referring to Figure 8.

[0079] Furthermore, if, for example, the value indicated by autonomic nervous system feature F3 is 0.8, the value indicated by autonomic nervous system feature F4 is 1.5, and the value indicated by autonomic nervous system feature F5 is 15, the model selection unit 207 will refer to Figure 8 and use model M2.

[0080] Furthermore, let's assume that model M4 is stored in the model memory unit 215, for example. Model M4 is selected when the conditions for autonomic nervous system feature F3, the conditions for autonomic nervous system feature F4, and the conditions for autonomic nervous system feature F5 are not met.

[0081] For example, if the value indicated by autonomic nervous system feature F3 is 3.5, the value indicated by autonomic nervous system feature F4 is 1.5, and the value indicated by feature F5 is 35, then it cannot be associated with any of the models M1 to M3 exemplified in Table 8. Therefore, the model selection unit 207 selects model M4 (not shown), which is not illustrated.

[0082] For example, suppose the value indicated by autonomic nervous system feature F3 is 0.8, the value indicated by autonomic nervous system feature F4 is 2.0, and the value indicated by autonomic nervous system feature F5 is 15. In that case, referring to the table illustrated in Figure 8, models M1 and M2 are linked. If multiple models are linked, a priority order may be set for each of the multiple models. Alternatively, the model selection unit 207 may select the model with the largest calculated value by multiplying the multiple models by weight coefficients and calculating the sum. The information used to select a model is stored in advance in the model storage unit 215. The information used to select a model includes, for example, the priority order set for each of the multiple models, the weight coefficient set for each of the multiple models, etc.

[0083] In this modified example, three models and three autonomic nervous system features are used, but this is not the only option; any number of models or features will suffice.

[0084] (Second embodiment) A second embodiment will now be described. In the drawings, elements that are the same as or similar to those in the first embodiment are denoted by the same reference numerals, and redundant explanations are omitted.

[0085] Figure 9 is a block diagram showing an example of the configuration of the measuring device 900 according to this embodiment. The difference between the measuring device 900 illustrated in Figure 9 and the measuring device 100 illustrated in Figure 2 is that it lacks the model selection unit 207 and, instead of the biometric information calculation unit 206 and model storage unit 215, it is equipped with the biometric information calculation unit 906 and model storage unit 915 shown in Figure 9.

[0086] In this embodiment, the model storage unit 915 has a model pre-stored that has been machine-trained using at least one of several types of autonomic nerve feature quantities 216a and feature quantity 216. In this embodiment, the biological information calculation unit 906 calculates biological information from feature quantity 216 and autonomic nerve feature quantity 216a using the model stored in the model storage unit 915.

[0087] In this embodiment, model selection is not performed, and the accuracy of calculating biological information is improved by using a machine learning model that utilizes the autonomic nervous system features 216a.

[0088] In machine learning, it is believed that features not explicitly stated as features can be inferred and captured from other features if the model learns them well. However, explicitly specifying the features that you want to focus on learning is effective for better learning. Therefore, the measurement device 900 according to this embodiment explicitly learns the autonomic nervous system features 216a during model creation to improve the accuracy of calculating biological information.

[0089] For example, the learning device generates training data for multiple learning organisms in a state of active autonomic nervous system activity and multiple learning organisms in a state of inactive autonomic nervous system activity. This data consists of autonomic nervous system features calculated from volume pulse waves, features calculated from those volume pulse waves, and biological information indicating the correct values ​​measured under the conditions in which the volume pulse waves were measured. The learning device then uses the training data, taking the autonomic nervous system features and features calculated from volume pulse waves as input, and performs machine learning to generate a model that outputs biological information indicating the correct values. Since the autonomic nervous system features reflect the state of autonomic nervous system activity in the organism, the model can accurately calculate biological information using the autonomic nervous system features. Here, we have explained how to create a model for two states: one with active autonomic nervous system activity and one with inactive autonomic nervous system activity. However, the data that the model will target is not necessarily limited to these two types. Also, here we have explained how to create one model as an example, but it is not necessary to create only one model. For example, if you are measuring a diverse user group with varying autonomic nervous system activity due to differences in age group, whether or not they use antihypertensive drugs, or whether or not they exercise, you may create multiple models.

[0090] Based on the above, the learning device generates a machine learning model using at least one of the multiple types of autonomic nervous system features 216a. The measurement device 100 then stores the model generated by the learning device in the model storage unit 915.

[0091] Figure 10 is a flowchart showing an example of the operation of the measuring device 900 according to this embodiment. Detailed explanations of steps similar to those illustrated in Figure 6 are omitted.

[0092] In step S1001, the imaging unit 101 acquires image 211. In step S1002, the feature calculation unit 205 acquires feature 216 from image 211 acquired in step S1001. In step S1003, the feature calculation unit 205 calculates autonomic nervous system feature 216a. In this flowchart, feature 216 is calculated before the autonomic nervous system feature 216a, but the order is not limited to this; the order can be reversed, or they can be calculated in the same step. The processing in steps S1001 to S1003 is the same as the processing in steps S601 to S603 illustrated in Figure 6, so a detailed explanation is omitted.

[0093] In step S1004, the biological information calculation unit 906 uses a machine learning model that utilizes at least one of the autonomic nerve feature quantities 216a stored in the model storage unit 915 to calculate biological information from the feature quantity 216 and the autonomic nerve feature quantity 216a. The processing in steps S1005 to S1006 is the same as the processing in steps S606 to S607 illustrated in Figure 6, so it is omitted.

[0094] The processes performed in the above embodiments are not limited to the processing modes exemplified in each embodiment. The functional blocks described above may be implemented using either logic circuits (hardware) formed on an integrated circuit or software using a CPU. The processes performed in the above embodiments may be executed on multiple computers. For example, some of the processes performed on the measuring device 100 may be performed on other computers, or all of the processes may be shared and executed on multiple computers.

[0095] Furthermore, each functional block or feature of the apparatus used in the embodiments described above may be implemented or executed by an electrical circuit, such as an integrated circuit or a combination of integrated circuits. An electrical circuit designed to perform the functions described herein may include a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or a combination thereof. The general-purpose processor may be a microprocessor, a conventional processor, controller, microcontroller, or state machine. The aforementioned electrical circuit may consist of digital circuits or analog circuits. Also, if advances in semiconductor technology lead to the emergence of integrated circuit technologies that replace current integrated circuits, one or more aspects of this disclosure may use new integrated circuits based on such technologies.

[0096] This disclosure is not limited to the embodiments described above, and may be replaced with configurations substantially identical to those shown in the embodiments, configurations that produce the same effects, or configurations that can achieve the same objectives. This disclosure also includes embodiments obtained by appropriately combining the technical means disclosed in different embodiments. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment. [Explanation of Symbols]

[0097] 100 Measurement device, 101 Imaging unit, 102 Biological data, 201 Input unit, 202 Output unit, 203 Storage unit, 204 Control unit, 205 Feature calculation unit, 206 Biological information calculation unit, 207 Model selection unit, 211 Image, 212 Report, 213 Measurement information storage unit, 214 Biological information storage unit, 215 Model storage unit, 216 Feature quantity, 216a Autonomic nervous system feature quantity, 217 Judgment result, 900 Measurement device, 903 Storage unit, 906 Biological information calculation unit, 915 Model storage unit

Claims

1. An imaging unit that captures images of living organisms and acquires images, A feature calculation unit that calculates the aforementioned biological feature quantities and the autonomic nerve feature quantities related to the autonomic nerves of the biological organism from the image, A storage unit that stores a model for calculating biological information relating to at least one of the blood vessels of the living organism and the blood of the living organism, A biological information calculation unit that calculates the biological information using at least one of the aforementioned feature quantities and the aforementioned autonomic nervous system feature quantities and the aforementioned model, A measuring device equipped with the following features.

2. The measuring device according to claim 1, wherein the feature calculation unit calculates a biological signal relating to the biological organism from the image, and calculates the feature quantity and the autonomic nervous system feature quantity from the biological signal.

3. The aforementioned biological signal shows a volume pulse wave. The measuring device according to claim 2, wherein the feature calculation unit calculates the autonomic nerve feature from at least one of the waveform of the volume pulse wave, the result of frequency analysis of the volume pulse wave, and an image showing the volume pulse wave.

4. The image showing the volume pulse wave includes at least one of the following: an image of the volume pulse wave waveform, a spectrogram calculated from the volume pulse wave, a scalogram, and a correlogram. The measuring device according to claim 3, wherein the feature calculation unit calculates the autonomic nerve feature from a pattern shown by at least one of the waveform image of the volume pulse wave, the spectrogram, the scalogram, and the correlogram.

5. Model Selection Section Furthermore, The memory unit stores multiple types of machine learning models. The measuring device according to claim 1 or 2, wherein the model selection unit selects a model for calculating the biological information from among the multiple types of machine-learned models based on the autonomic nerve features.

6. The feature calculation unit calculates a plurality of autonomic nerve features from the image, The model selection unit selects a model from a plurality of models based on an area identified from a plurality of autonomic nervous system features. The measuring device according to claim 5.

7. The model selection unit selects a model from a plurality of models based on conditions that are satisfied by the values ​​indicated by the autonomic nervous system features. The measuring device according to claim 5.

8. The measuring device according to claim 1 or 2, wherein the model is a machine learning model that uses at least one of the multiple types of autonomic nervous system features.

9. By imaging living organisms, we acquire images. The aforementioned biological features and autonomic nerve features related to the autonomic nerves of the biological organism are calculated from the image. A model for calculating biological information relating to at least one of the blood vessels and blood of the said living organism is read out. The biological information is calculated using at least one of the aforementioned features and the aforementioned autonomic nervous system features, along with the aforementioned model. Measurement method.

10. On the computer, The function of imaging living organisms and acquiring images, A function to calculate the aforementioned biological features and the autonomic nervous system features related to the autonomic nervous system of the biological organism from the image, A function to read a model for calculating biological information relating to at least one of the blood vessels and blood of the said living organism, A function to calculate the biological information using at least one of the aforementioned feature quantities and the aforementioned autonomic nervous system feature quantities, and the aforementioned model. A program that executes the command.

Images