Device, method and computer program for measuring muscle oxygen consumption

Abstract

The measuring device includes: a light source; a sensor including a plurality of photoelectric conversion elements; and a processing circuit. The processing circuit acquires a plurality of detection signals by performing a plurality of detection operations, wherein the detection operation includes: causing the light source to emit a light pulse; and causing the sensor to detect at least a portion of an internally scattered component, which is a component scattered inside the object part in a reflected light pulse generated from the object part due to the emission of the light pulse, and outputting a detection signal indicating a spatial distribution of the intensity of the at least a portion of the internally scattered component, wherein the plurality of detection signals include the detection signal. The processing circuit generates and outputs distribution data indicating a spatial distribution of muscle oxygen consumption in the object part based on the plurality of detection signals.

Description

Technical Field

[0001] The present invention relates to a device and a method for measuring muscle oxygen consumption. Background Art

[0002] In recent years, the demand for fitness clubs has been increasing around the world, including in Japan. According to a report by the International Health, Racquet & Sportsclub Association, the country with the largest number of fitness club members is the United States, with approximately 54 million members (approximately 17.4% of the population). Next is Germany, the United Kingdom, Brazil, and Japan, with 4.16 million members (3.3% of the population). In addition, the demand for functional recovery training for the elderly or rehabilitation training for the purpose of exerting potential is also increasing. In these muscle training and rehabilitation training, it is expected to improve the training effect by visualizing the training effect and providing feedback to the training menu.

[0003] On the other hand, vertebrates including humans have a blood circulation system. It is known that the blood circulation system significantly increases the oxygen transport capacity in the body, corresponding to the oxygen demand of cells headed by muscles. In the measurement of oxygen in the blood in a living body, invasive methods have been used in the past (for example, refer to non-patent document 1). In invasive methods, for example, the insertion of a catheter or an oxygen needle electrode can directly measure the oxygen dynamics in blood vessels and cells. However, due to its invasiveness, it is difficult to apply such a method to muscle training or rehabilitation exercises.

[0004] Therefore, in recent years, attempts have been made to use near-infrared spectroscopy (NIRS) for non-invasive measurement of oxygen dynamics. Near-infrared light easily penetrates the living body, and its absorption characteristics are different depending on the oxidation or deoxidation state of hemoglobin. By utilizing this property, it is possible to grasp the oxygen dynamics in the body. For example, Patent Documents 1 and 2 disclose devices that use NIRS to grasp the blood flow state in the muscle. Patent Document 1 discloses a system that analyzes changes in blood flow in the muscle that occur when the subject performs muscle load exercise. Patent Document 2 discloses an exercise monitoring device that detects the fatigue state of the muscle to indicate the optimal exercise load.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: International Publication No. 2020 / 004020

[0008] Patent Document 2: Japanese Patent Application Laid-Open No. 6-142087

[0009] Non-patent literature

[0010] Non-patent document 1: Gayeski, TEJ, Honig CR "Direct measurements of intracellular O2 gradients; role of convection and myoglobin", 1983, Adv Med Biol. 159, 613-621 Summary of the invention

[0011] Problems to be solved by the invention

[0012] The present disclosure provides a technology for obtaining information on the two-dimensional distribution of muscle oxygen consumption by a simple method.

[0013] Means used to solve problems

[0014] A measuring device according to a technical solution of the present disclosure measures muscle oxygen consumption in an object part of a user who performs muscle exercise. The measuring device comprises: a light source; a sensor including a plurality of photoelectric conversion elements; and a processing circuit. The processing circuit acquires a plurality of detection signals by performing a plurality of detection actions, wherein the detection actions include: causing the light source to emit a light pulse; and causing the sensor to detect at least a portion of the internal scattered component, which is a component scattered inside the object part in a reflected light pulse generated from the object part due to the emission of the light pulse, and outputting a detection signal representing the spatial distribution of the intensity of the at least a portion of the internal scattered component, wherein the plurality of detection signals include the detection signal. Based on the plurality of detection signals, the processing circuit generates and outputs distribution data representing the spatial distribution of muscle oxygen consumption in the object part.

[0015] The inclusive or specific form of the present disclosure may also be implemented by a system, device, method, integrated circuit, computer program or a recording medium such as a computer-readable recording disk, or may be implemented by any combination of a system, device, method, integrated circuit, computer program and recording medium. Computer-readable recording media may include, for example, non-volatile recording media such as CD-ROM (Compact Disc-Read Only Memory). A device may also be composed of more than one device. In the case where a device is composed of more than two devices, the two or more devices may be configured in one device or separately configured in two or more separate devices. In this specification and claims, "device" may refer not only to one device, but also to a system composed of multiple devices.

[0016] Effects of the Invention

[0017] According to a technical solution disclosed in the present invention, information on the two-dimensional distribution of muscle oxygen consumption can be obtained through a simple method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a diagram showing the configuration of a measurement system according to an exemplary embodiment.

[0019] Figure 2 FIG. 1 is a diagram showing an example of temporal change in the intensity of light reaching the image sensor.

[0020] Figure 3 This is a diagram showing an example of the relationship between the width of an input light pulse and the amount of light detected by an image sensor.

[0021] Figure 4 This is a diagram showing an example of a schematic structure of one pixel of an image sensor.

[0022] Figure 5 A diagram showing an example of the structure of an image sensor.

[0023] Fig. 6A This is a diagram showing an example of a detection operation based on light of two wavelengths.

[0024] Figure 6B FIG. 1 is a diagram showing another example of the detection operation using light of two wavelengths.

[0025] Figure 7 It is a diagram for explaining the waveform of the light intensity of the reflected light pulse.

[0026] Fig. 8A : is a timing chart showing an example of the operation of detecting the internal scattered component.

[0027] Figure 8B This is a timing chart showing an example of the operation of detecting the surface reflection component.

[0028] Fig. 9 This is a flowchart showing an example of the operation of controlling the light source and the image sensor by the control circuit.

[0029] Fig.10 This is a diagram schematically showing an example of temporal changes in muscle blood flow.

[0030] Fig.11 This is a diagram schematically showing an example in which a plurality of locations in a target part are measured simultaneously.

[0031] Fig.12 It is a diagram schematically showing an example of an irradiation area of ​​light.

[0032] Fig.13 It is a diagram schematically showing changes in the signal when the target portion is displaced in the lateral direction.

[0033] Fig.14This is a diagram showing the flow of one measurement operation of muscle oxygen consumption.

[0034] Fig.15 This is a flowchart showing an example of the operation of determining the training effect by the measuring device.

[0035] Fig.16 It is a diagram showing the status of an experiment for evaluating the training effect.

[0036] Fig.17 is a diagram showing the experimental scheme.

[0037] Fig.18 The diagram shows an example of an infrared image acquired by a NIRS camera.

[0038] Fig.19 This is a graph showing the temporal changes in the amount of hemoglobin change measured before training.

[0039] Fig. 20 This is a graph showing the temporal changes in the amount of hemoglobin change measured after training.

[0040] Fig.21 This is a diagram showing the analysis results of the behavior of Oxy-Hb before training when the analysis period is 60 seconds.

[0041] Fig. 22 This is a diagram showing the analysis results of the behavior of Oxy-Hb after training when the analysis period is 60 seconds.

[0042] Fig.23 This is a diagram showing the analysis results of the behavior of Oxy-Hb before training when the analysis period is 30 seconds.

[0043] Fig.24 This is a diagram showing the analysis results of the behavior of Oxy-Hb after training when the analysis period is 30 seconds. DETAILED DESCRIPTION

[0044] The embodiments described below all represent inclusive or specific examples. The numerical values, shapes, materials, constituent elements, configuration positions and connection forms of constituent elements, steps, the order of steps, etc. shown in the following embodiments are examples and are not intended to limit the technology disclosed herein. Among the constituent elements of the following embodiments, constituent elements that are not recorded in the independent claims representing the highest concept are described as arbitrary constituent elements. Each figure is a schematic diagram and is not necessarily a strict illustration. Furthermore, in each figure, the same reference numerals are given to substantially the same or similar structures. There are cases where repeated descriptions are omitted or simplified.

[0045] In the present disclosure, all or part of a circuit, unit, device, component or part, or all or part of a functional block in a block diagram may be performed by one or more electronic circuits including a semiconductor device, a semiconductor integrated circuit (IC) or an LSI (large scale integration). An LSI or an IC may be integrated into one chip or may be composed by combining multiple chips. For example, functional blocks other than storage elements may be integrated into one chip. Here, it is referred to as LSI or IC, but the name may change depending on the degree of integration, and may also be referred to as system LSI, VLSI (very large scale integration) or ULSI (ultra large scale integration). A Field Programmable Gate Array (FPGA) that can be programmed after the manufacture of the LSI, or a reconfigurable logic device that can reconfigure the bonding relationship within the LSI or set the circuit division within the LSI may also be used for the same purpose.

[0046] Furthermore, all or part of the functions or operations of a circuit, unit, device, component or part can be executed by software processing. In this case, the software is recorded in one or more non-transitory recording media such as ROMs, optical disks, hard disk drives, etc. When the software is executed by a processing device (processor), the functions determined by the software are executed by the processing device and peripheral devices. The system or device may also have one or more non-transitory recording media with software recorded, a processing device, and necessary hardware devices, such as interfaces.

[0047] (Understanding that is the basis of the present disclosure)

[0048] Before describing the embodiments of the present disclosure, the knowledge that is the basis of the present disclosure will be described.

[0049] In recent years, studies have been conducted to evaluate the effects of muscle exercise by measuring the blood flow dynamics of human skeletal muscle at rest and during exercise using near-infrared spectroscopy (NIRS). Blood flow dynamics measurement based on NIRS is based on the high tissue transmittance of near-infrared light and the wavelength dependence of the absorption coefficient of hemoglobin.

[0050] Near-infrared light has a relatively low scattering coefficient and absorption coefficient in the body, and the attenuation of light is small, so it can easily pass through the body. Due to this high tissue transmittance, near-infrared light is suitable for measuring blood flow dynamics in the body.

[0051] The major role of blood is to receive oxygen from the lungs and transport it to the tissues, and to receive carbon dioxide from the tissues and circulate it to the lungs. There is about 15g of hemoglobin in 100ml of blood. Hemoglobin bound to oxygen is called oxyhemoglobin, and hemoglobin not bound to oxygen is called deoxyhemoglobin. The absorption coefficients of these hemoglobins are wavelength-dependent. The absorbance of oxyhemoglobin and deoxyhemoglobin is equal at wavelengths around 805nm. At wavelengths shorter than 805nm, the absorption by deoxyhemoglobin increases, and at wavelengths longer than 805nm, the absorption by oxyhemoglobin increases. Using such characteristics, the concentrations of oxyhemoglobin and deoxyhemoglobin can be measured or estimated.

[0052] In the previous measurement of human skeletal muscle blood flow using NIRS, the probe of the NIRS device is installed on the skin of the object. Near-infrared light is irradiated from the light-transmitting part of the probe, and the light is absorbed and scattered when passing through the skin, subcutaneous fat and muscle, and then detected by the light-receiving part. Since it is difficult to determine the optical path length of the light detected by NIRS, the absolute value of the hemoglobin amount is not measured, but the relative value is measured. According to the intensity of the detected near-infrared light, the change in the concentration of each of oxygenated hemoglobin and deoxyhemoglobin in the tissue relative to the reference value can be calculated. In the following description, there are cases where oxygenated hemoglobin or its concentration is expressed as Oxy-Hb, and deoxyhemoglobin or its concentration is expressed as Deoxy-Hb. In addition, there are cases where the sum of Oxy-Hb and Deoxy-Hb is expressed as total hemoglobin or Total-Hb.

[0053] If muscles are active through exercise, the metabolism of muscle cells increases. Oxygen is required for the metabolism of muscle cells. Therefore, if muscles are active through exercise, oxygen consumption increases due to metabolism. The consumed oxygen is compensated by the oxygen supply in the blood. Therefore, the change in hemoglobin measured by NIRS reflects the balance between oxygen consumption and supply in the cells. In the early stage of exercise, Oxy-Hb decreases because oxygen consumption exceeds oxygen supply. If the balance between oxygen supply and consumption is maintained during exercise, Oxy-Hb becomes constant. After exercise, Oxy-Hb increases because oxygen supply exceeds oxygen consumption. In this way, based on the blood flow dynamics, especially the dynamics of Oxy-Hb, the oxygen consumption in active muscle cells (referred to as "muscle oxygen consumption" in this manual) can be measured or inferred.

[0054] However, there are some problems in the conventional method of measuring muscle oxygen consumption using NIRS.

[0055] The first problem is that in the previous methods, only the muscle oxygen consumption at a certain point of the target part is measured. Since the oxygen consumption in the muscle varies according to the part, it is not enough to only measure the muscle oxygen consumption at a certain point of a certain muscle in order to evaluate the effect of muscle training or rehabilitation exercise. In such a measurement method, it is difficult to determine the menu for the next effective training based on the measurement results. In order to solve this problem, a method of wearing sensors on multiple parts of the body can also be considered. However, it is not realistic to wear many sensors for measurement before, during and after exercise.

[0056] The second problem is that in the previous methods, the measurement results also reflect the behavior of skin blood flow, making it difficult to measure muscle oxygen consumption with high accuracy. In the previous methods, the probe of the NIRS device is installed on the skin of the target part. Near-infrared light is irradiated from the light-transmitting part of the probe, and the light is absorbed and scattered when passing through the skin, subcutaneous fat and muscle, and then detected by the light-receiving part. Therefore, the behavior of skin blood flow is also included in the signal representing the intensity of the detected light. It is also possible to consider using the distance data between the light-transmitting part and the light-receiving part to reduce the influence of skin blood flow, but for this purpose, more light-receiving parts are required to obtain data at one point. In this case, the measurement of oxygen consumption in multiple parts becomes more difficult.

[0057] The third problem is that the probe of the NIRS device needs to be attached to the skin of the subject. During and after exercise, sweat will occur. Therefore, if the device is attached to the skin for a long time, it is not only unhygienic, but also difficult to measure with high accuracy due to the influence of sweat.

[0058] The fourth problem is that it is difficult to compare the blood flow dynamics before and after exercise. In order to prevent sweat from sticking to the body and hindering exercise, a method of removing the NIRS device during exercise and measuring and comparing the blood flow dynamics before and after exercise can be considered. In this case, the NIRS device needs to be worn in the same position before and after exercise. However, it is difficult to perform the alignment correctly because the measurement accuracy is easily reduced due to the inconsistency of the wearing position.

[0059] If the NIRS device can be used to measure the dynamic changes in blood flow caused by exercise and visualize the effects of muscle exercise, more effective training can be expected. However, in the previous methods of using NIRS devices, there are various problems as mentioned above. Therefore, although there are reports on the research level, the use of NIRS devices to evaluate the effects of muscle exercise has not yet been popularized. In particular, there is no practical method for visualizing the effects of muscle exercise.

[0060] Therefore, the inventors of the present disclosure have studied a more effective method for visualizing the effect of muscle exercise and have come up with the configuration of the embodiment of the present disclosure. According to the embodiment of the present disclosure, information on the two-dimensional distribution of muscle oxygen consumption can be effectively obtained without bringing the measuring device into contact with the user's skin. Therefore, the effect of muscle exercise can be evaluated with higher accuracy.

[0061] Hereinafter, an outline of the embodiments of the present disclosure will be described.

[0062] The measuring device of the exemplary embodiment of the present disclosure measures the muscle oxygen consumption in the subject part of the user who performs muscle exercise. The measuring device comprises a light source, a sensor including a plurality of photoelectric conversion elements, and a processing circuit. The processing circuit acquires a plurality of detection signals including the following detection signals by performing the detection operation including the following steps (a) and (b) multiple times.

[0063] (a) The light source is caused to emit light pulses.

[0064] (b) causing the sensor to detect at least a portion of an internally scattered component, which is a component scattered inside the object portion in a reflected light pulse generated from the object portion due to emission of the light pulse, and output a detection signal representing a spatial distribution of the intensity of the at least a portion of the internally scattered component.

[0065] The processing circuit generates and outputs distribution data indicating spatial distribution of muscle oxygen consumption in the target part based on the plurality of detection signals.

[0066] Here, "muscle oxygen consumption" refers to the amount of oxygen consumed by active muscle cells. "Internal scattering component" may be a component of the reflected light pulse after the intensity of the reflected light pulse begins to decrease.

[0067] According to the above configuration, it is possible to obtain information on the two-dimensional distribution of muscle oxygen consumption without bringing the measuring device into contact with the user's skin, thereby enabling the effect of muscle exercise to be evaluated with high accuracy for each part.

[0068] The processing circuit may generate image data representing the spatial distribution of the muscle oxygen consumption in the target part in different colors according to the level of the muscle oxygen consumption as the distribution data. By generating such image data, the user can easily understand the effect of muscle exercise.

[0069] The processing circuit may also generate the following image data as the distribution data, wherein the image data represents an image in which the spatial distribution information of the muscle oxygen consumption is superimposed on an appearance image representing the appearance of the user including the object portion obtained by the sensor or other device. By generating such image data, the user can more easily grasp the effect of muscle exercise.

[0070] The processing circuit may also estimate the concentration distribution of oxyhemoglobin in the blood inside the target part based on the plurality of detection signals, and estimate the spatial distribution of the muscle oxygen consumption based on the temporal change of the concentration distribution of oxyhemoglobin. By such an operation, the spatial distribution of the muscle oxygen consumption generated by muscle movement can be estimated more accurately.

[0071] The processing circuit may estimate the muscle oxygen consumption based on the slope of the time change of the concentration of the oxyhemoglobin. By such an operation, the spatial distribution of the muscle oxygen consumption generated by the muscle movement can be estimated more accurately.

[0072] The measuring device may further include a pressurizer. The processing circuit may also perform the detection operation in a state where a part of the user's body is pressurized by the pressurizer and blood flow in the target part is suppressed. By pressurizing, muscle oxygen consumption can be estimated more accurately.

[0073] The pressurizer may also be controlled by the processing circuit. The processing circuit may also cause the pressurizer to start applying pressure to the part of the user's body before performing the detection action, and cause the pressurizer to end the pressurization after performing the detection action. By such actions, the start of pressurization, the detection action, and the end of pressurization can be automated, thereby improving convenience.

[0074] The processing circuit may also perform the detection operation a plurality of times in each of a first period before the user performs the muscle exercise and a second period after the user performs the muscle exercise, and generate the distribution data based on the plurality of detection signals obtained in the first period and the plurality of detection signals obtained in the second period. Through such an operation, the spatial distribution of muscle oxygen consumption caused by muscle exercise can be more accurately estimated.

[0075] The processing circuit may also obtain data of an appearance image representing the appearance of the user including the object portion, the appearance image being obtained by the sensor or other device during each of the first period and the second period, detect a change in the position of the object portion occurring between the first period and the second period by matching one or more feature points included in the appearance image obtained during the first period with one or more feature points included in the appearance image obtained during the second period, and generate the distribution data after compensating for the change in the position for the plurality of detection signals obtained during the first period and the plurality of detection signals obtained during the second period. By such an operation, a good measurement result can be obtained even if the position of the object portion changes between the first period and the second period.

[0076] The processing circuit may also generate first blood flow data indicating changes over time in the concentration distribution of oxygenated hemoglobin in the blood inside the object part based on the plurality of detection signals obtained during the first period, generate second blood flow data indicating changes over time in the concentration distribution of oxygenated hemoglobin in the blood inside the object part based on the plurality of detection signals obtained during the second period, and generate the distribution data based on the first blood flow data and the second blood flow data. Changes over time in the concentration of oxygenated hemoglobin depend on muscle oxygen consumption caused by muscle movement. Therefore, by comparing changes over time in the concentration distribution of oxygenated hemoglobin before and after exercise, the spatial distribution of muscle oxygen consumption can be inferred with high accuracy.

[0077] The processing circuit may determine a first change rate based on the first blood flow data, the first change rate indicating a decreasing slope of the time change of the concentration of the oxyhemoglobin at the plurality of points included in the object portion; determine a second change rate based on the second blood flow data, the second change rate indicating a decreasing slope of the time change of the concentration of the oxyhemoglobin at the plurality of points; and estimate the muscle oxygen consumption at the plurality of points based on a difference or ratio between the first change rate and the second change rate. By such an operation, the spatial distribution of the muscle oxygen consumption can be estimated more accurately.

[0078] The detection actions during the first period and the second period may be performed in a state where a part of the user's body is pressurized and the blood flow of the target part is suppressed. The processing circuit may also fit the time change of the concentration of the oxyhemoglobin during the prescribed period after the concentration of the oxyhemoglobin ends increasing to a function during the first period, and determine the first change rate based on the time change rate of the function, and fit the time change of the concentration of the oxyhemoglobin during the prescribed period after the concentration of the oxyhemoglobin ends increasing to the function during the second period, and determine the second change rate based on the time change rate of the function. By such actions, the spatial distribution of muscle oxygen consumption can be more accurately estimated.

[0079] The processing circuit may also add information indicating that the muscle oxygen consumption is relatively large to the distribution data and output it when the magnitude of the second change rate is a times or more (a is a real number greater than 1) the magnitude of the first change rate, and add information indicating that the muscle oxygen consumption is relatively small to the distribution data and output it when the magnitude of the second change rate is less than a times the magnitude of the first change rate. By such an operation, for example, a display device that has obtained the distribution data can display an image indicating at which point the muscle oxygen consumption is relatively large.

[0080] The processing circuit may determine a first region in the target part where the muscle oxygen consumption is relatively large based on the information indicating that the muscle oxygen consumption is relatively large, and determine a second region in the target part where the muscle oxygen consumption is relatively small based on the information indicating that the muscle oxygen consumption is relatively small, and the distribution data may include an image that highlights the first region or the second region. By displaying such an image, the user can easily grasp the effect of muscle exercise at each point of the target part.

[0081] The processing circuit may also generate and output data representing a training plan, wherein the training plan is a training plan for training muscles in an area of ​​the target portion where the oxygen consumption of the muscles is relatively low. Based on such data, the user can know a training plan for effectively training a muscle portion where the effect of muscle movement is low.

[0082] The processing circuit may also obtain history data indicating the content of the muscle exercise performed by the user, and adjust the training plan based on the history data. Through such an action, a more appropriate training plan can be presented to the user.

[0083] The processing circuit may also obtain identification data for identifying the user, and adjust the training plan based on the identification data, thereby providing an optimal training plan for each user.

[0084] The light source may also be configured to emit a first light pulse having a first wavelength of 650 nm or more and less than 805 nm, and a second light pulse having a second wavelength of 805 nm or more and 950 nm or less. The detection operation may also include: causing the light source to emit the first light pulse; causing the sensor to detect at least a portion of a first internally scattered component, which is a component scattered in the interior of the object portion, of a first reflected light pulse generated from the object portion due to the emission of the first light pulse, and outputting a first detection signal indicating a spatial distribution of the intensity of the at least a portion of the first internally scattered component; causing the light source to emit the second light pulse; and causing the sensor to detect at least a portion of a second internally scattered component, which is a component scattered in the interior of the object portion, of a second reflected light pulse generated from the object portion due to the emission of the second light pulse, and outputting a second detection signal indicating a spatial distribution of the intensity of the at least a portion of the second internally scattered component. The processing circuit may also estimate the concentration distribution of oxyhemoglobin in the blood inside the target part based on the first detection signal and the second detection signal, and estimate the spatial distribution of the muscle oxygen consumption based on the time change of the concentration distribution of the oxyhemoglobin. By such an operation, the spatial distribution of the muscle oxygen consumption can be estimated more accurately.

[0085] The measuring device may also include augmented reality (AR) glasses including a transparent display. The transparent display may also display the distribution image in a manner that the distribution image representing the distribution data is superimposed on the appearance of the user identified via the transparent display. Thus, the user can easily understand which part has a higher training effect.

[0086] A method according to another embodiment of the present disclosure is executed by a computer included in a measuring device for measuring muscle oxygen consumption in an object part of a user performing muscle exercise. The method includes the following processing: obtaining multiple detection signals by performing multiple detection actions, the detection actions including: causing a light source to emit a light pulse; and causing a sensor including multiple photoelectric conversion elements to detect at least a portion of the internal scattered component, which is a component scattered inside the object part in a reflected light pulse generated from the object part due to the emission of the light pulse, and outputting a detection signal representing the spatial distribution of the intensity of the at least a portion of the internal scattered component, the multiple detection signals including the detection signal; and generating and outputting distribution data representing the spatial distribution of muscle oxygen consumption in the object part based on the multiple detection signals.

[0087] A computer program according to another embodiment of the present disclosure is executed by a computer included in a measuring device for measuring muscle oxygen consumption in an object part of a user performing muscle exercise. The program causes the computer to perform the following processing: obtaining multiple detection signals by performing multiple detection actions, the detection actions including: causing a light source to emit a light pulse; and causing a sensor including multiple photoelectric conversion elements to detect at least a portion of the internal scattered components, which are components scattered inside the object part in a reflected light pulse generated from the object part due to the emission of the light pulse, and outputting a detection signal representing the spatial distribution of the intensity of the at least a portion of the internal scattered components, the multiple detection signals including the detection signal; and generating and outputting distribution data representing the spatial distribution of muscle oxygen consumption in the object part based on the multiple detection signals.

[0088] In another embodiment of the present disclosure, a non-temporary computer-readable recording medium records a computer program, which, when executed by a processor, causes the processor to perform processing, the processing comprising: obtaining multiple detection signals by performing multiple detection actions, the detection actions comprising: causing a light source to emit a light pulse; and causing a sensor including multiple photoelectric conversion elements to detect at least a portion of a component scattered inside the object part, i.e., an internal scattering component, in a reflected light pulse generated from the object part due to the emission of the light pulse, and output a detection signal representing a spatial distribution of the intensity of the at least a portion of the internal scattering component, the multiple detection signals including the detection signal; and generating and outputting distribution data representing the spatial distribution of muscle oxygen consumption in the object part based on the multiple detection signals.

[0089] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the drawings.

[0090] (Implementation Method)

[0091] [1. Composition]

[0092] Figure 1 1 is a diagram schematically showing the structure of a system for measuring muscle oxygen consumption according to an exemplary embodiment of the present disclosure. The system includes a measuring device 100, a pressurizer 40, a display 50, and AR glasses 90. The measuring device 100 includes a light source 20, an image sensor 30, a control circuit 60, a signal processing circuit 70, and a storage medium 80 such as a memory. Figure 1 2 also shows an arm of a user who uses the measuring device 100. In the present embodiment, the control circuit 60 and the signal processing circuit 70 function as the above-mentioned "processing circuit".

[0093] The pressurizer 40 is a device worn by the user, which temporarily cuts off the blood flow of a part of the body by applying pressure to a part of the user's body. In this embodiment, the pressurizer 40 is worn on the user's arm. The position where the pressurizer 40 is worn varies depending on the position of the target part. For example, when measuring the muscle oxygen consumption of the leg, the pressurizer 40 can be worn on the leg. In addition, when measuring the muscle oxygen consumption of the trunk, the pressurizer 40 can be worn on the trunk.

[0094] The light source 20 is a device that emits a light pulse to illuminate the target part of the user. The light source 20 includes one or more light emitting elements, and emits a light pulse toward the target part. The light source 20 can be configured to emit two light pulses with different wavelengths. For example, the light source 20 can be configured to emit a first light pulse having a first wavelength of 650 nm or more and less than 805 nm, and a second light pulse having a second wavelength of 805 nm or more and less than 950 nm.

[0095] The image sensor 30 detects at least a portion of the light pulses returning from the user's body and outputs a detection signal indicating the detection result. The image sensor 30 includes a plurality of light detection units arranged two-dimensionally. Each light detection unit includes a photoelectric conversion element and outputs an electrical signal corresponding to the amount of light received. In addition, although the image sensor 30 is used in this embodiment, other types of sensors including a plurality of photoelectric conversion elements arranged two-dimensionally may also be used.

[0096] The control circuit 60 controls the pressurizer 40 , the light source 20 , and the image sensor 30 . The control circuit 60 includes a pressurization control unit 64 that controls the pressurizer 40 , a light source control unit 62 that controls the light source, and a sensor control unit 63 that controls the image sensor 30 .

[0097] The pressurization control unit 64 controls the operation of the pressurizer 40. For example, it controls one or both of the pressure and the pressurization timing.

[0098] The light source control unit 62 controls the light emission operation of the light source 20 . For example, the light source control unit 62 controls at least one of the intensity, pulse width, emission timing, and wavelength of the light pulse emitted from the light source 20 .

[0099] The sensor control unit 63 controls the timing of signal accumulation in each light detection unit of the image sensor 30 .

[0100] The pressurization control unit 64, the light source control unit 62, and the sensor control unit 63 may be implemented by three separate circuits or by a single circuit. The pressurization control unit 64, the light source control unit 62, and the sensor control unit 63 may also be implemented by the control circuit 60 executing a control program stored in a memory (not shown).

[0101] The signal processing circuit 70 processes the detection signal output from the image sensor 30. The signal processing circuit 70 generates information indicating the blood flow state in the target part of the user (hereinafter referred to as "blood flow information") based on the detection signal. The blood flow information may include, for example, at least one information selected from the group consisting of blood flow rate, oxygenated hemoglobin concentration in blood, deoxygenated hemoglobin concentration in blood, total hemoglobin concentration which is the sum of oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration, and blood oxygen saturation. The signal processing circuit 70 generates and outputs distribution data indicating a two-dimensional distribution of muscle oxygen consumption based on the temporal change of the blood flow information in the target part.

[0102] In the present embodiment, the control circuit 60 and the signal processing circuit 70 are separated, but the functions of these circuits may be realized by a single electronic circuit.

[0103] The display 50 displays an image based on the distribution data generated by the signal processing circuit 70. For example, an image is displayed in which information on the spatial distribution of muscle oxygen consumption is superimposed on an image showing the user's appearance. The display 50 may be any display such as a liquid crystal or an organic EL. The display 50 may also be built into the measuring device 100.

[0104] The AR glasses 90 include a transparent display. The AR glasses 90 can cause the transparent display to display an image based on the distribution data generated by the signal processing circuit 70. For example, the AR glasses 90 causes the transparent display to display an image based on the distribution data in a manner superimposed on the appearance of the user recognized through the transparent display.

[0105] Hereinafter, each component will be described in more detail.

[0106] [1-1. Pressurizer 40]

[0107] The pressurizer 40 applies pressure to a part of the user's body. By applying pressure, the blood flow of a part of the user's body is cut off for a certain period of time. The pressurizer 40 may include a cuff, for example. The pressurization is performed, for example, with the cuff wrapped around the base of the arm or leg. The pressure is set to a specified value such as 40 mmHg or 200 mmHg. By applying pressure, the blood flow of the vein or artery is temporarily cut off. The pressurization time can be set to a predetermined time such as 1 minute. The pressurizer 40 can be connected to the control circuit 60 by wire or wirelessly. The pressurizer 40 operates according to the control signal input from the control circuit 60. The pressurizer 40 does not necessarily have to be connected to the control circuit 60, and can also be manual.

[0108] [1-2. Light source 20]

[0109] The light source 20 includes one or more light emitting elements. The light emitting element may include, for example, a laser diode that emits laser light. The light source 20 emits light pulses according to a control signal input from the light source control unit 62 of the control circuit 60 .

[0110] The light source 20 is configured to emit light toward a target part of the user. The target part is, for example, an arm or a leg of the user, more specifically, an upper arm, a forearm, a thigh, or a calf of the user. The target part is not limited to these parts, and may be, for example, the abdomen, the chest, the lower back, the upper back, or the shoulder.

[0111] The light emitted from the light source 20 and reaching the object part of the user is divided into a component reflected on the surface of the object part and a component scattered inside the object part. In this specification, the component reflected on the surface is referred to as the "surface reflection component", and the component scattered inside is referred to as the "internal scattering component". The internal scattering component is a component that is reflected or scattered once or multiple times inside the biological body. In the case of emitting light toward the arm or leg of the user, the internal scattering component reaches a position about 8mm to 16mm from the skin surface of the arm or leg to the inside. That is, the internal scattering component is a component of light that passes through the skin and subcutaneous fat and reaches the muscle, where it is scattered and returns to the measuring device 100 again. The surface reflection component includes three components: a direct reflection component, a diffuse reflection component, and a diffuse reflection component. The direct reflection component is a reflection component where the incident angle is equal to the reflection angle. The diffuse reflection component is a component that is diffused and reflected by the concave-convex shape of the surface. The diffuse reflection component is a component that is scattered and reflected by the internal tissue near the surface. In the case of emitting light toward the arm or leg of the user, the scattered reflection component is a component that is scattered and reflected inside the epidermis of the arm or leg. In the following, it is assumed that the surface reflection component reflected on the surface of the object part includes these three components. In addition, it is assumed that the internal scattering component does not include a component that is scattered and reflected by the internal tissue near the surface. Regarding the surface reflection component and the internal scattering component, the traveling direction of these components changes due to being reflected or scattered, and a part of them reaches the image sensor 30. The surface reflection component may include surface information of the object part, such as blood flow information of the epidermis. The internal scattering component may include internal information of the object part, such as blood flow information near muscle tissue. Therefore, by detecting the surface reflection component, it is possible to obtain surface information of the object part, such as blood flow information of the epidermis. In addition, by detecting the internal scattering component, it is possible to obtain internal information of the object part, such as blood flow information near muscle tissue.

[0112] In this embodiment, at least a portion of the internal scattered components of the light (collectively referred to as "reflected light" in this specification) returning from the target part such as the arm or leg of the user is detected. The internal scattered components reflect the activity of the user's muscle tissue and their intensity changes. Therefore, by analyzing the change of the internal scattered components over time, the state of the user's muscle activity can be inferred.

[0113] Hereinafter, an example of a method for detecting an internal scattered component will be described. The light source 20 repeatedly emits light pulses at a predetermined time interval or a predetermined timing according to an instruction from the control circuit 60. The light pulse emitted from the light source 20 may be, for example, a rectangular wave whose falling period is close to zero. In this specification, the "falling period" refers to the period from when the intensity of the light pulse starts to decrease to when the decrease ends. Usually, the light incident on the object part of the user propagates in the object part along various paths and is emitted from the object part with a time difference. Therefore, the rear end of the internal scattered component of the light pulse has a widening. In the case where the object part is an arm or a leg, the widening of the rear end of the internal scattered component is about 4ns. If this situation is taken into consideration, the length of the falling period of the light pulse emitted by the light source 20 can be set to, for example, less than 2ns, which is less than half of it. The length of the falling period can also be less than 1ns, which is another half of it. The length of the rising period of the light pulse emitted from the light source 20 is arbitrary. In this specification, the "rising period" is the period from when the intensity of the light pulse starts to increase to when the increase ends. In the detection of the internal scattered component of this embodiment, the falling part of the light pulse is used, and the rising part is not used. The rising part of the light pulse can be used to detect the surface reflection component. The light source 20 can include a laser element such as LD. The light emitted from the laser element has a steep time response characteristic in which the falling part of the light pulse is substantially at right angles to the time axis.

[0114] The light source 20 may emit light of a single wavelength or light of multiple wavelengths. It is also possible to emit light of multiple wavelengths from multiple light emitting elements. In the case where two different wavelengths of light are emitted from two light emitting elements, it is possible to design so that the optical path lengths of the two wavelengths of light returning to the image sensor 30 via the object portion of the user are approximately equal. For example, the distance between the image sensor 30 and the light emitting element on one side and the distance between the image sensor 30 and the light emitting element on the other side are the same, and the two light emitting elements are arranged at positions rotationally symmetrical with the image sensor 30 as the center.

[0115] The wavelength of the light emitted from the light source 20 can be, for example, any wavelength included in the wavelength range of more than 650nm and less than 950nm. This wavelength range is included in the wavelength range from red to near infrared. The above wavelength range is called the "biological window", and has the property that light is relatively difficult to be absorbed by water and skin in a living body. In the case of a living body as a measurement object, the detection sensitivity can be improved by using light in the above wavelength range. In the case of detecting changes in the user's blood flow as in the present embodiment, the light used is mainly absorbed by oxyhemoglobin and deoxyhemoglobin. Generally, if the blood flow changes, the concentrations of oxyhemoglobin and deoxyhemoglobin change. With this change, the degree of absorption of light also changes. Therefore, if the blood flow changes, the amount of light detected also changes over time.

[0116] The wavelength dependence of light absorption is different between oxygenated hemoglobin and deoxygenated hemoglobin. When the wavelength is 650 nm or more and shorter than 805 nm, the light absorption coefficient of deoxygenated hemoglobin is larger than that of oxygenated hemoglobin. At a wavelength of 805 nm, the light absorption coefficient of deoxygenated hemoglobin is about the same as that of oxygenated hemoglobin. When the wavelength is longer than 805 nm and less than 950 nm, the light absorption coefficient of oxygenated hemoglobin is larger than that of deoxygenated hemoglobin.

[0117] Therefore, the light source 20 is configured to emit a first light pulse having a wavelength of 650 nm or more and less than 805 nm, and a second light pulse having a wavelength of 805 nm or more and 950 nm or less. By irradiating the target portion with the first light pulse and the second light pulse having different wavelengths, the concentration of oxyhemoglobin and the concentration of deoxyhemoglobin contained in the blood inside the target portion can be estimated. In this way, by irradiating with two light pulses having different wavelengths, more detailed internal information of the target portion can be obtained.

[0118] In the present embodiment, the blood flow inside the muscles of the user is measured contactlessly. Considering the possibility that the light emitted from the light source 20 enters the eyes of the user, a light source 20 designed in consideration of the effect on the retina can be used. For example, a light source 20 that satisfies Level 1 of the laser safety standards established by various countries can be used. When Level 1 is satisfied, low-illuminance light with an achievable emission limit (AEL) of less than 1 mW is irradiated on the user. In addition, the light source 20 itself may not satisfy Level 1. For example, Level 1 of the laser safety standard may be satisfied by placing a diffuser plate or an ND filter in front of the light source 20 to diffuse or attenuate the light.

[0119] When the user's object part is irradiated with light in order to measure the blood flow near the muscle tissue, the light amount of the internal scattered component may become a very small value of about one thousandth to one tens of thousandsth of the light amount of the surface reflected component. Furthermore, if the safety standard of the laser is taken into consideration, the light amount of the light that can be irradiated becomes very small. Therefore, it is very difficult to detect the internal scattered component. In this case, as long as the light source 20 emits a light pulse with a relatively large pulse width, the cumulative amount of the internal scattered component accompanied by the time delay can be increased. As a result, the detection light amount can be increased, and the SN ratio can be improved.

[0120] The light source 20 emits a light pulse having a pulse width of, for example, 3 ns or more. Generally, the temporal spread of light scattered in a living tissue such as muscle is about 4 ns. Figure 2 is a diagram showing an example of a temporal change in the intensity of light reaching the image sensor 30. Figure 2 3 examples are shown in which the width of the light pulse emitted from the light source 20 is 0 ns, 3 ns, and 10 ns. Figure 2 As shown, as the width of the light pulse emitted from the light source 20 increases, the amount of light of the internal scattered component appearing at the rear end of the light pulse reaching the image sensor 30 from the user increases.

[0121] Figure 3 The horizontal axis of the graph shows the width of an input light pulse, which is a light pulse emitted from the light source 20, and the vertical axis shows the amount of light detected by the image sensor 30. The image sensor 30 includes an electronic shutter. Figure 3 The results are obtained under the condition that the electronic shutter is opened 1 ns after the rear end of the light pulse is reflected on the surface of the object and reaches the image sensor 30. The reason for selecting this condition is that the ratio of the surface reflection component is higher than the internal scattering component immediately after the rear end of the light pulse arrives. Figure 3 As shown, if the pulse width of the light pulse emitted from the light source 20 is set to 3 ns or more, the amount of detection light can be maximized.

[0122] The light source 20 may also emit a light pulse with a pulse width of more than 5 ns, and further more than 10 ns. On the other hand, if the pulse width is too large, it will also lead to an increase in unused light and become a waste. Therefore, the light source 20, for example, emits a light pulse with a pulse width of less than 50 ns. Alternatively, the light source may also emit a light pulse with a pulse width of less than 30 ns, and further less than 20 ns. If the pulse width of the rectangular pulse is several ns to several tens of ns, the light source 20 can be driven with a low voltage. Therefore, the miniaturization and low cost of the measuring device 100 can be achieved.

[0123] The irradiation pattern of the light source 20 may be, for example, a pattern having a uniform intensity distribution in the irradiation area. In this respect, the present embodiment is different from the conventional method. In the conventional method, the surface reflection component is spatially separated from the internal scattering component by separating the detector and the light source by about 3 cm. Therefore, discrete light irradiation has to be performed. In contrast, the measuring device 100 of the present embodiment can separate and reduce the surface reflection component from the internal scattering component in time. Therefore, a light source 20 having an irradiation pattern with a uniform intensity distribution can be used. An irradiation pattern with a uniform intensity distribution can also be formed by diffusing the light emitted from the light source 20 with a diffuser.

[0124] In this embodiment, the internal scattered component can be detected even immediately below the irradiation point of the target part. By irradiating the target part with light in a spatially wide range, the measurement resolution can be improved.

[0125] [1-3. Image sensor 30]

[0126] The image sensor 30 may be any imaging element such as a CCD image sensor or a CMOS image sensor. The image sensor 30 includes a plurality of light detection units two-dimensionally arranged on a light receiving surface. Each light detection unit may include, for example, a photoelectric conversion element such as a photodiode, and one or more charge accumulation units. The photoelectric conversion element generates a signal charge corresponding to the amount of light received by photoelectric conversion. The charge accumulation unit accumulates the signal charge generated from the photoelectric conversion element. The image sensor 30 is capable of acquiring two-dimensional information of the user at one time. In the following description, the light detection unit may be referred to as a "pixel".

[0127] The image sensor 30 may include an electronic shutter. The electronic shutter is a circuit that controls the timing of imaging. In the present embodiment, the sensor control unit 63 of the control circuit 60 has the function of an electronic shutter. The electronic shutter controls the period of signal accumulation in which the received light is converted into an effective electrical signal and accumulated, and the period in which the signal accumulation is stopped. The signal accumulation period is also referred to as an "exposure period". In the following description, the width of the exposure period is also referred to as a "shutter width". The time from the end of one exposure period to the beginning of the next exposure period is also referred to as a "non-exposure period". In the following, there are cases where the state of exposure is referred to as "OPEN", and the state of stopped exposure is referred to as "CLOSE".

[0128] The image sensor 30 can adjust the exposure period and the non-exposure period in the sub-nanosecond range, for example, 30ps to 1ns, through an electronic shutter. A TOF camera for the purpose of distance measurement detects all the light emitted from a light source and reflected by a subject and returned. Therefore, in a TOF camera, the shutter width needs to be larger than the pulse width of the light. In contrast, in the measuring device 100 of the present embodiment, the shutter width does not need to be larger than the pulse width. For example, the shutter width can be set to a value greater than 1ns and less than 30ns. According to the measuring device 100 of the present embodiment, since the shutter width can be reduced, the influence of the dark current contained in the detection signal can be reduced.

[0129] When the target part such as the user's arm or leg is irradiated with light to detect information such as blood flow, the attenuation rate of light inside the biological body is very large. For example, relative to the incident light, the emitted light may be attenuated to about 1 in 1,000,000. Therefore, in order to detect the internal scattered components, there may be a situation where the amount of light is insufficient by irradiation with only one pulse. In irradiation at level 1 of the laser safety standard, the amount of light is particularly weak. In this case, the light source 20 emits light pulses multiple times, and correspondingly, the image sensor 30 is also exposed multiple times through the electronic shutter, thereby accumulating the detection signal and improving the sensitivity.

[0130] Hereinafter, an example in which each pixel of the image sensor 30 includes a photoelectric conversion element such as a photodiode and a plurality of charge accumulation units is described. The plurality of charge accumulation units of each pixel may include a charge accumulation unit for accumulating signal charges generated by the surface reflection component of the light pulse and a charge accumulation unit for accumulating signal charges generated by the internal scattering component of the light pulse. The control circuit 60 causes the image sensor 30 to detect the surface reflection component by detecting the portion before the start of the decline in the reflected light pulse returned from the object part of the user. The control circuit 60 also causes the image sensor 30 to detect the portion after the start of the decline in the light pulse returned from the object part of the user. The light source 20 in this example emits light of two wavelengths.

[0131] Figure 4 2 is a diagram showing an example of a schematic configuration of one pixel 201 of the image sensor 30. Figure 4 The schematic diagram does not necessarily reflect the actual structure of a pixel 201. The pixel 201 in this example includes a photodiode 203 for photoelectric conversion, a first floating diffusion layer (Floating Diffusion: FD) 204, a second floating diffusion layer 205, a third floating diffusion layer 206, and a fourth floating diffusion layer 207 as charge accumulation parts, and a drain 202 for discharging signal charges.

[0132] The photons incident on each pixel due to the emission of one light pulse are converted into signal electrons as signal charges by the photodiode 203. The converted signal electrons are discharged to the drain 202 or distributed to any one of the first floating diffusion layer 204 to the fourth floating diffusion layer 207 according to the control signal input from the control circuit 60 to the image sensor 30.

[0133] The emission of pulse light from the light source 20, the accumulation of signal charges in the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207, and the discharge of signal charges to the drain 202 are repeated in this order. This repetitive action is high-speed, for example, it can be repeated tens of thousands to hundreds of millions of times within the time of one frame of a moving image. The time of one frame can be, for example, about 1 / 30 second. The pixel 201 finally generates and outputs four image signals based on the signal charges accumulated in the first floating diffusion layer 204 to the fourth floating diffusion layer 207.

[0134] The control circuit 60 in this example causes the light source 20 to repeatedly emit a first light pulse having a first wavelength and a second light pulse having a second wavelength in sequence. By selecting two wavelengths having different absorbances in the internal tissue of the target part as the first wavelength and the second wavelength, the internal state of the target part can be analyzed. For example, a wavelength longer than 805 nm can be selected as the first wavelength, and a wavelength shorter than 805 nm can be selected as the second wavelength. Thus, changes in the concentration of oxygenated hemoglobin and the concentration of deoxygenated hemoglobin in the blood can be detected.

[0135] The control circuit 60 first causes the light source 20 to emit a first light pulse. The control circuit 60 causes the first floating diffusion layer 204 to accumulate signal charges during a first period in which the surface reflection component of the first light pulse is incident on the photodiode 203. Next, the control circuit 60 causes the second floating diffusion layer 205 to accumulate signal charges during a second period in which the internal scattered component of the first light pulse is incident on the photodiode 203. Next, the control circuit 60 causes the light source 20 to emit a second light pulse. The control circuit 60 causes the third floating diffusion layer 206 to accumulate signal charges during a third period in which the surface reflection component of the second light pulse is incident on the photodiode 203. Next, the control circuit 60 causes the fourth floating diffusion layer 207 to accumulate signal charges during a fourth period in which the internal scattered component of the second light pulse is incident on the photodiode 203.

[0136] In this way, the control circuit 60 causes the first floating diffusion layer 204 and the second floating diffusion layer 205 to sequentially accumulate the signal charges from the photodiode 203 at a predetermined time difference after the start of emission of the first light pulse. Then, the control circuit 60 causes the third floating diffusion layer 206 and the fourth floating diffusion layer 207 to sequentially accumulate the signal charges from the photodiode 203 at a predetermined time difference after the start of emission of the second light pulse. The above operation is repeated multiple times.

[0137] In order to estimate the amount of disturbance light and ambient light, a period during which signal charges are accumulated in other floating diffusion layers (not shown) may be set in a state where the light source 20 is turned off. By subtracting the signal charge amount of the other floating diffusion layers from the signal charge amount of the first floating diffusion layer 204 to the fourth floating diffusion layer 207, a signal from which disturbance light and ambient light components have been removed can be obtained.

[0138] In addition, in the present embodiment, the number of charge storage units of each pixel is 4, but it can be designed to be any number more than 1 depending on the purpose. For example, when only one wavelength is used to detect the surface reflection component and the internal scattering component, the number of charge storage units can also be 2. In addition, when one wavelength is used and the surface reflection component is not detected, the number of charge storage units of each pixel can also be 1. In the case of using two wavelengths to detect only the internal scattering component, the number of charge storage units of each pixel can also be 2. In addition, even when more than two wavelengths are used, if the video recording using each wavelength is performed in different frames, the number of charge storage units can also be 1. Similarly, even when both the surface reflection component and the internal scattering component are detected, the number of charge storage units can also be 1 in a configuration in which both are detected using different frames.

[0139] Figure 5 is a diagram showing an example of the structure of the image sensor 30. Figure 4 In FIG. 1 , the area surrounded by the double-dashed line frame corresponds to one pixel 201. One photodiode is included in the pixel 201. Figure 5 Only four pixels arranged in two rows and two columns are shown, but more pixels may be configured in practice. The pixel 201 includes a first floating diffusion layer 204 to a fourth floating diffusion layer 207. The signals accumulated in the first floating diffusion layer 204 to the fourth floating diffusion layer 207 are processed like the signals of four pixels of a normal CMOS image sensor and output from the image sensor 30.

[0140] Each pixel 201 has four signal detection circuits. Each signal detection circuit includes a source follower transistor 309, a row selection transistor 308, and a reset transistor 310. In this example, the reset transistor 310 is connected to the Figure 4Corresponding to the drain 202 shown, a drain discharge pulse is input into the gate of the reset transistor 310. Each transistor is, for example, a field effect transistor formed on a semiconductor substrate, but is not limited to this. As shown in the figure, one of the input terminal and the output terminal of the source follower transistor 309 is connected to one of the input terminal and the output terminal of the row selection transistor 308. The above-mentioned one of the input terminal and the output terminal of the source follower transistor 309 is typically a source. The above-mentioned one of the input terminal and the output terminal of the row selection transistor 308 is typically a drain. The gate of the source follower transistor 309, which serves as a control terminal, is connected to the photodiode 203. The signal charge of holes or electrons generated by the photodiode 203 is accumulated in the floating diffusion layer, which serves as a charge accumulation part between the photodiode 203 and the source follower transistor 309.

[0141] Although in Figure 5 Although not shown, the first to fourth floating diffusion layers 204 to 207 are connected to the photodiode 203. One or more switches may be provided between the photodiode 203 and each of the first to fourth floating diffusion layers 204 to 207. The switch switches the conduction state between the photodiode 203 and each of the first to fourth floating diffusion layers 204 to 207 according to the signal accumulation pulse from the control circuit 60. Thus, the start and stop of the accumulation of signal charges to each of the first to fourth floating diffusion layers 204 to 207 are controlled. The electronic shutter of this embodiment has a mechanism for such exposure control.

[0142] The signal charges accumulated in the first floating diffusion layer 204 to the fourth floating diffusion layer 207 are read out by turning on the gate of the row selection transistor 308 by the row selection circuit 302. At this time, the current flowing from the source follower power supply 305 to the source follower transistor 309 and the source follower load 306 is amplified according to the signal potential of the first floating diffusion layer 204 to the fourth floating diffusion layer 207. The analog signal based on the current read out from the vertical signal line 304 is converted into digital signal data by the analog-to-digital (AD) conversion circuit 307 connected to each column. The digital signal data is read out by each column by the column selection circuit 303 and output from the image sensor 30. After reading out one row, the row selection circuit 302 and the column selection circuit 303 read out the next row, and the information of the signal charges of the floating diffusion layers of all rows is read out in the same manner. After reading out all the signal charges, the control circuit 60 resets all the floating diffusion layers by turning on the gate of the reset transistor 310. Thus, the imaging of one frame is completed. Similarly, by repeating the high-speed imaging of frames, the imaging of a series of frames by the image sensor 30 is completed.

[0143] Fig. 6Ais a diagram showing an example of an action within one frame. Fig. 6A As shown, the emission of the first light pulse and the emission of the second light pulse may be switched alternately multiple times within one frame. In this way, the time difference in the acquisition timing of the detection signals based on the two wavelengths can be reduced, and even when the user is moving, the first and second light pulses can be photographed at about the same time.

[0144] Figure 6B FIG. 2 is a diagram showing another example of a detection operation based on light of two wavelengths. Figure 6B As shown, the detection of the reflected light pulse of the first light pulse and the detection of the reflected light pulse of the second light pulse may be switched for each frame. Such an operation may be performed, for example, by switching the emission of the first light pulse and the emission of the second light pulse for each frame. In this case, each pixel 201 may also have a single charge accumulation unit. According to such a configuration, the number of charge accumulation units of each pixel 201 can be reduced, so the size of each pixel 201 can be increased, and the sensitivity can be improved.

[0145] In addition, the wavelength of the light emitted by the light source 20 may be one kind. In this case, the general state of muscle activity can be estimated.

[0146] In this embodiment, an example of a CMOS type image sensor 30 is described, but the image sensor 30 may be another type of imaging element. The image sensor 30 may be, for example, a CCD type, a single photon counting type element, or an amplified image sensor such as an EMCCD or ICCD. In addition, instead of the image sensor 30 having a plurality of light detection units arranged two-dimensionally, a plurality of sensors each having a single photoelectric conversion element may be used. Even in a configuration in which single-pixel sensors are arranged two-dimensionally, two-dimensional data of the object portion can be generated.

[0147] In this embodiment, the image sensor 30 can detect the surface reflection component and / or the internal scattering component of the light pulse. Based on the temporal or spatial changes of the surface reflection component, the first biological information such as the pulse of the user can be obtained. On the other hand, based on the temporal or spatial changes of the internal scattering component, the second biological information such as the muscle oxygen consumption of the user can be obtained.

[0148] The first biological information may be obtained by a method different from the method of detecting the surface reflection component, or may not be obtained at all. For example, the first biological information may be obtained using a detector of another type different from the image sensor 30. In this case, the image sensor 30 may detect only the internal scattering component. Other types of detectors may be, for example, radar or thermal cameras. The first biological information may be, for example, at least one selected from the group consisting of the user's pulse, sweating, breathing, and body temperature. The first biological information is biological information other than muscle oxygen consumption information obtained by detecting the internal scattering component of the light pulse irradiated on the object part of the user. Here, "biological information other than muscle oxygen consumption information" does not mean that it does not include information caused by muscle oxygen consumption activity at all. The first biological information may include biological information caused by biological activities different from muscle oxygen consumption activity. The first biological information may be, for example, biological information caused by autonomous or reflexive biological activities.

[0149] [1-4. Control Circuit 60 and Signal Processing Circuit 70]

[0150] The control circuit 60 adjusts the time difference between the emission timing of the light pulse of the light source 20 and the shutter timing of the image sensor 30. In this specification, this time difference is sometimes referred to as a "phase difference". The "emission timing" of the light source 20 is the timing when the light pulse emitted from the light source 20 starts to rise. The "shutter timing" is the timing when exposure starts. The control circuit 60 can adjust the phase difference by changing the emission timing or by changing the shutter timing.

[0151] The control circuit 60 may be, for example, a combination of a processor and a memory, or an integrated circuit such as a microcontroller having a built-in processor and a memory. The control circuit 60 adjusts, for example, the emission timing and the shutter timing by, for example, executing a program recorded in the memory by the processor.

[0152] The signal processing circuit 70 is a circuit that processes the detection signal output from the image sensor 30. The signal processing circuit 70 performs arithmetic processing such as image processing. The signal processing circuit 70 can be implemented by, for example, a programmable logic device (PLD) such as a digital signal processor (DSP) or a field programmable gate array (FPGA), or a combination of a central processing unit (CPU) or a graphics processing unit (GPU) and a computer program.

[0153] The control circuit 60 and the signal processing circuit 70 may be a combined circuit or separate circuits. The signal processing circuit 70 may also be a component of an external device such as a server located remotely. In this case, the external device such as the server transmits and receives data with the light source 20, the image sensor 30, and the control circuit 60 through wireless communication or wired communication.

[0154] The signal processing circuit 70 of this embodiment can generate motion image data representing the time changes of the concentrations of oxygenated hemoglobin (Oxy-Hb), deoxyhemoglobin (Deoxy-Hb), and total hemoglobin (Total-Hb) in the blood inside the object part based on the detection signal output from the image sensor 30. The signal processing circuit 70 is not limited to such motion image data, and other information can also be generated. For example, biological information such as muscle oxygen consumption or blood oxygen saturation can also be generated by synchronization with other equipment.

[0155] The signal processing circuit 70 can also estimate and remove the offset component caused by the disturbance light. The offset component is a signal component caused by the disturbance light such as sunlight or fluorescent light. By detecting the signal with the image sensor 30 in a state where the drive of the light source 20 is turned off and no light is emitted from the light source 20, the offset component caused by the ambient light or the disturbance light can be estimated.

[0156] [1-5. Others]

[0157] The measuring device 100 may also include an imaging optical system that forms a two-dimensional image of the user on the light receiving surface of the image sensor 30. The optical axis of the imaging optical system is substantially orthogonal to the light receiving surface of the image sensor 30. The imaging optical system may also include a zoom lens. If the position of the zoom lens changes, the magnification of the two-dimensional image of the user changes, and the resolution of the two-dimensional image on the image sensor 30 changes. Therefore, even if the distance to the user is far, the desired measurement area can be enlarged and observed in detail.

[0158] The measuring device 100 may also include a bandpass filter between the user and the image sensor 30 that allows only light of the wavelength band emitted from the light source 20 or light in the vicinity thereof to pass. This can reduce the influence of interference components such as ambient light. The bandpass filter may be composed of, for example, a multilayer film filter or an absorption filter. Considering the temperature change of the light source 20 and the shift of the wavelength band due to oblique incidence to the filter, the bandwidth of the bandpass filter may also have a width of, for example, about 20 to 100 nm.

[0159] The measuring device 100 may also include polarizing plates between the light source 20 and the user, and between the image sensor 30 and the user. In this case, the polarization directions of the polarizing plate disposed on the light source 20 side and the polarizing plate disposed on the image sensor 30 side may be in a cross-Nichol relationship. Thus, it is possible to prevent the regular reflection component of the surface reflection component of the user, that is, the component with the same incident angle and reflection angle, from reaching the image sensor 30. That is, it is possible to reduce the amount of light reaching the image sensor 30 from the surface reflection component.

[0160] [2.Signal detection action]

[0161] Next, the signal detection operation performed by the measuring device 100 will be described in more detail.

[0162] The measuring device 100 of this embodiment can detect the surface reflection component and the internal scattering component in the reflected light pulse from the object part by distinguishing them. In the case where the object part is an arm or a leg, the signal intensity of the internal scattering component to be detected becomes very small. This is because, as mentioned above, in addition to the very small amount of light that meets the laser safety standard, the scattering and absorption of light by the skin and subcutaneous fat are relatively large. Furthermore, the change in signal intensity caused by the change in blood flow or blood components during muscle activity is only equivalent to a few tens of tenths, which is very small. Therefore, in the case of detecting the internal scattering component, the surface reflection component with a size of several thousand to tens of thousands times the signal component of the internal scattering component to be detected is removed as much as possible.

[0163] An example of the operation of detecting the internal scattered component by the measuring device 100 will be described below.

[0164] As described above, when the light source 20 irradiates the object part of the user with a light pulse, a surface reflection component and an internal scattering component are generated. A part of each of the surface reflection component and the internal scattering component reaches the image sensor 30. The internal scattering component passes through the inside of the object part before being emitted from the light source 20 and reaching the image sensor 30. Therefore, the optical path length of the internal scattering component is longer than the optical path length of the surface reflection component. Therefore, the timing at which the internal scattering component reaches the image sensor 30 is, on average, later than the timing at which the surface reflection component reaches the image sensor 30.

[0165] Figure 7 2 is a diagram schematically showing the waveform of the light intensity of the reflected light pulse returned from the object part of the user when a rectangular wave light pulse is emitted from the light source 20. The horizontal axis represents time (t). The vertical axis represents Figure 7 Parts (a) to (c) indicate the intensity, and part (d) indicates the open (OPEN) or closed (CLOSE) state of the electronic shutter. Figure 7 Part (a) represents the surface reflection component I1. Figure 7Part (b) represents the internal scattering component I2. Figure 7 Part (c) represents the total component of the surface reflection component I1 and the internal scattering component I2. Figure 7 As shown in part (a), the waveform of the surface reflection component I1 is roughly rectangular. On the other hand, the internal scattering component I2 is the sum of light with various optical path lengths. Figure 7 As shown in part (b), the internal scattered component I2 exhibits a characteristic similar to the tailing of the light pulse. In other words, the falling period of the internal scattered component I2 is longer than the falling period of the surface reflected component I1. Figure 7 The internal scattered component I2 is extracted by increasing the ratio of the internal scattered component I2 in the optical signal shown in part (c). Figure 7 As shown in part (d) of FIG. 1 , the exposure of the electronic shutter starts after the time point when the rear end of the surface reflection component I1 arrives. In other words, the exposure starts when or after the waveform of the surface reflection component I1 falls. The shutter timing is adjusted by the control circuit 60.

[0166] When the target portion is not a plane, the timing at which light arrives differs depending on the pixel of the image sensor 30. In this case, the timing may be determined individually for each pixel. Figure 7 The shutter timing shown in part (d) of the image sensor 30. For example, let the direction perpendicular to the light receiving surface of the image sensor 30 be the z direction. The control circuit 60 may also obtain data representing the two-dimensional distribution of the z coordinate on the surface of the object portion, and change the shutter timing for each pixel based on the data. Thus, even when the surface of the object portion is curved, the optimal shutter timing can be determined at each position. The data representing the two-dimensional distribution of the z coordinate on the surface of the object portion is obtained, for example, by TOF (Time-of-Flight) technology. The time required for the reflected light generated by the irradiated light of the light source 20 reflected by the object portion to reach each pixel is measured. Therefore, based on the difference between the phase of the reflected light detected by each pixel and the phase of the irradiated light of the light source 20, the distance between each pixel and the object portion can be inferred. Thus, data representing the two-dimensional distribution of the z coordinate on the surface of the object portion can be obtained. The data representing the two-dimensional distribution can be obtained in advance before the measurement is performed.

[0167] exist Figure 7In the example shown in part (a) of the illustration, the rear end of the surface reflection component I1 drops vertically. In other words, the time from when the surface reflection component I1 starts to drop to when it stops is zero. However, in reality, there are cases where the rear end of the surface reflection component I1 does not drop vertically. For example, when the drop of the waveform of the light pulse emitted from the light source 20 is not completely vertical, when there are fine irregularities on the surface of the object, or when scattering occurs in the epidermis, the rear end of the surface reflection component I1 does not drop vertically. In addition, since the user is an opaque object, the amount of light of the surface reflection component I1 is very large compared to the amount of light of the internal scattering component I2. Therefore, even when the rear end of the surface reflection component I1 slightly exceeds the time point of vertical drop, the internal scattering component I2 may be masked. Furthermore, there are cases where a time delay accompanying the movement of electrons occurs during the readout period of the electronic shutter. Therefore, there is a possibility that the image quality cannot be achieved. Figure 7 The case of an ideal binary readout as shown in part (d) of FIG. 1 . In this case, the control circuit 60 may also delay the shutter start timing of the electronic shutter slightly after the surface reflection component I1 has just fallen. For example, the control circuit 60 may also delay the shutter start timing of the electronic shutter by about 0.5ns to 5ns compared to the falling point of the surface reflection component I1. Instead of adjusting the shutter timing of the electronic shutter, the control circuit 60 may also adjust the emission timing of the light source 20. In other words, the control circuit 60 may also adjust the time difference between the shutter timing of the electronic shutter and the emission timing of the light source 20. In the case of contactlessly measuring the blood flow or the change in the components in the blood in the object part, if the shutter timing is excessively delayed, the internal scattered component I2, which is originally small, is further reduced. Therefore, the shutter timing may also be stopped near the rear end of the surface reflection component I1. As mentioned above, the time delay caused by the internal scattering of the object part is about 4ns. In this case, the maximum delay amount of the shutter timing can be about 4ns.

[0168] You can also Fig. 6A and Figure 6B As in the example shown, a plurality of light pulses emitted from the light source 20 are exposed at shutter timings with the same time difference, and signals are accumulated. As a result, the detection light amount of the internal scattered component I2 is amplified.

[0169] Instead of or in addition to placing a bandpass filter between the user and the image sensor 30, the offset component may be estimated by performing photography with the same exposure time without emitting light from the light source 20. The estimated offset component is removed by subtracting it from the signal detected by each pixel of the image sensor 30. In this way, the dark current component generated in the image sensor 30 can be removed.

[0170] The internal scattered component I2 contains the user's internal characteristic information, such as muscle oxygen consumption information. According to the temporal changes in the user's muscle oxygen consumption, the amount of light absorbed by the blood changes. As a result, the amount of detection light obtained by the image sensor 30 also increases or decreases accordingly. Therefore, by monitoring the internal scattered component I2, the temporal changes in the concentrations of oxygenated hemoglobin (Oxy-Hb), deoxyhemoglobin (Deoxy-Hb), and total hemoglobin (Total-Hb) in the user's target part of the blood can be inferred. Based on the temporal changes in the concentrations of Oxy-Hb, Deoxy-Hb, and Total-Hb, muscle oxygen consumption can be inferred.

[0171] Fig. 8A 1 is a timing chart showing an example of the operation of monitoring the internal scattered component I2. In this example, the light source 20 repeatedly emits light pulses during one frame. The image sensor 30 sets the shutter to OPEN while the rear end of each reflected light pulse reaches the image sensor 30. Through this operation, the image sensor 30 accumulates the signal of the internal scattered component I2. When the predetermined number of signal accumulations are completed, the image sensor 30 outputs the signal accumulated for each pixel as a detection signal. The output detection signal is processed by the signal processing circuit 70.

[0172] In this way, the control circuit 60 repeatedly performs a detection operation of causing the light source 20 to emit a light pulse, causing the image sensor 30 to monitor at least a portion of the component after the start of the fall in the reflected light pulse, and outputting a detection signal representing the spatial distribution of the intensity of the internal scattered component. Through such an operation, the signal processing circuit 70 can generate and output distribution data representing the spatial distribution of the muscle oxygen consumption of the target part based on the repeatedly output detection signal.

[0173] Next, an example of a method for detecting the surface reflection component I1 is described. The surface reflection component I1 includes information indicating the surface characteristics of the user, such as skin blood flow information of the arms or legs. The information of the surface reflection component I1 does not necessarily need to be obtained, but can be obtained as needed.

[0174] Figure 8B: is a time chart showing an example of the action of detecting the surface reflection component I1. When detecting the surface reflection component I1, the image sensor 30 sets the shutter to OPEN before each reflected light pulse reaches the image sensor 30, and sets the shutter to CLOSE before the rear end of the reflected light pulse arrives. By controlling the shutter in this way, the mixing of the internal scattered component I2 can be suppressed, and the proportion of the surface reflection component I1 can be increased. The timing of the shutter CLOSE can also be set to just after the light reaches the image sensor 30. In this way, signal detection of the proportion of the surface reflection component I1 with a relatively short optical path length can be performed. By obtaining the signal of the surface reflection component I1, the user's pulse or the degree of oxidation of the epidermal blood flow can be inferred. As another method of obtaining the surface reflection component I1, the image sensor 30 can also detect the entire reflected light pulse, or detect the continuous light emitted from the light source 20.

[0175] The surface reflection component I1 may be detected by a device other than the measuring device 100 that obtains the internal scattering component I2. For example, other devices such as a pulse meter or a Doppler blood flow meter may be used. In this case, the other device is used in consideration of timing synchronization between devices, light interference, and coordination of detection sites. If time-sharing imaging is performed using the same measuring device 100 or the same sensor as in the present embodiment, temporal and spatial deviations are less likely to occur. In the case where the signals of both the surface reflection component I1 and the internal scattering component I2 are obtained by the same sensor, it is also possible to Fig. 8A and Figure 8B As shown, the components to be acquired are switched every frame. Alternatively, the components to be acquired may be switched alternately at high speed within one frame. In this case, the detection time difference between the surface reflection component I1 and the internal scattering component I2 can be reduced.

[0176] Furthermore, two wavelengths of light may be used to obtain the signals of the surface reflection component I1 and the internal scattering component I2. For example, two wavelengths of light pulses of 750 nm and 850 nm may be used. Thus, the concentration change of oxyhemoglobin and the concentration change of deoxyhemoglobin can be calculated based on the change of the detected light amount at each wavelength. When the surface reflection component I1 and the internal scattering component I2 are obtained at two wavelengths, for example, as shown in FIG. Figures 4 to 6A As described above, a method of switching four types of charge accumulation at high speed within one frame can be used. This method can reduce the time variation of the detection signal.

[0177] Fig. 91 is a flowchart showing an example of the operation of the control circuit 60 controlling the light source 20 and the image sensor 30. Here, an example of the operation in the case where the light source 20 emits light pulses of two wavelengths and the image sensor 30 detects only the internal scattered component is described. The light source 20 emits a first light pulse having a first wavelength and a second light pulse having a second wavelength. The first wavelength is longer than 650 nm and shorter than 805 nm. The second wavelength is longer than 805 nm and shorter than 950 nm.

[0178] In step S101, the control circuit 60 causes the light source 20 to emit the first light pulse at a predetermined time. At this time, the electronic shutter of the image sensor 30 is in a state of stopping exposure. The control circuit 60 causes the electronic shutter to stop exposing until the surface reflection component in the first reflected light pulse generated by the irradiation of the first light pulse reaches the image sensor 30. Next, in step S102, the control circuit 60 causes the electronic shutter to start exposing when the internal scattered component in the first reflected light pulse reaches the image sensor 30. After the predetermined time has passed, in step S103, the control circuit 60 causes the electronic shutter to stop exposing. Through steps S102 and S103, Figure 5 Signal charges are accumulated in one of the first floating diffusion layer 204 to the fourth floating diffusion layer 207. This signal charge is referred to as "first signal charge".

[0179] In step S104, the control circuit 60 causes the light source 20 to emit the second light pulse at a predetermined time. At this time, the electronic shutter of the image sensor 30 is in a state of stopping exposure. The control circuit 60 causes the electronic shutter to stop exposing until the surface reflection component in the second reflected light pulse generated by the irradiation of the second light pulse reaches the image sensor 30. Next, in step S105, the control circuit 60 causes the electronic shutter to start exposing when the internal scattered component in the second reflected light pulse reaches the image sensor 30. After the predetermined time has passed, in step S106, the control circuit 60 causes the electronic shutter to stop exposing. Through steps S105 and S106, Figure 5 The signal charge is accumulated in the other floating diffusion layer among the first floating diffusion layer 204 to the fourth floating diffusion layer 207. This signal charge is referred to as "second signal charge".

[0180] Next, in step S107, the control circuit 60 determines whether the number of times the above-mentioned signal accumulation has been executed has reached a predetermined number of times. If the determination in step S107 is "No", steps S101 to S106 are repeated until a determination is "Yes".

[0181] If the determination in step S107 is “YES”, the process proceeds to step S108 . In step S108 , the control circuit 60 causes the image sensor 30 to generate and output a first signal and a second signal based on the first signal charge and the second signal charge, respectively.

[0182] In this way, the control circuit 60 performs the first action of causing the light source 20 to emit the first light pulse and causing the image sensor 30 to detect at least a part of the components during the falling period of the first reflected light pulse. The control circuit 60 performs the second action of causing the light source 20 to emit the second light pulse and causing the image sensor 30 to detect at least a part of the components during the falling period of the second reflected light pulse. The control circuit 60 repeats a series of actions including the first action and the second action for a predetermined number of times. Alternatively, the control circuit 60 may repeat the first action for a predetermined number of times and then repeat the second action for a predetermined number of times. In addition, the order of the first action and the second action may be replaced.

[0183] pass Fig. 9 The action shown can detect internal scattered components with high sensitivity. When a living body such as a human is irradiated with light to obtain information such as blood flow, the attenuation rate of light inside the living body is very large. For example, the emitted light is attenuated to about 1 in 1,000,000 relative to the incident light. Therefore, if you want to detect internal scattered components, there may be insufficient light through irradiation with only one pulse. The light intensity is particularly weak in irradiation under Level 1 of the laser safety standard. Therefore, in Fig. 9 In the example, the light source 20 emits light pulses multiple times, and correspondingly, the image sensor 30 is exposed multiple times through the electronic shutter. Through such an action, the detection signal can be accumulated to improve the sensitivity. In addition, multiple light emission and exposure are not necessary, but are performed as needed.

[0184] exist Fig. 9 In the example of detecting the internal scattered component, the surface reflected component may also be detected. When both the surface reflected component and the internal scattered component are detected, a step of accumulating the signal charge of the surface reflected component based on the first reflected light pulse is added between step S101 and step S102, and a step of accumulating the signal charge of the surface reflected component based on the second reflected light pulse is added between step S104 and step S105. These signal charges are accumulated in Figure 5 The remaining two floating diffusion layers are shown as the first floating diffusion layer 204 to the fourth floating diffusion layer 207. By detecting the surface reflection component, information indicating the appearance of the user or the state of skin blood flow, for example, can be obtained.

[0185] Next, an example of the operation of estimating muscle oxygen consumption by the signal processing circuit 70 will be described.

[0186] As mentioned above, oxygenated hemoglobin and deoxygenated hemoglobin have different light absorption characteristics. Oxygenated hemoglobin absorbs near-infrared rays with wavelengths exceeding about 805nm relatively well. In contrast, deoxygenated hemoglobin absorbs near-infrared rays or red light with wavelengths shorter than 805nm relatively well. Regarding near-infrared rays with wavelengths near 805nm, the absorption rates of both are at the same level. Therefore, a first wavelength longer than 650nm and shorter than 805nm, and a second wavelength longer than 805nm and shorter than 950nm can be used. For example, the above-mentioned two wavelengths of 750nm and 850nm can be used. Based on the detected light amounts of these lights, the time changes in the concentrations of oxygenated hemoglobin and deoxygenated hemoglobin in the blood can be detected.

[0187] The signal processing circuit 70 generates and outputs a signal indicating the state of the blood inside the object part based on the first detection signal and the second detection signal output from the image sensor 30. The first detection signal is a signal indicating the detection result of the internal scattered component of the light of the first wavelength (for example, 750nm). The second detection signal is a signal indicating the detection result of the internal scattered component of the light of the second wavelength (for example, 850nm). The signal processing circuit 70 can calculate the change amount of each concentration of oxygenated hemoglobin (HbO2) and deoxygenated hemoglobin (Hb) in the blood relative to the initial value by solving a preset simultaneous equation using the first detection signal and the second detection signal. The simultaneous equations are represented by, for example, the following equations (1) and (2).

[0188] [Formula 1]

[0189]

[0190] [Formula 2]

[0191]

[0192] ΔHbO2 and ΔHb represent the changes in the concentrations of HbO2 and Hb in the blood, respectively, from their initial values. 750 OXY and ε 750 deOXY Respectively represent the molar absorption coefficients of HbO2 and Hb at a wavelength of 750nm. 850 OXY and ε 850 deOXY Respectively represent the molar absorption coefficients of HbO2 and Hb at a wavelength of 850nm. 750 ini and I 750 now The detection intensities at the initial time point and the measurement time point with respect to a wavelength of 750 nm are shown respectively. 850 iniand I 850 now The signal processing circuit 70 can calculate the change amount ΔHbO2 and ΔHb of each concentration of HbO2 and Hb in the blood relative to the initial value for each pixel based on the above-mentioned formulas (1) and (2). In this way, it is possible to generate data of a two-dimensional distribution of the change amount of each concentration of HbO2 and Hb in the blood of the target part.

[0193] The signal processing circuit 70 can also obtain the oxygen saturation of hemoglobin. Oxygen saturation is a value indicating the proportion of hemoglobin in the blood that is bound to oxygen. Let the concentration of deoxyhemoglobin be C (Deoxy-Hb), let the concentration of oxyhemoglobin be C (Oxy-Hb), and the oxygen saturation be defined by the following formula.

[0194] Oxygen saturation = C(Oxy-Hb) / [C(Oxy-Hb)+C(Deoxy-Hb)]×100(%)

[0195] In addition to blood, the body also contains components that absorb red light and near-infrared light. However, the light absorption rate varies over time mainly due to hemoglobin in arterial blood. Therefore, blood oxygen saturation can be measured with high accuracy based on the change in absorption rate.

[0196] Light passes through the skin and subcutaneous tissue and reaches the muscle. Therefore, the blood flow changes in the skin and subcutaneous tissue are also superimposed and detected. In order to remove or reduce its influence, the signal processing circuit 70 can also perform a process of subtracting the surface reflection component I1 from the internal scattering component I2 detected by the image sensor 30. In this way, pure muscle blood flow information can be obtained without the blood flow information of the skin and subcutaneous tissue. In the subtraction method, for example, a method can be used in which a value obtained by multiplying the signal of the surface reflection component I1 by one or more coefficients determined in consideration of the optical path length difference is subtracted from the signal of the internal scattering component I2. This coefficient can be calculated by simulation or experiment based on the average value of the optical constant of a normal human arm or leg. Such a subtraction process can be easily performed when the same measuring device 100 or sensor is used to measure light of the same wavelength. This is because it is easy to reduce the deviation in time and space, and it is easy to make the blood flow components of the skin and subcutaneous tissue contained in the internal scattering component I2 consistent with the characteristics of the surface reflection component I1.

[0197] The two-dimensional distribution of blood flow in the skin and subcutaneous tissue is independent of the two-dimensional distribution of blood flow in the muscles. Therefore, based on the signal output from the image sensor 30, the two-dimensional distribution of the internal scattered component I2 and the two-dimensional distribution of the surface reflected component I1 can be separated using a statistical method such as independent component analysis or principal component analysis.

[0198] [3. Example of detecting changes in muscle blood flow]

[0199] Next, an example of a method of detecting changes in the user's muscle blood flow rate will be described.

[0200] Fig.10 is a diagram schematically showing an example of the temporal changes in muscle blood flow. Fig.10 As shown, the object part 501 of the user 500 is illuminated with light from the light source 20, and the return light is detected. In this case, the surface reflection component is much larger than the internal scattering component. However, by adjusting the shutter as described above, only the internal scattering component can be extracted. Fig.10 The graph shown shows an example of temporal changes in the concentrations of oxyhemoglobin (Oxy-Hb) and deoxyhemoglobin (Deoxy-Hb) and their sum (Total-Hb) in the blood inside the object part 501. The internal scattered component in this example is acquired using light of two wavelengths. Fig.10 The concentration shown indicates a change amount relative to the normal concentration amount, which is calculated by the signal processing circuit 70 based on the light intensity signal.

[0201] The blood flow in the muscle reflects the balance between oxygen consumption and oxygen supply in the muscle tissue. Therefore, compared with resting state, the oxygen concentration decreases at the beginning of exercise because oxygen consumption exceeds oxygen supply. If the balance between oxygen supply and consumption is maintained during exercise, the oxygen concentration becomes constant. After exercise, the oxygen concentration rises because oxygen supply exceeds oxygen consumption. Therefore, in this embodiment, the change of muscle blood flow over time is measured at the same position in the object part 501 of the user 500. When observing the change of muscle activity over time, even if the absolute amount of muscle blood flow is unknown, the state of the user's muscle activity can be inferred based on the relative change of muscle blood flow over time.

[0202] Fig.11 1 is a diagram schematically showing an example of a case where multiple parts in the object part 501 of the user 500 are measured simultaneously. In this example, light is irradiated to a two-dimensionally expanded area, and light scattered inside the two-dimensional area is detected. Therefore, data representing the two-dimensional distribution of the oxidation state of the blood inside the object part 501 can be obtained. In this case, the irradiation pattern of the light source 20 can be, for example, a uniform distribution of uniform intensity, a point distribution, or a ring distribution. When the light source 20 irradiates with a uniform distribution of uniform intensity, it is unnecessary or simple to adjust the irradiation position on the object part 501. If the irradiation is the same distribution, light is incident on the object part 501 of the user 500 from a wide range. Therefore, the signal detected by the image sensor 30 can be enhanced. Furthermore, it is possible to measure any part in the irradiation area.

[0203] Fig.12 1 is a diagram schematically showing an example of the irradiation area 22 of light. In the non-contact biological measurement, the detection light amount decays in inverse proportion to the square of the distance from the measurement device 100 to the object part. Therefore, the signal of each pixel generated by the image sensor 30 can also be enhanced by accumulating the signals of multiple pixels in the vicinity. In this way, the number of accumulated pulses can be reduced while maintaining the SN ratio. As a result, the frame rate can be increased.

[0204] Fig.13 Schematically shows the change of the signal when the object part of the user 500 is displaced in the lateral direction. As described above, the change of muscle activity is read by detecting the change in the concentration of oxygenated hemoglobin or deoxygenated hemoglobin in the blood when the muscle activity state changes from the resting state. When using the image sensor 30 having a plurality of photoelectric conversion elements arranged two-dimensionally, it is possible to obtain information on the two-dimensional distribution of muscle activity. In this embodiment, since the measurement is performed non-contactly, Fig.13 As shown in the figure below, the position of the object part will change during the measurement. This may also occur, for example, when the user 500 moves slightly due to breathing. Usually, the two-dimensional distribution of the concentrations of oxyhemoglobin and deoxyhemoglobin in muscle blood does not change dramatically in a small period of time. Therefore, for example, by pattern matching between frames of the detected two-dimensional distribution, the position deviation of the object part can be corrected. Alternatively, if it is a periodic movement such as breathing, only the frequency component can be extracted and corrected or removed. The object part does not need to be a single area, but can be a plurality of areas. The plurality of areas can also be, for example, 1 on each side, or a 2×6 matrix point distribution.

[0205] In addition, muscle activity can also be inferred based on changes in blood conditions during the two periods. Specifically, measurements can be performed before training, training can be performed, and measurements can be performed again after training. By comparing and analyzing the blood conditions in the muscles before and after training, the training effect can be determined or evaluated. In this case, the changes in oxygenated hemoglobin and the like in the muscle blood at the same point are compared before and after training. When performing such a measurement method, the signal processing circuit 70 extracts one or more feature points from an appearance image representing the appearance of a user including an object portion, detects the position deviation of the object portion based on the relative positional relationship between the feature points and the measurement area or the analysis area, and can reduce or compensate for the influence of the position deviation by correcting the measurement area. Such position deviation correction processing can be performed either during measurement or during analysis after the measurement is completed.

[0206] The image of the object part including the user may be, for example, an infrared image generated based on a light pulse emitted from the light source 20 and reflected by the object part. The image may be generated based on a component including at least a part of the front-end component or the rear-end component of the light pulse reaching the image sensor 30. In addition, the image of the object part may be a visible image obtained by a camera or a sensor provided independently of the image sensor 30 provided in the measuring device 100.

[0207] As a feature point, for example, a mole, the shape of the object part, the front end of the fingertip, the shape of the nail or blood vessel, etc. can be used, but it is not limited to these. In addition, a mark as a feature point can also be given to the object part. The signal processing circuit 70 can detect the change in the position of the object part before and after the movement by using one or more of these feature points for pattern matching. The signal processing circuit 70 can generate distribution data representing the spatial distribution of muscle oxygen consumption after compensating for the detected change in position.

[0208] [4. Determination of training effect]

[0209] Next, a specific example of a method of determining the training effect of the user 500 using the measurement device 100 will be described.

[0210] As described above, based on the detection result of the internal scattered component of the reflected light, it is possible to infer the state of the blood around the muscle tissue of the user 500. Using this characteristic, it is possible to measure the muscle oxygen consumption of the user 500. Based on the measurement result of the muscle oxygen consumption, it is possible to determine the effect of muscle exercise such as muscle training or rehabilitation training.

[0211] In the present embodiment, as an example of a method for measuring muscle oxygen consumption of the user 500, cuff pressurization by the pressurizer 40 is used to cut off blood flow. The light source 20 repeatedly emits light pulses at a certain time interval toward the arm or leg of the user 500. The image sensor 30 accumulates signals of internal scattered components of the reflected light pulses for each pixel. In the present embodiment, the same operation is performed in each period of rest before the start of cuff pressurization, during cuff pressurization, and after the cuff pressurization is released.

[0212] Fig.14 This is a diagram showing the flow of one measurement operation of muscle oxygen consumption in this embodiment on the time axis. One measurement operation in this embodiment includes three periods: a pre-pressurization period, a pressurization period, and a post-pressurization period. The pre-pressurization period is the period before the cuff pressurization begins. The pressurization period is the period during the cuff pressurization. The post-pressurization period is the period immediately after the cuff pressurization is released.

[0213] During these three periods, the measuring device 100 repeatedly generates detection signals indicating the user's muscle activity. During the pressurization period, the user applies pressurization to a predetermined part of the arm or leg, etc., using a cuff. The magnitude of the pressurization is set to a predetermined value, such as 40 mmHg or 200 mmHg. Fig.14 In the example of , the length of each of the pre-pressurization period and the post-pressurization period is 30 seconds, and the length of the pressurization period is 120 seconds. This example is not limited to the example, and the length of each period can be adjusted appropriately. The length of the pre-pressurization period and the post-pressurization period can also be different.

[0214] Fig.15 1 is a flowchart showing an example of a determination action of a training effect performed by the measuring device 100. The control circuit 60 of the present embodiment repeatedly performs the above-mentioned signal detection action in each of the first period before the user performs muscle exercise and the second period after the user performs muscle exercise. The signal processing circuit 70 generates distribution data representing the spatial distribution of muscle oxygen consumption based on the detection signal repeatedly output from the image sensor 30 in the first period and the detection signal repeatedly output from the image sensor 30 in the second period. More specifically, the signal processing circuit 70 generates first blood flow data representing the temporal change of the concentration distribution of oxygenated hemoglobin in the blood inside the object part based on the detection signal repeatedly output in the first period. In addition, the second blood flow data representing the temporal change of the concentration distribution of oxygenated hemoglobin in the blood inside the object part is generated based on the detection signal repeatedly output in the second period. The signal processing circuit 70 generates distribution data representing the two-dimensional distribution of muscle oxygen consumption based on the first blood flow data and the second blood flow data. For example, the signal processing circuit 70 determines a first change rate indicating a slope of a decrease in a time change in the concentration of oxyhemoglobin at a plurality of points included in the target portion based on the first blood flow data. In addition, the signal processing circuit 70 determines a second change rate indicating a slope of a decrease in a time change in the concentration of oxyhemoglobin at a plurality of points included in the target portion based on the second blood flow data. The muscle oxygen consumption at each point of the target portion can be estimated based on the difference or ratio between the first change rate and the second change rate.

[0215] exist Fig.15In the example of , if the user performs an operation to start measurement, the measuring device 100 starts the measuring action (step S701). In this stage, the pressurization of a part of the user's body by the cuff of the pressurizer 40 is not performed. In a state where the user does nothing, the detection signal is repeatedly obtained for a predetermined time (for example, 30 seconds). This is performed to obtain the blood flow data of the user at rest. This period is the pre-pressurization period mentioned above. As described above, the measurement is performed by repeatedly performing the action of the light source 20 emitting a light pulse toward the target part such as the arm or leg of the user, and the image sensor 30 detecting the rear end component of the reflected light pulse. Thus, the internal scattered component of the reflected light pulse is repeatedly detected. The image sensor 30 outputs a detection signal corresponding to the accumulated amount of the reflected light pulse for each pixel. The signal processing circuit 70 repeatedly generates blood flow data representing the concentration distribution of oxygenated hemoglobin based on the detection signal output from the image sensor 30. The blood flow data can be generated, for example, for each pixel or for each pixel group.

[0216] When a predetermined time has passed since the start of measurement, the control circuit 60 causes the pressurizing device 40 to start pressurizing a portion of the user's body (step S702). The generation of blood flow data continues as it is during the pressurization period. The signal processing circuit 70 repeatedly generates blood flow data based on the detection signal output from the image sensor 30 while the user is being pressurized.

[0217] When a certain time (e.g., 120 seconds) has passed since the start of the pressurization by the cuff, the pressurization by the cuff ends, and a part of the user's body is released from the pressurization by the cuff (step S703). The control circuit 60 stops the pressurization by the pressurizer 10. Even after the pressurization stops, the measurement is continued as it is for a certain time (e.g., 30 seconds). Thus, blood flow data is repeatedly generated even after the pressurization by the cuff ends.

[0218] If a certain time has passed since the end of the pressurization by the cuff, the control circuit 60 ends the measurement (step S704). Then, the signal processing circuit 70 analyzes the blood flow data obtained in each of the pre-pressurization period, the pressurization period, and the post-pressurization period. Thus, the time change of the concentration of Oxy-Hb during the pressurization period is analyzed (step S705). For example, the signal processing circuit 70 uses the maximum point of the increase of Oxy-Hb just after the cuff pressurization as the analysis starting point, and calculates the slope of the decrease of Oxy-Hb, that is, the rate of change. Specifically, the signal processing circuit 70 performs the following processing.

[0219] (1) Determine the maximum point of Oxy-Hb during pressurization.

[0220] (2) The maximum point is used as the analysis start point, and a linear fit is performed on the behavior of Oxy-Hb for a predetermined period from this point. The predetermined period can be set to a value such as 30 seconds or 60 seconds.

[0221] (3) Calculate the slope of the straight line after linear fitting.

[0222] Through the above processing, the signal processing circuit 70 calculates the slope of the decrease of Oxy-Hb during the pressurization period. The slope of the decrease of Oxy-Hb during the pressurization period represents the muscle oxygen consumption. If the slope of the decrease is small, it means that the muscle oxygen consumption is small, and if the slope of the decrease is large, it means that the muscle oxygen consumption is large. So far, it is the measurement operation of the muscle oxygen consumption before training.

[0223] Then, the user performs training such as muscle training or rehabilitation exercises (step S706). Then, the muscle oxygen consumption after training is measured by the same method as the above method (steps S707 to S711). The actions of steps S707 to S711 are respectively the same as the actions of steps S701 to S705. In step S711, the signal processing circuit 70 calculates the slope of the decrease of Oxy-Hb during the pressurization period after training.

[0224] Thus, it is possible to compare the muscle oxygen consumption before and after training. The signal processing circuit 70 uses an existing test method to test for significant errors based on the muscle oxygen consumption before and after training (step S712). In the case of a significant error, the signal processing circuit 70 determines that there is a training effect (S713). On the contrary, in the case of no significant error, the signal processing circuit 70 determines that there is no training effect (step S714). For example, when the result of comparing the muscle oxygen consumption before training with the muscle oxygen consumption after training is that the increase in muscle oxygen consumption exceeds the reference value, it can be determined that there is a training effect, and when the increase in muscle oxygen consumption does not exceed the reference value, it can be determined that there is no training effect. The determination of whether there is a training effect can be performed on a pixel-by-pixel basis. The signal processing circuit 70 can also determine the increase in muscle oxygen consumption as a result of the determination on a pixel-by-pixel basis, and generate image data represented by different colors according to the level of the increase. For example, an image may be generated in which pixels whose increase in muscle oxygen consumption exceeds a reference value are considered to have a high training effect and are displayed in red, and pixels whose increase in muscle oxygen consumption does not exceed a reference value are considered to have a low training effect and are displayed in blue. Alternatively, an image may be generated in which colors are differentiated in three or more levels according to the level of increase in muscle oxygen consumption. The image may be displayed on the display 50 or the AR glasses 90. A color bar indicating the correspondence between the color and the training effect may also be displayed together with the image.

[0225] Through the above processing, the user can visually grasp the training effect as a two-dimensional distribution based on whether the muscle oxygen consumption has increased. The signal processing circuit 70 can also generate an image in which an image of a two-dimensional distribution representing an increase in muscle oxygen consumption is superimposed on an appearance image representing the user's appearance obtained by the image sensor 30 or other devices. By displaying such an image, the user can clearly grasp the training effect of each muscle part. In addition, an image of a two-dimensional distribution representing an increase in muscle oxygen consumption can also be displayed in a manner overlapping with a real image of the user that can be seen through the AR glasses 90. Through such a display, the training effect of each muscle part can be more clearly grasped.

[0226] As a method for determining whether muscle oxygen consumption has increased, the signal processing circuit 70 can also determine based on the size of the slope of the decrease of Oxy-Hb before and after exercise. For example, a real number greater than 1 can also be set to a. When the size of the slope of Oxy-Hb after exercise (i.e., the second rate of change) is more than a times the size of the slope of Oxy-Hb before exercise (i.e., the first rate of change), it is determined that the muscle oxygen consumption is relatively large or the training effect is high, and the information to this effect is included in the distribution data of muscle oxygen consumption and output. a can be set to an appropriate value such as 2. In addition, an image that emphasizes the area with a larger increase in muscle oxygen consumption or a smaller area can also be generated. By displaying such an image, the user can more accurately grasp the training effect.

[0227] According to the present embodiment, the user can grasp the muscle parts with training effects and the muscle parts without training effects based on the two-dimensional distribution of the increase in muscle oxygen consumption, and feedback to the next training menu. In this way, training can be performed more effectively. The signal processing circuit 70 can also generate a training plan for training the area determined to have a relatively small increase in muscle oxygen consumption based on the two-dimensional distribution of the increase in muscle oxygen consumption, and display it on the display 50 or AR glasses 90. In this case, data such as a table specifying the correspondence between multiple muscle parts and one or more training items for stimulating each muscle part is pre-recorded in the storage medium 80. By referring to the data, the signal processing circuit 70 can generate a training plan for exercising the muscle parts with a relatively small increase in muscle oxygen consumption. The user can improve the training effect by executing the suggested training plan. According to the present embodiment, even users who do not have sufficient knowledge about training can implement more effective training.

[0228] When providing a training plan, the signal processing circuit 70 may also refer to the training history information of the user, compare the training history with the training effect, and create and provide a more optimized training plan. Furthermore, when providing a training plan, the signal processing circuit 70 may also refer to the user identification information to provide a more effective training plan suitable for each user. In such a configuration, the user identification information and the training history information of each user are pre-recorded in the storage medium 80.

[0229] (Example)

[0230] Next, an example of a method for evaluating a training effect based on the measurement results of muscle oxygen consumption will be described.

[0231] Fig.16 1 is a diagram showing the state of an experiment for evaluating the training effect using a NIRS camera 400. In this experiment, the blood flow dynamics when the blood flow was cut off before and after the training were measured using the NIRS camera 400. The NIRS camera 400 is equivalent to the measuring device 100 of the above-mentioned embodiment. The measuring part is set to the muscles of the forearm, the training is set to 100 times of the handgrip 450 (load 2kg), and the blood flow is cut off by applying pressure to the upper arm with a pressure of 200mmHg using a cuff.

[0232] Fig.17 : is a diagram showing the experimental scheme. The experimental scheme is: rest for 30 seconds → pressurization with a cuff for 120 seconds → rest for 30 seconds, blood flow dynamics are measured using a NIRS camera 400 before training, and then training is performed using a handgrip 450, and blood flow dynamics are measured again using the above scheme immediately after. In order to make the measurement sites before and after training consistent, the forearm is marked with an oil-based marker. Fig.18 shows an infrared image acquired by the NIRS camera 400. Fig.18 The marking part 510 in the infrared image shown is aligned. In this way, it is possible to accurately perform alignment based on the position of the feature point in the image obtained by the NIRS camera 400. This is one of the major advantages of the measurement method disclosed herein over the conventional method of wearing the NIRS device on the skin.

[0233] Fig.19 and Fig. 20 These are graphs showing the temporal changes in the amount of hemoglobin change measured by the NIRS camera 400 before and after training, respectively. Fig.19 and Fig. 20Indicates the temporal changes in the amount of change in the concentration of each of oxygenated hemoglobin (Oxy-Hb), deoxyhemoglobin (Deoxy-Hb) and total hemoglobin (Total-Hb) relative to the baseline value. Before and after training, Total-Hb increases significantly by pressurization, gradually increases during pressurization, and decreases after the pressurization is released. Oxy-Hb increases significantly by pressurization, gradually decreases during pressurization, increases significantly just after the pressurization is released, and then decreases. Deoxy-Hb increases slightly at the beginning of pressurization, gradually increases during pressurization, and decreases significantly after the pressurization is released. In this way, different behaviors are shown between Oxy-Hb and Deoxy-Hb during the start of pressurization, during pressurization, and during the release of pressurization. If we focus on the pressurization, Oxy-Hb gradually decreases and Deoxy-Hb gradually increases. It is considered that this is because the oxygen in the blood retained in the forearm is consumed by the metabolism of muscle cells due to the interruption of blood flow by the cuff pressurization of the upper arm. Regarding the above general blood flow dynamics, the same trend was shown before and after training.

[0234] This experiment was conducted based on the assumption that "muscle oxygen consumption increases with training, and thus the slope of the decrease in Oxy-Hb during cuff pressurization increases." Fig.19 and Fig. 20 Comparing the slopes of decrease in Oxy-Hb during cuff pressurization before and after training shown, it can be confirmed that the slope of decrease in Oxy-Hb during pressurization has indeed increased in the measurement after training.

[0235] In order to quantitatively study the slope of the decrease of Oxy-Hb in the cuff pressurization before and after training, Fig.19 and Fig. 20 The behavior of Oxy-Hb during cuff pressurization shown is analyzed in detail. Figure 21 to Figure 24 The results of analysis of the behavior of Oxy-Hb during cuff pressurization are shown. Fig.21 and Fig. 22 The analysis results of the behavior of Oxy-Hb before and after training are shown when the analysis period is 60 seconds. Fig.23 and Fig.24 The analysis results of the behavior of Oxy-Hb before and after training are shown in the case where the analysis period is 30 seconds. The analysis start point is the time point when the Oxy-Hb change is the largest in all examples. The analysis end point is Fig.21 and Fig. 22 In the example, 60 seconds after the parsing start point, Fig.23 and Fig.24 In the example, it is 30 seconds after the parsing start point. Figure 21 to Figure 24As shown, the cuff pressurization start point is different from the analysis start point. Since there is a deviation between the cuff pressurization start time point and the time point when the Oxy-Hb change amount is the largest, if the cuff pressurization start point is used as the analysis start point, the slope of the decrease in Oxy-Hb cannot be calculated correctly.

[0236] like Figure 21 to Figure 24 As shown in the figure, since the decrease of Oxy-Hb during pressurization is not linear, the fitting accuracy is low when the analysis period is 60 seconds, whereas it is confirmed that a good fitting is possible when the analysis period is 30 seconds. As described above, it is confirmed that the slope of the decrease of Oxy-Hb during cuff pressurization can be analyzed with high accuracy by setting the time point when the change amount of Oxy-Hb is the largest as the analysis start point and setting the analysis period to 30 seconds.

[0237] like Fig.21 and Fig. 22 As shown in the figure, when the analysis period is 60 seconds, the slope of the decrease of Oxy-Hb during pressurization is -0.00052 before training, while it is -0.0013 after training, and the absolute value increases by about 2.5 times. If the plot shown in the figure is compared with the fitting line, the two deviate greatly, especially after training, and it can be seen that the fitting line does not correctly represent the actual slope of the initial stage of Oxy-Hb decrease. On the other hand, Fig.23 and Fig.24 As shown in the figure, when the analysis period is 30 seconds, the slope of the decrease of Oxy-Hb during pressurization is -0.000081 before training, while it is -0.0024 after training, which is a significant increase of about 30.0 times. In addition, when the analysis period is 30 seconds, the slope of the initial decrease of Oxy-Hb is well consistent with the fitting line. Based on the above results, it is confirmed that the slope of the decrease of Oxy-Hb during pressurization can be calculated with higher accuracy by setting the analysis period to be shorter, such as 30 seconds.

[0238] In addition, although linear fitting is performed in this embodiment, fitting based on a function different from a straight line may also be performed. The signal processing circuit may also fit the time change of the concentration of oxyhemoglobin in a predetermined period after the concentration of oxyhemoglobin stops increasing to a predetermined function in each measurement period before and after training, and determine the change rate based on the time change rate of the function.

[0239] (Other embodiments)

[0240] In the above embodiments and examples, muscle oxygen consumption is measured in a state where the blood flow of the target part is cut off using the pressurizer 40. However, the present disclosure is not necessarily limited to such a method. Other examples of the method of measuring muscle oxygen consumption are described below.

[0241] As a method different from the blood flow interception method, there is a method of inferring muscle oxygen consumption based on the difference between Oxy-Hb and Deoxy-Hb after training. Since muscle oxygen consumption increases during training compared to before training, oxygen consumption increases compared to the oxygen supply in the blood flow. As a result, Oxy-Hb decreases compared to before training, and Deoxy-Hb increases compared to before training. After training, in order to compensate for the lack of oxygen in muscle blood, Oxy-Hb increases and Deoxy-Hb decreases. If the difference between the value of Oxy-Hb and the value of Deoxy-Hb after training is set to ΔHb (=Oxy-Hb-Deoxy-Hb), ΔHb can be used as an indicator value representing the height of the training effect. That is, the higher the muscle load for training, the greater the ΔHb. Therefore, based on the two-dimensional distribution of ΔHb, it is possible to visualize effectively trained parts and ineffectively trained parts in two dimensions.

[0242] As another different method, there is a method of using the proportion of Oxy-Hb in Total-Hb (=Oxy-Hb+Deoxy-Hb) after training as an indicator of the training effect. During training, since muscle oxygen consumption increases, oxygen consumption increases compared to the oxygen supply in the bloodstream. As a result, the proportion of Oxy-Hb in Total-Hb during training decreases compared to before training. After training, in order to make up for the lack of oxygen in muscle blood, Oxy-Hb increases, so the proportion of Oxy-Hb in Total-Hb increases. At this time, if the time until the proportion of Oxy-Hb in Total-Hb before training, which serves as a benchmark, is returned to is set as recovery time, information about muscle oxygen consumption can be obtained based on the recovery time. Specifically, recovery time indicates aerobic capacity, that is, exercise capacity. If aerobic capacity is improved by continuous training, recovery time is shortened. Thus, by continuously measuring the recovery time of each muscle part, it is possible to visualize two-dimensionally which muscle part has improved or not improved in aerobic capacity after a certain period of time.

[0243] Industrial Applicability

[0244] The technology disclosed in the present invention can obtain information on the user's muscle activity without contact. The technology disclosed in the present invention can be used in various devices such as cameras, measuring equipment, smartphones, tablet computers, and head-mounted devices.

[0245] Description of symbols

[0246] 20 Light Source

[0247] 22 Irradiation area

[0248] 30 Image Sensor

[0249] 40 Pressurizer

[0250] 50 Display

[0251] 60 Control circuit

[0252] 62 Light source control unit

[0253] 63 Photodetector control unit

[0254] 64 Pressurization control unit

[0255] 70 Signal processing circuit

[0256] 80 Storage Media

[0257] 90 AR glasses

[0258] 100 Measuring device

[0259] 201 pixels

[0260] 202 Drain

[0261] 203 Photodiode

[0262] 204, 205, 206, 207 floating diffusion layer

[0263] 302 row selection circuit

[0264] 303 Column selection circuit

[0265] 304 vertical signal line

[0266] 305 Source Follower Power Supply

[0267] 306 Source Follower Load

[0268] 307 conversion circuit

[0269] 308 row selection transistor

[0270] 309 Source Follower Transistor

[0271] 310 Reset transistor

[0272] 400 NIRS Camera

[0273] 450 Grip Strengthener

[0274] 500 users

[0275] 501 Object Department

[0276] 510 Marking location

Claims

1. A measuring device for measuring muscle oxygen consumption in a target part of a user performing muscle exercise, wherein: have: light source; A sensor including a plurality of photoelectric conversion elements; as well as Processing circuit, The above processing circuit, Acquiring a plurality of detection signals by performing a plurality of detection actions, the detection actions comprising: causing the light source to emit a light pulse; and causing the sensor to detect at least a portion of an internally scattered component, which is a component scattered inside the object portion, of a reflected light pulse generated from the object portion due to the emission of the light pulse, and outputting a detection signal indicating a spatial distribution of the intensity of the at least a portion of the internally scattered component, the plurality of detection signals including the detection signal; and Based on the plurality of detection signals, distribution data indicating spatial distribution of muscle oxygen consumption in the target part is generated and output.

2. The measuring device according to claim 1, wherein: The processing circuit generates, as the distribution data, image data representing the spatial distribution of the muscle oxygen consumption in the target portion in different colors according to levels of the muscle oxygen consumption.

3. The measuring device according to claim 1 or 2, wherein: The processing circuit generates the following image data as the distribution data, wherein the image data represents an image in which the information on the spatial distribution of the muscle oxygen consumption is superimposed on an appearance image representing the appearance of the user including the object portion obtained by the sensor or other device.

4. The measuring device according to claim 1 or 2, wherein: The above processing circuit, estimating the concentration distribution of oxyhemoglobin in the blood inside the target part based on the plurality of detection signals, The spatial distribution of the muscle oxygen consumption is inferred based on the temporal change of the concentration distribution of the oxyhemoglobin.

5. The measuring device according to claim 4, wherein: The processing circuit estimates the muscle oxygen consumption based on a slope of the time change in the concentration of the oxyhemoglobin.

6. The measuring device according to claim 1 or 2, wherein: It also has a pressurizer. The processing circuit performs the detection operation in a state where a part of the user's body is pressurized by the pressurizer and blood flow in the target part is suppressed.

7. The measuring device according to claim 6, wherein: The above-mentioned pressurizer is controlled by the above-mentioned processing circuit, The above processing circuit, Before performing the detection operation, the pressurizing device starts to pressurize the part of the user's body. After the detection operation is performed, the pressurizing device is caused to terminate the pressurizing.

8. The measuring device according to claim 1 or 2, wherein: The above processing circuit, The detection action is performed multiple times in each of a first period before the user performs the muscle movement and a second period after the user performs the muscle movement. The distribution data is generated based on the plurality of detection signals acquired during the first period and the plurality of detection signals acquired during the second period.

9. The measuring device according to claim 8, wherein: The above processing circuit, acquiring data of an appearance image representing the appearance of the user including the object portion, the appearance image being acquired by the sensor or other device during each of the first period and the second period, By matching one or more feature points included in the appearance image obtained during the first period with one or more feature points included in the appearance image obtained during the second period, a change in the position of the object portion occurring between the first period and the second period is detected. The distribution data is generated after a process of compensating for the position change is performed on the plurality of detection signals obtained in the first period and the plurality of detection signals obtained in the second period.

10. The measuring device according to claim 8, wherein: The above processing circuit, generating first blood flow data indicating temporal changes in the concentration distribution of oxyhemoglobin in the blood inside the target portion based on the plurality of detection signals acquired during the first period, generating second blood flow data indicating a temporal change in the concentration distribution of oxyhemoglobin in the blood inside the target portion based on the plurality of detection signals acquired during the second period, The distribution data is generated based on the first blood flow data and the second blood flow data.

11. The measuring device according to claim 10, wherein: The above processing circuit, determining a first change rate indicating a slope of a decrease in a time change in the concentration of the oxyhemoglobin at a plurality of points included in the target portion based on the first blood flow data, determining a second change rate indicating a slope of a decrease in a time change in the concentration of the oxyhemoglobin at the plurality of points based on the second blood flow data, The muscle oxygen consumption at the plurality of points is estimated based on the difference or ratio between the first rate of change and the second rate of change.

12. The measuring device according to claim 11, wherein: The detection operation during the first period and the second period is performed in a state where a part of the user's body is pressurized and blood flow in the target part is suppressed, The above processing circuit, During the first period, the time change of the concentration of the oxyhemoglobin during a predetermined period after the concentration of the oxyhemoglobin stops increasing is fitted to a function, and the first change rate is determined based on the time change rate of the function. In the second period, the time change of the concentration of the oxyhemoglobin in the predetermined period after the concentration of the oxyhemoglobin stops increasing is fitted to the function, and the second change rate is determined based on the time change rate of the function.

13. The measuring device according to claim 11, wherein: The above processing circuit, When the magnitude of the second rate of change is a times or more the magnitude of the first rate of change, information indicating that the oxygen consumption of the muscle is relatively large is added to the distribution data and output, where a is a real number greater than 1. When the magnitude of the second change rate is smaller than a times the magnitude of the first change rate, information indicating that the muscle oxygen consumption is relatively small is added to the distribution data and outputted.

14. The measuring device according to claim 13, wherein: The processing circuit determines a first region in the target portion where the oxygen consumption of the muscle is relatively large based on the information indicating that the oxygen consumption of the muscle is relatively large, and determines a second region in the target portion where the oxygen consumption of the muscle is relatively small based on the information indicating that the oxygen consumption of the muscle is relatively small. The distribution data includes an image that highlights the first area or the second area.

15. The measuring device according to claim 1 or 2, wherein: The processing circuit further generates and outputs data indicating a training plan for training muscles in a region of the target portion where oxygen consumption of the muscles is relatively low.

16. The measuring device according to claim 15, wherein: The above processing circuit, acquiring history data indicating the content of the muscle movement performed by the user, Based on the above historical data, adjust the above training plan.

17. The measuring device according to claim 15, wherein: The above processing circuit, obtaining identification data identifying said user, Based on the above recognition data, the above training plan is adjusted.

18. The measuring device according to claim 1 or 2, wherein: The light source is configured to emit a first light pulse having a first wavelength of 650 nm or more and less than 805 nm, and a second light pulse having a second wavelength of 805 nm or more and less than 950 nm. The above detection actions include: causing the light source to emit the first light pulse; causing the sensor to detect at least a portion of a first internally scattered component, which is a component scattered in the interior of the object portion, of a first reflected light pulse generated from the object portion due to emission of the first light pulse, and output a first detection signal representing a spatial distribution of the intensity of the at least a portion of the first internally scattered component; causing the light source to emit the second light pulse; and causing the sensor to detect at least a portion of a second internally scattered component, which is a component scattered within the interior of the object portion, of a second reflected light pulse generated from the object portion due to emission of the second light pulse, and output a second detection signal indicating a spatial distribution of the intensity of the at least a portion of the second internally scattered component, the processing circuit estimates the concentration distribution of oxygenated hemoglobin in the blood inside the object part based on the first detection signal and the second detection signal, The spatial distribution of the muscle oxygen consumption is inferred based on the temporal change of the concentration distribution of the oxyhemoglobin.

19. The measuring device according to claim 1 or 2, wherein: It also has extended reality glasses that include a transparent display. The transparent display displays the distribution image in a manner in which the distribution image representing the distribution data is superimposed on the appearance of the user recognized via the transparent display.

20. The measuring device according to claim 1 or 2, wherein: The internal scattered component is a component in the reflected light pulse after the intensity of the reflected light pulse starts to decrease.

21. A method executed by a computer included in a measuring device for measuring muscle oxygen consumption in a target part of a user performing muscle exercise, wherein: The above method includes the following processing: Acquiring a plurality of detection signals by performing a plurality of detection actions, the detection actions comprising: causing a light source to emit a light pulse; and causing a sensor including a plurality of photoelectric conversion elements to detect at least a portion of an internally scattered component, which is a component scattered inside the object portion in a reflected light pulse generated from the object portion due to the emission of the light pulse, and outputting a detection signal indicating a spatial distribution of the intensity of the at least a portion of the internally scattered component, the plurality of detection signals including the detection signal; and Based on the plurality of detection signals, distribution data indicating spatial distribution of muscle oxygen consumption in the target part is generated and output.

22. A recording medium having a computer program recorded thereon, the computer program being executed by a computer included in a measuring device for measuring muscle oxygen consumption in a target part of a user performing muscle exercise, wherein: The computer program causes the computer to execute the following processing: Acquiring a plurality of detection signals by performing a plurality of detection actions, the detection actions comprising: causing a light source to emit a light pulse; and causing a sensor including a plurality of photoelectric conversion elements to detect at least a portion of an internally scattered component, which is a component scattered inside the object portion in a reflected light pulse generated from the object portion due to the emission of the light pulse, and outputting a detection signal indicating a spatial distribution of the intensity of the at least a portion of the internally scattered component, the plurality of detection signals including the detection signal; and Based on the plurality of detection signals, distribution data indicating spatial distribution of muscle oxygen consumption in the target part is generated and output.

Images