Heartbeat detector

Abstract

To detect information on a heartbeat of a crew member of a movable body with a high degree of precision.SOLUTION: A heartbeat detector includes: a transmission unit for transmitting a transmission wave subjected to FMCW modulation to a room where a person is present in a movable body; a reception unit for receiving a reflection wave generated when the transmission wave is reflected by an object in the room; a calculation unit for calculating the velocity of the movement of a body site at a heartbeat detection position of the person on the basis of the reflection wave; and a determination unit for determining the aforesaid velocity as a heartbeat velocity if the aforesaid velocity is within a range of a predetermined heartbeat velocity threshold.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] FIELD OF THE INVENTION An embodiment of the present invention relates to a heart rate detection device. [Background technology]

[0002] Conventionally, research and development has been conducted on technologies for detecting biometric information of vehicle (passenger car, etc.) occupants. Examples of biometric information include information related to heartbeats. In this case, for example, it is conceivable to irradiate (transmit) radio waves (transmitted waves) into the vehicle interior and calculate the speed of movement of the part of the occupant at the heartbeat detection position (heart position) based on the received reflected waves. Summary of the Invention [Problem to be solved by the invention]

[0003] However, in the above-mentioned conventional technology, when a radio wave sensor that emits radio waves of 24 GHz or 60 GHz is used, the velocity resolution is low, and therefore the heartbeat velocity detection performance is also low. Specifically, for example, the lower limit of the detection velocity of a 60 GHz radio wave sensor is about 2.5×10 3 The heart rate is approximately 1 to 6 mm / s, making it difficult to detect the heart rate with a 60 GHz radio wave sensor. Furthermore, the heart rate detection performance of a 24 GHz radio wave sensor is even lower.

[0004] Therefore, one of the problems to be solved by the present invention is to provide a heartbeat detection device that can detect information related to the heartbeat of an occupant of a moving body with high accuracy. [Means for solving the problem]

[0005] The heart rate detection device of the present invention comprises a transmitting unit that transmits an FMCW modulated transmission wave into a room in a moving body where a person is present, a receiving unit that receives a reflected wave generated when the transmission wave is reflected by an object in the room, a calculating unit that calculates the speed of movement of a part of the person at the heart rate detection position based on the reflected wave, and a determining unit that determines the speed to be the heart rate if the speed is within a predetermined heart rate threshold range.

[0006] According to this configuration, by using data on the reflected wave of the FMCW modulated transmitted wave, it is possible to achieve a velocity resolution sufficient for detecting the heart rate, and the velocity of the vibration component of the moving body (which is significantly faster than the heart rate) can be removed as noise, so information related to the heart rate of the occupant of the moving body can be detected with high accuracy. [Effects of the Invention]

[0007] According to the heartbeat detecting device of the present invention, information relating to the heartbeat of an occupant of a moving body can be detected with high accuracy. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing the configuration of a vehicle according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the sensor and the control device according to the embodiment. [Figure 3] FIG. 3 is a diagram showing an outline of signal processing by the FMCW method according to the embodiment. [Figure 4] FIG. 4 is a diagram showing the interior of the vehicle according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a transmitted wave, a reflected wave, and a beat frequency according to the embodiment. [Figure 6] FIG. 6 is a graph showing the velocity resolution and the like of each of a plurality of sensors. [Figure 7] FIG. 7 is an explanatory diagram of a heartbeat detection method according to an embodiment. [Figure 8] FIG. 8 is an explanatory diagram of a heartbeat detection method according to an embodiment. [Figure 9] FIG. 9 is a flowchart showing the processing performed by the heartbeat detection device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Exemplary embodiments of the heartbeat detection device of the present invention are disclosed below. The configurations of the embodiments shown below, as well as the actions, results, and advantages brought about by the configurations, are examples. The present invention can be realized with configurations other than those disclosed in the following embodiments, and it is possible to obtain at least one of various advantages and derivative advantages based on the basic configurations.

[0010] FIG. 1 is a diagram showing the configuration of a vehicle C according to an embodiment. A LiDAR (Light Detection and Ranging) sensor 2 and a control device 3 constituting a heartbeat detection device 1 are disposed inside the vehicle C (an example of a moving body). The LiDAR sensor 2 is installed on the ceiling of the vehicle. The control device 3 is installed inside a dashboard located at the front end of the vehicle. Note that the installation positions and number of the LiDAR sensor 2 and the control device 3 are not limited to those described above. For example, the LiDAR sensor 2 may be installed (built-in) in an overhead console, a rearview mirror, or the like.

[0011] FIG. 1 illustrates an example of a state in which an occupant M (person) is sitting in a seat S. The following describes how the heartbeat detection device 1 detects the presence of the occupant M and detects the heartbeat of the occupant M. Note that FIG. 1 illustrates an example of a state in which the occupant M is sitting in the rear seat, but the position of the occupant M to be detected is not limited to this. For example, the occupant sitting in the driver's seat or the passenger seat may be the detection target.

[0012] 2 is a block diagram showing the functional configuration of the LiDAR sensor 2 and the control device 3 according to the embodiment. The LiDAR sensor 2 includes a transmitter 21 and a receiver 22.

[0013] The transmitter 21 transmits a frequency modulated continuous wave (FMCW) modulated transmission wave into a room in the vehicle C where a person is present. Specifically, the transmitter 21 transmits, as the transmission wave, a laser beam (electromagnetic wave) that has been frequency modulated so as to repeat a gradual increase period in which the frequency gradually increases and a gradual decrease period in which the frequency gradually decreases. The transmitter 21 may transmit the transmission wave over a wide range or may transmit it over a narrow range (for example, by irradiating a single point).

[0014] The receiving unit 22 receives a reflected wave generated when the transmitted wave is reflected by an object in the room.

[0015] The control device 3 is configured by, for example, an MCU (Micro Controller Unit) having an integrated circuit equipped with a hardware processor, a memory, etc. The control device 3 includes an ADC (Analog-to-Digital Converter) 31, a processing unit 32, and a storage unit 33.

[0016] The ADC 31 converts the analog signal acquired by the receiving unit 22 of the LiDAR sensor 2 into a digital signal and outputs the digital signal to the processing unit 32.

[0017] The storage unit 33 is, for example, a storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), or an HDD (Hard Disk Drive). The storage unit 33 stores various information such as programs executed by the processing unit 32, data required for executing the programs, and data generated by executing the programs.

[0018] Here, an overview of signal processing using the FMCW method will be described with reference to Fig. 3. Fig. 3 is a diagram showing an overview of signal processing using the FMCW method according to an embodiment. First, as shown in state (A), for example, FMCW-modulated laser light is transmitted from the transmitter 21 of the LiDAR sensor 2 so as to scan the entire interior of the vehicle C. Then, the receiver 22 of the LiDAR sensor 2 receives the reflected wave.

[0019] Next, as shown in state (B), a voxel diagram showing the arrangement (including position, size, shape, etc.) of objects in three-dimensional space (indoor vehicle) is created based on the intensity distribution of the reflected waves in the vehicle. The voxel diagram includes a labeling area D showing the arrangement of objects present in the vehicle (occupant M, seat S, luggage, other vehicle body structures, etc.). The voxel diagram is updated as the reflected wave information acquired by the LiDAR sensor 2 is updated. Information regarding the position and speed of objects can be obtained by analyzing such changes in the voxel diagram. For example, the Doppler shift of the reflected waves can be calculated by performing FFT (Fast Fourier Transform) analysis on the difference between adjacent frames of the voxel diagram.

[0020] Next, as shown in state (C), an occupant M present in the cabin is detected based on the analysis results of the voxel values ​​of the voxels that make up the labeling area D. For example, of multiple objects present in the cabin, an object that satisfies predetermined conditions (size, shape, speed, etc.) can be determined to be the occupant M. Furthermore, biometric information of the occupant M can be detected based on the movement of a part (e.g., face, neck, chest, back, etc.) of the object determined to be the occupant M. The biometric information is information about the body of the occupant M present in the cabin, such as information about at least one of the heartbeat, pulse, and breathing of the occupant M. In the following, information about the heartbeat will be taken as an example.

[0021] Next, the premise of the following description will be described with reference to Fig. 4. Fig. 4 is a diagram showing the interior of a vehicle C according to an embodiment. In the following, the driver DR will be taken as an example of the occupant M.

[0022] Measurement point P1 is a measurement point for measuring data related to the movement of the driver DR, including the heartbeat component and the vibration component of the vehicle C. Note that the laser light emitted from the transmitter 21 of the LiDAR sensor 2 is not transparent, and if the laser light is irradiated onto the clothing at a position other than the seat belt 42, the movement of the driver DR's body (heartbeat) cannot be detected accurately. Therefore, the position of measurement point P1 is set to the position of the driver DR's heart and the position of the seat belt 42. At this position of measurement point P1, the movement of the driver DR's body due to the heartbeat is transmitted to the seat belt 42 due to the fastening of the seat belt 42, and therefore, the movement can be detected accurately.

[0023] Next, the transmitted wave, the reflected wave, and the beat frequency will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the transmitted wave, the reflected wave, and the beat frequency of an embodiment. In Fig. 5, the upper graph illustrates a line Lt showing the relationship between the elapsed time and the frequency of the transmitted wave transmitted from the transmitter 21 of the LiDAR sensor 2, and a line Lr showing the relationship between the elapsed time and the frequency of the reflected wave received by the receiver 22 of the LiDAR sensor 2. In Fig. 5, the lower graph illustrates a line Lb showing the relationship between the elapsed time and the beat frequency.

[0024] As shown by line Lt, the transmission wave in this embodiment is a chirp signal in which up periods T1 (gradual increase periods), in which the frequency increases over time, and down periods T2 (gradual decrease periods), in which the frequency decreases over time, are repeated alternately. One up period T1 and one down period T2 make up one chirp period T. As shown by line Lr, the frequency of the reflected wave changes with a delay corresponding to the distance from the LiDAR sensor 2 to the object, relative to the change in the frequency of the transmission wave over time. In other words, a difference occurs between the frequency of the transmission wave and the frequency of the reflected wave at the same time.

[0025] The beat frequency is a value determined according to this difference, is proportional to the distance from the LiDAR sensor 2 to the object, and changes according to the movement of the target object. When the object is stationary, the beat frequency in the up period T1 and the beat frequency in the down period T2 will be the same. On the other hand, when the object is moving, the beat frequency in the up period T1 and the beat frequency in the down period T2 will be different, and the speed of the object's movement can be calculated from the magnitude of the difference.

[0026] 2, the processing of the processing unit 32 will be described. The processing unit 32 is configured by a hardware processor such as a CPU (Central Processing Unit), for example. The processing unit 32 reads a program stored in the storage unit 33 and executes arithmetic processing. The processing unit 32 includes, as functional units, an acquisition unit 321, a calculation unit 322, a determination unit 323, a calculation unit 324, a detection unit 325, and a control unit 326. Note that some or all of the units 321 to 326 may be configured by hardware such as circuits including an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

[0027] The acquisition unit 321 acquires various types of information. The acquisition unit 321 acquires a digital signal from the ADC 31, for example.

[0028] The calculation unit 322 calculates various types of information. For example, the calculation unit 322 calculates the displacement (distance) and speed of movement of the part (measurement point P1) where the person's heartbeat is detected, based on the reflected wave, using the method described above.

[0029] 6 is a graph showing the velocity resolution of each of a plurality of sensors, etc. Region R1 is the velocity resolution (range in which velocity can be detected) in the case of a 24 GHz radio wave sensor.

[0030] Moreover, region R2 is the velocity resolution in the case of a 60 GHz radio wave sensor. Moreover, region R3 is the velocity resolution in the case of a LiDAR sensor 2 equipped with a transmitter 21 that emits radio waves of 194 THz. Region R4 is an approximate range of the speed of body surface movement due to heartbeat.

[0031] Therefore, it can be seen that the LiDAR sensor 2 can detect the heart rate with high accuracy as shown in region R5, but the 24 GHz radio wave sensor and the 60 GHz radio wave sensor cannot detect the heart rate with high accuracy.

[0032] However, the speed calculated based on the data of the reflected waves acquired using the LiDAR sensor 2 may contain not only the heartbeat component but also the vibration component of the vehicle C as noise. In this case, the speed of the vibration component of the vehicle C is generally significantly greater than the speed of the heartbeat component, so it is possible to distinguish them. This will be explained using FIG. 7.

[0033] 7A and 7B are explanatory diagrams of a heartbeat detection method according to an embodiment. Fig. 7A is a graph showing the relationship between the distance and the speed from the LiDAR sensor 2 at each time. Region R11 is a range of distances within which the body surface of the occupant is located. This range of distances can be known, for example, through a previous experiment.

[0034] Region R12 is the speed of the heartbeat component. Region R13 is the speed of the vibration component of vehicle C. Therefore, based on the magnitude of the speed, the speed corresponding to region R12 can be identified as the speed of the heartbeat component.

[0035] Moreover, region R14 is the speed of the vibration component of vehicle C. The speed in region R14 can be identified as not being the speed of the heartbeat component based on the magnitude of the speed, and can also be identified as not being the speed of the heartbeat component because it is outside region R11.

[0036] 7(b) corresponds to region R11 in FIG. 7(a), and is a graph in which the detected velocity portion is re-plotted. Region R21 is the heart rate velocity range. For example, the black square in region R22 is included in region R21 and can be identified as the velocity of the heart rate component.

[0037] Fig. 8 is an explanatory diagram of a heartbeat detection method according to an embodiment. Fig. 8(a) is a graph showing how the distance calculated by the calculation unit 322 changes over time. Fig. 8(b) is a graph showing how the speed (symbols G1 to G14) changes over time, calculated from the displacement of two adjacent transitions between increasing and decreasing distances identified for the region R31 in Fig. 8(a). Details of Fig. 8(b) will be described later.

[0038] 2, the determination unit 323 performs various determinations. For example, if the speed calculated by the calculation unit 322 is within a predetermined heartbeat speed threshold range (area R5 in FIG. 6), the determination unit 323 determines the speed to be the heartbeat speed.

[0039] Furthermore, the determination unit 323 may determine that the velocity is the heart rate velocity if the velocity is a predetermined velocity (for example, 3 mm / s) within the heart rate velocity threshold range (area R5 in FIG. 6) (hereinafter referred to as a first determination method).

[0040] Alternatively, the following may be performed: First, the calculation unit 322 calculates the displacement of the heartbeat detection position based on the reflected wave over time, identifies the transition points between an increase and a decrease in the displacement (FIG. 8(a)), and calculates the velocity ((symbols G1 to G14) in FIG. 8(b)) from the displacement at two adjacent transition points.

[0041] Then, if the length of time between two adjacent transitions corresponding to the speed is within a predetermined time threshold range, the determination unit 323 determines that the speed is the heartbeat speed (hereinafter referred to as the second determination method). For example, if a predetermined time threshold range including the time length of symbol G3 in FIG. 8(b) is set, the determination unit 323 determines that the speed of symbol G3 is the heartbeat speed.

[0042] Furthermore, the determination unit 323 determines the speed as the heartbeat speed if the difference between the speed and the immediately preceding speed is within a predetermined difference threshold range (hereinafter referred to as a third determination method). For example, if a predetermined difference threshold range is set that includes the difference between the speed of code G3 and the speed of the immediately preceding code G2, the determination unit 323 determines the speed of code G3 as the heartbeat speed.

[0043] Furthermore, the determination unit 323 may execute a combination of two or three of the first, second, and third determination methods.

[0044] The above-mentioned heart rate threshold range, time threshold range, and difference threshold range can be determined in advance by experiment, simulation, or the like.

[0045] For example, the heart rate threshold range can be determined using statistical methods as follows. First, the heart rate data actually contains positive, 0, and negative values, but the absolute values ​​are taken and all are set to 0 or greater. The average rate was 1.301. Then, assuming the standard deviation is σ, σ, 2σ, and 3σ were calculated, resulting in the following:

[0046] σ:1.604 2σ:3.208 3σ:4.813

[0047] In that case, the sum of these and the average value is as follows: (1) Mean value + σ: 2.905 (2) Average value + 2σ: 4.509 (3) Average value + 3σ: 6.114

[0048] If you consider using one of (1) to (3) as the heart rate threshold range (the upper limit of the absolute value of the heart rate), for example, if you want to prioritize not missing any heart rate over removing noise, you should choose (3).

[0049] For example, if you want to prioritize noise removal over not missing the heart rate, you should choose (1). If you want something in between, you should choose (2).

[0050] Although the case where the absolute value of the heartbeat velocity data is taken has been described here, the present invention is not limited to this. For example, instead of taking the absolute value of the heartbeat velocity data, the boundary values ​​of the heartbeat velocity threshold range may be found for both positive and negative values ​​using the statistical method described above.

[0051] The detector 325 detects the heartbeat based on the rate determined by the determiner 323 to be the heartbeat rate.

[0052] The control unit 326 executes various controls. For example, the control unit 326 controls the transmitter 21 of the LiDAR sensor 2.

[0053] Next, the processing performed by the heartbeat detecting device 1 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the processing performed by the heartbeat detecting device 1 according to the embodiment.

[0054] In step S1, the detection unit 325 detects an occupant in the interior of the vehicle C. A specific detection method may be, for example, a detection method using 3D mapping using sensing data from the LiDAR sensor 2, as described above with reference to Fig. 3. However, the detection method is not limited to this, and may also be a detection method using image recognition of an image captured by a camera (not shown), or other detection methods.

[0055] Next, in step S2, the detection unit 325 detects the position of the measurement point P1 (FIG. 4) using data from various sensors (LiDAR sensor 2, camera, etc.).

[0056] Next, in step S3, the transmitter 21 of the LiDAR sensor 2 transmits a transmission wave into the interior of the vehicle C.

[0057] Next, in step S4, the receiver 22 of the LiDAR sensor 2 receives the reflected wave of the transmitted wave.

[0058] Next, in step S5, the acquisition unit 321 acquires data of the measurement point P1 based on the sensing data obtained using the LiDAR sensor 2.

[0059] Next, in step S6, the calculation unit 322 calculates the displacement (distance) and speed of movement of the part (measurement point P1) where the person's heartbeat is detected, based on the reflected wave or the like.

[0060] Next, in step S7, the determination unit 323 determines the heartbeat rate. Specifically, if the rate calculated in step S6 is within a predetermined heartbeat rate threshold range (area R5 in FIG. 6), the determination unit 323 determines that rate as the heartbeat rate. Alternatively, the determination unit 323 determines the heartbeat rate using any of the methods described above.

[0061] Next, in step S8, the detection unit 325 detects information about the heartbeat based on the rate determined to be the heartbeat rate in step S7.

[0062] Next, in step S9, the calculation unit 324 estimates an RRI (RR Interval) based on the information about the heart rate detected in step S8.

[0063] Next, in step S10, the calculation unit 324 executes various processes using the RRI estimation result. For example, the calculation unit 324 uses the RRI estimation result to estimate the state of the driver DR, predict state hazards, estimate emotions, etc. Then, the calculation unit 324 notifies the driver DR (by sound or display) as necessary.

[0064] The processes in steps S8 to S10 can be realized by, for example, publicly known techniques.

[0065] As described above, the heartbeat detection device 1 of this embodiment calculates the speed of movement of the part of the person's heartbeat detection position based on reflected wave data acquired using the LiDAR sensor 2, and if the speed is within a predetermined heartbeat speed threshold range, determines the speed as the heartbeat speed. As a result, the use of the LiDAR sensor 2 makes it possible to achieve speed resolution sufficient for heartbeat detection and to remove the speed of the vibration component of the vehicle C as noise, making it possible to detect information related to the heartbeat of the occupant of the vehicle C with high accuracy.

[0066] Furthermore, if the determining unit 323 determines that the velocity is a predetermined velocity (for example, 3 mm / s) within the heartbeat velocity threshold range (area R5 in FIG. 6) as the heartbeat velocity, the accuracy of heartbeat detection can be further improved.

[0067] Furthermore, if the determination unit 323 determines that the speed is a heartbeat speed if the length of time between two adjacent transitions corresponding to the speed is within a predetermined time threshold range (FIG. 8(b)), the accuracy of heartbeat detection can be further improved.

[0068] Furthermore, if the determining unit 323 determines that the speed is the heartbeat speed if the difference between the speed and the immediately preceding speed is within a predetermined difference threshold range (FIG. 8(b)), the accuracy of heartbeat detection can be further improved.

[0069] The program executed by the control device 3 may be provided as a computer program product stored in an installable or executable file format on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD). Alternatively, the program may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the program may be provided or distributed via a network such as the Internet.

[0070] Although the embodiments of the present invention have been described above, the above embodiments are presented as examples and are not intended to limit the scope of the invention. This novel embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as set forth in the claims. [Explanation of symbols]

[0071] 1... heartbeat detection device, 21... transmission unit, 22... reception unit, 322... calculation unit, 323... determination unit

Claims

1. a transmitter that transmits a frequency modulated continuous wave (FMCW) modulated transmission wave into a room in which a person is present in the mobile object; a receiving unit that receives a reflected wave generated when the transmitted wave is reflected by an object in the room; a calculation unit that calculates a speed of movement of a part of the person at a heartbeat detection position based on the reflected wave; a determination unit that determines the speed as a heartbeat speed if the speed is within a predetermined heartbeat speed threshold range; A heart rate detection device comprising:

2. The heartbeat detection device according to claim 1 , wherein the determination unit determines the speed as a heartbeat speed if the speed is a predetermined speed within the heartbeat speed threshold range.

3. the calculation unit calculates a displacement of the heartbeat detection position based on the reflected wave over time, identifies a transition point between an increase and a decrease in the displacement, and calculates the velocity from the displacement at two adjacent transition points; The heartbeat detection device according to claim 1 , wherein the determination unit determines the speed as a heartbeat speed if a length of time between two adjacent transitions corresponding to the speed is within a predetermined time threshold range.

4. the calculation unit calculates a displacement of the heartbeat detection position based on the reflected wave over time, identifies a transition point between an increase and a decrease in the displacement, and calculates the velocity from the displacement at two adjacent transition points; The heartbeat detection device according to claim 1 , wherein the determination unit determines the speed to be a heartbeat speed if a difference between the speed and the immediately preceding speed is within a predetermined difference threshold range.

Images