Electronic device, method for controlling an electronic device, and program

Abstract

This electronic apparatus comprises: a transmission unit which transmits a transmission wave; a reception unit which receives a reflection wave from a target of the transmission wave; and a signal processing unit which detects the distance, the direction, and the velocity of the target on the basis of transformed signals after beat signals of the transmission wave and the reception wave are Fourier-transformed. The signal processing unit: extracts, from the transformed signals, signal components corresponding to a heart sound vibration accompanying the heart beat of the target by means of a plurality of window functions; performs a frequency analysis on the envelope signal obtained by performing envelope processing on the signal components corresponding to the heart sound vibration; calculates a statistical result with respect to the frequency analysis result; and outputs, on the basis of the calculation result, time series data of the heart beat frequency of the target.

Description

Cross - reference to related applications

[0001] This application claims the priority of Japanese Patent Application No. 2022 - 148633 filed in Japan on September 16, 2022, and incorporates the entire disclosure of the prior application herein by reference.

Technical Field

[0002] The present disclosure relates to an electronic device, a control method for an electronic device, and a program.

Background Art

[0003] For example, in fields such as industries related to automobiles, technologies for measuring the distance between a host vehicle and a predetermined object are regarded as important. In particular, in recent years, technologies for radar (Radio Detecting and Ranging) that measure the distance to an object by transmitting radio waves such as millimeter waves and receiving the reflected waves reflected by an object such as an obstacle have been variously studied. The importance of such technologies for measuring distances and the like is expected to increase further in the future with the development of technologies for assisting a driver's driving and technologies related to autonomous driving that automate part or all of the driving.

[0004] In addition, various proposals have been made for technologies for detecting the presence of an object by receiving the reflected wave when the transmitted radio wave is reflected by a predetermined object. For example, Patent Document 1 proposes a device that can detect the presence of a person and the biometric information of the person by using microwaves. Also, for example, Patent Document 2 proposes a device that detects vital signs such as the frequency of a living body's respiration or heartbeat based on the reflected signal of a microwave radar.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

[0006] An electronic device according to an embodiment includes a transmission unit that transmits a transmission wave, a reception unit that receives a reflected wave from a target of the transmission wave, and a signal processing unit that detects the distance, direction, and speed of the target based on a conversion signal obtained by performing a Fourier transform on a beat signal of the transmission wave and the reception wave. The signal processing unit extracts a signal component corresponding to heart sound vibration associated with the heartbeat of the target from the conversion signal using a plurality of window functions, , centered around multiple distance ranges, performs frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration, performs calculation processing of statistical results on the result of the frequency analysis, and outputs time-series data of the frequency of the heartbeat of the target based on the calculation result.

[0007] A control method for an electronic device according to an embodiment includes a step of transmitting a transmission wave from a transmission unit, a step of receiving, by a reception unit, a reflected wave from a target of the transmission wave Step and a step of detecting the distance, direction, and speed of the target based on a conversion signal obtained by performing a Fourier transform on a beat signal of the transmission wave and the reception wave, a step of extracting a signal component corresponding to heart sound vibration associated with the heartbeat of the target from the conversion signal , centered around multiple distance ranges, using a plurality of window functions, a step of performing frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration, a step of performing calculation processing of statistical results on the result of the frequency analysis, and a step of outputting time-series data of the frequency of the heartbeat of the target based on the calculation result.

[0008] A program according to an embodiment causes an electronic device to transmit a transmission wave from a transmission unit, receive, at a reception unit, a reflected wave from a target of the transmission wave Step and detect the distance, direction, and speed of the target based on a conversion signal obtained by performing Fourier transform on a beat signal of the transmission wave and the reception wave, extract a signal component corresponding to heart sound vibration associated with the heartbeat of the target from the conversion signal , centered around multiple distance ranges, using a plurality of window functions, perform frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration, perform a calculation process of statistical results on the result of the frequency analysis, output time-series data of the frequency of the heartbeat of the target based on the calculation result, and execute.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

MODE FOR CARRYING OUT THE INVENTION

[0010] If a weak vibration such as a heartbeat of a human body or the like can be detected by transmitting and receiving radio waves such as millimeter waves, and the heart rate of the human body can be detected with high robustness in a good system, it can be expected to be useful in a wide variety of fields. An object of the present disclosure is to provide an electronic device, a control method for the electronic device, and a program capable of detecting the heart rate of a human body or the like with high robustness and good accuracy by transmitting and receiving radio waves. According to one embodiment, an electronic device, a control method for the electronic device, and a program capable of detecting the heart rate of a human body or the like with high robustness and good accuracy by transmitting and receiving radio waves can be provided. Hereinafter, one embodiment will be described in detail with reference to the drawings.

[0011] In the present disclosure, the “electronic device” may be a device driven by electric power. Further, the “user” may be a person (typically a human) or an animal who uses the system and / or the electronic device according to an embodiment. The user may include a person who monitors an object such as a human by using the electronic device according to an embodiment. Further, the “object” may be a person (for example, a human or an animal) to be monitored by the electronic device according to an embodiment. Furthermore, the user may include the object.

[0012] In the present disclosure, “heart beat” may be the pulsation of the heart. Further, the “pulsation” may be the rhythmic contraction movement performed by the heart.

[0013] In the present disclosure, the "heart rate" refers to the number of heartbeats within a certain period of time. For example, the heart rate may be the number of beats per minute. When the heart pumps blood, pulsations occur in the arteries. Therefore, the number of arterial pulsations may also be referred to as the pulse rate or simply the pulse.

[0014] Furthermore, in the present disclosure, the "heart sound" may refer to the sound of the heart's pulsation. That is, the heart sound may be the sound that occurs when the heart contracts and expands. Here, the "heart sound" may consist of a low and long first sound based on the tension of the ventricular muscle, the closure of the mitral valve, the start of blood ejection into the artery, and / or the acceleration of blood flow, and a high and short second sound following this, which is derived from the closure of the aortic valve and / or the pulmonary valve.

[0015] In the present disclosure, the heart sound is not necessarily limited to a physical sound based on air vibration, and may also mean the vibration itself caused by the pulsation (beating) of the heart. For example, in the present disclosure, the heart rate may sometimes imply the vibration source, and the heart sound may sometimes imply the vibration itself caused by the vibration source. Furthermore, in the present disclosure, the heart rate may imply the heart sound depending on the situation.

[0016] Also, in the present disclosure, the heart sound may be the vibration of the body accompanying the movement of the heart. The heart sound may be, for example, the vibration of the whole body, or the head, neck, chest, throat, arms, legs, wrists, or other parts of the body. The heart sound may be, for example, the vibration of the skin on the surface of the whole body, or the head, neck, chest, throat, arms, legs, wrists, or other parts of the body. The heart sound may be, for example, the vibration of the clothes, underwear, glasses, or other accessories worn by the subject accompanying the vibration of the skin on the surface of the whole body, or the head, neck, chest, throat, arms, legs, wrists, or other parts of the body. The heart rate may be the pulsation of the heart itself. The heart rate interval or heart rate may be calculated from the movement of the heart rate. In the present disclosure, the pulsation of the heart may also be referred to as the beating.

[0017] The electronic device according to one embodiment can detect the heartbeat of a target such as a person existing around the electronic device. Therefore, the scenarios where the electronic device according to one embodiment is used can be assumed to be, for example, specific facilities used by those engaged in social activities, such as companies, hospitals, nursing homes, schools, sports gyms, and nursing facilities. For example, in a company, it is extremely important to grasp and / or manage the health status of employees and the like. Similarly, in a hospital, it is extremely important to grasp and / or manage the health status of patients and medical staff, etc., and in a nursing home, it is extremely important to grasp and / or manage the health status of residents and staff, etc. The scenarios where the electronic device according to one embodiment is used are not limited to the above-mentioned facilities such as companies, hospitals, and nursing homes, but can be any facility where it is desired to grasp and / or manage the health status of the target. The arbitrary facility may include, for example, a non-commercial facility such as the user's home. Also, the scenarios where the electronic device according to one embodiment is used are not limited to indoors, but can also be outdoors. For example, the scenarios where the electronic device according to one embodiment is used can be inside a moving body such as a train, bus, and airplane, as well as stations and boarding areas. Also, as scenarios where the electronic device according to one embodiment is used, a moving body such as an automobile, airplane, or ship, a hotel, the user's home, the living room at home, the bathroom, the toilet, or the bedroom, etc. may be used.

[0018] An electronic device according to an embodiment may be used, for example, in a nursing facility or the like for detecting or monitoring the heartbeat of a target such as a person requiring nursing care or a person requiring care. Further, when an abnormality is recognized in the heartbeat of a target such as a person requiring nursing care or a person requiring care, the electronic device according to an embodiment may issue a predetermined warning to, for example, the person himself / herself and / or other persons. Therefore, according to the electronic device according to an embodiment, for example, the person himself / herself and / or the staff of a nursing facility or the like can recognize that an abnormality is recognized in the pulse of a target such as a person requiring nursing care or a person requiring care. On the other hand, when no abnormality is recognized (for example, recognized as normal) in the heartbeat of a target such as a person requiring nursing care or a person requiring care, the electronic device according to an embodiment may notify the person himself / herself and / or other persons to that effect. Therefore, according to the electronic device according to an embodiment, for example, the person himself / herself and / or the staff of a nursing facility or the like can recognize that the pulse of a target such as a person requiring nursing care or a person requiring care is normal.

[0019] Further, the electronic device according to an embodiment may detect the pulse of other animals other than humans. The electronic device according to an embodiment described below will be described as an example of detecting the pulse of a human by a sensor based on a technology such as a millimeter-wave radar.

[0020] The electronic device according to an embodiment may be installed on any stationary object or any moving object. The electronic device according to an embodiment can transmit a transmission wave from a transmission antenna to the surroundings of the electronic device. Further, the electronic device according to an embodiment can receive a reflected wave obtained by reflecting the transmission wave from a reception antenna. At least one of the transmission antenna and the reception antenna may be provided in the electronic device, or may be provided in, for example, a radar sensor or the like.

[0021] Hereinafter, as a typical example, the electronic device according to one embodiment will be described as being stationary. On the other hand, the object (human) to be detected for the pulse by the electronic device according to one embodiment may be stationary, may be moving, or may be moving the body in a stationary state. The electronic device according to one embodiment can measure the distance between the electronic device and the object, etc. in a situation where the objects around the electronic device can move, similar to a normal radar sensor. Further, the electronic device according to one embodiment can measure the distance between the electronic device and the object, etc. even when both the electronic device and the object are stationary.

[0022] The electronic device according to one embodiment will be described in detail below with reference to the drawings. First, an example of object detection by the electronic device according to one embodiment will be described.

[0023] FIG. 1 is a diagram for explaining an example of the usage mode of the electronic device according to one embodiment. FIG. 1 shows an example of an electronic device having the functions of a sensor including a transmission antenna and a reception antenna according to one embodiment.

[0024] As shown in FIG. 1, the electronic device 1 according to one embodiment may include a transmission unit and a reception unit described later. As will be described later, the transmission unit may include a transmission antenna array 24. Further, the reception unit may include a reception antenna array 31. The specific configurations of the electronic device 1, the transmission unit, and the reception unit will be described later. FIG. 1 shows a situation where the electronic device 1 includes a transmission antenna array 24 and a reception antenna array 31 for ease of viewing. Further, the electronic device 1 may appropriately include at least a part of other functional units such as at least a part of the signal processing unit 10 (FIG. 2) included in the electronic device 1. Further, the electronic device 1 may include at least a part of other functional units such as at least a part of the signal processing unit 10 (FIG. 2) included in the electronic device 1 outside the electronic device 1. In FIG. 1, the electronic device 1 may be moving, but may be stationary without moving.

[0025] In the example shown in FIG. 1, the electronic device 1 is schematically shown with a transmission unit including a transmission antenna array 24 and a reception unit including a reception antenna array 31. The electronic device 1 may include, for example, a plurality of transmission units and / or a plurality of reception units. The transmission unit may include a transmission antenna array 24 composed of a plurality of transmission antennas. Also, the reception unit may include a reception antenna array 31 composed of a plurality of reception antennas. Here, the positions where the transmission unit and / or the reception unit are installed in the electronic device 1 are not limited to the positions shown in FIG. 1, and may be other positions as appropriate. Also, the number of the transmission unit and / or the reception unit may be any number of one or more according to various conditions (or requirements) such as the detection range and / or detection accuracy of the heartbeat by the electronic device 1.

[0026] As will be described later, the electronic device 1 transmits an electromagnetic wave as a transmission wave from the transmission antenna array 24. For example, when a predetermined object (for example, the target 200 shown in FIG. 1) exists around the electronic device 1, at least a part of the transmission wave transmitted from the electronic device 1 is reflected by the object to become a reflected wave. Then, by receiving such a reflected wave by, for example, the reception antenna array 31 of the electronic device 1, the electronic device 1 can detect the object as a target.

[0027] The electronic device 1 including the transmission antenna array 24 may typically be a radar (Radio Detecting and Ranging) sensor that transmits and receives radio waves. However, the electronic device 1 is not limited to a radar sensor. The electronic device 1 according to one embodiment may be, for example, a sensor based on the technology of LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) using light waves. Such sensors can be configured to include, for example, patch antennas. Since technologies such as RADAR and LIDAR are already known, detailed descriptions may be appropriately simplified or omitted. Further, the electronic device 1 according to one embodiment may be, for example, a sensor based on a technology that transmits and receives sound waves or ultrasonic waves to detect an object.

[0028] The electronic device 1 shown in FIG. 1 receives a reflected wave of a transmission wave transmitted from the transmission antenna array 24 from the reception antenna array 31. In this way, the electronic device 1 can detect a predetermined target 200 existing within a predetermined distance from the electronic device 1. For example, as shown in FIG. 1, the electronic device 1 can measure the distance L between the electronic device 1 and the predetermined target 200. Further, the electronic device 1 can also measure the relative speed between the electronic device 1 and the predetermined target 200. Furthermore, the electronic device 1 can also measure the direction (arrival angle θ) in which the reflected wave from the predetermined target 200 arrives at the electronic device 1.

[0029] In FIG. 1, the XY plane may be, for example, a plane substantially parallel to the ground surface. In this case, the positive direction of the Z axis shown in FIG. 1 may indicate vertically upward. In FIG. 1, the electronic device 1 may be arranged on a plane parallel to the XY plane. Also, in FIG. 1, the target 200 may be, for example, in a state of standing on the ground surface substantially parallel to the XY plane.

[0030] Here, the target 200 may be, for example, a human being existing around the electronic device 1. Also, the target 200 may be a living organism other than a human being, such as an animal existing around the electronic device 1. As described above, the target 200 may be moving or may be stopped or stationary. In the present disclosure, the object detected by the electronic device 1 includes not only inanimate objects such as arbitrary objects but also living organisms such as humans, dogs, cats, horses, and other animals. The object detected by the electronic device 1 of the present disclosure may include a target including a person, an object, an animal, etc., detected by radar technology. In the present disclosure, the target may include a person, an object, an animal, etc. Hereinafter, an object such as the target 200 existing around the electronic device 1 will be described assuming that it is a human (or an animal). Hereinafter, the "target 200" will also be referred to as the "subject 200" as appropriate. In the present disclosure, the target may be the above-mentioned target 200.

[0031] In FIG. 1, the ratio between the size of the electronic device 1 and the size of the target 200 does not necessarily indicate the actual ratio. Also, in FIG. 1, the transmission antenna array 24 of the transmission unit and the reception antenna array 31 of the reception unit are shown in a state of being installed outside the electronic device 1. However, in one embodiment, the transmission antenna array 24 of the transmission unit and / or the reception antenna array 31 of the reception unit may be installed at various positions of the electronic device 1. For example, in one embodiment, the transmission antenna array 24 of the transmission unit and / or the reception antenna array 31 of the reception unit may be installed inside the electronic device 1 so as not to appear on the exterior of the electronic device 1.

[0032] Hereinafter, as a typical example, the transmission antenna of the electronic device 1 will be described as transmitting radio waves in a frequency band such as millimeter waves (30 GHz or higher) or quasi-millimeter waves (for example, around 20 GHz to 30 GHz). On the other hand, the transmission antenna of the electronic device 1 may transmit radio waves having a frequency bandwidth of 4 GHz, such as, for example, 77 GHz to 81 GHz.

[0033] FIG. 2 is a functional block diagram schematically showing a configuration example of the electronic device 1 according to one embodiment. Hereinafter, an example of the configuration of the electronic device 1 according to one embodiment will be described.

[0034] When measuring the distance and the like by a millimeter-wave radar, a frequency-modulated continuous-wave radar (hereinafter referred to as an FMCW radar (Frequency Modulated Continuous Wave radar)) is often used. The FMCW radar generates a transmission signal by sweeping the frequency of the transmitted radio wave. Therefore, for example, in a millimeter-wave FMCW radar using radio waves in the 79 GHz frequency band, the frequency of the radio wave to be used has a frequency bandwidth of 4 GHz, such as, for example, 77 GHz to 81 GHz. The radar in the 79 GHz frequency band has a characteristic that the available frequency bandwidth is wider than that of other millimeter-wave / quasi-millimeter-wave radars such as, for example, the 24 GHz, 60 GHz, and 76 GHz frequency bands. Hereinafter, such an embodiment will be described as an example.

[0035] The FMCW radar method used in the present disclosure may include the FCM method (Fast-Chirp Modulation) that transmits a chirp signal at a period shorter than normal. The signal generated by the electronic device 1 is not limited to the FMCW signal. The signal generated by the electronic device 1 may be a signal of various methods other than the FMCW method. The transmission signal sequence stored in any storage unit may be different depending on these various methods. For example, in the case of the above-described FMCW radar signal, signals whose frequencies increase and decrease for each time sample may be used. Since the above-described various methods can appropriately apply known techniques, a more detailed description will be omitted.

[0036] As shown in FIG. 2, an electronic device 1 according to an embodiment includes a signal processing unit 10. The signal processing unit 10 may include a signal generation processing unit 11, a received signal processing unit 12, a heartbeat extraction unit 13, and a calculation unit 14. The heartbeat extraction unit 13 may execute a process of extracting, for example, a micro-Doppler component. Also, the heartbeat extraction unit 13 may execute a process of extracting an envelope of the heart sound of the subject 200. The calculation unit 14 may execute a process of calculating, for example, the R-R interval (RRI) of the subject 200. Also, the calculation unit 14 may execute a process of calculating, for example, the heartbeat of the subject 200. Further, the calculation unit 14 may execute a process of calculating, for example, the heart rate variability (HRV) of the subject 200. In this case, the calculation unit 14 may execute a process of performing a frequency analysis of the time-series data of the extracted R-R interval of the subject 200. Also, the calculation unit 14 may execute a process of calculating the heart rate variability of the subject 200 based on the frequency analysis of the time-series data of the R-R interval. In the present disclosure, the calculation unit 14 may calculate the R-R interval and use it to calculate the heart rate variability. The signal generation processing unit 11, the received signal processing unit 12, the heartbeat extraction unit 13, and the calculation unit 14 will be described in more detail as appropriate later. In the present disclosure, the heart sound may be, for example, a chest vibration waveform (see FIG. 20, etc.) directly observed by a radar, or may be a chest vibration. The heartbeat is the beating of the heart itself. It may be assumed that the R-R interval, the heart rate, etc. are calculated from the movement of the heartbeat. The R-R interval may be the time interval between the beating of the heart and the next beating of the heart.

[0037] Also, the electronic device 1 according to one embodiment includes, as a transmission unit, a transmission DAC 21, a transmission circuit 22, a millimeter-wave transmission circuit 23, and a transmission antenna array 24. Further, the electronic device 1 according to one embodiment includes, as a reception unit, a reception antenna array 31, a mixer 32, a reception circuit 33, and a reception ADC 34. The electronic device 1 according to one embodiment may not include at least any of the functional units shown in FIG. 2, or may include functional units other than the functional units shown in FIG. 2. The electronic device 1 shown in FIG. 2 may be configured using a circuit configured in basically the same manner as a general radar using electromagnetic waves in a millimeter-wave band or the like. On the other hand, in the electronic device 1 according to one embodiment, the signal processing by the signal processing unit 10 may include processing different from that of a conventional general radar.

[0038] The signal processing unit 10 included in the electronic device 1 according to one embodiment can control the operation of the entire electronic device 1, including the control of each functional unit constituting the electronic device 1. In particular, the signal processing unit 10 performs various processes on the signals handled by the electronic device 1. The signal processing unit 10 may include at least one processor, such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), to provide control and processing capabilities for executing various functions. The signal processing unit 10 may be realized by a single processor, may be realized by several processors, or may be realized by individual processors respectively. The processor may be realized as a single integrated circuit. The integrated circuit is also referred to as an IC (Integrated Circuit). The processor may be realized as a plurality of communicably connected integrated circuits and discrete circuits. The processor may be realized based on various other known technologies. In one embodiment, the signal processing unit 10 may be configured, for example, as a CPU (hardware) and a program (software) executed by the CPU. The signal processing unit 10 may appropriately include a storage unit (memory) necessary for the operation of the signal processing unit 10.

[0039] The signal generation processing unit 11 of the signal processing unit 10 generates a signal to be transmitted from the electronic device 1. In the electronic device 1 according to an embodiment, the signal generation processing unit 11 may generate a transmission signal (transmission chirp signal) such as a chirp signal. In particular, the signal generation processing unit 11 may generate a signal whose frequency linearly changes periodically (linear chirp signal). For example, the signal generation processing unit 11 may generate a chirp signal whose frequency periodically and linearly increases from 77 GHz to 81 GHz as time elapses. Also, for example, the signal generation processing unit 11 may generate a signal whose frequency periodically repeats linear increase (up chirp) and decrease (down chirp) from 77 GHz to 81 GHz as time elapses. The signal generated by the signal generation processing unit 11 may be preset, for example, in the signal processing unit 10. Also, the signal generated by the signal generation processing unit 11 may be pre-stored, for example, in an arbitrary storage unit in the signal processing unit 10. Since the chirp signal used in the technical field such as radar is known, a more detailed description will be appropriately simplified or omitted. The signal generated by the signal generation processing unit 11 is supplied to the transmission DAC 21. For this reason, the signal generation processing unit 11 may be connected to the transmission DAC 21.

[0040] The transmission DAC (Digital - Analog Converter) 21 has a function of converting the digital signal supplied from the signal generation processing unit 11 into an analog signal. The transmission DAC 21 may be configured to include a general digital - analog converter. The signal analogized by the transmission DAC 21 is supplied to the transmission circuit 22. For this reason, the transmission DAC 21 may be connected to the transmission circuit 22.

[0041] The transmission circuit 22 has a function of converting the signal analogized by the transmission DAC 21 into a band of an intermediate frequency (IF). The transmission circuit 22 may be configured to include a general IF - band transmission circuit. The signal processed by the transmission circuit 22 is supplied to the millimeter - wave transmission circuit 23. For this reason, the transmission circuit 22 may be connected to the millimeter - wave transmission circuit 23.

[0042] The millimeter-wave transmission circuit 23 has a function of transmitting the signal processed by the transmission circuit 22 as a millimeter wave (RF wave). The millimeter-wave transmission circuit 23 may be configured to include a general millimeter-wave transmission circuit. The signal processed by the millimeter-wave transmission circuit 23 is supplied to the transmission antenna array 24. Therefore, the millimeter-wave transmission circuit 23 may be connected to the transmission antenna array 24. Also, the signal processed by the millimeter-wave transmission circuit 23 is also supplied to the mixer 32. Therefore, the millimeter-wave transmission circuit 23 may also be connected to the mixer 32.

[0043] The transmission antenna array 24 is formed by arranging a plurality of transmission antennas in an array. In FIG. 2, the configuration of the transmission antenna array 24 is shown in a simplified manner. The transmission antenna array 24 transmits the signal processed by the millimeter-wave transmission circuit 23 to the outside of the electronic device 1. The transmission antenna array 24 may be configured to include a transmission antenna array used in a general millimeter-wave radar.

[0044] In this way, the electronic device 1 according to one embodiment includes a transmission antenna (transmission antenna array 24), and can transmit a transmission signal (for example, a transmission chirp signal) as a transmission wave from the transmission antenna array 24.

[0045] For example, as shown in FIG. 2, assume a case where an object such as a subject 200 exists around the electronic device 1. In this case, at least a part of the transmission wave transmitted from the transmission antenna array 24 is reflected by an object such as the subject 200. At least a part of the transmission wave transmitted from the transmission antenna array 24 that is reflected by an object such as the subject 200 can be reflected toward the reception antenna array 31.

[0046] The reception antenna array 31 receives the reflected wave. Here, the reflected wave may be at least a part of the transmission wave transmitted from the transmission antenna array 24 that is reflected by an object such as the subject 200.

[0047] The receiving antenna array 31 is formed by arranging a plurality of receiving antennas in an array. In FIG. 2, the configuration of the receiving antenna array 31 is shown in a simplified manner. The receiving antenna array 31 receives a reflected wave obtained by reflecting the transmitted wave transmitted from the transmitting antenna array 24. The receiving antenna array 31 may be configured to include a receiving antenna array used in a general millimeter-wave radar. The receiving antenna array 31 supplies the received signal received as a reflected wave to the mixer 32. For this reason, the receiving antenna array 31 may be connected to the mixer 32.

[0048] The mixer 32 converts the signal (transmission signal) processed by the millimeter-wave transmission circuit 23 and the received signal received by the receiving antenna array 31 into an intermediate frequency (IF) band. The mixer 32 may be configured to include a mixer used in a general millimeter-wave radar. The mixer 32 supplies the signal generated as a synthesized result to the receiving circuit 33. For this reason, the mixer 32 may be connected to the receiving circuit 33.

[0049] The receiving circuit 33 has a function of analog-processing the signal converted into the IF band by the mixer 32. The receiving circuit 33 may be configured to include a receiving circuit that converts to a general IF band. The signal processed by the receiving circuit 33 is supplied to the receiving ADC 34. For this reason, the receiving circuit 33 may be connected to the receiving ADC 34.

[0050] The receiving ADC (analog-digital converter) 34 has a function of converting the analog signal supplied from the receiving circuit 33 into a digital signal. The receiving ADC 34 may be configured to include a general analog-digital converter. The signal digitized by the receiving ADC 34 is supplied to the receiving signal processing unit 12 of the signal processing unit 10. For this reason, the receiving ADC 34 may be connected to the signal processing unit 10.

[0051] The reception signal processing unit 12 of the signal processing unit 10 has a function of performing various processes on the digital signal supplied from the reception DAC 34. For example, the reception signal processing unit 12 calculates the distance from the electronic device 1 to an object such as the subject 200 (range measurement) based on the digital signal supplied from the reception DAC 34. Also, the reception signal processing unit 12 calculates the relative speed of an object such as the subject 200 with respect to the electronic device 1 based on the digital signal supplied from the reception DAC 34 (speed measurement). Further, the reception signal processing unit 12 calculates the azimuth angle of an object such as the subject 200 as seen from the electronic device 1 based on the digital signal supplied from the reception DAC 34 (angle measurement). Specifically, I / Q converted data may be input to the reception signal processing unit 12. When such data is input, the reception signal processing unit 12 performs two-dimensional fast Fourier transforms (2D-FFT) in the range direction and the velocity direction, respectively. Then, the reception signal processing unit 12 performs suppression of false alarms and constant probability by removing noise points by a process such as CFAR (Constant False Alarm Rate). And the reception signal processing unit 12 obtains the position of an object such as the subject 200 by performing arrival angle estimation on points that satisfy the CFAR criterion. The information generated as a result of range measurement, speed measurement, and angle measurement by the reception signal processing unit 12 may be supplied to the heartbeat extraction unit 13.

[0052] The heartbeat extraction unit 13 extracts information related to the heartbeat from the information generated by the reception signal processing unit 12. The operation of extracting information related to the heartbeat by the heartbeat extraction unit 13 will be described later in more detail. The information related to the heartbeat extracted by the heartbeat extraction unit 13 may be supplied to the calculation unit 14.

[0053] The calculation unit 14 performs various calculation processes and / or arithmetic operations on the information regarding the heartbeat supplied from the heartbeat extraction unit 13. The various calculation processes and / or arithmetic operations by the calculation unit 14 will be described in further detail later. The various information calculated and / or arithmetically processed by the calculation unit 14 may be supplied to, for example, the communication interface 50. For this reason, the calculation unit 14 and / or the signal processing unit 10 may be connected to the communication interface 50. The various information calculated and / or arithmetically processed by the calculation unit 14 may be supplied to other functional units other than the communication interface 50.

[0054] The communication interface 50 is configured to include an interface that outputs the information supplied from the signal processing unit 10 to, for example, an external device 60. The communication interface 50 may output information on at least any one of the position, speed, and angle of an object such as the subject 200 to the external device 60 as a signal such as, for example, CAN (Controller Area Network). For example, information on at least any one of the position, speed, and angle of an object such as the subject 200 may be supplied to the external device 60 via the communication interface 50. For this reason, the communication interface 50 may be connected to the external device 60.

[0055] As shown in FIG. 2, the electronic device 1 according to one embodiment may be connected to an external device 60 by wire or wirelessly via the communication interface 50. In one embodiment, the external device 60 may be configured to include any computer and / or any control device, etc. Also, the electronic device 1 according to one embodiment may be configured to include the external device 60. The external device 60 can be configured in various ways according to the mode in which the information on the heartbeat and / or heart sound detected by the electronic device 1 is utilized. Therefore, a more detailed description of the external device 60 will be omitted.

[0056] FIG. 3 is a diagram for explaining an example of a chirp signal generated by the signal generation processing unit 11 of the signal processing unit 10.

[0057] Figure 3 shows the time structure of one frame when the FCM (Fast-Chirp Modulation) method is used. Figure 3 shows an example of the received signal of the FCM method. FCM is a method of repeating chirp signals shown as c1, c2, c3, c4, …, cn in Figure 3 at short intervals (for example, longer than the round-trip time between the radar and the target of the electromagnetic wave calculated from the maximum ranging distance). In FCM, for the convenience of signal processing of the received signal, it is often divided into sub-frame units as shown in Figure 3 and the transmission and reception processes are performed.

[0058] In Figure 3, the horizontal axis represents the elapsed time and the vertical axis represents the frequency. In the example shown in Figure 3, the signal generation processing unit 11 generates a linear chirp signal whose frequency changes linearly periodically. In Figure 3, each chirp signal is shown as c1, c2, c3, c4, …, cn. As shown in Figure 3, in each chirp signal, the frequency increases linearly with the passage of time.

[0059] In the example shown in Figure 3, several chirp signals such as c1, c2, c3, c4, …, cn are included as one sub-frame. That is, the sub-frame 1 and sub-frame 2 shown in Figure 3 are each composed of several chirp signals such as c1, c2, c3, c4, …, cn. Also, in the example shown in Figure 3, several sub-frames such as sub-frame 1, sub-frame 2, …, sub-frame N are included as one frame (one frame). That is, one frame shown in Figure 3 is composed of N sub-frames. Also, taking the one frame shown in Figure 3 as frame 1, then frame 2, frame 3, …, etc. may follow. These frames may each be composed of N sub-frames in the same way as frame 1. Also, a frame interval of a predetermined length may be included between the frames. One frame shown in Figure 3 may have a length of about 30 milliseconds to 50 milliseconds, for example.

[0060] In the electronic device 1 according to one embodiment, the signal generation processing unit 11 may generate a transmission signal as any number of frames. Also, in FIG. 3, some of the chirp signals are shown omitted. Thus, the relationship between the time and frequency of the transmission signal generated by the signal generation processing unit 11 may be stored, for example, in the storage unit of the signal processing unit 10 or the like.

[0061] As described above, the electronic device 1 according to one embodiment may transmit a transmission signal composed of sub-frames including a plurality of chirp signals. Also, the electronic device 1 according to one embodiment may transmit a transmission signal composed of frames including a predetermined number of sub-frames.

[0062] Hereinafter, the electronic device 1 will be described as transmitting a transmission signal having a frame structure as shown in FIG. 3. However, the frame structure as shown in FIG. 3 is an example, and for example, the chirp signals included in one sub-frame may be arbitrary. That is, in one embodiment, the signal generation processing unit 11 may generate a sub-frame including an arbitrary number (for example, an arbitrary plurality) of chirp signals. Also, the sub-frame structure as shown in FIG. 3 is an example, and for example, the sub-frames included in one frame may be arbitrary. That is, in one embodiment, the signal generation processing unit 11 may generate a frame including an arbitrary number (for example, an arbitrary plurality) of sub-frames. The signal generation processing unit 11 may generate signals of different frequencies. The signal generation processing unit 11 may generate a plurality of discrete signals having different bandwidths with each frequency f being different.

[0063] FIG. 4 is a diagram showing a part of the sub-frame shown in FIG. 3 in another aspect. FIG. 4 shows each sample of the received signal obtained by receiving the transmission signal shown in FIG. 3 as a result of performing 2D-FFT (Two Dimensional Fast Fourier Transform), which is the processing performed in the reception signal processing unit 12 (FIG. 2) of the signal processing unit 10.

[0064] As shown in FIG. 4, in each sub-frame such as sub-frame 1, …, sub-frame N, each chirp signal c1, c2, c3, c4, …, cn is stored. In FIG. 4, each chirp signal c1, c2, c3, c4, …, cn is composed of each sample indicated by the grids arranged horizontally. The received signal shown in FIG. 4 is subjected to 2D-FFT, CFAR, and / or integrated signal processing of each sub-frame by the received signal processing unit 12 shown in FIG. 2.

[0065] FIG. 5 is a diagram showing an example in which a point group on a range-Doppler (distance-velocity) plane is calculated as a result of performing 2D-FFT, CFAR, and integrated signal processing of each sub-frame in the received signal processing unit 12 shown in FIG. 2.

[0066] In FIG. 5, the horizontal direction represents the range (distance), and the vertical direction represents the velocity. The filled grid s1 shown in FIG. 5 indicates a point group of signals that exceed the threshold processing of CFAR. The unfilled grid s2 shown in FIG. 5 indicates a bin (2D-FFT sample) without a point group that does not exceed the threshold of CFAR. The point group on the range-Doppler plane calculated in FIG. 5 has its azimuth from the radar calculated by direction estimation, and the position and velocity on the two-dimensional plane are calculated as a point group indicating an object such as the subject 200. Here, the direction estimation may be calculated by a beamformer and / or a subspace method. Representative subspace method algorithms include MUSIC (MUltiple SIgnal Classification) and ESPRIT (Estimation of Signal Parameters via Rotation Invariance Technique).

[0067] FIG. 6 is a diagram showing an example of the result of the conversion of the point group coordinates from the range-Doppler plane shown in FIG. 5 to the XY plane after the reception signal processing unit 12 performs direction estimation. As shown in FIG. 6, the reception signal processing unit 12 can plot the point group PG on the XY plane. Here, the point group PG is composed of each point P. Also, each point P has an angle θ and a radial velocity Vr in polar coordinates.

[0068] The reception signal processing unit 12 detects an object existing in the range where the transmission wave T is transmitted based on at least one of the results of 2D-FFT and angle estimation. The reception signal processing unit 12 may perform object detection, for example, by performing clustering processing based on the respectively estimated distance information, velocity information, and angle information. As an algorithm used when clustering data, for example, DBSCAN (Density-based spatial clustering of applications with noise) is known. This is an algorithm that performs density-based clustering. In the clustering process, for example, the average power of the points constituting the detected object may be calculated. The distance information, velocity information, angle information, and power information of the object detected by the reception signal processing unit 12 may be supplied to an external device 60 or the like, for example, via the communication interface 50.

[0069] As described above, the electronic device 1 may include a transmission antenna (transmission antenna array 24), a reception antenna (reception antenna array 31), and a signal processing unit 10. The transmission antenna array 24 transmits the transmission wave T. The reception antenna array 31 receives the reflected wave R obtained by reflecting the transmission wave T. Then, the signal processing unit 10 detects an object (such as an object like the subject 200) that reflects the transmission wave T based on the transmission signal transmitted as the transmission wave T and the reception signal received as the reflected wave R.

[0070] Next, the direction estimation of the incoming wave by the antenna array of the electronic device 1 according to an embodiment will be further described.

[0071] FIG. 7 is a diagram for explaining the configuration of the receiving antenna array 31 of the electronic device 1 according to an embodiment and the principle of estimating the direction of an incoming wave by the receiving antenna array 31. FIG. 7 shows an example of radio wave reception by the receiving antenna array 31.

[0072] As shown in FIG. 7, the receiving antenna array 31 may be formed by arranging sensors such as receiving antennas in a straight line. As shown in FIG. 7, in one embodiment, the receiving antenna array 31 may be configured to include a plurality of receiving antennas arranged in a straight line. In FIG. 7, the receiving antenna array 31 is shown with a plurality of antennas such as antenna x1, x2, x3, …, x M indicated by small circles. The receiving antenna array 31 may be configured by any plurality of antennas. Also, as shown in FIG. 7, the plurality of antennas constituting the receiving antenna array 31 are assumed to be arranged at intervals of an array pitch d. In this way, a sensor array in which sensors (such as antennas, ultrasonic transducers, and microphones) corresponding to various physical waves are arranged in an array is also called a Uniform Linear Array (ULA). As shown in FIG. 7, physical waves (such as electromagnetic waves and sound waves) arrive from various directions such as θ1 and θ2, for example. Here, θ1 and θ2 may be the incoming angles described above. In this way, a sensor array such as the receiving antenna array 31 can estimate the incoming direction (incoming angle) by using the phase difference generated in the measurements between the sensors according to the incoming direction of the physical wave. In this way, the method of estimating the incoming direction of a wave is also referred to as incoming angle estimation or Direction of Arrival (DoA).

[0073] In the electronic device 1 according to one embodiment, at least one of the transmission antenna array 24 and the reception antenna array 31 may be configured by arranging a plurality of antennas linearly. Thereby, for example, in a millimeter-wave radar, the directivity during transmission and reception of radio waves can be appropriately narrowed. When transmitting a transmission wave, the direction of the transmission beam is often controlled by a beamformer. On the other hand, when receiving a reflected wave, the arrival direction of the reflected wave is often estimated by a subspace method (such as MUSIC and ESPRIT described above) rather than a beamformer. In the beamformer and the subspace method, in a ULA as shown in FIG. 7, for electromagnetic waves arriving from various directions, a phase difference occurs in the measurement values between sensors according to the arrival direction. Therefore, the arrival direction of the reflected wave can be estimated using the phase difference.

[0074] Next, the two-direction angle estimation of the arrival wave by the antenna array of the electronic device 1 according to one embodiment will be further described.

[0075] FIG. 8 is a diagram showing an example of the arrangement of antennas for estimating the arrival directions of two orthogonal angles.

[0076] As shown in FIG. 8, in the electronic device 1 according to one embodiment, the transmission antenna array 24 and / or the reception antenna array 31 may be configured to include an array of a plurality of patch antenna units.

[0077] In the transmission antenna array 24 shown in FIG. 8, one patch antenna unit may be configured to include a plurality of elements electrically connected in the direction of direction 1 shown in the figure. In each patch antenna unit, the plurality of elements may be electrically connected by wiring such as a strip line on a substrate. In each patch antenna unit, each of the plurality of elements may be spaced apart by a distance d shorter than half of the wavelength λ of the transmission wave. 1,t In FIG. 8, each patch antenna unit may be configured by electrically connecting two or more arbitrary numbers of elements.

[0078] Also, as shown in FIG. 8, the transmission antenna array 24 may be arranged by arraying a plurality of patch antenna units in the direction 2 shown in the figure. Each patch antenna unit may be arranged at an interval d shorter than half of the wavelength λ of the transmission wave. 2,t In one embodiment, the transmission antenna array 24 may include any number of two or more patch antenna units.

[0079] As shown in FIG. 8, in one embodiment, the reception antenna array 31 may be obtained by changing the arrangement of a plurality of elements in the transmission antenna array 24. That is, in the reception antenna array 31 shown in FIG. 8, one patch antenna unit may be configured to include a plurality of elements electrically connected in the direction 2 shown in the figure. In each patch antenna unit, the plurality of elements may be electrically connected by wiring such as a strip line on a substrate, for example. In each patch antenna unit, each of the plurality of elements may be arranged at an interval d shorter than half of the wavelength λ of the transmission wave. 2,s In FIG. 8, each patch antenna unit may be configured by electrically connecting any number of two or more elements.

[0080] Also, as shown in FIG. 8, the reception antenna array 31 may be arranged by arraying a plurality of patch antenna units in the direction 1 shown in the figure. Each patch antenna unit may be arranged at an interval d shorter than half of the wavelength λ of the transmission wave. 1,s In one embodiment, the reception antenna array 31 may include any number of two or more patch antenna units.

[0081] All the elements included in the transmission antenna array 24 and the reception antenna array 31 may be arranged on the same plane (for example, on the surface layer of the same substrate). Also, the transmission antenna array 24 and the reception antenna array 31 may be arranged close to each other (monostatic). Further, the direction 1 and the direction 2 shown in FIG. 8 may be geometrically orthogonal.

[0082] The directivities of the transmitting antenna and the receiving antenna can be appropriately narrowed by the transmitting antenna array 24 and the receiving antenna array 31 as shown in FIG. 8. Also, by using the transmitting antenna array 24 as shown in FIG. 8 and controlling the direction of each transmitted wave at each timing of transmitting the transmitted wave (transmission signal), a beamformer for the direction of direction 2 shown in FIG. 8 can be realized. Further, by using the receiving antenna array 31 as shown in FIG. 8, it is possible to estimate the arrival direction of the reflected wave for the direction of direction 1 shown in FIG. 8. In this way, it becomes possible to estimate the arrival direction of the reflected wave for two substantially orthogonal angles. Therefore, it becomes possible to three-dimensionally acquire a point group indicating an object such as the subject 200.

[0083] Next, a method for detecting the heartbeat of the subject 200 by the electronic device 1 according to an embodiment will be described.

[0084] The electronic device 1 according to an embodiment transmits a transmitted wave such as a millimeter-wave radar to the subject 200, and measures (estimates) the heartbeat of the subject 200 based on the result of receiving the reflected wave reflected from the chest where the heart of the subject 200 exists. As described above, the subject 200 may be a human or an animal. In this case, for example, by frequency-filtering the vibration of the existence position of the subject 200 detected by the radar, a component assumed to be the envelope of the heartbeat can be extracted. Then, when the component assumed to be the envelope of the heartbeat is extracted, the approximate heartbeat interval can be calculated by using the interval between the peaks of the envelope as the heartbeat interval. Here, an approximation that the peak of the heartbeat envelope approximately coincides with the R peak of the electrocardiogram can be used. For this reason, the "heartbeat interval" is also denoted as the RR interval or RRI (RR interval), similar to the term used in an electrocardiogram or the like.

[0085] Here, a method for estimating the heart rate interval of subject 200 will be examined based on the results of performing the above-described 2D-FFT, CFAR processing, direction-of-arrival estimation, etc. First, the results of the 2D-FFT performed in FIGS. 4 and 5 will be used to explain the manifestation patterns of human heartbeats or body movements.

[0086] FIG. 9 is a diagram showing an example of the result of receiving the reflected wave of the transmitted wave transmitted to subject 200 and performing 2D-FFT processing. FIG. 9 shows a spectrum indicating the heart sound and body movement of subject 200 as a result of the 2D-FFT. In FIG. 9, the horizontal axis represents distance (Range), and the vertical axis represents velocity (Velocity). The signal processing unit 10 (for example, the heartbeat extraction unit 13) of the electronic device 1 according to one embodiment may extract, for example, a peak Hm as shown in FIG. 9 as a body movement such as the heartbeat of subject 200. Here, the spectral components indicated by the peak Hm in FIG. 9 include not only the heart sound of subject 200 and the envelope of the heart sound but also body movement. In order to extract the heart rate interval, it is necessary to extract body movement, etc., so it is assumed that, for example, frequency filtering is performed. The frequency filtering performed here is assumed to be, for example, a band-pass filter, a high-pass filter, and / or a low-pass filter targeting 0.5 Hz or more and 10 Hz or less.

[0087] FIG. 10 is a diagram for explaining a method of detecting a peak based on the envelope waveform of the heart sound obtained by the above-described frequency filtering. The graph shown in FIG. 10 shows an example of the time change of the envelope waveform of the heart sound extracted by the above-described frequency filtering. The envelope waveform shown in FIG. 10 includes a large number of peaks as shown. On the other hand, it is known that the pulsation of subject 200 falls within a range of about 50 to 130 per minute. Therefore, by selecting peaks having an interval of 0.4 seconds to 0.8 seconds, which is the reciprocal of the pulsation rate per minute, from among the large number of peaks shown in FIG. 10, the approximate heart sound interval of subject 200 can be calculated. For example, the peak indicated by the downward arrow in FIG. 10 may be selected as the approximate heart sound interval of subject 200.

[0088] FIG. 11 is a flowchart showing an example of the operation of estimating the heartbeat interval described above. Hereinafter, with reference to FIG. 11, the operation of estimating the heartbeat interval described above will be outlined.

[0089] FIG. 11 shows the operation of the electronic device 1 according to an embodiment after receiving the reflected wave. That is, as a premise of the operation shown in FIG. 11, the electronic device 1 shown in FIG. 2 transmits a transmission wave (transmission signal) from the transmission antenna array 24. Then, at least a part of the transmission wave transmitted from the electronic device 1 is reflected by the subject 200 (e.g., the chest) to become a reflected wave. Then, the electronic device 1 shown in FIG. 2 receives such a reflected wave from the reception antenna array 31. Then, the operation shown in FIG. 11 starts.

[0090] When the operation shown in FIG. 11 starts, first, in step S110, the signal processing unit 10 of the electronic device 1 processes the received signal (received signal). The signal processing performed in step S110 may include, for example, the above-described 2D-FFT, CFAR processing, and / or direction-of-arrival estimation. Such an operation may be performed, for example, by the received signal processing unit 12 of the signal processing unit 10.

[0091] Next, in step S120, the signal processing unit 10 extracts the vibration source from the information processed in step S110. The signal processing performed in step S120 may include, for example, a process of filtering the data obtained as a result of performing the 2D-FFT process. In step S120, the signal processing unit 10 may extract the spectral components only at the position where the subject 200 exists. Here, the position where the subject 200 exists may be specified by various known methods. Such an operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.

[0092] Next, in step S130, the signal processing unit 10 converts the processing result of the previous stage into a vibration waveform. The processing performed in step S130 may include, for example, a process of extracting phase information from IQ data. In step S130, the signal processing unit 10 may include a process of extracting vibration data including heart sounds from the spectral components of the 2D-FFT processing of the subject 200 extracted in step S120. Such an operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.

[0093] Next, in step S140, the signal processing unit 10 extracts vibration data from the processing result of the previous stage. The processing performed in step S140 may include, for example, frequency filtering. In step S140, the signal processing unit 10 may extract a low-frequency signal including the envelope of the heart sound of the subject 200 by performing frequency filtering. Such an operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.

[0094] Next, in step S150, the signal processing unit 10 detects the peak of the pulsation of the subject 200 from the processing result of the previous stage, and calculates the RR interval (RRI) of the subject 200 by calculating the interval between the peaks. In step S150, the signal processing unit 10 may detect the peak of the low-frequency signal including the envelope of the heart sound of the subject 200. Also, in step S150, the signal processing unit 10 may calculate and / or extract the interval (interval) between each peak. Such an operation may be performed, for example, by the calculation unit 14 of the signal processing unit 10. As described above, in step S150, the signal processing unit 10 may extract the interval between heart sounds.

[0095] Next, in step S160, the signal processing unit 10 may calculate the heart rate variability (HRV) of the subject 200. In step S160, the signal processing unit 10 may calculate the HRV of the subject 200 by calculating the spectral density of the time series data of the RRI. When calculating the power spectral density of the time series data of the RRI, frequency analysis of the waveform of the time series of the RRI may be performed by using, for example, Welch's method or the like. Such an operation may be performed by, for example, the calculation unit 14 of the signal processing unit 10. Also, in step S160, the signal processing unit 10 may analyze the spectrum from the processing result of the previous stage.

[0096] As described above, the electronic device 1 according to one embodiment extracts a component that seems to be the envelope of the heartbeat by frequency filtering the vibration at the position where the subject 200 is present, and sets the interval between the peaks of the envelope as the heartbeat interval. In this way, the electronic device 1 according to one embodiment can calculate the approximate heartbeat interval of the subject 200.

[0097] In the calculation of the heartbeat interval as described above, for example, as shown in FIG. 10, several peaks exist during a time of several 10 ms. Therefore, a certain degree of uncertainty is included when selecting the peak of the heartbeat. For this reason, the accuracy of the calculated heartbeat interval also has an error of about several 10 ms. For example, in the calculation of the heartbeat interval described above, when compared with the instantaneous RRI acquired by an electrocardiograph, an error of at least about 20 ms occurs. With this accuracy, it is difficult to calculate the heart rate variability (HRV) and perform human autonomic nerve and / or emotion analysis. That is, in the calculation of the heartbeat interval described above, it is difficult to perform more advanced medical analysis by extracting heart sounds. Also, the calculation of the heartbeat interval described above tends to generate errors if the measurement environment is not good. For this reason, in the calculation of the heartbeat interval described above, depending on the measurement environment, it may be difficult to obtain a highly robust pulse rate.

[0098] Therefore, for example, it is also conceivable to further cut the peak on the high-frequency side by frequency filtering so that the time interval between heartbeats is about 0.4 to 0.8 seconds. However, since the accuracy of the signal information shown in FIG. 10 cannot be exceeded, it seems difficult to reduce the error by about several tens of milliseconds.

[0099] Also, in the calculation of the above-mentioned heartbeat interval, heart sounds are not extracted. Therefore, in the above-mentioned method, it seems difficult to obtain information contributing to the diagnosis based on the acoustic properties of the heart sound itself (for example, auscultation in medical treatment, etc.).

[0100] To measure the heart rate of a person or animal by a technology such as radar, it is conceivable to perform frequency filtering on the vibration of the existence position of the detected target (person or animal) to extract a component that seems to be the envelope of the heartbeat, and then perform frequency analysis on the result. However, to obtain the heart rate with high accuracy and high robustness, it is necessary to appropriately search the data space formed by the radar data and estimate the heart rate. That is, it is necessary to estimate the heart rate for the results of extracting chest vibrations corresponding to heart sounds from a plurality of locations (radar ranges), calculate candidates for time series data of a plurality of heart rates, and then determine the optimal heart rate data from among them. The chest vibration corresponding to the heart sound may be simply referred to as "heart sound" in the present disclosure. It is desirable to extract the heart sound by a radar using a high-frequency band of millimeter waves or higher, analyze the heart sound itself, and realize accurate extraction of the heartbeat interval.

[0101] Therefore, the electronic device 1 according to an embodiment further improves the above-mentioned method. As a result, the electronic device 1 according to an embodiment extracts the heart sound by a radar using a high-frequency band of millimeter waves or higher, for example, analyzes the heart sound itself, and extracts an accurate heartbeat interval. In this way, the electronic device 1 according to an embodiment detects the heart rate of a human body or the like with good accuracy and high robustness by transmitting and receiving radio waves. Hereinafter, such a method will be described.

[0102] The electronic device 1 according to one embodiment narrows down the subspace and subspace basis where the signal component of the heart sound of the subject 200 exists by performing appropriate signal processing on the signal (chirp signal) received by the electronic device 1 in an appropriate order in order to extract the heart sound of the subject 200 with high precision. The electronic device 1 according to one embodiment may adopt different basis vectors in the linear space where the signal of the heart sound of the subject 200 exists, describe the space using an appropriate coordinate system, and extract subspaces based on each coordinate. By such processing, the electronic device 1 according to one embodiment can search for the subspace where the heart sound of the subject 200 exists. In one embodiment, such a coordinate system may be appropriately selected according to the purpose. For example, a coordinate system of a space processed by 2D-FFT, a coordinate system of a time-series signal obtained by time reduction of a chirp signal, a coordinate system by its Fourier transform, or a coordinate system by continuous / discrete wavelet may be used.

[0103] The electronic device 1 according to one embodiment may execute the following characteristic processing on the received signal according to a step-by-step procedure. Hereinafter, the outline of the characteristic processing by the electronic device 1 according to one embodiment is shown. The electronic device 1 according to one embodiment executes two characteristic processes. That is, the electronic device 1 according to one embodiment executes (1) extraction of a subspace and (2) a process of calculating a time-series vector as time-series data of the heart rate. Hereinafter, each process will be described in more detail.

[0104] (1) Extraction of subspace First, the electronic device 1 according to one embodiment executes a process of removing data considered unnecessary in the linear space for the received signal. The electronic device 1 according to one embodiment may execute a process of leaving the necessary subspace by removing the unnecessary subspace. Specifically, the following process may be executed.

[0105] (1-1) First stage: Dimension reduction In the first stage, an appropriate window function is applied to the 2D-FFT processing of the chirp signal received by the electronic device 1, and only the micro-Doppler components of the point group where the subject 200 (for example, a person or an animal) exists are extracted by direction-of-arrival estimation. Here, around the approximate position of the target, a plurality of window functions centered on a plurality of distance ranges are generated and the plurality of window functions are applied. Hereinafter, the process of applying such a plurality of window functions is referred to as "multi-window processing". Here, for one window function, that is, for one center distance range, a time-series signal of one vibration may be generated.

[0106] (1-2) Second stage: Reduction of the dimension of the subspace In the second stage, principal component analysis and / or singular value decomposition of the set of time-series waveforms of the plurality of vibrations extracted in the first-stage processing are performed on the micro-Doppler signal.

[0107] (1-3) Third stage: Reduction of the dimension of the subspace In the third stage, frequency filtering is performed by using at least any one of short-time Fourier transform, continuous wavelet transform, and band-pass filter.

[0108] (1-4) Fourth stage: Reduction of the dimension of the subspace In the fourth stage, the heart sound is extracted by performing multi-resolution analysis by discrete wavelet transform using a wavelet function and a scaling function appropriate for the heart sound.

[0109] (2) Process of calculating a time-series vector as time-series data of the heart rate Next, in order to facilitate processing of the extracted heart sound data, the electronic device 1 according to one embodiment may perform a process of generating an envelope from the waveform of the extracted heart sound. Further, the electronic device 1 according to one embodiment may calculate a time-series vector as time-series data of the heart rate based on the result of performing frequency analysis on the generated envelope. Specifically, the following process may be performed.

[0110] (2-1) Generation of envelope Here, an envelope is generated (extracted) from the heart sound extracted in the above (1) subspace extraction. For the envelope generation process, for example, any one of continuous wavelet transform, discrete wavelet transform, wavelet scattering coefficient, mel-frequency cepstral coefficient, or moving variance may be used.

[0111] (2-2) Frequency analysis of heartbeat Apply a window function to the envelope of the heart sound extracted by the process of (2-1) above, and perform frequency analysis (frequency analysis) by fast Fourier transform (FFT) processing or the like to analyze (analyze) the frequency of the heartbeat. Here, while applying a time-moving window to the envelope of the heart sound, for example, frequency characteristics for each time are generated by the MUSIC method or the like. Also, such frequency characteristics for each time generate a multi-dimensional array that is overlapped by the number of multi-windows described in the above “(1) Subspace extraction”. This multi-dimensional array can be represented by the following tensor (multi-dimensional array). (Peak frequency vector) × (Array of time windows) × (Multi-window array) Here, × represents a tensor product. In the present disclosure, the tensor as described above is also referred to as a heartbeat frequency data tensor.

[0112] (2-3) Extraction of time series vector Based on the heartbeat frequency data tensor generated by the process of (2-2) above, perform dimensionality reduction of the frequency vector and the multi-window. Thereby, a time series vector of the heartbeat frequency and a time series vector of the heart rate may be calculated.

[0113] According to the electronic device 1 according to an embodiment, accurate time series data of the heart rate can be calculated by going through the processes of each stage described above.

[0114] Next, the operation of the electronic device 1 according to an embodiment will be described in more detail.

[0115] FIG. 12 is a flowchart showing an example of operations performed by the electronic device 1 according to one embodiment. FIG. 13 is a flowchart showing in more detail an example of the operation in step S16 in FIG. 12. Hereinafter, with reference to FIGS. 12 and 13, the flow of operations performed by the electronic device 1 according to one embodiment will be described.

[0116] Step S11 shown in FIG. 12 can be performed in the same manner as the operation in step S110 shown in FIG. 11. That is, when the operation shown in FIG. 12 starts, first, in step S11, the signal processing unit 10 of the electronic device 1 processes the received signal (received signal). The signal processing performed in step S11 may include, for example, the above-described 2D-FFT, CFAR processing, and / or direction-of-arrival estimation. Such an operation may be performed by, for example, the received signal processing unit 12 of the signal processing unit 10.

[0117] In step S12, the signal processing unit 10 performs multi-window processing. In step S12, the signal processing unit 10 may perform processing for determining candidates for clusters of humans or animals and / or filtering processing (multi-window processing) on data subjected to 2D-FFT processing by a plurality of window functions.

[0118] In step S12, the signal processing unit 10 may perform multi-window processing for generating a plurality of heart sound candidates as a pre-stage for selecting the best heart sound candidate. Here, the signal processing unit 10 may perform processing for grouping point groups corresponding to humans or animals as clusters on the data subjected to 2D-FFT processing. Further, the signal processing unit 10 may perform multi-window processing by applying a plurality of window functions to the selected clusters. The multi-window processing performed here may include processing for generating data subjected to 2D-FFT processing, which is the basis for a plurality of heart sound candidates.

[0119] In step S12, the signal processing unit 10 may generate a plurality of candidate heart sounds by extracting only the region where the subject 200 exists on the range-Doppler plane calculated by 2D-FFT using an appropriate window function. In step S12, the signal processing unit 10 may provide a plurality of window functions as appropriate window functions. Here, the signal processing unit 10 may use, for example, window functions such as a Hanning window, a Hamming window, and a Blackman-Harris window.

[0120] In step S12, the signal processing unit 10 may detect a vibration source (target object) including body movements such as the heart sound and / or respiration of the subject 200 based on, for example, the following first to third procedures.

[0121] (First procedure) The signal processing unit 10 classifies the point group exceeding the CFAR threshold on the range-Doppler plane into a predetermined angular area based on the result of arrival direction estimation (see FIGS. 5 and 6). For example, assuming that the angle on the xy plane as shown in FIG. 6 is θ, it may be classified into angular areas A to C as follows. Area A: -10deg. < θ < 10deg. Area B: -20deg. < θ ≤ -10deg. Area C: 10deg. ≤ θ < 20deg.

[0122] (Second procedure) The signal processing unit 10 applies clustering to the point group exceeding the CFAR threshold on the range-Doppler plane as shown in S1 of FIG. 5 within a group of areas of a certain angle classified in the first procedure. Here, as the clustering, for example, a method such as DBSCAN may be applied.

[0123] (Third procedure) Assuming that L clusters are processed in the second procedure, the signal processing unit 10 calculates the deviation D dev [l] of the number of bins in the Doppler direction for the l-th cluster with respect to a certain threshold D dev,th and compares them. As a result of this comparison, the signal processing unit 10 determines Ddev [l] ≥ D dev,th Only those that satisfy this condition are determined as the subject 200, and the flag HF[l] is set. That is, the signal processing unit 10 may perform processing as shown in the following [Pseudo-code 1], for example.

[0124] [Pseudo-code 1] for l = 1 to L do if D dev [l] ≥ D dev,th HF[l] = 1 else HF[l] = 0 end if end do

[0125] In step S13, the signal processing unit 10 restores the set of chirp signals. In step S13, the signal processing unit 10 may restore the set of chirp signals by performing an inverse discrete Fourier transform (2D-IFFT (inverse fast Fourier transform)) on the plurality of generated 2D-FFT data.

[0126] FIG. 14 is a diagram for explaining multi-window processing by the electronic device 1 according to an embodiment. FIG. 14 explains multi-window processing for generating a plurality of time-series signals by window functions moved in a plurality of range directions from the point group belonging to the cluster selected by the above-described Pseudo-code 1.

[0127] In the upper part of FIG. 14, the horizontal axis represents distance (Range), and the vertical axis represents Doppler velocity. The upper part of FIG. 14 shows the result of clustering the point cloud of 2D-FFT data at a certain time. The upper part of FIG. 14 shows the state where three clusters, namely Cluster 1, Cluster 2, and Cluster 3, are generated as a result of clustering. The middle part of FIG. 14 conceptually shows the state where the signal processing unit 10 performs multi-window processing on each of Cluster 1, Cluster 2, and Cluster 3. The middle part of FIG. 14 shows the state where the signal processing unit 10 performs multi-window processing by applying a window function while shifting it little by little in the distance (Range) direction for each cluster. Also, the lower part of FIG. 14 shows the result of the signal processing unit 10 calculating the vibration velocity for each of Cluster 1, Cluster 2, and Cluster 3 by performing 2D-IFFT. The lower part of FIG. 14 shows the state where a time-series signal of vibration is generated by performing 2D-IFFT on each cluster. Actually, by applying a plurality of window functions while shifting them little by little in the distance (Range) direction for each cluster shown in FIG. 14, a plurality of time-series signals of vibration can be obtained.

[0128] Figure 12 in step S 12 In this case, in step S13, the signal processing unit 10 applies a plurality of window functions to the cluster composed of the point cloud selected with flag HF[l]=1 by the above [Pseudo-code 1]. By this process, 2D-FFT data corresponding to each window function is generated. Here, assuming that the 2D-FFT data generated by the l-th window function is represented by the following equation (1).

[0129] In this case, in step S13, the signal processing unit 10 can calculate the time-series signal waveform of the vibration velocity according to the following equation (2). By performing two-dimensional inverse fast Fourier transform (2D-IFFT), it is restored to a set of chirp signals.

Equation

[0130] Hereinafter, as an example, only one cluster, for example, only cluster 1 shown in FIG. 14 will be described.

[0131] In step S14, the signal processing unit 10 performs analysis of the main components of the signal and / or removal of noise. Step S14 may be singular value decomposition (SVD) for preprocessing and removing the noise of the signal. This process may be performed for the purpose of removing noise with low energy and / or low-level noise mixed due to the uncertainty of the Fourier transform.

[0132] In step S14, the signal processing unit 10 may perform singular value decomposition on the set S l of received chirp signals extracted in the 2D-FFT plane. Denoting the matrix formed by arranging the left singular vectors as U, the matrix formed by arranging the singular values diagonally as Σ, and the matrix formed by arranging the right singular vectors as V, the following equation (3) is obtained by singular value decomposition of the set S l of received chirp signals.

Equation

[0133] Next, the signal processing unit 10 may limit the number of row vectors in the rows of the left singular vectors in the above equation (3) (U ext ). Also, the signal processing unit 10 may limit the number of diagonal elements in the diagonal matrix of the singular values in the above equation (3) (Σ ext ). As a result, unnecessary noise signals and vibration components other than the desired positions are removed. As a result, the signal S l projected onto the subspace of the target signal of S l extIt is expressed as the following formula (4). [Number]

[0134] FIG. 15 is a diagram illustrating the relationship of the respective ranks representing the target signal and the noise signal in singular value decomposition. FIG. 15 is a diagram in which the results of SVD are arranged with the horizontal axis being the rank and the vertical axis being the singular value in descending order of the singular values (corresponding to energy). In the present disclosure, the concept is adopted that a signal having a singular value equal to or greater than a certain value becomes the target signal. In this case, the rank of the singular value that serves as the boundary between whether it is the target signal or not is the portion of the rank region of 30 or less indicated by the dark gray portion in the graph of FIG. 15. As shown in FIG. 15, the portion of the rank region of 30 or less becomes the region of the target signal. In FIG. 15, the rank portion of this target signal is set to be 1 or more and 30 or less. Therefore, in the present disclosure, for example, the singular vectors corresponding to the left and right singular values in the rank region of 1 or more and 30 or less may be extracted.

[0135] In the above example, the rank portion of the target signal is set to be 1 or more and 30 or less. The maximum rank may basically be determined empirically. Also, the maximum rank (rank 30 in the present disclosure) may be determined by a statistical method. In the graph of FIG. 15, the singular value rapidly decreases at rank 130, and the signals of the ranks after that basically become noise. Therefore, the subsequent rank signals may be considered unnecessary.

[0136] In the graph of FIG. 15, the left and right singular vectors (vectors that span the signal space) corresponding to the singular values in the area of ranks 31 to 130 have some target signal components. These left and right singular vectors are presumably mainly due to unnecessary fine vibrations and / or artificial noise (artifacts) generated by radar signal processing. Therefore, such elements may be discarded.

[0137] If noise is removed by SVD in step S14, the signal processing unit 10 converts the result of the removed noise into a signal waveform (step S15). In step S15, the signal processing unit 10 may convert the beat signal (IQ data) into a signal of the vibration time series.

[0138] In step S15, the signal processing unit 10 obtains the vector shown in the following formula (5) by taking the sum in the column vector of each chirp signal based on the above formula (4).

Equation

[0139] In step S15, finally, s l ext,sum By taking the argument and derivative on the Gaussian plane of, a vector representing the vibration velocity as shown in the following formula (6) is calculated.

Equation

[0140] Next, in step S16, the signal processing unit 10 performs extraction of heart sound and / or analysis of the R-R interval on the obtained plurality of vibration waveforms. Specifically, in step S16, the signal processing unit 10 extracts the heart sound from the vibration waveform shown in the above formula (6). In step S16, the signal processing unit 10 may execute a process of generating a waveform of the heart sound, a process of calculating the R-R interval (RRI), and / or a process of calculating the heart rate variability (HRV). Thereby, the signal processing unit 10 can calculate the R-R interval. This process may be executed on the waveforms of a plurality of vibration velocities corresponding to each window function.

[0141] As described above, the process of step S16 shown in FIG. 12 may include at least a part of the processes of steps S21 to S25 shown in FIG. 13 in more detail. Hereinafter, each step of steps S21 to S25 shown in FIG. 13 will be described in more detail.

[0142] First, in step S21 shown in FIG. 13, the signal processing unit 10 performs a process (denoising) of removing noise from a time-series waveform. Here, in the process of removing noise, an empirical Bayes method, or a wavelet method such as a continuous wavelet transform (CWT) may be used.

[0143] More specifically, in step S21, the signal processing unit 10 performs preprocessing to further remove unnecessary noise from the vibration velocity vector v corresponding to the l-th window function. l vib with respect to.

[0144] In step S21, the signal processing unit 10 may perform a noise removal process by an empirical Bayes method and / or by restricting the band by a continuous wavelet. Also, in step S21, the signal processing unit 10 may perform a noise removal process by frequency subtraction using a noise profile for artifact noise and the like associated with the non-linear process from step S11 to step S14 shown in FIG. 12.

[0145] Next, in step S22, the signal processing unit 10 extracts the waveform of the target signal, that is, the waveform of the heart sound. In this case, for example, a method using a discrete wavelet such as a maximum overlap discrete wavelet transform (MODWT) may be used.

[0146] More specifically, in step S22, the signal processing unit 10 may perform multi-resolution analysis by discrete wavelet transform using a wavelet waveform having a waveform similar to the heart sound waveform on the signal denoised as pre-processing. In this way, in step S22, the signal processing unit 10 extracts only the subspace at the level of multi-resolution analysis where the heart sound of the subject 200 empirically exists. In this way, in step S22, the signal processing unit 10 may extract the heart sound of the subject 200. Specifically, the signal processing unit 10 may use maximum overlap discrete wavelet transform (MODWT) or the like to improve the time resolution. Also, the wavelet basis suitable for heart beat extraction may be, for example, Symlet and Daubechies. Also, these orders may be appropriately set each time.

[0147] Perform the denoising process of step S21 on the signal velocity vector v l vib to obtain the signal v l vib,dn and further perform the multi-resolution analysis shown in FIG. 16 on the real part Re(v l vib,dn ). FIG. 16 is a diagram conceptually showing multi-resolution analysis by discrete wavelet transform. Then, the signal processing unit 10 can obtain the heart sound waveform h by limiting and reconstructing the empirically appropriate level to j0∈N in the following formula (7).

Equation

[0148] In this way, the electronic device 1 according to an embodiment can obtain a heart sound waveform h such as the waveforms shown in FIGS. 22 and FIGS. 23 described later. This result can be directly utilized for heart sound diagnosis and the like.

[0149] Next, in step S23, the signal processing unit 10 generates an envelope waveform. For the generation process of the envelope waveform, for example, any one of continuous wavelet transform, discrete wavelet transform, wavelet scattering coefficient, mel-frequency cepstral coefficient, or moving variance may be used. In step S23, in order to detect (extract) the peak of the energy of the target signal (heart sound waveform) extracted in step S22, the signal processing unit 10 performs a scalogram by continuous wavelet transform and envelope waveform extraction by one-dimensionalization thereof. Here, discrete / continuous wavelet transform, moving variance, and / or Hilbert transform may be used.

[0150] More specifically, in step S23, the signal processing unit 10 may obtain a scalogram by continuous wavelet transform in order to obtain the envelope waveform of the heart sound waveform h shown in the above formula (7). The signal processing unit 10 takes the sum for a certain range f1 to f2 on the frequency axis at each time of the scalogram. Thereby, the signal processing unit 10 obtains a one-dimensional waveform s as shown in FIGS. 17 and 18. FIGS. 17 and 18 are diagrams showing examples of the analysis results of continuous wavelet transform. The continuous wavelet transform may be replaced with a discrete wavelet transform by appropriately setting the resolution. l l h l l

[0151] FIG. 17 is a diagram showing the scalogram of the heart sound waveform h. FIG. 17 is a diagram showing the time change of the normalized vibration frequency. The horizontal axis of FIG. 17 indicates time in units of the number of samples, and the vertical axis of FIG. 17 indicates the normalized vibration frequency for each time. l l

[0152] l

[0153]

[0153] Here, assume that the matrix representing the absolute value of the scalogram satisfies the following equation (8).

Equation

[0154] [Pseudo Code 2] for m=1:M

Equation

[0155] Next, in step S24, the signal processing unit 10 analyzes the heart rate. The envelope waveform (heart sound envelope data) generated by the process of step S23 exists in the number of window functions as shown in FIG. 14. Here, frequency analysis is performed for one of the plurality of window functions

[0156] FIG. 19 is a diagram showing the arrangement of time windows for the signal processing unit 10 to analyze the frequency of the heart sound envelope. In the heart sound envelope shown in the upper part of FIG. 19, the time windows cut out for time frames t = 1 and t = 2 are shown. In the heart sound envelope shown in the lower part of FIG. 19, the time windows cut out for time frames t = T - 1 and t = T are shown. Here, T and t are discrete times and represent non-negative integers. Also, in the heart sound envelope shown in the upper part of FIG. 19, the width w win of the time window is shown, and the width w o by which the time windows overlap is shown. Here, the moving step of the time window is w step = w win - w o and is as follows.

[0157] Generally, the heart rate is very low-frequency. Therefore, a high-resolution method is required for frequency analysis. Here, as one of the subspace methods, the processing by the MUSIC (MUltiple SIgnal Classification) method will be described. Hereinafter, the process of performing frequency analysis using MUSIC will be explained. This process may include the following four processes from [First Process] to [Fourth Process].

[0158] [First Process] First, the signal processing unit 10 arranges the time-series signals for each snapshot to generate an observation matrix X = [x1, x2, …, x N . Here, the observation matrix X can be obtained by dividing the samples in one window shown in FIG. 19 into equal sizes.

[0159] [Second Process] Based on the observation matrix generated in the above [First Process], the signal processing unit 10 obtains the autocorrelation matrix R x = 1 / N·X·X H . Here, the superscript H on the right indicates the Hermitian transpose.

[0160] [Third Process] The signal processing unit 10 performs eigenvalue decomposition on the autocorrelation matrix Rx obtained in the above [Second Process]. In this eigenvalue decomposition, the signal processing unit 10 classifies the eigenvectors with large eigenvalues as the basis vectors of the signal space and the eigenvectors with small eigenvalues as the basis vectors of the noise space. Here, the eigenvectors are sorted in descending order of eigenvalues, and the eigenvectors from the 1st to the p-th are regarded as the signal space, and the eigenvectors from the (p + 1)-th to the M-th are regarded as the basis vectors v m (p + 1 ≤ m ≤ M).

[0161] [Fourth Process] The signal processing unit 10 calculates the frequency vector e = [1, e j ω, e j2 ω, …, ej(M-1) ω] T For ω, take the inner product of the basis vectors of the noise space. In the above manner, the MUSIC spectrum shown in the following equation (10) can be obtained.

Number

[0162] FIG. 20 is a diagram showing an example of the calculated MUSIC spectrum (frequency analysis) calculated by the above equation (10). The vertical axis of FIG. 20 indicates the MUSIC spectrum, and the horizontal axis of FIG. 20 indicates the frequency. In the MUSIC spectrum shown in FIG. 20, there are peaks at three frequencies indicated as g1, g2, and g3. The frequency peaks g1, g2, and g3 are approximately 1 Hz, approximately 1.7 Hz, and approximately 3.1 Hz, respectively.

[0163] As a result of the above processing, finally, an appropriate one as the heart rate frequency may be picked up. The process of picking up an appropriate one as the heart rate frequency may be performed by the following two-stage process. These processes may be performed by the signal processing unit 10 in the electronic device 1 according to an embodiment.

[0164] [Process A] First, a plurality of frequency ranges for detecting the heart rate are provided. Then, the peaks of the frequencies within each frequency range are extracted. Also, for each of the frequency ranges for detecting the heart rate, a time-series vector of the frequency peak is generated.

[0165] Generally, the normal heart rate of a human is about 40 to 160 times per minute (BPM = 40 to 160). Therefore, first, the range of the human heart rate is divided into three classes with an interval of 40 as follows. Class 1: BPM = 40 to 80 Class 2: BPM = 80 to 120 Class 3: BPM = 120 to 160

[0166] In each of these classes, when the BPM is converted to frequency, the frequency classes of the heartbeats are as follows. Class 1: 0.67 [Hz] to [1.33] Hz Class 2: 1.33 [Hz] to [2.0] Hz Class 3: 2.0 [Hz] to [2.67] Hz

[0167] Therefore, among the frequency peaks of the MUSIC spectrum as shown in Fig. 20, the peaks corresponding to the above-mentioned Class 1, Class 2, and Class 3 are extracted for each time frame. For example, since the frequency of the peak g1 shown in Fig. 20 is about 1 Hz, it falls into Class 1. Since the frequency of the peak g2 shown in Fig. 20 is about 1.7 Hz, it falls into Class 2. Also, since the frequency of the peak g3 shown in Fig. 20 is about 3.1 Hz, it does not fall into any class. Summarizing the above, it is as follows. Class 1: Peak g1 Class 2: Peak g2 Class 3: None applicable

[0168] The above process can be shown by the following [Pseudo-code 3]. Here, let the number of time windows (time frames) to be processed be T, and the index be t. Let the number of classes of the heart rate frequency be J, and the index be j. Let the minimum value of the frequency in class j of the heart rate frequency be f j min and the maximum value be f j max Let the number of window functions on the 2D-FFT shown in Fig. 14 be L, and the index be l. Also, let the vector of the peaks of the MUSIC spectrum P MU (ω) at time frame t be p MU,peak And let the heart rate frequency data tensor to be obtained be F l,t,j

[0169] [Pseudo-code 3] for l = 1:L for t = 1:T for j = 1:J [Equation] ​ end end end

[0170] In the above formula (11), find(Condition) is a function that searches for a value sandwiched between the maximum value and the minimum value among p MU,peak . If such a value is not found, for example, NaN (Not a number: standardized in the IEEE 754 floating-point standard) etc. may be defined to indicate that the value does not exist.

[0171] [Process B] Next, the optimal one is selected from the frequency peak time series vectors generated for the detected frequency ranges of multiple heartbeats, and this is used as the final frequency vector of the heartbeats.

[0172] Here, a process is performed to determine the indices l and j of F l,t,j to a single natural number. That is, a process of selecting the best one from the L×J frequency vectors of the heartbeats is performed. This process is achieved by selecting a frequency vector that satisfies the conditions of (1) having few missing values (NaN) and (2) having the smallest change in the heart rate frequency.

[0173] The above condition (1) is based on the principle that when there are many missing values, there is no peak in the MUSIC spectrum corresponding to the class j. Also, the above condition (1) is based on the principle that when the window position in the 2D-FFT and the class of the heart rate frequency are not appropriate, the peak frequency of the MUSIC spectrum is not determined and has extreme changes.

[0174] FIG. 21 is a diagram for explaining the selection (good / bad determination) of the heart rate frequency vector by the electronic device 1 according to an embodiment. The vertical axis in FIG. 21 indicates the heart rate frequency, and the horizontal axis in FIG. 21 indicates time. In FIG. 21, the graph h1 shows an example of the heart rate frequency vector determined to be optimal by the electronic device 1 according to an embodiment. Also, in FIG. 21, the graph h2 shows an example of the heart rate frequency vector determined not to be appropriate by the electronic device 1 according to an embodiment. As shown by the graph h2 in FIG. 21, the heart rate frequency vector calculated based on an inappropriate window position on the 2D FFT and / or an inappropriate heart rate frequency class causes unnaturally large fluctuations.

[0175] The electronic device 1 according to an embodiment can statistically select a heart rate frequency vector such as the graph h1 and / or the graph h2 shown in FIG. 21, for example, by taking a variance value. In this case, the selection conditional expressions can be described as, for example, the following expressions (12) and (13). Expression (12) shows the above condition (1). Also, expression (13) shows the above condition (1).

Equation

Equation

[0176] Through the above processing, the electronic device 1 according to an embodiment can obtain an optimal heart rate frequency vector as shown in the following expression (14).

Equation

[0177] Also, through the above processing, the electronic device 1 according to an embodiment can obtain a heart rate data vector r as shown in the following expression (15) by taking the reciprocal of the optimal heart rate frequency vector as shown in expression (14) and performing an integerization process.

Number

[0178] By performing the processing as described so far, the electronic device 1 according to one embodiment can obtain a time-series waveform of heart sounds and can also obtain a time-series waveform of heart rate.

[0179] Therefore, in step S25 shown in FIG. 13, the signal processing unit 10 can generate a time-series vector of heart rate.

[0180] FIGS. 22, 23, and 24 are diagrams showing an example of data obtained by the electronic device 1 according to one embodiment. FIGS. 22 and 23 are diagrams showing examples of time-series waveforms of heart sounds obtained by the electronic device 1 according to one embodiment. FIG. 24 is a diagram showing an example of a time-series waveform of heart sounds obtained by the electronic device 1 according to one embodiment. In FIGS. 22 and 23, the horizontal axis represents time, and the vertical axis represents vibration velocity. Also, in FIG. 24, the horizontal axis represents time, and the vertical axis represents heart rate. FIG. 22 is a diagram showing a time-series waveform of heart sounds obtained by the electronic device 1 according to one embodiment. FIG. 23 is a diagram showing an enlarged view of the time span of the region surrounded by the broken line in FIG. 22.

[0181] As shown in FIG. 23, according to the time-series waveform of heart sounds obtained by the electronic device 1 according to one embodiment, the first heart sound S1 and the second heart sound S2 can be clearly identified. Also, as shown in FIG. 23, according to the time-series waveform of heart sounds obtained by the electronic device 1 according to one embodiment, the RRI calculated from the interval between the first heart sound S1 and the second heart sound S2 can also be clearly identified. Here, RRI and the heart beat interval are not exactly the same, but are considered to approximately coincide. For this reason, in the present disclosure, RRI and the heart beat interval are described as indicating the same thing.

[0182] FIG. 24 is a diagram showing an example of time-series data of the heart rate calculated from the time-series data of heart sounds shown in FIGS. 22 and 23. The result shown in FIG. 24 is calculated based on the heart sounds accurately extracted in FIGS. 22 and 23.

[0183] As described above, according to the electronic device 1 according to one embodiment, for example, in FIGS. 22 and FIGS. 23 as shown, detailed heart sound waves shape can be obtained. According to the electronic device 1 according to one embodiment, the heartbeat of a human body or the like can be detected with good accuracy by transmitting and receiving radio waves. Therefore, according to the electronic device 1 according to one embodiment, weak vibrations such as the heartbeat of a human body or the like can be detected with good accuracy by transmitting and receiving radio waves such as millimeter waves, and it is expected to be useful in a wide variety of fields.

[0184] According to the electronic device 1 according to one embodiment, heart sounds can be extracted by a radar using a high-frequency band of millimeter waves or higher, the heart sounds themselves can be analyzed, and accurate extraction of the heart beat interval can be realized. Here, in order to obtain the heart rate with high accuracy and high robustness, it is necessary to appropriately search the data space formed by the radar data and estimate the heart rate. According to the electronic device 1 according to one embodiment, the heart rate can be obtained with high accuracy and high robustness.

[0185] Here, in the examples of FIGS. 22 to 24, only the prominent first heart sound S1 and second heart sound S2 of a healthy person are described. However, according to the electronic device 1 according to one embodiment, even when the third heart sound and / or the fourth heart sound generated due to an abnormal heartbeat occur, the same processing can be performed.

[0186] (Other Embodiments) Hereinafter, other embodiments will be described.

[0187] The electronic device 1 according to the above-described embodiment executed processing by the MUSIC method in step S24 of FIG. 13. However, the electronic device 1 according to other embodiments may execute processing by another subspace method such as the Root MUSIC method in step S24 of FIG. 13.

[0188] In [Processing A] described in step S24 of FIG. 13, a plurality of classes of the heart rate frequency and the heart rate were set. However, in other embodiments, if the heart rate range of the user is known, such as being stored in a database, the classes of the heart rate frequency and the heart rate may be set to one class. In this case, the heart rate frequency data tensor is F l,t,j =F l,t and is simplified. Therefore, in this case, the index for narrowing down by the above formulas (12) and (13) is only the index l of the window function in the 2D-FFT.

[0189] In other embodiments, the processing in step S14 shown in FIG. 12 and / or the processing in step S21 shown in FIG. 13 may be omitted depending on the situation. Also, in other embodiments, the processing in step S14 shown in FIG. 12 (processing for performing singular value decomposition / principal component analysis) may be substituted by another subspace method.

[0190] In other embodiments, in step S24 shown in FIG. 13, [Processing A] and [Processing B] were described for the case of performing processing on the heart rate frequency vector. However, in other embodiments, before [Processing A], the heart rate frequency vector may be converted into a heart rate vector for processing.

[0191] In one embodiment, the transmission antenna array 24 and / or the reception antenna array 31 included in the electronic device 1 is not limited to the arrangement shown in FIG. 8. For example, in one embodiment, the reception antenna array 31 included in the electronic device 1 may have a configuration as shown in FIG. 25. FIG 25is a diagram showing an example of a URA (Uniform rectangular array) receiving antenna. The figure 25 employs a URA receiving antenna as shown in FIG. 2, and in the transmitting antenna array 24, the direction of arrival estimation for two angles may be performed only by the URA receiving antenna without changing the directivity by the beamformer.

[0192] In the electronic device 1 shown in FIG. 2, the signal processing unit 10 has been described as including functional units such as a heartbeat extraction unit 13 and a calculation unit 14. However, in one embodiment, the processing performed by the heartbeat extraction unit 13 and / or the calculation unit 14 may be performed by an external computer or processor.

[0193] Also, in one embodiment, the processing (singular value decomposition / principal component analysis processing) in step S14 shown in FIG. 12 may be omitted when the number of samples N of one chirp can be set to a large number. Also, in one embodiment, the processing (singular value decomposition / principal component analysis processing) in step S14 shown in FIG. 12 may be omitted when other noise mixing can be excluded by hardware techniques or the like.

[0194] In the processing (generation of heart sound envelope waveform) of step S23 shown in FIG. 13, it was calculated by adding the results of the continuous wavelet transform in the frequency axis direction according to [Pseudo-code 2]. However, an envelope waveform may be generated by other methods, for example, moving average or Hilbert transform.

[0195] As described above, the electronic device 1 according to one embodiment detects weak vibrations such as a heartbeat using, for example, a millimeter-wave sensor including a plurality of transmission antennas and a plurality of reception antennas. When no target is detected, the electronic device 1 according to one embodiment detects the body movement of the target while changing the beamforming pattern of the transmission antenna by changing the transmission phase of the antenna. On the other hand, when the electronic device 1 according to one embodiment detects the body movement of the target, it performs beamforming in the direction of the body movement to detect the heartbeat. Thus, according to the electronic device 1 according to one embodiment, the signal quality can be improved by automatically detecting the direction of the human body. Therefore, according to the electronic device 1 according to one embodiment, the detection accuracy and / or the detection range of the heartbeat can be improved. For this reason, according to the electronic device 1 according to one embodiment, the human heartbeat can be detected with high accuracy.

[0196] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications or amendments based on the present disclosure. Therefore, it should be noted that these modifications or amendments are included in the scope of the present disclosure. For example, the functions included in each functional unit can be rearranged so as not to be logically contradictory. A plurality of functional units and the like may be combined into one or divided. Each of the embodiments according to the present disclosure described above is not limited to being faithfully implemented in each of the described embodiments, and can be implemented by appropriately combining each feature or omitting a part thereof. That is, those skilled in the art can make various modifications and amendments based on the present disclosure to the content of the present disclosure. Therefore, these modifications and amendments are included in the scope of the present disclosure. For example, in each embodiment, each functional unit, each means, each step, etc. can be added to other embodiments so as not to be logically contradictory, or replaced with each functional unit, each means, each step, etc. of other embodiments. Also, in each embodiment, a plurality of each functional unit, each means, each step, etc. can be combined into one or divided. Also, each of the embodiments of the present disclosure described above is not limited to being faithfully implemented in each of the described embodiments, and can also be implemented by appropriately combining each feature or omitting a part thereof.

[0197] The above-described embodiments are not limited to being implemented only as the electronic device 1. For example, the above-described embodiments may be implemented as a control method for a device such as the electronic device 1. Further, the above-described embodiments may be implemented as a program executed by a device such as the electronic device 1, or a storage medium or a recording medium on which the program is recorded.

[0198] Also, the electronic device 1 according to the above-described embodiment has been described as including components that constitute a so-called radar sensor, such as the transmission antenna array 24 and the reception antenna array 31. However, the electronic device according to an embodiment may be implemented as a configuration such as the signal processing unit 10, for example. In this case, the signal processing unit 10 may be implemented as having a function of processing signals handled by, for example, the transmission antenna array 24 and the reception antenna array 31.

Explanation of Signs

[0199] 1 Electronic device 10 Signal processing unit 11 Signal generation processing unit 12 Received signal processing unit 13 Heartbeat extraction unit 14 Calculation unit 21 Transmission DAC 22 Transmission circuit 23 Millimeter-wave transmission circuit 24 Transmission antenna array 31 Reception antenna array 32 Mixer 33 Reception circuit 34 Reception ADC 50 Communication interface 60 External device

Claims

1. A transmission unit that transmits a transmission wave, A reception unit that receives a reflected wave from a target of the transmission wave, A signal processing unit that detects the distance, direction, and speed of the target based on a conversion signal obtained by performing a Fourier transform on a beat signal of the transmission wave and the reception wave, An electronic device comprising: The signal processing unit extracts a signal component corresponding to a heart sound vibration associated with the heartbeat of the target from the conversion signal by using a plurality of window functions centered on a plurality of distance ranges, performs frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration, performs a calculation process of statistical results on the result of the frequency analysis, and outputs time-series data of the frequency of the heartbeat of the target based on the calculation result. An electronic device.

2. The signal processing unit performs super-resolution frequency analysis as the frequency analysis. The electronic device according to claim 1.

3. The signal processing unit performs analysis based on a subspace method as the frequency analysis. The electronic device according to claim 1.

4. The signal processing unit performs analysis based on the MUSIC method as the frequency analysis. The electronic device according to claim 3.

5. A step of transmitting a transmission wave from a transmission unit, A step of receiving, by a reception unit, a reflected wave from a target of the transmission wave, A step of detecting the distance, direction, and speed of the target based on a conversion signal obtained by performing a Fourier transform on a beat signal of the transmission wave and the reception wave, A step of extracting a signal component corresponding to a heart sound vibration associated with the heartbeat of the target from the conversion signal by using a plurality of window functions centered on a plurality of distance ranges, A step of performing frequency analysis on an envelope signal obtained by performing envelope processing on a signal component corresponding to the heart sound vibration; A step of performing a calculation process of a statistical result on the result of the frequency analysis; A step of outputting time-series data of the frequency of the heartbeat of the target based on the calculation result; A control method for an electronic device including the above.

6. In an electronic device, A step of transmitting a transmission wave from a transmission unit; A step of receiving, by a receiving unit, a reflected wave from the target of the transmission wave; A step of detecting the distance, direction, and speed of the target based on a conversion signal after performing Fourier transform on a beat signal of the transmission wave and the reception wave; A step of extracting a signal component corresponding to the heart sound vibration associated with the heartbeat of the target from the conversion signal by a plurality of window functions centered on a plurality of distance ranges; A step of performing frequency analysis on an envelope signal obtained by performing envelope processing on a signal component corresponding to the heart sound vibration; A step of performing a calculation process of a statistical result on the result of the frequency analysis; A step of outputting time-series data of the frequency of the heartbeat of the target based on the calculation result; A program for causing the above to be executed.

Images