Electronic apparatus, method for controlling electronic apparatus, and program

Abstract

To provide an electronic apparatus that can detect the heart rate of a human body with good accuracy and high robustness through transmission and reception of radio waves, a method for controlling an electronic apparatus, and a program.SOLUTION: An electronic apparatus comprises: a transmission unit that transmits a transmission wave; a reception unit that receives a reflected wave from a target of the transmission wave; and a signal processing unit that detects the distance, direction, and speed of the target on the basis of conversion signals after performing Fourier transformation on beat signals of the transmission wave and received wave. The signal processing unit extracts a signal component corresponding to heart sound vibration along with the heartbeat of the target from the conversion signals by using a plurality of window functions centered on a plurality of distance ranges, performs frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration, performs processing of calculating a statistical result for a result of the frequency analysis, and outputs time-series data of the frequency of the heartbeat of the target on the basis of a calculation result.SELECTED DRAWING: Figure 2

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent Application No. 2022-148633, filed on September 16, 2022, the entire disclosure of which is incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to an electronic device, a control method for an electronic device, and a program. [Background technology]

[0003] For example, in fields such as the automobile industry, technology for measuring the distance between a vehicle and a predetermined object has become increasingly important. In particular, in recent years, various RADAR (Radio Detecting and Ranging) technologies have been researched, which measure the distance between a vehicle and an object by transmitting radio waves such as millimeter waves and receiving the waves reflected by the object, such as an obstacle. The importance of such technology for measuring distance is expected to increase in the future along with the development of technologies for assisting drivers in driving and technologies related to autonomous driving, which automates driving partially or completely.

[0004] Various proposals have also been made regarding technologies for detecting the presence of a specific object by receiving a reflected wave of a transmitted radio wave reflected by the object. For example, Patent Document 1 proposes a device that can detect the presence of a person and their biological information by using microwaves. Also, for example, Patent Document 2 proposes a device that detects vital signs such as the frequency of a living body's breathing or heartbeat based on a reflected signal from microwave radar. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2002-71825 [Patent Document 2] Patent Publication No. 2021-32880 Summary of the Invention

[0006] An electronic device according to an embodiment includes: a transmitting unit that transmits a transmission wave; a receiving unit that receives a reflected wave of the transmitted wave from a target; a signal processing unit that detects the distance, direction, and speed of the target based on a converted signal obtained by Fourier transforming beat signals of the transmitted wave and the received wave; Equipped with. The signal processing unit extracting signal components corresponding to cardiac vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; performing frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration; performing a calculation process of statistical results on the results of the frequency analysis; Based on the calculation result, time series data of the heartbeat frequency of the target is output.

[0007] A method for controlling an electronic device according to an embodiment includes: transmitting a transmission wave from a transmitting unit; receiving, at a receiving unit, a reflected wave of the transmitted wave from a target; detecting a distance, a direction, and a velocity of the target based on a converted signal obtained by performing a Fourier transform on beat signals of the transmitted wave and the received wave; extracting signal components corresponding to cardiac vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; a step of performing frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration; performing a statistical calculation process on the result of the frequency analysis; a step of outputting time series data of the heartbeat frequency of the target based on the calculation result; Includes:

[0008] A program according to an embodiment includes: For electronic devices, transmitting a transmission wave from a transmitting unit; receiving, at a receiving unit, a reflected wave of the transmitted wave from a target; detecting a distance, a direction, and a velocity of the target based on a converted signal obtained by performing a Fourier transform on beat signals of the transmitted wave and the received wave; extracting signal components corresponding to cardiac vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; a step of performing frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration; performing a statistical calculation process on the result of the frequency analysis; a step of outputting time series data of the heartbeat frequency of the target based on the calculation result; Execute the following. [Brief explanation of the drawings]

[0009] [Figure 1] 1A and 1B are diagrams illustrating a usage mode of an electronic device according to an embodiment. [Figure 2] FIG. 1 is a functional block diagram illustrating a schematic configuration of an electronic device according to an embodiment. [Figure 3] FIG. 2 is a diagram illustrating the configuration of a signal processed by an electronic device according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating signal processing by an electronic device according to an embodiment. [Figure 5] FIG. 2 is a diagram illustrating signal processing by an electronic device according to an embodiment. [Figure 6] FIG. 2 is a diagram illustrating signal processing by an electronic device according to an embodiment. [Figure 7] 1A and 1B are diagrams illustrating an example of an antenna arrangement and an operation principle in an antenna array of an electronic device according to an embodiment. [Figure 8]1A and 1B are diagrams illustrating examples of antenna arrangements in an antenna array of an electronic device according to an embodiment. [Figure 9] FIG. 1 is a diagram illustrating an example of signal processing by an electronic device according to an embodiment. [Figure 10] FIG. 1 is a diagram illustrating an example of signal processing by an electronic device according to an embodiment. [Figure 11] 10 is a flowchart illustrating a comparative example of the operation of the electronic device according to the embodiment. [Figure 12] 10 is a flowchart illustrating an operation of an electronic device according to an embodiment. [Figure 13] 10 is a flowchart illustrating an operation of an electronic device according to an embodiment. [Figure 14] FIG. 10 is a diagram illustrating a multi-window process performed by an electronic device according to an embodiment. [Figure 15] 10A and 10B are diagrams illustrating the relationship between ranks representing a target signal and a noise signal in singular value decomposition by an electronic device according to an embodiment. [Figure 16] FIG. 1 is a diagram conceptually illustrating multi-resolution analysis using a discrete wavelet transform performed by an electronic device according to an embodiment. [Figure 17] FIG. 10 is a diagram conceptually illustrating the process of classifying and identifying the best heart sound envelope by an electronic device according to an embodiment. [Figure 18] FIG. 10 is a diagram showing a result of one-dimensional processing performed by continuous wavelet transform in an electronic device according to an embodiment. [Figure 19] FIG. 10 illustrates the placement of time windows for analyzing the frequency of a heart sound envelope by an electronic device according to one embodiment. [Figure 20] FIG. 10 is a diagram showing a MUSIC spectrum of a heart sound envelope generated by an electronic device according to an embodiment. [Figure 21] 10A and 10B are diagrams illustrating selection of a frequency vector of a heartbeat by an electronic device according to an embodiment. [Figure 22] FIG. 1 is a diagram showing a time-series waveform of a heart sound obtained by an electronic device according to an embodiment. [Figure 23]FIG. 1 is a diagram showing a time-series waveform of a heart sound obtained by an electronic device according to an embodiment. [Figure 24] FIG. 10 is a diagram illustrating an example of a time-series waveform of a heart rate obtained by an electronic device according to an embodiment. [Figure 25] 1A and 1B are diagrams illustrating examples of antenna arrangements in an antenna array of an electronic device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] If it were possible to detect weak vibrations such as the heartbeat of a human body or the like with high accuracy and high robustness by detecting the heart rate of the human body by transmitting and receiving radio waves, such as millimeter waves, this would be useful in a wide variety of fields. An object of the present disclosure is to provide an electronic device, a control method for an electronic device, and a program that can detect the heart rate of a human body or the like with high accuracy and high robustness by transmitting and receiving radio waves. According to one embodiment, it is possible to provide an electronic device, a control method for an electronic device, and a program that can detect the heart rate of a human body or the like with high accuracy and high robustness by transmitting and receiving radio waves. Hereinafter, one embodiment will be described in detail with reference to the drawings.

[0011] In the present disclosure, an "electronic device" may refer to a device that is powered by electricity. Furthermore, a "user" may refer to a person (typically a human) or an animal that uses a system and / or an electronic device according to an embodiment. A user may include a person who monitors a target, such as a human, by using an electronic device according to an embodiment. Furthermore, a "target" may refer to a person (e.g., a human or an animal) that is monitored by an electronic device according to an embodiment. Furthermore, a user may include a target.

[0012] In this disclosure, a "heart beat" may refer to the beating of the heart, and a "beat" may refer to the rhythmic contraction of the heart.

[0013] In addition, in this disclosure, the term "heart rate" refers to the number of times the heart beats within a certain period of time. For example, the heart rate may be the number of beats per minute. When the heart pumps blood, pulsations occur in the arteries. Therefore, the number of times the arteries beat may be referred to as the pulse rate, or simply as pulses.

[0014] Furthermore, in the present disclosure, a "heart sound" may refer to the sound of the heart beating. That is, a heart sound may refer to the sound that occurs when the heart contracts and expands. Here, a "heart sound" may be composed of a first low, long sound resulting from ventricular muscle tension, mitral valve closure, the start of blood ejection into the arteries, and / or acceleration of blood flow, followed by a second high, short sound resulting from aortic valve closure and / or pulmonary valve closure.

[0015] In the present disclosure, heart sounds are not necessarily limited to physical sounds based on air vibrations, but may refer to the vibrations themselves caused by the heartbeat (pulsation). For example, in the present disclosure, heartbeats may imply a vibration source, and heart sounds may imply the vibrations themselves caused by the vibration source. Furthermore, in the present disclosure, heartbeats may also imply heart sounds depending on the situation.

[0016] Furthermore, in the present disclosure, heart sounds may refer to vibrations of the body that accompany the movement of the heart. Heart sounds may refer to, for example, vibrations of the entire body, or the head, neck, chest, throat, arms, legs, wrists, or other body parts. Heart sounds may refer to, for example, vibrations of the entire body, or the skin on the surface of the head, neck, chest, throat, arms, legs, wrists, or other body parts. Heart sounds may refer to, for example, vibrations of clothing, underwear, glasses, or other attachments worn by the subject that accompany vibrations of the entire body, or the skin on the surface of the head, neck, chest, throat, arms, legs, wrists, or other body parts. Heartbeats may refer to the beating of the heart itself. Beat-to-beat intervals, heart rate, etc. may be calculated from the movement of the heartbeat. In the present disclosure, heartbeats may be referred to as pulses.

[0017] An electronic device according to an embodiment can detect the heartbeat of a human or other target present in the vicinity of the electronic device. Therefore, the electronic device according to an embodiment may be used in specific facilities used by socially active individuals, such as companies, hospitals, nursing homes, schools, sports gyms, and care facilities. For example, in a company, it is extremely important to understand and / or manage the health of employees. Similarly, it is extremely important to understand and / or manage the health of patients and medical professionals in a hospital, and residents and staff in a nursing home. The electronic device according to an embodiment may be used in any facility where it is desirable to understand and / or manage the health of a target, without being limited to the aforementioned facilities such as companies, hospitals, and nursing homes. Such facilities may also include non-commercial facilities, such as a user's home. Furthermore, the electronic device according to an embodiment may be used not only indoors but also outdoors. For example, the electronic device according to an embodiment may be used inside a moving vehicle, such as a train, bus, or airplane, or at a station or platform. Furthermore, the electronic device according to one embodiment may be used in a moving object such as an automobile, an airplane, or a ship, a hotel, a user's home, a living room, a bathroom, a toilet, or a bedroom.

[0018] An electronic device according to an embodiment may be used, for example, in a nursing facility or the like, to detect or monitor the heartbeat of a subject, such as a person requiring nursing care or care. Furthermore, when an abnormality is detected in the heartbeat of a subject, such as a person requiring nursing care or care, the electronic device according to an embodiment may issue a predetermined warning to the subject and / or other persons. Therefore, the electronic device according to an embodiment may allow the subject and / or staff at a nursing facility or the like to recognize that an abnormality is detected in the pulse of a subject, such as a person requiring nursing care or care. On the other hand, when no abnormality is detected in the heartbeat of a subject, such as a person requiring nursing care or care, (e.g., recognized as normal), the electronic device according to an embodiment may notify the subject and / or other persons to that effect. Therefore, the electronic device according to an embodiment may allow the subject and / or staff at a nursing facility or the like to recognize that the pulse of a subject, such as a person requiring nursing care or care, is normal.

[0019] Furthermore, the electronic device according to an embodiment may detect the pulse of animals other than humans. As an example, the electronic device according to an embodiment described below will be described as detecting the pulse of a human using a sensor based on technology such as millimeter-wave radar.

[0020] An electronic device according to an embodiment may be installed on any stationary object or any mobile object. The electronic device according to an embodiment can transmit a transmission wave to the surroundings of the electronic device from a transmission antenna. The electronic device according to an embodiment can receive a reflected wave of the transmission wave from a reception antenna. At least one of the transmission antenna and the reception antenna may be provided in the electronic device, or may be provided in, for example, a radar sensor.

[0021] Hereinafter, as a typical example, an electronic device according to an embodiment will be described as being stationary. Meanwhile, the subject (human) whose pulse is detected by the electronic device according to an embodiment may be stationary, moving, or moving while stationary. The electronic device according to an embodiment can measure the distance between the electronic device and an object in a situation where the object around the electronic device may move, similar to a normal radar sensor. Furthermore, the electronic device according to an embodiment can measure the distance between the electronic device and an object even when both the electronic device and the object are stationary.

[0022] An electronic device according to an embodiment will be described in detail below with reference to the drawings. First, an example of object detection by the electronic device according to an embodiment will be described.

[0023] Fig. 1 is a diagram illustrating an example of a usage mode of an electronic device according to an embodiment. Fig. 1 shows an example of an electronic device having a sensor function and including a transmitting antenna and a receiving antenna according to an embodiment.

[0024] As shown in FIG. 1, an electronic device 1 according to an embodiment may include a transmitter and a receiver, which will be described later. As will be described later, the transmitter may include a transmitter antenna array 24. The receiver may include a receiver antenna array 31. Specific configurations of the electronic device 1, the transmitter, and the receiver will be described later. For ease of viewing, FIG. 1 illustrates the electronic device 1 including the transmitter antenna array 24 and the receiver antenna array 31. The electronic device 1 may also include at least one of the other functional units, such as at least a part of the signal processing unit 10 (FIG. 2) included in the electronic device 1, as appropriate. The electronic device 1 may also include at least one of the other functional units, such as at least a part of the signal processing unit 10 (FIG. 2), external to the electronic device 1. In FIG. 1, the electronic device 1 may be moving or may be stationary.

[0025] In the example shown in FIG. 1 , the electronic device 1 is shown in a simplified form, with a transmitter having a transmitting antenna array 24 and a receiver having a receiving antenna array 31. The electronic device 1 may include, for example, multiple transmitters and / or multiple receivers. The transmitter may include a transmitting antenna array 24 consisting of multiple transmitting antennas. The receiver may include a receiving antenna array 31 consisting of multiple receiving antennas. Here, the locations at which the transmitters and / or receivers are installed in the electronic device 1 are not limited to the locations shown in FIG. 1 , and may be other locations as appropriate. The number of transmitters and / or receivers may be any number greater than or equal to one, depending on various conditions (or requirements) such as the range and / or accuracy of heartbeat detection by the electronic device 1.

[0026] As will be described later, the electronic device 1 transmits electromagnetic waves as transmission waves from the transmitting antenna array 24. For example, if a predetermined object (e.g., the target 200 shown in FIG. 1 ) is present around the electronic device 1, at least a portion of the transmission waves transmitted from the electronic device 1 is reflected by the object and becomes a reflected wave. Then, by receiving such a reflected wave, for example, by the receiving antenna array 31 of the electronic device 1, the electronic device 1 can detect the object as a target.

[0027] The electronic device 1 including the transmitting antenna array 24 may typically be a RADAR (Radio Detecting and Ranging) sensor that transmits and receives radio waves. However, the electronic device 1 is not limited to a radar sensor. The electronic device 1 according to an embodiment may be a sensor based on, for example, a light wave-based LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) technology. Such sensors may include, for example, a patch antenna. Technologies such as RADAR and LIDAR are already known, so detailed descriptions may be appropriately simplified or omitted. Furthermore, the electronic device 1 according to an embodiment may be a sensor based on a technology that detects objects by transmitting and receiving, for example, sound waves or ultrasonic waves.

[0028] The electronic device 1 shown in FIG. 1 receives, from the receiving antenna array 31, reflected waves of transmitted waves transmitted from the transmitting antenna array 24. In this way, the electronic device 1 can detect a predetermined object 200 that exists within a predetermined distance from the electronic device 1 as a target. For example, as shown in FIG. 1, the electronic device 1 can measure the distance L between the electronic device 1 and the predetermined object 200. The electronic device 1 can also measure the relative speed between the electronic device 1 and the predetermined object 200. Furthermore, the electronic device 1 can also measure the direction (arrival angle θ) in which the reflected wave from the predetermined object 200 arrives at the electronic device 1.

[0029] In Fig. 1, the XY plane may be, for example, a plane substantially parallel to the ground surface. In this case, the positive direction of the Z axis shown in Fig. 1 may indicate a vertically upward direction. In Fig. 1, the electronic device 1 may be disposed on a plane parallel to the XY plane. Also, in Fig. 1, the target 200 may be, for example, standing on the ground surface substantially parallel to the XY plane.

[0030] Here, the target 200 may be, for example, a human being present around the electronic device 1. The target 200 may also be a non-human living thing, such as an animal present around the electronic device 1. As described above, the target 200 may be moving, stationary, or static. In the present disclosure, the object detected by the electronic device 1 includes not only inanimate objects such as any object, but also living things such as people, dogs, cats, horses, and other animals. The object detected by the electronic device 1 of the present disclosure may include targets detected using radar technology, including people, objects, and animals. In the present disclosure, targets may include people, objects, and animals. In the following description, it is assumed that an object such as the target 200 present around the electronic device 1 is a human being (or an animal). Hereinafter, the "target 200" may also be referred to as the "subject 200" as appropriate. In the present disclosure, the target may be the above-mentioned target 200.

[0031] 1, the ratio between the size of the electronic device 1 and the size of the target 200 does not necessarily represent the actual ratio. Also, in FIG. 1, the transmitting antenna array 24 of the transmitting unit and the receiving antenna array 31 of the receiving unit are shown installed outside the electronic device 1. However, in one embodiment, the transmitting antenna array 24 of the transmitting unit and / or the receiving antenna array 31 of the receiving unit may be installed at various positions on the electronic device 1. For example, in one embodiment, the transmitting antenna array 24 of the transmitting unit and / or the receiving antenna array 31 of the receiving unit may be installed inside the electronic device 1 so as not to be visible from the outside of the electronic device 1.

[0032] In the following, as a typical example, the transmitting antenna of the electronic device 1 will be described as transmitting radio waves in a frequency band such as millimeter waves (30 GHz or higher) or quasi-millimeter waves (for example, around 20 GHz to 30 GHz). On the other hand, the transmitting antenna of the electronic device 1 may transmit radio waves having a frequency bandwidth of 4 GHz, for example, 77 GHz to 81 GHz.

[0033] 2 is a functional block diagram schematically illustrating an example of the configuration of the electronic device 1 according to an embodiment. An example of the configuration of the electronic device 1 according to an embodiment will be described below.

[0034] When measuring distances and the like using millimeter-wave radar, frequency modulated continuous wave radar (hereinafter referred to as FMCW radar) is often used. FMCW radar generates a transmission signal by sweeping the frequency of the radio waves to be transmitted. Therefore, in a millimeter-wave FMCW radar using radio waves in the 79 GHz frequency band, for example, the frequency of the radio waves used has a frequency bandwidth of 4 GHz, such as 77 GHz to 81 GHz. Radar using the 79 GHz frequency band is characterized by a wider usable frequency bandwidth than other millimeter-wave / quasi-millimeter-wave radars, such as those in the 24 GHz, 60 GHz, and 76 GHz frequency bands. Hereinafter, such an embodiment will be described as an example.

[0035] The FMCW radar system used in the present disclosure may include an FCM (Fast-Chirp Modulation) system that transmits chirp signals at a shorter period than normal. The signals generated by the electronic device 1 are not limited to FMCW signals. The signals generated by the electronic device 1 may be signals of various systems other than FMCW. The transmission signal sequence stored in any storage unit may differ depending on these various systems. For example, in the case of the above-mentioned FMCW radar signal, signals whose frequency increases and decreases with each time sample may be used. Since known technologies can be applied as appropriate to the above-mentioned various systems, further detailed explanations will be omitted.

[0036] As shown in FIG. 2 , the electronic device 1 according to an embodiment includes a signal processing unit 10. The signal processing unit 10 may include a signal generation processing unit 11, a received signal processing unit 12, a heartbeat extraction unit 13, and a calculation unit 14. The heartbeat extraction unit 13 may, for example, execute a process of extracting a micro-Doppler component. The heartbeat extraction unit 13 may also execute a process of extracting an envelope of the heart sounds of the subject 200. The calculation unit 14 may, for example, execute a process of calculating the heartbeat interval (RRI) of the subject 200. The calculation unit 14 may also execute a process of calculating the heartbeat of the subject 200. The calculation unit 14 may also execute a process of calculating the heart rate validity (HRV) of the subject 200. In this case, the calculation unit 14 may execute a process of performing frequency analysis on the extracted time series data of the heartbeat interval of the subject 200. The calculation unit 14 may also execute a process of calculating the heartbeat variability of the subject 200 based on the frequency analysis of the time series data of the heartbeat interval. In the present disclosure, the calculation unit 14 may calculate the heartbeat interval and use it to calculate heartbeat variability. The signal generation processing unit 11, the received signal processing unit 12, the heartbeat extraction unit 13, and the calculation unit 14 will be described further below as appropriate. In the present disclosure, the heart sound may be, for example, a chest vibration waveform (see FIG. 20, etc.) directly observed by radar, or may be chest vibration. The heartbeat is the heartbeat itself. The heartbeat interval, heart rate, etc. may be calculated from the movement of the heartbeat. The heartbeat interval may be the time interval between one heartbeat and the next.

[0037] Moreover, the electronic device 1 according to an embodiment includes a transmitting unit that includes a transmitting DAC 21, a transmitting circuit 22, a millimeter-wave transmitting circuit 23, and a transmitting antenna array 24. Moreover, the electronic device 1 according to an embodiment includes a receiving unit that includes a receiving antenna array 31, a mixer 32, a receiving circuit 33, and a receiving ADC 34. The electronic device 1 according to an embodiment may not include at least one of the functional units shown in FIG. 2, or may include functional units other than the functional units shown in FIG. 2. The electronic device 1 shown in FIG. 2 may be configured using a circuit that is basically configured similarly to a general radar that uses electromagnetic waves in the millimeter-wave band or the like. Meanwhile, in the electronic device 1 according to an embodiment, the signal processing by the signal processing unit 10 may include processing that differs from that of conventional general radar.

[0038] The signal processing unit 10 included in the electronic device 1 according to an embodiment can control the overall operation of the electronic device 1, including the control of each functional unit constituting the electronic device 1. In particular, the signal processing unit 10 performs various processes on signals handled by the electronic device 1. The signal processing unit 10 may include at least one processor, such as a central processing unit (CPU) or a digital signal processor (DSP), to provide control and processing capabilities for executing various functions. The signal processing unit 10 may be implemented as a single processor, several processors, or individual processors. The processor may be implemented as a single integrated circuit. An integrated circuit is also called an IC (integrated circuit). The processor may be implemented as multiple integrated circuits and discrete circuits connected to each other in a communicative manner. The processor may be implemented based on various other known technologies. In an embodiment, the signal processing unit 10 may be configured as, for example, a CPU (hardware) and a program (software) executed by the CPU. The signal processing unit 10 may also include a storage unit (memory) necessary for the operation of the signal processing unit 10, as appropriate.

[0039] The signal generation processing unit 11 of the signal processing unit 10 generates a signal to be transmitted from the electronic device 1. In the electronic device 1 according to one embodiment, the signal generation processing unit 11 may generate a transmission signal (transmission chirp signal) such as a chirp signal. In particular, the signal generation processing unit 11 may generate a signal whose frequency changes periodically and linearly (linear chirp signal). For example, the signal generation processing unit 11 may generate a chirp signal whose frequency periodically and linearly increases from 77 GHz to 81 GHz over time. Alternatively, the signal generation processing unit 11 may generate a signal whose frequency periodically and linearly increases (up-chirp) and decreases (down-chirp) from 77 GHz to 81 GHz over time. The signal generated by the signal generation processing unit 11 may be preset in the signal processing unit 10, for example. Alternatively, the signal generated by the signal generation processing unit 11 may be pre-stored in a storage unit in the signal processing unit 10, for example. Chirp signals used in technical fields such as radar are well known, and therefore a detailed description thereof will be appropriately simplified or omitted. The signal generated by the signal generating processing unit 11 is supplied to the transmitting DAC 21. For this reason, the signal generating processing unit 11 may be connected to the transmitting DAC 21.

[0040] The transmission DAC (digital-to-analog converter) 21 has a function of converting the digital signal supplied from the signal generation processing unit 11 into an analog signal. The transmission DAC 21 may be configured to include a general digital-to-analog converter. The signal converted into an analog signal by the transmission DAC 21 is supplied to the transmission circuit 22. For this reason, the transmission DAC 21 may be connected to the transmission circuit 22.

[0041] The transmission circuit 22 has a function of converting the analog signal converted by the transmission DAC 21 into an intermediate frequency (IF) band. The transmission circuit 22 may be configured to include a general IF band transmission circuit. The signal processed by the transmission circuit 22 is supplied to the millimeter wave transmission circuit 23. For this reason, the transmission circuit 22 may be connected to the millimeter wave transmission circuit 23.

[0042] The millimeter-wave transmission circuit 23 has the function of transmitting the signal processed by the transmission circuit 22 as a millimeter wave (RF wave). The millimeter-wave transmission circuit 23 may be configured to include a general millimeter-wave transmission circuit. The signal processed by the millimeter-wave transmission circuit 23 is supplied to the transmission antenna array 24. For this reason, the millimeter-wave transmission circuit 23 may be connected to the transmission antenna array 24. The signal processed by the millimeter-wave transmission circuit 23 is also supplied to the mixer 32. For this reason, the millimeter-wave transmission circuit 23 may also be connected to the mixer 32.

[0043] The transmitting antenna array 24 is an array of multiple transmitting antennas. In Fig. 2, the configuration of the transmitting antenna array 24 is shown in a simplified form. The transmitting antenna array 24 transmits the signal processed by the millimeter-wave transmitting circuit 23 to the outside of the electronic device 1. The transmitting antenna array 24 may be configured to include a transmitting antenna array used in a general millimeter-wave radar.

[0044] In this way, the electronic device 1 according to the embodiment includes a transmitting antenna (transmitting antenna array 24), and can transmit a transmitting signal (for example, a transmitting chirp signal) from the transmitting antenna array 24 as a transmitting wave.

[0045] 2, assume that an object such as a subject 200 is present around the electronic device 1. In this case, at least a portion of the transmission waves transmitted from the transmitting antenna array 24 is reflected by the object such as the subject 200. Of the transmission waves transmitted from the transmitting antenna array 24, at least a portion of those reflected by the object such as the subject 200 may be reflected toward the receiving antenna array 31.

[0046] The receiving antenna array 31 receives the reflected waves, which may be at least a portion of the transmitted waves transmitted from the transmitting antenna array 24 that are reflected by an object such as the subject 200.

[0047] The receiving antenna array 31 is an array of multiple receiving antennas. In FIG. 2, the configuration of the receiving antenna array 31 is shown in a simplified form. The receiving antenna array 31 receives reflected waves that are the result of reflection of the transmitted waves transmitted from the transmitting antenna array 24. The receiving antenna array 31 may be configured to include a receiving antenna array used in a general millimeter-wave radar. The receiving antenna array 31 supplies the received signals received as reflected waves to the mixer 32. For this reason, the receiving antenna array 31 may be connected to the mixer 32.

[0048] The mixer 32 converts the signal (transmission signal) processed by the millimeter-wave transmission circuit 23 and the reception signal received by the reception antenna array 31 into an intermediate frequency (IF) band. The mixer 32 may be configured to include a mixer used in a general millimeter-wave radar. The mixer 32 supplies the signal generated as a result of the combination to the reception circuit 33. For this reason, the mixer 32 may be connected to the reception circuit 33.

[0049] The receiving circuit 33 has a function of performing analog processing on the signal converted to the IF band by the mixer 32. The receiving circuit 33 may be configured to include a receiving circuit that converts to a general IF band. The signal processed by the receiving circuit 33 is supplied to the receiving ADC 34. For this reason, the receiving circuit 33 may be connected to the receiving ADC 34.

[0050] The receiving ADC (analog-to-digital converter) 34 has a function of converting the analog signal supplied from the receiving circuit 33 into a digital signal. The receiving ADC 34 may be configured to include a general analog-to-digital converter. The signal digitized by the receiving ADC 34 is supplied to the receiving signal processing unit 12 of the signal processing unit 10. For this reason, the receiving ADC 34 may be connected to the signal processing unit 10.

[0051] The reception signal processing unit 12 of the signal processing unit 10 has a function of performing various processes on the digital signal supplied from the reception DAC 34. For example, the reception signal processing unit 12 calculates the distance from the electronic device 1 to an object such as the subject 200 based on the digital signal supplied from the reception DAC 34 (distance measurement). The reception signal processing unit 12 also calculates the relative velocity of the object such as the subject 200 with respect to the electronic device 1 based on the digital signal supplied from the reception DAC 34 (velocity measurement). The reception signal processing unit 12 also calculates the azimuth angle of the object such as the subject 200 as seen from the electronic device 1 based on the digital signal supplied from the reception DAC 34 (angle measurement). Specifically, I / Q converted data may be input to the reception signal processing unit 12. By inputting such data, the reception signal processing unit 12 performs fast Fourier transforms (2D-FFT) in the range direction and the velocity direction, respectively. The received signal processing unit 12 then suppresses false alarms by removing noise points using processing such as CFAR (Constant False Alarm Rate) and makes the probability constant. The received signal processing unit 12 then estimates the angle of arrival for points that satisfy the CFAR criteria, thereby obtaining the position of an object such as the subject 200. Information generated as a result of measuring the distance, speed, and angle by the received signal processing unit 12 may be supplied to the heart rate extraction unit 13.

[0052] The heartbeat extraction unit 13 extracts information related to the heartbeat from the information generated by the received signal processing unit 12. The operation of extracting information related to the heartbeat by the heartbeat extraction unit 13 will be described further below. The information related to the heartbeat extracted by the heartbeat extraction unit 13 may be supplied to the calculation unit 14.

[0053] The calculation unit 14 performs various calculation processes and / or arithmetic processes on the information related to the heartbeat supplied from the heartbeat extraction unit 13. The various calculation processes and / or arithmetic processes performed by the calculation unit 14 will be described further below. The various pieces of information calculated and / or processed by the calculation unit 14 may be supplied to, for example, a communication interface 50. For this reason, the calculation unit 14 and / or the signal processing unit 10 may be connected to the communication interface 50. The various pieces of information calculated and / or processed by the calculation unit 14 may be supplied to functional units other than the communication interface 50.

[0054] The communication interface 50 includes an interface that outputs information supplied from the signal processing unit 10 to, for example, an external device 60. The communication interface 50 may output at least one of information regarding the position, velocity, and angle of an object such as the subject 200 to the external device 60 as a signal such as a CAN (Controller Area Network). For example, at least one of information regarding the position, velocity, and angle of an object such as the subject 200 may be supplied to the external device 60 via the communication interface 50. For this reason, the communication interface 50 may be connected to the external device 60.

[0055] 2, the electronic device 1 according to an embodiment may be connected to an external device 60 via a communication interface 50 in a wired or wireless manner. In an embodiment, the external device 60 may include any computer and / or any control device. The electronic device 1 according to an embodiment may also include the external device 60. The external device 60 may have various configurations depending on how the information on the heartbeat and / or heart sounds detected by the electronic device 1 is used. Therefore, a detailed description of the external device 60 will be omitted.

[0056] FIG. 3 is a diagram illustrating an example of a chirp signal generated by the signal generation processing unit 11 of the signal processing unit 10. In FIG.

[0057] FIG. 3 shows the time structure of one frame when using the FCM (Fast-Chirp Modulation) method. FIG. 3 shows an example of a received signal using the FCM method. FCM is a method in which chirp signals shown as c1, c2, c3, c4, ..., cn in FIG. 3 are repeated at short intervals (for example, equal to or longer than the round-trip time between the electromagnetic wave radar and the target, calculated from the maximum measured distance). In FCM, for convenience of signal processing of the received signal, transmission and reception processing is often performed by dividing the signal into subframe units as shown in FIG. 3.

[0058] In Fig. 3, the horizontal axis represents elapsed time, and the vertical axis represents frequency. In the example shown in Fig. 3, the signal generation processing unit 11 generates linear chirp signals whose frequencies change periodically and linearly. In Fig. 3, each chirp signal is represented as c1, c2, c3, c4, ..., cn. As shown in Fig. 3, the frequency of each chirp signal increases linearly with the passage of time.

[0059] In the example shown in FIG. 3, several chirp signals such as c1, c2, c3, c4, ..., cn are included in one subframe. That is, subframe 1 and subframe 2 shown in FIG. 3 are each configured to include several chirp signals such as c1, c2, c3, c4, ..., cn. Also, in the example shown in FIG. 3, several subframes such as subframe 1, subframe 2, ..., subframe N are included in one frame (1 frame). That is, 1 frame shown in FIG. 3 is configured to include N subframes. Also, 1 frame shown in FIG. 3 may be frame 1, followed by frame 2, frame 3, ..., etc. Each of these frames may be configured to include N subframes, just like frame 1. Also, a frame interval of a predetermined length may be included between frames. One frame shown in FIG. 3 may be, for example, 30 to 50 milliseconds long.

[0060] In the electronic device 1 according to one embodiment, the signal generation processing unit 11 may generate a transmission signal as any number of frames. Also, some chirp signals are omitted from the illustration in Fig. 3. In this manner, the relationship between time and frequency of the transmission signal generated by the signal generation processing unit 11 may be stored in, for example, a storage unit of the signal processing unit 10.

[0061] In this way, the electronic device 1 according to one embodiment may transmit a transmission signal consisting of subframes each including a plurality of chirp signals. Also, the electronic device 1 according to one embodiment may transmit a transmission signal consisting of a frame each including a predetermined number of subframes.

[0062] Hereinafter, the electronic device 1 will be described as transmitting a transmission signal having a frame structure as shown in FIG. 3. However, the frame structure as shown in FIG. 3 is merely an example, and for example, the number of chirp signals included in one subframe may be arbitrary. That is, in one embodiment, the signal generation processing unit 11 may generate subframes including any number of chirp signals (for example, any plural number). Also, the subframe structure as shown in FIG. 3 is merely an example, and for example, the number of subframes included in one frame may be arbitrary. That is, in one embodiment, the signal generation processing unit 11 may generate a frame including any number of subframes (for example, any plural number). The signal generation processing unit 11 may generate signals of different frequencies. The signal generation processing unit 11 may generate multiple discrete signals, each having a frequency f with a different bandwidth.

[0063] Fig. 4 is a diagram showing, in another aspect, part of the subframe shown in Fig. 3. Fig. 4 shows each sample of the received signal obtained by receiving the transmitted signal shown in Fig. 3 as a result of performing 2D-FFT (Two Dimensional Fast Fourier Transform), which is processing performed in the received signal processing unit 12 (Fig. 2) of the signal processing unit 10.

[0064] As shown in Fig. 4, chirp signals c1, c2, c3, c4, ..., cn are stored in each subframe, such as subframe 1, ..., subframe N. In Fig. 4, each chirp signal c1, c2, c3, c4, ..., cn is composed of samples, each represented by a square arranged in the horizontal direction. The received signal shown in Fig. 4 is subjected to 2D-FFT, CFAR, and / or integrated signal processing of each subframe by the received signal processing unit 12 shown in Fig. 2.

[0065] FIG. 5 is a diagram showing an example of a point group calculated on a range-Doppler (distance-velocity) plane as a result of 2D-FFT, CFAR, and integrated signal processing of each subframe being performed in the received signal processing unit 12 shown in FIG. 2.

[0066] In FIG. 5, the horizontal direction represents range (distance), and the vertical direction represents velocity. The filled squares s1 in FIG. 5 represent point clouds indicating signals that exceed the CFAR threshold processing. The unfilled squares s2 in FIG. 5 represent bins (2D-FFT samples) without point clouds that do not exceed the CFAR threshold. The point clouds on the range-Doppler plane calculated in FIG. 5 have their azimuth from the radar calculated by direction estimation, and the position and velocity on a two-dimensional plane are calculated as a point cloud indicating an object such as the subject 200. Here, the direction estimation may be calculated using a beamformer and / or a subspace method. Representative subspace method algorithms include MUSIC (MUltiple SIgnal Classification) and ESPRIT (Estimation of Signal Parameters via Rotation Invariance Technique).

[0067] Fig. 6 is a diagram showing an example of the result of the reception signal processing unit 12 converting the point cloud coordinates from the range-Doppler plane shown in Fig. 5 to the XY plane after performing direction estimation. As shown in Fig. 6, the reception signal processing unit 12 can plot the point cloud PG on the XY plane. Here, the point cloud PG is made up of points P. Furthermore, each point P has an angle θ and a radial velocity Vr in polar coordinates.

[0068] The received signal processing unit 12 detects an object present within the range where the transmitted wave T is transmitted, based on at least one of the results of the 2D-FFT and the angle estimation. The received signal processing unit 12 may perform object detection by, for example, clustering processing based on the estimated distance information, speed information, and angle information. Known algorithms used for clustering data include DBSCAN (Density-based spatial clustering of applications with noise). This is an algorithm that performs clustering based on density. In the clustering processing, for example, the average power of points constituting the detected object may be calculated. Information on the distance, speed, angle, and power of the object detected by the received signal processing unit 12 may be supplied to an external device 60, for example, via a communication interface 50.

[0069] As described above, the electronic device 1 may include a transmitting antenna (transmitting antenna array 24), a receiving antenna (receiving antenna array 31), and a signal processing unit 10. The transmitting antenna array 24 transmits a transmission wave T. The receiving antenna array 31 receives a reflected wave R resulting from reflection of the transmission wave T. The signal processing unit 10 then detects an object (such as the subject 200) that reflects the transmission wave T based on the transmission signal transmitted as the transmission wave T and the reception signal received as the reflected wave R.

[0070] Next, estimation of the direction of an incoming wave by the antenna array of the electronic device 1 according to an embodiment will be further described.

[0071] 7 is a diagram illustrating the configuration of the receiving antenna array 31 of the electronic device 1 according to one embodiment and the principle of estimating the direction of an incoming wave by the receiving antenna array 31. FIG. 7 shows an example of reception of radio waves by the receiving antenna array 31.

[0072] As shown in Figure 7, the receive antenna array 31 may be a linear arrangement of sensors such as receive antennas. As shown in Figure 7, in one embodiment, the receive antenna array 31 may include multiple receive antennas arranged in a linear arrangement. In Figure 7, the receive antenna array 31 includes antennas x1, x2, x3, ..., x M In the figure, multiple antennas such as those shown in the figure are represented by small circles. The receiving antenna array 31 may be composed of any number of antennas. As shown in FIG. 7, the multiple antennas constituting the receiving antenna array 31 are arranged at an array pitch d. A sensor array in which sensors (antennas, ultrasonic transducers, microphones, etc.) corresponding to various physical waves are arranged in an array is also called a uniform linear array (ULA). As shown in FIG. 7, physical waves (electromagnetic waves, sound waves, etc.) arrive from various directions, such as θ1 and θ2. Here, θ1 and θ2 may be the angles of arrival described above. In this way, a sensor array such as the receiving antenna array 31 can estimate the direction of arrival (angle of arrival) by utilizing the phase difference that occurs in the measurements between sensors depending on the direction of arrival of the physical wave. This method of estimating the direction of arrival of a wave is also referred to as angle of arrival estimation or direction of arrival (DoA).

[0073] In the electronic device 1 according to an embodiment, at least one of the transmitting antenna array 24 and the receiving antenna array 31 may be configured with multiple antennas arranged in a line. This allows, for example, millimeter-wave radar to appropriately narrow the directivity when transmitting and receiving radio waves. When transmitting a transmitted wave, the direction of the transmitted beam is often controlled by a beamformer. On the other hand, when receiving a reflected wave, the direction of arrival of the reflected wave is often estimated by a subspace method (such as the above-mentioned MUSIC and ESPRIT) rather than a beamformer. In the beamformer and subspace method, in a ULA such as that shown in FIG. 7, a phase difference occurs in the measurements between sensors depending on the direction of arrival of electromagnetic waves arriving from various directions. Therefore, the phase difference can be used to estimate the direction of arrival of the reflected wave.

[0074] Next, estimation of angles of incoming waves in two directions by the antenna array of the electronic device 1 according to one embodiment will be further described.

[0075] FIG. 8 is a diagram showing an example of an antenna arrangement for estimating the directions of arrival at two orthogonal angles.

[0076] As shown in FIG. 8, in the electronic device 1 according to one embodiment, the transmitting antenna array 24 and / or the receiving antenna array 31 may be configured to include an array of a plurality of patch antenna units.

[0077] In the transmitting antenna array 24 shown in FIG. 8, one patch antenna unit may be configured to include a plurality of elements electrically connected in the direction 1 shown in the figure. In each patch antenna unit, the plurality of elements may be electrically connected by wiring such as a stripline on a substrate. In each patch antenna unit, the plurality of elements are spaced apart at intervals d that are shorter than half the wavelength λ of the transmitting wave. 1,t 8, each patch antenna unit may have any number of elements greater than or equal to two electrically connected together.

[0078] 8, the transmitting antenna array 24 may be configured by arranging a plurality of patch antenna units in a direction 2 shown in the figure. The patch antenna units are spaced apart at intervals d, which are shorter than half the wavelength λ of the transmitting wave. 2,t In one embodiment, the transmit antenna array 24 may include any number of patch antenna units greater than or equal to two.

[0079] As shown in Fig. 8, in one embodiment, the receiving antenna array 31 may be configured by changing the arrangement of the multiple elements in the transmitting antenna array 24. That is, in the receiving antenna array 31 shown in Fig. 8, one patch antenna unit may be configured to include multiple elements electrically connected in the direction 2 shown in the figure. In each patch antenna unit, the multiple elements may be electrically connected by wiring such as a stripline on a substrate. In each patch antenna unit, the multiple elements are spaced apart at intervals d that are shorter than half the wavelength λ of the transmission wave. 2,s 8, each patch antenna unit may have any number of elements greater than or equal to two electrically connected together.

[0080] 8, the receiving antenna array 31 may be configured by arraying a plurality of patch antenna units in a direction 1 shown in the figure. The patch antenna units are spaced apart at intervals d, which are shorter than half the wavelength λ of the transmission wave. 1,s In one embodiment, the receive antenna array 31 may include any number of patch antenna units greater than or equal to two.

[0081] The elements included in the transmitting antenna array 24 and the receiving antenna array 31 may all be arranged on the same plane (for example, on the surface layer of the same substrate). The transmitting antenna array 24 and the receiving antenna array 31 may also be arranged close to each other (monostatic). Furthermore, directions 1 and 2 shown in FIG. 8 may be geometrically orthogonal to each other.

[0082] The transmitting antenna array 24 and the receiving antenna array 31 shown in FIG. 8 can appropriately narrow the directivity of each of the transmitting antennas and the receiving antennas. Furthermore, by using the transmitting antenna array 24 shown in FIG. 8 to control the direction of transmission of each transmission wave (transmission signal) at each timing of transmission, a beamformer for direction 2 shown in FIG. 8 can be realized. Furthermore, by using the receiving antenna array 31 shown in FIG. 8, the arrival direction of the reflected wave can be estimated for direction 1 shown in FIG. 8. In this way, it is possible to estimate the arrival direction of the reflected wave for two angles that are substantially orthogonal to each other. Therefore, it is possible to acquire a point cloud representing an object such as the subject 200 in three dimensions.

[0083] Next, a method for detecting the heartbeat of the subject 200 using the electronic device 1 according to an embodiment will be described.

[0084] An electronic device 1 according to an embodiment transmits a transmission wave, such as a millimeter-wave radar wave, to a subject 200 and measures (estimates) the heart rate of the subject 200 based on the result of receiving a reflected wave reflected from the chest where the heart of the subject 200 is located. As described above, the subject 200 may be a human or an animal. In this case, for example, a component assumed to be the envelope of the heart rate can be extracted by frequency filtering vibrations at the location of the subject 200 detected by radar. Once the component assumed to be the envelope of the heart rate is extracted, the interval between peaks of the envelope can be taken as the heart rate interval, thereby calculating an approximate heart rate interval. Here, an approximation can be used in which the peak of the heart rate envelope roughly coincides with the R peak of an electrocardiogram. For this reason, the "heart rate interval" is also referred to as the RR interval or RRI (RR interval), similar to the term used in electrocardiograms.

[0085] Here, we will consider a method for estimating the heartbeat interval of subject 200 from the results of the above-mentioned 2D-FFT, CFAR processing, and direction-of-arrival estimation. First, we will explain how a person's heartbeat or body movement appears as a result of the 2D-FFT performed in Figs. 4 and 5.

[0086] FIG. 9 is a diagram showing an example of the results of receiving a reflected wave of a transmitted wave transmitted to the subject 200 and performing 2D-FFT processing. FIG. 9 shows a spectrum indicating the heart sounds and body movement of the subject 200 as a result of the 2D-FFT. In FIG. 9, the horizontal axis represents distance (range) and the vertical axis represents velocity. The signal processing unit 10 (e.g., the heartbeat extraction unit 13) of the electronic device 1 according to an embodiment may extract, for example, a peak Hm as shown in FIG. 9 as body movement such as the heartbeat of the subject 200. Here, the spectral components indicated by the peak Hm in FIG. 9 include not only the heart sounds and the envelope of the heart sounds of the subject 200 but also body movement. Extracting the heartbeat interval requires extracting body movement, etc., so frequency filtering, for example, may be performed. The frequency filtering performed here may be, for example, a band-pass filter, a high-pass filter, and / or a low-pass filter targeting a frequency range of 0.5 Hz to 10 Hz.

[0087] FIG. 10 is a diagram illustrating a method for detecting peaks based on the envelope waveform of heart sounds obtained by the above-described frequency filtering. The graph in FIG. 10 shows an example of the time change of the envelope waveform of heart sounds extracted by the above-described frequency filtering. As shown, the envelope waveform in FIG. 10 contains many peaks. Meanwhile, it is known that the heartbeat of subject 200 falls within a range of approximately 50 to 130 beats per minute. Therefore, by selecting, from the many peaks shown in FIG. 10, peaks with a time interval between 0.4 and 0.8 seconds, which is the reciprocal of the number of heartbeats per minute, the approximate heartbeat interval of subject 200 can be calculated. For example, the peak indicated by the downward arrow in FIG. 10 may be selected as the approximate heartbeat interval of subject 200.

[0088] 11 is a flowchart showing an example of the above-mentioned operation of estimating a heartbeat interval. The above-mentioned operation of estimating a heartbeat interval will be outlined below with reference to FIG.

[0089] Fig. 11 shows the operation of the electronic device 1 according to one embodiment after receiving a reflected wave. That is, as a premise for the operation shown in Fig. 11, the electronic device 1 shown in Fig. 2 transmits a transmission wave (transmission signal) from the transmitting antenna array 24. Then, at least a part of the transmission wave transmitted from the electronic device 1 is reflected by the subject 200 (for example, the chest) and becomes a reflected wave. Then, the electronic device 1 shown in Fig. 2 receives such a reflected wave from the receiving antenna array 31. Then, the operation shown in Fig. 11 starts.

[0090] 11 starts, first, in step S110, the signal processing unit 10 of the electronic device 1 processes the received signal (received signal). The signal processing performed in step S110 may include, for example, the above-mentioned 2D-FFT, CFAR processing, and / or direction-of-arrival estimation. Such operations may be performed by, for example, the received signal processing unit 12 of the signal processing unit 10.

[0091] Next, in step S120, the signal processing unit 10 extracts a vibration source from the information processed in step S110. The signal processing performed in step S120 may include, for example, filtering the data resulting from 2D-FFT processing. In step S120, the signal processing unit 10 may extract spectral components only at the position where the subject 200 is present. Here, the position where the subject 200 is present may be identified by various known methods. This operation may be performed, for example, by the heart rate extraction unit 13 of the signal processing unit 10.

[0092] Next, in step S130, the signal processing unit 10 converts the result of the previous processing into a vibration waveform. The processing performed in step S130 may include, for example, a process of extracting phase information from the IQ data. In step S130, the signal processing unit 10 may also include a process of extracting vibration data including heart sounds from the spectral components of the 2D-FFT processing of the subject 200 extracted in step S120. This operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.

[0093] Next, in step S140, the signal processing unit 10 extracts vibration data from the result of the previous processing. The processing performed in step S140 may include, for example, frequency filtering. In step S140, the signal processing unit 10 may extract a low-frequency signal including an envelope of the heart sounds of the subject 200 by performing frequency filtering. This operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.

[0094] Next, in step S150, the signal processing unit 10 detects peaks of the heartbeat of the subject 200 from the results of the previous processing, and calculates the interval between the peaks to calculate the RR interval (RRI) of the subject 200. In step S150, the signal processing unit 10 may detect peaks of a low-frequency signal including an envelope of the heart sounds of the subject 200. Also, in step S150, the signal processing unit 10 may calculate and / or extract the interval between each peak. Such an operation may be performed by, for example, the calculation unit 14 of the signal processing unit 10. As described above, in step S150, the signal processing unit 10 may extract the interval between heart sounds.

[0095] Next, in step S160, the signal processing unit 10 may calculate the heart rate variability (HRV) of the subject 200. In step S160, the signal processing unit 10 may calculate the HRV of the subject 200 by calculating the spectral density of the time-series data of the RRI. When calculating the power spectral density of the time-series data of the RRI, a frequency analysis of the time-series waveform of the RRI may be performed, for example, by using the Welch method. This operation may be performed by, for example, the calculation unit 14 of the signal processing unit 10. Furthermore, in step S160, the signal processing unit 10 may analyze the spectrum from the processing result of the previous stage.

[0096] As described above, the electronic device 1 according to an embodiment extracts components that are thought to be the envelope of the heartbeat by frequency filtering the vibrations at the position where the subject 200 is present, and determines the interval between the peaks of the envelope as the heartbeat interval. In this way, the electronic device 1 according to an embodiment can calculate the approximate heartbeat interval of the subject 200.

[0097] In the calculation of the heartbeat interval as described above, several peaks are present over a period of several tens of milliseconds, as shown in FIG. 10 . Therefore, there is a certain degree of uncertainty in selecting the heartbeat peaks. Therefore, the accuracy of the calculated heartbeat interval also has an error of several tens of milliseconds. For example, in the calculation of the heartbeat interval as described above, an error of at least 20 milliseconds occurs when compared with the instantaneous RRI obtained by an electrocardiograph. With this accuracy, it is difficult to calculate heart rate variability (HRV) and perform autonomic nervous and / or emotional analysis of a person. In other words, in the calculation of the heartbeat interval as described above, it is difficult to perform more advanced medical analysis by extracting heart sounds. Furthermore, the calculation of the heartbeat interval as described above is prone to errors if the measurement environment is not favorable. Therefore, in the calculation of the heartbeat interval as described above, it may be difficult to obtain a highly robust pulse rate depending on the measurement environment.

[0098] Therefore, it is conceivable to further cut the peaks on the high frequency side by using frequency filtering so that the time interval between heartbeats becomes approximately 0.4 to 0.8 seconds. However, since this does not exceed the accuracy of the signal information shown in Figure 10, it seems difficult to reduce the error of approximately several tens of milliseconds.

[0099] Furthermore, the calculation of the heartbeat intervals described above does not involve extracting heart sounds, and therefore it is difficult to obtain information that contributes to diagnosis (for example, auscultation during medical treatment) based on the acoustic properties of the heart sounds themselves.

[0100] To measure the heart rate of a person or animal using technology such as radar, one possible approach is to perform frequency filtering on the vibrations at the location of the detected target (person or animal) to extract components that appear to represent the heart rate envelope, and then perform frequency analysis on the results. However, to obtain the heart rate with high accuracy and robustness, it is necessary to appropriately search the data space formed by the radar data and estimate the heart rate. That is, it is necessary to estimate the heart rate from the results of extracting chest vibrations corresponding to heart sounds from multiple locations (radar ranges) from the radar data, calculate multiple heart rate time series data candidates, and then select the optimal heart rate data from among them. In this disclosure, chest vibrations corresponding to heart sounds may be simply referred to as "heart sounds." It would be desirable to extract heart sounds using radar using high-frequency bands of millimeter waves or higher, analyze the heart sounds themselves, and accurately extract heart rate intervals.

[0101] Therefore, the electronic device 1 according to an embodiment further improves on the above-described method. As a result, the electronic device 1 according to an embodiment extracts heart sounds using, for example, a radar using a high-frequency band of millimeter waves or higher, thereby analyzing the heart sounds themselves and extracting accurate heartbeat intervals. In this way, the electronic device 1 according to an embodiment detects the heart rate of a human body or the like with high accuracy and robustness by transmitting and receiving radio waves. Such a method will be described below.

[0102] In order to extract the heart sounds of the subject 200 with high accuracy, the electronic device 1 performs appropriate signal processing on the signal (chirp signal) received by the electronic device 1 in an appropriate order to narrow down the subspace and subspace basis in which the signal components of the heart sounds of the subject 200 exist. The electronic device 1 according to an embodiment may employ different basis vectors in the linear space in which the heart sound signals of the subject 200 exist, describe the space using an appropriate coordinate system, and extract the subspace based on the respective coordinates. Through such processing, the electronic device 1 according to an embodiment can search for the subspace in which the heart sounds of the subject 200 exist. In an embodiment, an appropriate coordinate system may be used depending on the purpose. For example, a coordinate system of a space obtained by 2D-FFT processing, a coordinate system of a time-series signal obtained by temporally contracting a chirp signal, a coordinate system based on the Fourier transform of the time-series signal, or a coordinate system based on continuous / discrete wavelets may be used.

[0103] The electronic device 1 according to an embodiment may perform the following characteristic processes on the received signal in a step-by-step procedure. The characteristic processes performed by the electronic device 1 according to an embodiment will be outlined below. The electronic device 1 according to an embodiment performs two characteristic processes. That is, the electronic device 1 according to an embodiment performs (1) extraction of a subspace, and (2) calculation of a time series vector as time series data of the heart rate. Each process will be described in more detail below.

[0104] (1) Extraction of subspace First, the electronic device 1 according to an embodiment executes a process for removing data that is considered unnecessary in a linear space from a received signal. The electronic device 1 according to an embodiment may execute a process for removing unnecessary subspaces to leave necessary subspaces. Specifically, the electronic device 1 may execute the following process.

[0105] (1-1) First step: Dimension reduction In the first stage, an appropriate window function is applied to the 2D-FFT processing of the chirp signal received by the electronic device 1, and only the micro-Doppler components of the point cloud where the subject 200 (e.g., a person or animal) is present are extracted by estimating the direction of arrival. Here, multiple window functions are generated around the approximate position of the target, with multiple distance ranges at their centers, and these multiple window functions are applied. Hereinafter, this process of applying multiple window functions is referred to as "multi-window processing." Here, one vibration time series signal may be generated for one window function, i.e., one center distance range.

[0106] (1-2) Second stage: Reducing the dimension of the subspace In the second stage, principal component analysis and / or singular value decomposition of the set of time-series waveforms of the vibrations extracted in the processing of the first stage is performed on the micro-Doppler signals.

[0107] (1-3) Third step: Reducing the dimension of the subspace In the third stage, frequency filtering is performed using a short-time Fourier transform, a continuous wavelet transform, and / or a band-pass filter.

[0108] (1-4) Fourth stage: Reducing the dimension of the subspace In the fourth step, heart sounds are extracted by performing multi-resolution analysis using discrete wavelet transforms with wavelet and scaling functions appropriate for the heart sounds.

[0109] (2) Calculating the time series vector as time series data of the heart rate Next, to facilitate processing of the extracted heart sound data, the electronic device 1 according to an embodiment may execute a process of generating an envelope from the waveform of the extracted heart sound. Furthermore, the electronic device 1 according to an embodiment may calculate a time series vector as time series data of the heart rate based on the results of frequency analysis of the generated envelope. Specifically, the electronic device 1 may execute the following process.

[0110] (2-1) Envelope generation Here, an envelope is generated (extracted) from the heart sounds extracted in the above (1) subspace extraction. The envelope generation process may use any of the processes such as continuous wavelet transform, discrete wavelet transform, wavelet scattering coefficients, Mel-frequency cepstrum coefficients, or moving variance.

[0111] (2-2) Heartbeat frequency analysis The heartbeat frequency is analyzed by applying a window function to the heart sound envelope extracted by the process (2-1) above and performing frequency analysis using a fast Fourier transform (FFT) or similar process. Here, a time-varying window is applied to the heart sound envelope, and time-varying frequency characteristics are generated using, for example, the MUSIC method. Furthermore, a multidimensional array is generated by overlapping these time-varying frequency characteristics for the multiple windows described in "(1) Extracting subspace" above. This multidimensional array can be represented by the following tensor (multidimensional array). (Peak frequency vector) × (Time window array) × (Multi-window array) Here, × represents a tensor product. In this disclosure, the above-described tensor is also referred to as a heartbeat frequency data tensor.

[0112] (2-3) Extraction of time series vectors Based on the heartbeat frequency data tensor generated by the process (2-2) above, the dimension of the frequency vector and the multi-window is reduced, thereby calculating a time series vector of the heartbeat frequency and a time series vector of the heart rate.

[0113] According to the electronic device 1 of the embodiment, accurate time-series data of the heart rate can be calculated by going through the processes at the above-mentioned stages.

[0114] Next, the operation of the electronic device 1 according to the embodiment will be described in more detail.

[0115] Fig. 12 is a flowchart showing an example of the operation performed by the electronic device 1 according to an embodiment. Fig. 13 is a flowchart showing in more detail an example of the operation of step S16 in Fig. 12. Hereinafter, the flow of the operation performed by the electronic device 1 according to an embodiment will be described with reference to Figs. 12 and 13.

[0116] Step S11 shown in Fig. 12 can be performed in the same manner as the operation in step S110 shown in Fig. 11. That is, when the operation shown in Fig. 12 starts, first, in step S11, the signal processing unit 10 of the electronic device 1 processes a received signal (received signal). The signal processing performed in step S11 may include, for example, the above-mentioned 2D-FFT, CFAR processing, and / or direction-of-arrival estimation. Such an operation may be performed by, for example, the received signal processing unit 12 of the signal processing unit 10.

[0117] In step S12, the signal processing unit 10 performs multi-window processing. In step S12, the signal processing unit 10 may perform a process of determining candidates for human or animal clusters and / or a filtering process (multi-window processing) on ​​data that has been subjected to 2D-FFT processing using multiple window functions.

[0118] In step S12, the signal processing unit 10 may perform multi-window processing to generate multiple heart sound candidates as a preliminary step to selecting the best heart sound candidate. Here, the signal processing unit 10 may perform processing on the 2D-FFT-processed data to group point clouds corresponding to people or animals into clusters. The signal processing unit 10 may also perform multi-window processing by applying multiple window functions to the selected clusters. The multi-window processing performed here may include processing to generate 2D-FFT-processed data that serves as the basis for the multiple heart sound candidates.

[0119] In step S12, the signal processing unit 10 may generate multiple candidate heart sounds by extracting only the region where the subject 200 is present on the range-Doppler plane calculated by 2D-FFT using an appropriate window function. In step S12, the signal processing unit 10 may provide multiple window functions as the appropriate window function. Here, the signal processing unit 10 may use window functions such as a Hanning window, a Hamming window, and a Blackman-Harris window, for example.

[0120] In step S12, the signal processing unit 10 may detect a vibration source (target) including heart sounds and / or body movements including breathing of the subject 200, based on, for example, the following first to third procedures.

[0121] (Step 1) The signal processing unit 10 classifies the points on the range-Doppler plane that exceed the CFAR threshold into areas of a predetermined angle based on the result of direction-of-arrival estimation (see FIGS. 5 and 6). For example, if the angle on the xy plane shown in FIG. 6 is θ, the points may be classified into areas A to C of the following angles. Area A: -10deg.<θ<10deg. Area B: -20deg.<θ≦-10deg. Area C: 10deg.≦θ<20deg.

[0122] (Second step) The signal processing unit 10 applies clustering to a group of points that exceed the CFAR threshold on a range-Doppler plane such as S1 in Fig. 5, within a group of areas of a certain angle classified in the first step. Here, a method such as DBSCAN may be applied as the clustering method.

[0123] (Third step) Assuming that L clusters have been processed in the second step, the signal processing unit 10 calculates the deviation D of the number of bins in the Doppler direction for the l-th cluster. dev [l] is a threshold D dev,th As a result of this comparison, the signal processing unit 10 determines that Ddev [l]≧D dev,th Only those who satisfy the condition are determined to be the subject 200 and a flag HF[l] is set. That is, the signal processing unit 10 may perform the process shown in the following [Pseudo Code 1], for example.

[0124] [Pseudocode 1] for l = 1 to L do if D dev [l] ≧ D dev,th HF[l] = 1 else HF[l] = 0 end if end do

[0125] In step S13, the signal processing unit 10 restores the set of chirp signals. In step S13, the signal processing unit 10 may restore the set of chirp signals by performing an inverse discrete Fourier transform (2D-IFFT (inverse fast Fourier transform)) on the plurality of generated 2D-FFT data.

[0126] Fig. 14 is a diagram illustrating multi-window processing by the electronic device 1 according to one embodiment. Fig. 14 illustrates multi-window processing for generating multiple time-series signals from a point cloud belonging to a cluster selected by the above-described pseudocode 1 using window functions shifted in multiple range directions.

[0127] In the upper part of FIG. 14, the horizontal axis indicates distance (Range) and the vertical axis indicates Doppler velocity. The upper part of FIG. 14 shows the results of clustering performed on a point cloud of 2D-FFT data at a certain time. The upper part of FIG. 14 shows how three clusters, Cluster 1, Cluster 2, and Cluster 3, are generated as a result of the clustering. The middle part of FIG. 14 conceptually shows how the signal processing unit 10 performs multi-window processing on each of Cluster 1, Cluster 2, and Cluster 3. The middle part of FIG. 14 shows how the signal processing unit 10 performs multi-window processing on each of Cluster 1, Cluster 2, and Cluster 3 by gradually shifting the window function in the distance (Range) direction. The lower part of FIG. 14 shows the results of calculating vibration velocities by the signal processing unit 10 performing 2D-IFFT on each of Cluster 1, Cluster 2, and Cluster 3. The lower part of FIG. 14 shows how time-series vibration signals are generated by performing 2D-IFFT on each of the clusters. In practice, a window function is applied to each cluster shown in FIG. 14 multiple times while being shifted slightly in the range direction, thereby obtaining multiple time-series vibration signals.

[0128] In step S12 of Fig. 12, the signal processing unit 10 applies multiple window functions to a cluster formed from a group of points for which flag HF[l] = 1 has been selected in accordance with the above-mentioned [Pseudo Code 1]. This process generates 2D-FFT data corresponding to each window function. Here, the 2D-FFT data generated by the l-th window function is expressed as in the following equation (1).

number

[0129] In this case, in step S13, the signal processing unit 10 can calculate the time-series signal waveform of the vibration velocity according to the following equation (2): The signal waveform is restored to a set of chirp signals by a two-dimensional inverse fast Fourier transform (2D-IFFT).

number

[0130] Hereinafter, only one cluster, for example, cluster 1 shown in FIG. 14, will be described as an example.

[0131] In step S14, the signal processing unit 10 analyzes the principal components of the signal and / or removes noise. Step S14 may be singular value decomposition (SVD) for pre-processing the signal noise. This process may be performed to remove low-energy noise and / or low-level noise that has been introduced due to the uncertainty of the Fourier transform.

[0132] In step S14, the signal processing unit 10 calculates a set S of received chirp signals extracted in the 2D-FFT plane. l Let U denote the matrix of left singular vectors, Σ denote the matrix of diagonal singular values, and V denote the matrix of right singular vectors. Then, the set of received chirp signals S l By singular value decomposition of the above, the following equation (3) is obtained.

number

[0133] Next, the signal processing unit 10 limits the number of row vectors in the row of the left singular vectors in the above formula (3) (U ext ) The signal processing unit 10 may also limit the number of diagonal elements of the diagonal matrix of the singular values ​​of the above equation (3) (Σ ext ) can be used. This eliminates unwanted noise signals and vibration components other than the desired position. As a result, Sl The signal S projected onto the subspace of the target signal l ext is expressed as the following equation (4).

number

[0134] FIG. 15 is a diagram illustrating the relationship between the ranks representing the target signal and the noise signal in singular value decomposition. FIG. 15 is a diagram in which the results of SVD are arranged in descending order of singular value (corresponding to energy), with the horizontal axis representing the rank and the vertical axis representing the singular value. The present disclosure adopts the idea that a signal extracted that is equal to or greater than a certain singular value becomes the target signal. In this case, the rank of the singular value that determines whether or not it is the target signal is the rank region of 30 or less, indicated by the dark gray area in the graph of FIG. 15. As shown in FIG. 15, the region of 30 or less is the region of the target signal. In FIG. 15, the rank portion of this target signal is 1 or more and 30 or less. Therefore, in the present disclosure, for example, singular vectors corresponding to left and right singular values ​​in the rank region of 1 or more and 30 or less may be extracted.

[0135] In the above example, the rank portion of the target signal is set to 1 or more and 30 or less. The maximum rank may basically be determined empirically. The maximum rank (rank 30 in this disclosure) may also be determined using a statistical method. In the graph of FIG. 15, the singular value suddenly decreases at rank 130, and signals at ranks after that are basically noise. Therefore, signals at ranks after that may be unnecessary.

[0136] In the graph of Fig. 15, the left and right singular vectors (vectors spanning the signal space) corresponding to the singular values ​​in the area of ​​rank 31 to 130 also contain some target signal components. These left and right singular vectors are assumed to be mainly caused by unnecessary minute vibrations and / or artificial noise (artifacts) generated by radar signal processing. Therefore, such elements may be discarded.

[0137] After noise is removed by SVD in step S14, the signal processing unit 10 converts the noise-removed result into a signal waveform (step S15). In step S15, the signal processing unit 10 may convert the beat signal (IQ data) into a time-series signal of vibration.

[0138] In step S15, the signal processing unit 10 obtains the vector shown in the following equation (5) by taking the sum of the column vectors of the chirp signals based on the above equation (4).

number

[0139] Finally, in step S15, s l ext,sum By taking the argument and derivative on the Gaussian plane, the vector representing the vibration velocity is calculated as shown in the following equation (6).

number

[0140] Next, in step S16, the signal processing unit 10 extracts heart sounds and / or analyzes heartbeat intervals from the obtained vibration waveforms. Specifically, in step S16, the signal processing unit 10 extracts heart sounds from the vibration waveform shown in the above equation (6). In step S16, the signal processing unit 10 may execute a process to generate a heart sound waveform, a process to calculate the heartbeat interval (RRI), and / or a process to calculate the heart rate variability (HRV). This enables the signal processing unit 10 to calculate the heartbeat interval. This process may be executed on waveforms of multiple vibration velocities corresponding to each window function.

[0141] As described above, the process of step S16 shown in Fig. 12 may, in more detail, include at least a part of the processes of steps S21 to S25 shown in Fig. 13. Each of steps S21 to S25 shown in Fig. 13 will be described in more detail below.

[0142] 13, the signal processing unit 10 performs a process of removing noise from a time-series waveform (denoising). Here, the process of removing noise may use an Empirical Bayes method or a wavelet method such as a Continuous Wavelet Transform (CWT).

[0143] More specifically, in step S21, the signal processing unit 10 calculates the vibration velocity vector v corresponding to the l-th window function. l vib Further preprocessing is performed to remove unnecessary noise.

[0144] In step S21, the signal processing unit 10 may perform noise removal processing by limiting the band using an empirical Bayes method and / or a continuous wavelet. Also, in step S21, the signal processing unit 10 may perform noise removal processing by frequency subtraction using a noise profile for artifact noise and the like accompanying the nonlinear processing from step S11 to step S14 shown in Fig. 12.

[0145] Next, in step S22, the signal processing unit 10 extracts the waveform of the target signal, i.e., the waveform of the heart sound. In this case, for example, a discrete wavelet technique such as maximum overlap discrete wavelet transform (MODWT) may be used.

[0146] More specifically, in step S22, the signal processing unit 10 may perform multiresolution analysis using a discrete wavelet transform on the denoised signal as preprocessing, employing a wavelet waveform having a waveform similar to the heart sound waveform. In this way, in step S22, the signal processing unit 10 empirically extracts only the subspace of the level of multiresolution analysis in which the heart sounds of the subject 200 exist. In this way, in step S22, the signal processing unit 10 may extract the heart sounds of the subject 200. Specifically, the signal processing unit 10 may use a maximum overlap multiresolution analysis (MODWT) or the like to improve the temporal resolution. Furthermore, wavelet bases suitable for extracting heartbeats may be, for example, Symlet and Daubechies. Furthermore, the order of these may be set appropriately each time.

[0147] The denoising process in step S21 is performed on the signal velocity vector v l vib The signal sent to v l vib,dn Furthermore, the real part Re(v l vib,dn ) may be subjected to multi-resolution analysis as shown in Fig. 16. Fig. 16 is a diagram conceptually showing multi-resolution analysis using discrete wavelet transform. Then, the signal processing unit 10 obtains the cardiac sound waveform h by reconstructing the waveform by limiting the empirically appropriate level to j0∈N in the following equation (7).

number

[0148] In this way, the electronic device 1 according to one embodiment can obtain a cardiac sound waveform h such as the waveforms shown in Figures 22 and 23, which will be described later. This result can be directly used in cardiac diagnosis and treatment.

[0149] Next, in step S23, the signal processing unit 10 generates an envelope waveform. The envelope waveform generation process may use any of processes such as continuous wavelet transform, discrete wavelet transform, wavelet scattering coefficients, Mel-frequency cepstrum coefficients, or shift variance. In step S23, the signal processing unit 10 extracts a scalogram using continuous wavelet transform and an envelope waveform by converting the scalogram into a one-dimensional form in order to detect (extract) the energy peak of the target signal (cardiac sound waveform) extracted in step S22. Here, discrete / continuous wavelet transform, shift variance, and / or Hilbert transform may be used.

[0150] More specifically, in step S23, the signal processing unit 10 calculates the cardiac waveform h l To obtain the envelope waveform of , a scalogram may be obtained by continuous wavelet transform. The signal processing unit 10 calculates the sum of the scalogram over a certain range of the frequency axis from f1 to f2 at each time. As a result, the signal processing unit 10 obtains the one-dimensional waveform s as shown in Figs. 17 and 18. l h 17 and 18 are diagrams showing examples of analysis results of continuous wavelet transform. The continuous wavelet transform may be replaced with a discrete wavelet transform by appropriately setting the resolution.

[0151] Figure 17 shows the heart sound waveform h l Fig. 17 is a diagram showing the time change of the normalized vibration frequency. The horizontal axis of Fig. 17 represents time in units of the number of samples, and the vertical axis of Fig. 17 represents the normalized vibration frequency for each time.

[0152] FIG. 18 shows the heart sound waveform h shown in FIG. l Fig. 18 is a diagram showing the result of applying a continuous wavelet transform to one-dimensionalize the scalogram of Fig. 18. Fig. 18 is a diagram showing the change in vibration velocity over time. The horizontal axis of Fig. 18 represents time in units of the number of samples, and the vertical axis of Fig. 18 represents vibration velocity.

[0153] Here, the matrix representing the absolute value of the scalogram satisfies the following equation (8).

number

[0154] [Pseudocode 2] for m=1:M

number

[0155] Next, in step S24, the signal processing unit 10 analyzes the frequency of the heartbeat. The envelope waveforms (heart sound envelope data) generated by the process of step S23 exist in the same number as the number of window functions, as shown in Fig. 14. Here, frequency analysis is performed for one of the multiple window functions.

[0156] Fig. 19 is a diagram showing the arrangement of time windows used by the signal processing unit 10 to analyze the frequency of the heart sound envelope. In the heart sound envelope shown in the upper part of Fig. 19, the time windows cut out for time frames t=1 and t=2 are shown. In the heart sound envelope shown in the lower part of Fig. 19, the time windows cut out for time frames t=T-1 and t=T are shown. Here, T and t are discrete times and represent non-negative integers. In addition, in the heart sound envelope shown in the upper part of Fig. 19, the width of the time window w win and the width of overlap between the time windows w o Here, the time window movement step is w step =w win -w o This becomes:

[0157] Generally, the heart rate is a very low frequency. Therefore, a high-resolution method must be used for frequency analysis. Here, processing using the MUSIC (MUltiple SIgnal Classification) method as one of the subspace methods will be described. Below, processing for frequency analysis using MUSIC will be explained. This processing may include the following four processing steps: [First Processing] to [Fourth Processing].

[0158] [First process] First, the signal processing unit 10 arranges the time-series signals for each snapshot to obtain an observation matrix X=[x1, x2, ..., x N ], where the observation matrix X can be obtained by dividing the samples in one window shown in FIG. 19 into equal sizes.

[0159] [Second process] The signal processing unit 10 calculates the autocorrelation matrix R based on the observation matrix generated in the above [first process]. x =1 / N X X H Here, the subscript H in the upper right corner indicates the Hermitian transpose.

[0160] [Third Processing] The signal processing unit 10 performs eigenvalue decomposition on the autocorrelation matrix Rx obtained in the above [second processing]. In this eigenvalue decomposition, the signal processing unit 10 classifies the eigenvectors with larger eigenvalues ​​as the basis vectors of the signal space and those with smaller eigenvalues ​​as the basis vectors of the noise space. Here, the eigenvectors are sorted in descending order of eigenvalue, and the eigenvectors up to 1≦m≦pth are set as the signal space, and the eigenvectors up to p+1≦m≦Mth are set as the basis vectors v of the noise space. m Let (p+1≦m≦M).

[0161] [Fourth Process] The signal processing unit 10 calculates a frequency vector e=[1, e jω ,e j2ω ,…,e j(M-1)ω ]T The inner product of the basis vectors in the noise space is calculated for [mathematical formula - see original document]. In this way, the MUSIC spectrum shown in the following equation (10) can be obtained.

number

[0162] Fig. 20 is a diagram showing an example of a music spectrum (frequency analysis) calculated using equation (10) above. The vertical axis of Fig. 20 represents the music spectrum, and the horizontal axis of Fig. 20 represents frequency. The music spectrum shown in Fig. 20 has three frequency peaks indicated as g1, g2, and g3. The frequency peaks g1, g2, and g3 are approximately 1 Hz, 1.7 Hz, and 3.1 Hz, respectively.

[0163] As a result of the above processing, an appropriate heartbeat frequency may finally be selected. The processing for selecting an appropriate heartbeat frequency may be performed by the following two-stage processing. These processes may be performed by the signal processing unit 10 in the electronic device 1 according to one embodiment.

[0164] [Process A] First, multiple frequency ranges of heartbeats to be detected are set, and frequency peaks within each frequency range are extracted.Furthermore, a time series vector of frequency peaks is generated for each frequency range of heartbeats to be detected.

[0165] Generally, the normal heart rate of a human is about 40 to 160 beats per minute (BPM = 40 to 160). Therefore, the range of human heart rates is first divided into three ranges of 40 beats each, as follows: Class 1: BPM=40 to 80 Class 2: BPM=80 to 120 Class 3: BPM=120 to 160

[0166] If we convert BPM to frequency in each of these classes, the heart rate frequency classes are as follows: Class 1: 0.67[Hz] to [1.33]Hz Class 2: 1.33[Hz] to [2.0]Hz Class 3: 2.0[Hz] to [2.67]Hz

[0167] Therefore, from the frequency peaks of the MUSIC spectrum as shown in Figure 20, peaks that fall into the above-mentioned classes 1, 2, and 3 are extracted for each time frame. For example, peak g1 shown in Figure 20 has a frequency of about 1 Hz, so it falls into class 1. Peak g2 shown in Figure 20 has a frequency of about 1.7 Hz, so it falls into class 2. Furthermore, peak g3 shown in Figure 20 has a frequency of about 3.1 Hz, so it does not fall into any of the classes. The above can be summarized as follows. Grade 1: Peak G1 Grade 2: Peak G2 Class 3: Not applicable

[0168] The above processing can be shown by the following [Pseudo Code 3]. Here, the number of time windows (time frames) to be processed is T, and the index is t. The number of heartbeat frequency classes is J, and the index is j. The minimum frequency value in the heartbeat frequency class j is f j min Let the maximum value be f j max Let L be the number of window functions on the 2D-FFT shown in Figure 14, and let l be the index. Also, let the MUSIC spectrum P MU The peak vector of (ω) is p MU,peak Then, the heartbeat frequency data tensor to be obtained is F l,t,j Let's say.

[0169] [Pseudocode 3] for l=1:L for t=1:T for j=1:J

number

[0170] In the above formula (11), find(Condition) is p MU,peak This function searches for a value between the maximum and minimum values ​​in the range. If no such value is found, a value such as NaN (Not a number, as standardized in the IEEE 754 floating-point standard) may be used to indicate that the value does not exist.

[0171] [Process B] Next, the optimum frequency peak time series vector is selected from the frequency peak time series vectors generated for the frequency ranges of the detected multiple heartbeats, and this is used as the final heartbeat frequency vector.

[0172] Here, the calculated heartbeat frequency data tensor is F l,t,j The indexes l and j are determined to be natural numbers. In other words, the best one is selected from the L x J heartbeat frequency vectors. This process is achieved by selecting a frequency vector that satisfies the following conditions: (1) the number of missing values ​​(NaN) is small, and (2) the change in heartbeat frequency is smallest.

[0173] The above condition (1) is based on the principle that if there are many missing values, there will be no peak in the MUSIC spectrum corresponding to that class j. In addition, the above condition (1) is based on the principle that if the window position in the 2D-FFT and the class of the heartbeat frequency are not appropriate, the peak frequency of the MUSIC spectrum will not be stable and will have extreme changes.

[0174] FIG. 21 is a diagram illustrating the selection (pass / fail determination) of a heartbeat frequency vector by the electronic device 1 according to an embodiment. The vertical axis of FIG. 21 indicates the heartbeat frequency, and the horizontal axis of FIG. 21 indicates time. In FIG. 21, graph h1 shows an example of a heartbeat frequency vector determined to be optimal by the electronic device 1 according to an embodiment. Also, in FIG. 21, graph h2 shows an example of a heartbeat frequency vector determined to be inappropriate by the electronic device 1 according to an embodiment. As shown in graph h2 in FIG. 21, a heartbeat frequency vector calculated based on an inappropriate window position on the 2D FFT and / or an inappropriate heartbeat frequency class exhibits unnaturally large fluctuations.

[0175] The electronic device 1 according to one embodiment can statistically select a heartbeat frequency vector such as graph h1 and / or graph h2 shown in Fig. 21 by, for example, calculating the variance. In this case, the selection condition can be written as, for example, the following equations (12) and (13). Equation (12) represents the above-mentioned condition (1). Equation (13) represents the above-mentioned condition (1).

number

number

[0176] Through the above processing, the electronic device 1 according to one embodiment can obtain an optimal heartbeat frequency vector as shown in the following equation (14).

number

[0177] Furthermore, through the above processing, the electronic device 1 according to one embodiment can obtain the heart rate data vector r as shown in the following equation (15) by processing the inverse of the optimal heart rate frequency vector as shown in equation (14) and converting it to an integer.

number

[0178] By performing the processing described above, the electronic device 1 according to one embodiment can obtain a time series waveform of heart sounds, and can also obtain a time series waveform of heart rate.

[0179] Therefore, in step S25 shown in FIG. 13, the signal processing unit 10 can generate a time-series vector of the heart rate.

[0180] 22, 23, and 24 are diagrams showing an example of data obtained by the electronic device 1 according to an embodiment. FIGS. 22 and 23 are diagrams showing an example of a time-series waveform of a heart sound obtained by the electronic device 1 according to an embodiment. FIG. 24 is a diagram showing an example of a time-series waveform of a heart sound obtained by the electronic device 1 according to an embodiment. In FIGS. 22 and 23, the horizontal axis represents time, and the vertical axis represents vibration velocity. In FIG. 24, the horizontal axis represents time, and the vertical axis represents heart rate. FIG. 22 is a diagram showing a time-series waveform of a heart sound obtained by the electronic device 1 according to an embodiment. FIG. 23 is a diagram showing an enlarged time span of the area surrounded by a dashed line in FIG. 22.

[0181] As shown in Fig. 23, the first heart sound S1 and the second heart sound S2 can be clearly identified from the time-series waveform of heart sounds obtained by the electronic device 1 according to an embodiment. Furthermore, as shown in Fig. 23, the heart beat interval RRI, calculated from the interval between the first heart sound S1 and the second heart sound S2, can also be clearly identified from the time-series waveform of heart sounds obtained by the electronic device 1 according to an embodiment. While the RRI and the heart beat interval are not strictly the same, they are considered to be approximately the same. Therefore, in the present disclosure, the RRI and the heart beat interval will be described as being the same thing.

[0182] Fig. 24 is a diagram showing an example of time-series data of heart rate calculated from the time-series data of heart sounds shown in Fig. 22 and Fig. 23. The results shown in Fig. 24 were calculated based on the heart sounds accurately extracted in Fig. 22 and Fig. 23.

[0183] As described above, the electronic device 1 according to one embodiment can obtain detailed cardiac waveforms such as those shown in Figs. 22 and 23. The electronic device 1 according to one embodiment can detect the heartbeat of a human body or the like with high accuracy by transmitting and receiving radio waves. Therefore, the electronic device 1 according to one embodiment can detect weak vibrations such as the heartbeat of a human body with high accuracy by transmitting and receiving radio waves such as millimeter waves, and is expected to be useful in a wide variety of fields.

[0184] According to an embodiment of the electronic device 1, heart sounds are extracted using radar using a high-frequency band of millimeter waves or higher, and the heart sounds themselves are analyzed, enabling accurate heartbeat interval extraction. To obtain the heart rate with high accuracy and robustness, it is necessary to appropriately search the data space formed by the radar data and estimate the heart rate. According to the embodiment of the electronic device 1, the heart rate can be obtained with high accuracy and robustness.

[0185] 22 to 24, only the first heart sound S1 and the second heart sound S2, which are prominent in healthy individuals, are described. However, according to the electronic device 1 of one embodiment, similar processing can be performed even when a third and / or fourth heart sound occurs due to an abnormal heartbeat.

[0186] (Other embodiments) Other embodiments will be described below.

[0187] The electronic device 1 according to the above-described embodiment performs the process using the MUSIC method in step S24 of Fig. 13. However, the electronic device 1 according to other embodiments may perform the process using another subspace method, such as the Root MUSIC method, in step S24 of Fig. 13.

[0188] In the [Process A] described in step S24 of FIG. 13, multiple classes of heartbeat frequency and heart rate are set. However, in other embodiments, if the user's heart rate range is known in a database or the like, only one class of heartbeat frequency and heart rate may be set. In this case, the heartbeat frequency data tensor is F l,t,j =F l,t Therefore, in this case, the index narrowed down by the above formula (12) and formula (13) is only the index 1 of the window function in the 2D-FFT.

[0189] In other embodiments, the process in step S14 shown in Fig. 12 and / or the process in step S21 shown in Fig. 13 may be omitted depending on the situation. Also, in other embodiments, the process in step S14 shown in Fig. 12 (the process of analyzing singular value decomposition / principal components) may be substituted by another subspace method.

[0190] In another embodiment, in step S24 shown in Fig. 13, [Process A] and [Process B] are described as being performed on the heartbeat frequency vector. However, in another embodiment, before [Process A], the heartbeat frequency vector may be converted into a heart rate vector and then processed.

[0191] In one embodiment, the transmitting antenna array 24 and / or the receiving antenna array 31 included in the electronic device 1 are not limited to the arrangement shown in Fig. 8. For example, in one embodiment, the receiving antenna array 31 included in the electronic device 1 may have a configuration as shown in Fig. 25. Fig. 25 is a diagram showing an example of a URA (Uniform Rectangular Array) receiving antenna. By employing a URA receiving antenna as shown in Fig. 25, it is possible to estimate the directions of arrival at two angles using only the URA receiving antenna without changing the directivity using a beamformer in the transmitting antenna array 24.

[0192] 2, the signal processing unit 10 has been described as including functional units such as the heartbeat extraction unit 13 and the calculation unit 14. However, in one embodiment, the processing performed by the heartbeat extraction unit 13 and / or the calculation unit 14 may be performed by an external computer or processor.

[0193] In one embodiment, the process in step S14 shown in Fig. 12 (the process of analyzing the singular value decomposition / principal components) may be omitted if the number of samples N per chirp can be set to a large number. In one embodiment, the process in step S14 shown in Fig. 12 (the process of analyzing the singular value decomposition / principal components) may also be omitted if the inclusion of other noise can be eliminated by a hardware technique or the like.

[0194] In the process of step S23 (generation of the heart sound envelope waveform) shown in Fig. 13, the results of the continuous wavelet transform are added in the frequency axis direction according to [Pseudo Code 2]. However, the envelope waveform may be generated by other methods, such as moving average or Hilbert transform.

[0195] As described above, the electronic device 1 according to an embodiment detects weak vibrations such as heartbeats using, for example, a millimeter-wave sensor including multiple transmitting antennas and multiple receiving antennas. When the electronic device 1 according to an embodiment does not detect a target, it detects the target's body movement by changing the transmission phase of the antenna's beamforming pattern. On the other hand, when the electronic device 1 according to an embodiment detects the target's body movement, it performs beamforming in the direction of the body movement and detects the heartbeat. In this way, the electronic device 1 according to an embodiment can improve signal quality by automatically detecting the direction of the human body. Therefore, the electronic device 1 according to an embodiment can improve the heartbeat detection accuracy and / or detection range. Therefore, the electronic device 1 according to an embodiment can detect a human heartbeat with high accuracy.

[0196] While the present disclosure has been described based on various drawings and examples, it should be noted that those skilled in the art would easily be able to make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of the present disclosure. For example, the functions contained in each functional unit can be rearranged so as not to cause logical inconsistencies. Multiple functional units may be combined into one or divided. The above-described embodiments of the present disclosure are not limited to faithful implementation of each of the described embodiments, but may be implemented by combining features or omitting some features as appropriate. In other words, those skilled in the art can make various modifications and alterations based on the present disclosure. Therefore, these modifications and alterations are within the scope of the present disclosure. For example, in each embodiment, each functional unit, means, step, etc. can be added to other embodiments so as not to cause logical inconsistencies, or can be replaced with each functional unit, means, step, etc. of other embodiments. Furthermore, in each embodiment, multiple functional units, means, steps, etc. can be combined into one or divided into two or more. Furthermore, each of the above-described embodiments of the present disclosure is not limited to being implemented faithfully according to each of the described embodiments, but can also be implemented by combining each feature or omitting some of them as appropriate.

[0197] The above-described embodiment is not limited to being implemented only as the electronic device 1. For example, the above-described embodiment may be implemented as a control method for a device such as the electronic device 1. Furthermore, the above-described embodiment may be implemented as a program executed by a device such as the electronic device 1, or as a storage medium or recording medium on which a program is recorded.

[0198] The electronic device 1 according to the above-described embodiment has been described as including components constituting a so-called radar sensor, such as the transmitting antenna array 24 and the receiving antenna array 31. However, the electronic device according to an embodiment may be implemented as, for example, a configuration such as the signal processing unit 10. In this case, the signal processing unit 10 may be implemented as having a function of processing signals handled by, for example, the transmitting antenna array 24 and the receiving antenna array 31. [Explanation of symbols]

[0199] 1 Electronic equipment 10 Signal Processing Section 11 Signal generation processing section 12 Received signal processing section 13 Heart rate extraction unit 14 Calculation section 21 Transmit DAC 22 Transmitting circuit 23 Millimeter wave transmitter circuit 24 Transmitting Antenna Array 31 Receiving Antenna Array 32 Mixer 33 Receiving circuit 34 Receive ADC 50 Communication Interface 60 External equipment

Claims

1. a transmitting unit that transmits a transmission wave; a receiving unit that receives a reflected wave of the transmitted wave from a target; a signal processing unit that detects the distance, direction, and speed of the target based on a converted signal obtained by Fourier transforming beat signals of the transmitted wave and the received wave; An electronic device comprising: The signal processing unit extracting signal components corresponding to cardiac vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; performing frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration; performing a calculation process of statistical results on the results of the frequency analysis; and an electronic device that outputs time series data of the heartbeat frequency of the target based on the calculation result.

2. The electronic device according to claim 1 , wherein the signal processing unit performs super-resolution frequency analysis as the frequency analysis.

3. The electronic device according to claim 1 , wherein the signal processing unit performs the frequency analysis based on a subspace method.

4. The electronic device according to claim 3 , wherein the signal processing unit performs the frequency analysis based on a MUSIC method.

5. transmitting a transmission wave from a transmitting unit; receiving, at a receiving unit, a reflected wave of the transmitted wave from a target; detecting a distance, a direction, and a velocity of the target based on a converted signal obtained by performing a Fourier transform on beat signals of the transmitted wave and the received wave; extracting signal components corresponding to cardiac vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; a step of performing frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration; performing a statistical calculation process on the result of the frequency analysis; a step of outputting time series data of the heartbeat frequency of the target based on the calculation result; A method for controlling an electronic device, including:

6. For electronic devices, transmitting a transmission wave from a transmitting unit; receiving, at a receiving unit, a reflected wave of the transmitted wave from a target; detecting a distance, a direction, and a velocity of the target based on a converted signal obtained by performing a Fourier transform on beat signals of the transmitted wave and the received wave; extracting signal components corresponding to cardiac vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; a step of performing frequency analysis on an envelope signal obtained by performing envelope processing on the signal component corresponding to the heart sound vibration; performing a statistical calculation process on the result of the frequency analysis; a step of outputting time series data of the heartbeat frequency of the target based on the calculation result; A program that executes the following.

Images